Differences

This shows you the differences between two versions of the page.

--- eolab:openhype:start [2023/05/21 11:03] – [2.4 Observation Data in the Database] sina001
+++ eolab:openhype:start [2024/03/15 17:21] (current) – [2.1 Data Flow] rolf.becker
@@ Line 111: / Line 111: @@
-|  {{ :eolab:openhype:data_flow.png?600 |}}  |
+|  {{ :eolab:openhype:data_flow.png?600&direct |}}  |
 |  // Image 1- Data flow//               |
@@ Line 145: / Line 145: @@
 ===== 2.3 Data Engineering =====
-===== 2.3.1 Downloading the data =====
+===== 2.3.1 Downloading the Data =====
 In the first step, The data must be downloaded from [[ https://www.opengeodata.nrw.de/produkte/umwelt_klima/wasser/grundwasser/hygrisc/ |here]].
@@ Line 405: / Line 405: @@
 As we can see in Image 11, the minimum date is ** 1951-04-30**
-** Create geometry column in messstelle table:** In this section, we want to create a geometry column from **e32** and **n32** columns from the messstelle table. with the below code, we are able to create a new column and we set the name as **geom**
+** Create geometry column in messstelle table:** In this section, we want to create a geometry column from **e32** and **n32** columns from the messstelle table. With the below code, we are able to create a new column and we set the name as **geom**
 <code>
@@ Line 416: / Line 416: @@
-Now the messstelle table has one more column (geom) which consists of the geometry information of the location of each station.
+The "messstelle" table has been enhanced with an additional column called "geom," which contains the geometry information representing the location of each station.
 **Merge two tables:**
-In this section, we want to merge the two tables (messwert and messstelle) based on the same column which is "messstelle_id". We need to select columns that we need from each tables and then merge them based on the "messstelle_id"
+In this section, we aim to merge the "messwert" and "messstelle" tables based on the common column, "messstelle_id." To achieve this, we will select the desired columns from each table and then perform the merge based on the "messstelle_id" column.
 <code>
@@ Line 431: / Line 431: @@
 |  // Image 13- Merge tables//  |
-Now we need to create a view and save this SQL command as a new view. the name of this new view is "nitrat_geom".
+In order to have the above SQL command available as a new view for the subsequent section in QGIS, we should store it accordingly.
 <code>
@@ Line 439: / Line 440: @@
 </code>
-We need this new view for the next section in QGIS.
 ===== 2.5 QGIS=====
-QGIS is an open-source and free application that can support viewing, editing and analysis of geospatial data.
+QGIS  is a free and open-source geographic information system (GIS) software. It provides a wide range of tools and functionalities for visualizing, analyzing, and managing geospatial data. QGIS supports various data formats and allows users to create, edit, and publish maps.
 You can download QGIS for free from the below link.
@@ Line 449: / Line 450: @@
 [[https://qgis.org/en/site/]]
-The below video shows how to download and install QGIS for windows which are highly recommended to watch before installing it.
+The below video shows how to download and install QGIS for Windows which is highly recommended to watch before installing it.
 |  {{ youtube>LDjY-9ucvys? }}                       |
 |  // Video 7- How to download and install QGIS //  |
-Now is the time to get to know about QGIS and the below video can help so much.
+To gain a better understanding of QGIS, I recommend watching the following video, which provides valuable insights and guidance on using the software.
 |  {{youtube>kCnNWyl9qSE?}}         |
@@ Line 460: / Line 461: @@
-**Create a time series video:** In this section, we want to create a time series video to see how the nitrate concentration has been changed over time in North Rhine-Westphalia which is the most crowded state in Germany. First,  we need to download the shapefile of the North Rhine-Westphalia state and load it into QGIS.
+**Create a time series video:** In this section, our objective is to generate a time series video depicting the changes in nitrate concentration over time in North Rhine-Westphalia, the most populous state in Germany. To begin, we must download the shapefile for North Rhine-Westphalia and import it into QGIS for further analysis and visualization.
-We will download three shapefiles,
+Three below shapefiles need to be downloaded
-  * Whole state shapefile (dvg1bld_nw.shp)
+  * entire state shapefile (dvg1bld_nw.shp)
   * kreis shapefile (dvg1krs_nw.shp)
   * Gemeinde shapefile (dvg1gem_nw.shp)
-All the three shapefiles can be downloaded from [[https://github.com/sina7272/LANUV_Groundwater/tree/main/data/DVG | here]]. After downloading, we need to load them to the QGIS to see them. The below video shows how to load the shapefile in QGIS.
+All three shapefiles can be downloaded from [[https://github.com/sina7272/LANUV_Groundwater/tree/main/data/DVG | here]].
+After downloading the shapefiles, we can proceed to load them into QGIS for visualization and analysis. The below video shows how to load the shapefile in QGIS.
 |  {{youtube>Fg11ZDV2HYI?}}                        |
@@ Line 475: / Line 477: @@
 Now we can see the map of NRW, kreis and Gemeinde. There are two options to create a video for time series.
-**Locally with shapefile:** In here,  we need to have a shapefile that consists of the nitrate concentration over time. download the notebook from  [[https://github.com/sina7272/LANUV_Groundwater/blob/main/Notebook/shape_file_nitrat.ipynb| here]] and run the python codes to create two shapefiles. then we should load these two shapefile to the QGIS. The first shapefile is consist of all stations in NRW and the second one is consist of the nitrate concentration.
+**Locally with shapefile:** In here,  we need to have a shapefile that consists of the nitrate concentration over time. download the notebook from  [[https://github.com/sina7272/LANUV_Groundwater/blob/main/Notebook/shape_file_nitrat.ipynb| here]] and run the Python codes to create two shapefiles. then we should load these two shapefiles to the QGIS. The first shapefile consists of all stations in NRW and the second one consists of the nitrate concentration.
 The below video shows how we can load shapefiles to QGIS.
@@ Line 484: / Line 486: @@
 **Connect to Database:**
- The below video shows how we can connect our QGIS to Database and load the file.
+ The below video shows how we can connect our QGIS to Database and load the file from Database.
 |  {{youtube>UL4nBRshNIA?}}                                           |
 |  // Video 11- How to connect database to QGIS and load the files//  |
@@ Line 493: / Line 495: @@
-This section will discuss how we can create an interactive dashboard for our data. An interactive dashboard is a tool that users can interrelate with data by analyzing, visualizing as well as monitoring the data.
+In this section, the process of creating an interactive dashboard for our data will be explored. An interactive dashboard is a versatile tool that allows data to be interacted with, analyzed, visualized, and key information to be monitored by users.
-Two approaches to creating a dashboard will be discussed in this section. The aim of this dashboard is a simple interactive dashboard in which users with no knowledge of programming can easily consider the data as well as a map. This kind of dashboard will help users to understand data better. Nowadays dashboards are widely used in several ways to help managers and decision-makers to make decision easier. one good example of such a kind of dashboard is the Covid-19 dashboard which each country also here in Germany people are widely used. The Covid-19 Dashboard aids people in noticing how many new cases and how many new deaths have been recorded in different periods of time.
+This section discusses two approaches to creating a dashboard. The objective of this dashboard is to provide a user-friendly and interactive interface for data exploration and visualization, accessible even to non-programmers. Such a dashboard plays a crucial role in enhancing data comprehension and is widely utilized by managers and decision-makers to facilitate informed decision-making processes.
-In our case, we want to create a simple dashboard which shows the map of NRW as well as Nitrate and Sulfate concentration rates at different times.
+ One notable example of this type of dashboard is the Covid-19 dashboard, which has gained widespread usage worldwide, including in Germany. The Covid-19 dashboard provides users with valuable insights into the number of new cases and deaths reported over various time periods. It helps individuals track the progression of the pandemic and understand the impact it has had on different regions and countries.
-** 3.1 Plotly Dash: **
+In our specific case, the objective is to develop a straightforward dashboard that showcases the map of North Rhine-Westphalia (NRW) alongside the concentration rates of Nitrate and Sulfate at different time intervals. This dashboard will provide a visual representation of the spatial distribution of these pollutants and enable users to observe any temporal variations in their concentrations within NRW.
+** Plotly Dash: **
 ** Plotly: ** Plotly is a computing company located in Montreal, Canada. They develop online data analytics and visualization tools.
 Plotly offers online graphing, analytics, and statistics tools for their users, as well as scientific graphing libraries for Python, R, MATLAB, Perl, Julia, Arduino, and REST. Plotly offers several open-source and enterprise products such as Dash which have been used for creating simple and interactive dashboards in this project.
-** Dash: ** Dash is a framework to build data apps rapidly not only in Python but also in R, Julia, and F#. According to Plotly official website, Dash is downloaded 800,000 times per month which shows that nowadays Dash getting more popular. Dash is a great framework for anyone who uses data with a customised user interface. Through a couple of simple patterns, Dash eliminated all of the technologies as well as protocols that are needed to make a full-stack web app with interactive data considerations. Another good feature is that Dash is running on web browsers so it means that no other application needs to run it.
+** Dash: ** Dash is a framework to build data apps rapidly not only in Python but also in R, Julia, and F#. According to Plotly's official website, Dash is downloaded 800,000 times per month which shows that nowadays Dash getting more popular. Dash is a great framework for anyone who uses data with a customised user interface. Through a couple of simple patterns, Dash eliminated all of the technologies as well as protocols that are needed to make a full-stack web app with interactive data considerations. Another good feature is that Dash is running on web browsers so it means that no other application needs to run it.
-If you would like to know more about Dashboard with Plotly Dash, click the link below. In the below link, you will find full tutorials about how to create a simple dashboard with Plotly Dash.
+To learn more about creating a dashboard with Plotly Dash, you can follow the link provided below. This resource contains comprehensive tutorials that guide you through the process of building a simple dashboard using Plotly Dash. These tutorials will provide you with step-by-step instructions and examples to help you create interactive and visually appealing dashboards using Plotly Dash.
@@ Line 515: / Line 519: @@
 All the source codes of the dash gallery are available in [[ https://github.com/plotly/dash-sample-apps/|| here]]
+** Dashboard Design: **
+I have developed a web application dashboard that effectively visualizes time series data for Nitrate and Sulfate. The image below shows the main page of our dashboard.
+|  {{:eolab:openhype:main_page.jpg?600|}}  |
+|  // Image 14- Main page of the dashboard //  |
+In order to run the above dashboard on your local system, you need to do the following:
+[[ https://github.com/sina7272/LANUV_Groundwater |Click here and pull the repository ]]
+direct to the "Dashboard" folder, then all the Python codes and SQL queries are available to run the application.
+There is a yml file inside the folder which name is "dashboard_environment". This yml file will create an environment with all the necessary packages which you will need to run the application. The first step needs to load this file to the Anaconda, if you are not sure how to do that please refer to the 2.3.3 Anaconda section.
+The next step is to add the credentials (Username, DB name, password and ...) of the database to the "credential_temp" file.
+And the last step is to run the "app.py" inside the new environment that you have already created with the help of "dashboard_environment.yml"
-** 3.2 Panel: **
 ===== 4. Result=====
@@ Line 523: / Line 547: @@
 ==== Nitrate concentration 2000-2010 ====
-The below video has shown the concentration of nitrate in NRW from 2000 to 2010.
+The video below shows the concentration of nitrate in North Rhine-Westphalia (NRW) from 2000 to 2010. This visualization was created using QGIS 3.16. By watching the video, you can observe the temporal changes in nitrate levels across NRW during the specified time period.
-The video is created with QGIS 3.16
 {{ youtube>IMznT88l2VQ?medium }}
@@ Line 532: / Line 555: @@
 ==== Nitrate concentration 2010-2020 ====
-The below video has shown the concentration of nitrate in NRW from 2010 to 2020.
+The video below shows the concentration of nitrate in NRW from 2010 to 2020.
-The video is created with QGIS 3.16
+The video was created with QGIS 3.16
@@ Line 544: / Line 567: @@
 ==== Sulfat concentration 2000-2010 ====
-The below video has shown the concentration of sulfate in NRW from 2000 to 2010. The video is created with QGIS 3.16
+The video below shows the concentration of sulfate in NRW  from 2000 to 2010. The video was created with QGIS 3.16
 {{ youtube>Jz2cINIOwWs?medium }}
@@ Line 551: / Line 574: @@
 ==== Sulfat concentration 2010-2020 ====
-The below video has shown the concentration of sulfate in NRW from 2010 to 2020. The video is created with QGIS 3.16
+The video below shows the concentration of sulfate in NRW from 2010 to 2020. The video is created with QGIS 3.16
 {{ youtube>0SqfLi2lPH8?medium }}