ILMSinfo / RBIS: Import of time series data
From ILMS-Wiki
(Difference between revisions)
(→Single Import) |
|||
| (32 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
There are different options to import time series data: | There are different options to import time series data: | ||
# Single import | # Single import | ||
| − | # Mass import including | + | # Mass import including metadata creation with or without preprocessing |
# Contentious automatic import | # Contentious automatic import | ||
(For 2. and 3. please contact your RBIS administrator.) | (For 2. and 3. please contact your RBIS administrator.) | ||
= First steps = | = First steps = | ||
| − | == | + | == Metadata creation == |
| − | Before you can upload new time series data you have to '''create''' the corresponding | + | #Before you can upload new time series data you have to '''create''' the corresponding [[ILMSinfo_/_RBIS:_Station|station]], where the data was measured. The station can be a real or virtual station (simulated data). To make sure to be able to use the full functionality of RBIS fill out at least the coordinates and spatial reference system. |
| − | + | #The second step is to '''create''' the '''metadata''' of your '''time series data'''. Please make sure to fill out at least Dataset Responsible Person or Organization and / or Distributor and the Dataset Use Limitation. | |
| − | The second step is to '''create''' the ''' | + | |
== Data preprocessing == | == Data preprocessing == | ||
| + | |||
To import time series data you have to convert your data in the following structure: | To import time series data you have to convert your data in the following structure: | ||
| − | + | * One or more columns of values | |
| − | Date and time [tab] value [tab] value | + | : ''Date and time [tab] value [tab] value'' |
| − | + | ||
| − | + | : [[File:file_ts_multi_parameter.jpg]] | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | * With or without time | |
| − | Date and time [tab] value | + | :''Date and time [tab]'' value or ''Date and time [tab] value'' |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | :[[File:file_ts_with_time.jpg]] | |
| − | + | ||
| − | + | :[[File:file_ts_without_time.jpg]] | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
* The format of your date can be: [year/month/day] [day/month/year] [month/day/year] | * The format of your date can be: [year/month/day] [day/month/year] [month/day/year] | ||
| Line 47: | Line 33: | ||
* Missing values or dates do not have to be filled | * Missing values or dates do not have to be filled | ||
| − | + | : [[File:file_ts_gaps.jpg]] | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| + | :will be detected as one gap -> start: 02.01.1990 end: 03.01.1990 length: 2 | ||
* comments need to be introduced with # or * | * comments need to be introduced with # or * | ||
| − | + | : [[File:file_ts_comment.jpg]] | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
* Make sure that there are no empty lines between your comments and the data | * Make sure that there are no empty lines between your comments and the data | ||
| Line 100: | Line 73: | ||
'''Gap detection'''<br> | '''Gap detection'''<br> | ||
Please note that gaps will only be detected during upload if the interval is regular based on seconds (hourly, daily, … ). Gaps of e.g. monthly or yearly data will be not detected. | Please note that gaps will only be detected during upload if the interval is regular based on seconds (hourly, daily, … ). Gaps of e.g. monthly or yearly data will be not detected. | ||
| + | |||
| + | [[Category:ILMSinfo]] | ||
| + | [[Category:RBISts]] | ||
Latest revision as of 15:42, 6 December 2011
There are different options to import time series data:
- Single import
- Mass import including metadata creation with or without preprocessing
- Contentious automatic import
(For 2. and 3. please contact your RBIS administrator.)
Contents |
First steps
Metadata creation
- Before you can upload new time series data you have to create the corresponding station, where the data was measured. The station can be a real or virtual station (simulated data). To make sure to be able to use the full functionality of RBIS fill out at least the coordinates and spatial reference system.
- The second step is to create the metadata of your time series data. Please make sure to fill out at least Dataset Responsible Person or Organization and / or Distributor and the Dataset Use Limitation.
Data preprocessing
To import time series data you have to convert your data in the following structure:
- One or more columns of values
- Date and time [tab] value [tab] value
- With or without time
- Date and time [tab] value or Date and time [tab] value
- The format of your date can be: [year/month/day] [day/month/year] [month/day/year]
- The date needs to be in ascending order
- The measurement values need to be numeric (4.5 is valid; x a ‘ + non valid)
- The decimal delimiter is “,” or “.” (4,5 and 4.5 is valid)
- The value for missing entries needs to be consistent (either -9999 or 9999 or -99 or …)
- Missing values or dates do not have to be filled
- will be detected as one gap -> start: 02.01.1990 end: 03.01.1990 length: 2
- comments need to be introduced with # or *
- Make sure that there are no empty lines between your comments and the data
- Save your file as tab stop separated text file or as CSV file.
In case of a low traffic internet connection you can zip your file (*.zip) before uploading it.
Single Import
To import new time series data follow the link “Import”.
Follow the steps in the upload form
- General content information
- Here you can select the number of columns (do not count the date column) and select the type of the values. According to the selected type (quality or physical) a list of parameter and units will appear in the selection list below. If a parameter or unit is missing please contact your RBIS administrator.
- Information about the file
- Select your file
- Specify the file format (text or CSV file)
- Specify the type of the date
- The function “time shift” corrects the given input time. The time shift in a time interval is corrected. The reference for shifting is the first given time. (For example: 5-minutes values: 12:30 12:35 12:41 12:48 => 12:30 12:35 12:40 12:50)
- Specify the used character for the beginning of a comment line
- Specify the time interval and format of the missing values
- Upload file
Template
You can save your filled form. Plaese type a name and click on “Save as template”.
The saved template will be availible in the list of predefined templates at the top of the form.
Gap detection
Please note that gaps will only be detected during upload if the interval is regular based on seconds (hourly, daily, … ). Gaps of e.g. monthly or yearly data will be not detected.
