ILMSinfo / RBIS: Import of time series data

From ILMS-Wiki
Revision as of 11:02, 2 December 2011 by Fzander (Talk | contribs)
Jump to: navigation, search

There are different options to import time series data:

  1. Single import
  2. Mass import including meta-data creation with or without preprocessing
  3. Contentious automatic import

(For 2. and 3. please contact your RBIS administrator.)

Contents

First steps

Meta-data creation

Before you can upload new time series data you have to create the corresponding station, where the data was measured. The station can be a real or virtual station (simulated data). To make sure to be able to use the full functionality of RBIS fill out at least the coordinates and spatial reference system.

The second step is to create the meta-data of your time series data. Please make sure to fill out at least Dataset Responsible Person or Organization and / or Distributor and the Dataset Use Limitation.

Data preprocessing

To import time series data you have to convert your data in the following structure:

a) One or more columns of values
Date and time [tab] value [tab] value

File ts multi parameter.jpg

b) With or without time
Date and time [tab] value

File ts with time.jpg

or Date and time [tab] value

File ts without time.jpg

  • The format of your date can be: [year/month/day] [day/month/year] [month/day/year]
  • The date needs to be in ascending order
  • The measurement values need to be numeric (4.5 is valid; x a ‘ + non valid)
  • The decimal delimiter is “,” or “.” (4,5 and 4.5 is valid)
  • The value for missing entries needs to be consistent (either -9999 or 9999 or -99 or …)
  • Missing values or dates do not have to be filled

File ts gaps.jpg

will be detected as one gap -> start: 02.01.1990 end: 03.01.1990 length: 2


  • comments need to be introduced with # or *

#time series data precipitation (mm) and relative humidity (%) at Jena Sternwarte

01.01.1990 9:00 4.5 45
01.01.1990 10:00 3.6 50
  • Make sure that there are no empty lines between your comments and the data
  • Save your file as tab stop separated text file or as CSV file.

In case of a low traffic internet connection you can zip your file (*.zip) before uploading it.

Single Import

To import new time series data follow the link “Import”.

Import ts.png

Upload ts form.png

Follow the steps in the upload form

  1. General content information
    • Here you can select the number of columns (do not count the date column) and select the type of the values. According to the selected type (quality or physical) a list of parameter and units will appear in the selection list below. If a parameter or unit is missing please contact your RBIS administrator.
  2. Information about the file
    • Select your file
    • Specify the file format (text or CSV file)
    • Specify the type of the date
    • The function “time shift” corrects the given input time. The time shift in a time interval is corrected. The reference for shifting is the first given time. (For example: 5-minutes values: 12:30 12:35 12:41 12:48 => 12:30 12:35 12:40 12:50)
    • Specify the used character for the beginning of a comment line
    • Specify the time interval and format of the missing values
  3. Upload file


Template
You can save your filled form. Plaese type a name and click on “Save as template”. The saved template will be availible in the list of predefined templates at the top of the form.


Gap detection
Please note that gaps will only be detected during upload if the interval is regular based on seconds (hourly, daily, … ). Gaps of e.g. monthly or yearly data will be not detected.

Personal tools