Autodesk

Development site for Square One's software and teaching resources


WeatherTool: Importing Data

Unfortunately, accurate and comprehensive weather data is often difficult to obtain. Even when purchased in electronic form from official meteorological offices it can be incomplete and in the strangest of formats. To assist with this, the Weather Tool recognises a wide range of known file formats and even lets you tailor your own to match almost any ASCII format available. A list of formats that the Weather Tool already automatically recognises is as follows:

  • TMY Climate Data (TMY)
  • TMY2 Climate Data (TM2)
  • TRNSYS TMY Variant (TRY)
  • Aus. BOM Hourly Data (LST)
  • CSIRO Weather Data (DAT)
  • NatHERS Climate Data
  • ASHRAE WYEC2 Data
  • The Weather Tool v1.10

Introduction

For weather data to be useful in most analysis routines, you really need a full year of hourly weather data. This cannot be monthly averages or even daily ranges, but actual data recordings or design values for every hour of the day. The data required includes:

  • Air Temperature
  • Relative Humidity
  • Global or Direct Solar Radiation
  • Diffuse Horizontal Solar Radiation
  • Wind speed

It is also desirable but not essential to include:

  • Wind direction
  • Cloudiness
  • Rainfall

Obtaining the right data for The Weather Tool means obtaining digital hourly data sets from some source or other. If you overcome that hurdle, then the software can import almost any ASCII file format as well as some binary files. As there are so many different 'standards' for weather data, you will most likely also need to know the format of the data within the file(s) in case you need to build up an import/export profile, a relatively simple process within the software itself.

The main source of localised weather data is always going to be the local government weather station or the closest airport. We have tried to provide a range of data on the Square One website, however it is impossible to collect data for all the major population centres let alone smaller towns and cities.

Weather File Formats

There are a plethora of weather file formats available. It seems that almost every new analysis tool takes pride in using its own 'standard' file format - the Weather Tool included. Unfortunately though, like every other developer before me, I reckon mine is the only logical way of doing it and cannot possibly understand why anyone else would want to store it any other way. That said, there are generally two very distinct types of weather data available in digital form, ASCII and binary files.

ASCII Weather Files

ASCII stands for American Standard Code for Information Interchange. An ASCII file has all its data stored as text in the file, or as a series of human readable alphanumeric characters. I say human readable here because they consist of recognisable characters, however there are very few ASCII weather file formats that are actually human comprehensible.

Such files almost invariably record a single hour's worth of climate data on a single line within the file, with a full year requiring at least 8760 lines (365x24). The order in which values for date, time, temperature, humidity or cloud cover actually occur within that line is determined by the file's format (not its 'type' which is ASCII). In addition to the values themselves, many formats include data uncertainty flags to indicate whether the values are actual measurements, interpolated from measures made at other times or mathematically generated because recorded data was not available.

Within ASCII weather files there are two further distinct groups:

  1. Fixed Format Data
    In this type of file, values occupy fixed columns within each line. For example, the first two characters may represent the day number, the next two the month number, the next two the hour of the day, the next five the dry bulb temperature in C, etc. Such files are characteristic of computer programs written in the FORTRAN programming language as it is particularly adept at handling such a record format. This means that each line is exactly the same length and each value is therefore right-padded to the required length with zeros or spaces.
  2. Column Separated Data
    These files use a special character, usually a coma or tab, to separate or delimit each value in the line. This means that each values can be as long or as short as needed, provided the delimiting character is present between each one. See the related links section below for detailed information on some of the commonly used examples of ASCII weather file formats.

ASCII weather files tend to be quite large (usually more than 800K each) and, as they require a lot of parsing by the computer program as they are read in or written out, tend to be quite slow to work with.

Binary Weather File

Binary files tend to be stored as on disk in almost the same way as they are stored in memory and accessed by the computer program that uses them. As such, they are usually relatively small (< 200K) and very quick to read and write. However, they are completely unreadable by humans and, unless the detailed binary format is made available by the developer, unreadable by other programs as well.

The Weather Tool uses its own proprietary binary file format called a .WEA file - simply because it is quick to read in and relatively small in size. However, even though its format is readily available, it is doubtful few programs other than the Square One suite of analysis tools will ever use it. What it does is provide an efficient means of storing and transferring large amounts of weather data for use by software that can convert it to any other ASCII format required.

The point is, however, that unless you know for sure that the software you have can read a particular binary file format, it will likely end up of absolutely no use to you. You will not be able to simply work out its format by loading it into a spreadsheet program and simply looking at it.

WeatherTool: Technical
WeatherTool: Tutorial Fixed

HOW WOULD YOU RATE THIS PAGE/DEFINITION ?

No votes yet