Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

Please read this post before posting

Latest Cumulus release v1.9.4 (build 1099) - Nov 28 2014
Latest Cumulus MX release - v3.0.0 build 3043 Jan 20 2017. See this post for download

Decoding METARS

Talk about the weather
User avatar
saratogaWX
Posts: 863
Joined: Wed May 06, 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows XP SP3
Location: Saratoga, CA, USA
Contact:

Re: Decoding METARS

Postby saratogaWX » Fri Sep 29, 2017 5:55 pm

Wow, thanks for the detailed report about the changes!

I'm interested in your PHP7 code if you're willing to share.

I will be updating the get-metar-conditions-inc.php script to accommodate the 2017 changes (and a couple of bug fixes that have emerged lately).

Thanks!

Best regards,
Ken

sfws
Posts: 374
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Maplin based, but customised
Operating System: off-line Vista and W10 PCs
Location: UK

Re: Decoding METARS

Postby sfws » Sat Sep 30, 2017 8:55 pm

saratogaWX wrote:I'm interested in your PHP7 code if you're willing to share.

Shared (September 2017 version) with Ken by PM. I have a new version, but the changes relate to coping better with just a few foreign METAR which use RMK group or push the boundaries a bit (e.g. a colour state of 'BLU+')!

Although I cannot guarantee my script will always work, I have tested my script with around a thousand METAR, most are UK sourced, some are from North Sea rigs, some from European aerodromes nearby, and other examples have been taken from a wide selection of countries in all the continents. To test the standard international format a few of my test METAR have been made-up. Unlike many scripts found on-line, my script does attempt to cope with many malformed METARs I have encountered, and it does decode not just the mandatory key parts of a METAR report (that is all that the current version of Ken's script can handle) but also the more rarely used conditional elements. There is a Group within the METAR definition for general remarks. As the allowed content is set by each country and often is abbreviated plain text (not always in English) I can only decode a selection of the possible content there.

Any Canadians familiar with their METAR content can help me as I do not yet have ability to translate your two letter codes for cloud and general weather, (in RMK Group) nor understand your 'BBBB' system with RR?, CC?, AA? or P?? that appears before the Surface Wind Group.

The previous most comprehensive internationally-focussed METAR decoding option in a freely available source code was written (2002 to 2007) in JavaScript by Manuel Heras and is available at "http://www.heras-gilsanz.com/manuel/METAR-Decoder.html". North American use of Altimeter readings in "in Hg" instead of QNH in "hPa" and prevailing visibility in "statute miles" instead of "metres" was added to this script by Ricardo Fernández. This script is widely used on many commercial web sites (JavaScript is processed by a browser so your browser can therefore display it, and allow it to be downloaded) but that script does less than my script as it does not cope with malformed METAR entry, nor Colour State codes, and it excludes any RMK group decoding. Those unable to use PHP might want to use that script instead.
Last edited by sfws on Mon Oct 23, 2017 7:15 pm, edited 1 time in total.

User avatar
PaulMy
Posts: 1625
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Mon Oct 02, 2017 11:12 pm

In trying to reply to PM I get
"Some users couldn’t be added as they have disabled private message receipt."

You can email me.

Paul

User avatar
PaulMy
Posts: 1625
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Mon Oct 02, 2017 11:48 pm

Hi sfws;
Any Canadians familiar with their METAR content can help me as I do not yet have ability to translate your two letter codes for cloud and general weather, (in RMK Group) nor understand your 'BBBB' system with RR?, CC?, AA? or P?? that appears before the Surface Wind Group.


I am not knowledgeable about METAR but a couple of years ago I started a spreadsheet by regularly copying in the local CYXU METAR (London, ON) and comparing my station's data to the METAR, especially pressure. At that time I also copied into my spreadsheet the METAR definitions that was linked at the CYXU site. I haven't updated this for about 18 months and now see that the METAR definitions are no longer linked at the CYXU station.

I can share my spreadsheet if that could be helpful.

I presume you have looked at https://ec.gc.ca/manobs/default.asp?lan ... 3a380ded-1
If there is something specific in this that is not clear I would gladly get in touch with an EC Meteorologist who I communicate with on occasion (CANWARN) and see if he has the information or where to get it.

Regards,
Paul

sfws
Posts: 374
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Maplin based, but customised
Operating System: off-line Vista and W10 PCs
Location: UK

Re: Decoding METARS

Postby sfws » Tue Oct 03, 2017 8:22 am

PaulMy wrote:I presume you have looked at https://ec.gc.ca/manobs/default.asp?lan ... 3a380ded-1

Paul

That is exactly what I was looking for, I had tried a search for 'METAR' on a different part of the EC site, but it gave me access to METAR reports, not the definition. I should have thought of adding 'publications' or something similar to my search.

Thank you very much. From a quick glance, that almost certainly contains everything I wanted. When I'm next able to spend time coding, that will allow me to really make my decoder the most comprehensive one available, accepting METAR from anywhere.

Since other people successfully send me (and reply to) PMs, and I have not changed any settings since I cannot account for the error you saw.

User avatar
PaulMy
Posts: 1625
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Tue Oct 03, 2017 1:23 pm

Hi sfws, and glad that will be helpful, and look forward to your finished work. Let me know if I can be of any further help.

Since other people successfully send me (and reply to) PMs, and I have not changed any settings since I cannot account for the error you saw.
Your PM was received but could not send a reply and that message came up. Tried several times and got the same message. Today I couldn't even find a PM button on your profile but does show up for other users. Here is what I get from my control panel when I enter your user name to add recipient.

Enjoy,
Paul
You do not have the required permissions to view the files attached to this post.

sfws
Posts: 374
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Maplin based, but customised
Operating System: off-line Vista and W10 PCs
Location: UK

Re: Decoding METARS

Postby sfws » Tue Oct 03, 2017 6:32 pm

PaulMy wrote:when I enter your user name to add recipient
Maybe what others have been doing is just hitting 'send reply' button (i.e. using button when my PM is displayed), rather than attempting to create a new message, maybe I just clicked the wrong place in the user panel sometime or maybe when Steve migrated to a new bulletin board version a while ago now some setting defaults changed! Although I have been contributing quite a bit recently, there was a period when I was focussed on more active outdoor activities. Anyway, I have reset all my user panel settings, so that should cure any PM sending difficulties.

PaulMy wrote: look forward to your finished work. Let me know if I can be of any further help.

I realise now that looking at other decoders or at places like Wipedia to see what codes exist, muddles me as the information there is often incomplete or out of date. That is why it is critical that I look at detailed reference books written by the appropriate office in each country. But each time I look at the script I am improving it and I remain confident that I will soon have the best possible script to parse METAR and decode almost everything that is not free text!

I've just added Canada's "Variable Visibility" type to my decoder, I made a few errors (for me it is the end of a busy day) while typing in the extra code, but when I corrected my carelessness that new addition works on the METAR quoted as an example and gives an equivalent explanation. It will be a while before I work through all the extras as I have some more busy days coming, but I do feel cheered up!

User avatar
PaulMy
Posts: 1625
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Tue Oct 03, 2017 8:06 pm

Success I think, PM sent.

Paul

sfws
Posts: 374
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Maplin based, but customised
Operating System: off-line Vista and W10 PCs
Location: UK

Re: Decoding METARS

Postby sfws » Wed Nov 08, 2017 3:42 pm

sfws wrote:If anyone is interested, I have written in PHP 7 a METAR decoding script that takes into account all the 2017 changes

Since I wrote that quote, I changed my decoder to use "pow(base, value)" instead of "base ** value", and changed all my array declarations to use "x = array (value_list)" instead of "x =[value_list]", I made sure that I do not restrain type of return value in my user defined functions. As a result my scripts are now compatible with PHP versions from 5.4.0 to 7.1.14 (but not earlier versions of PHP that do not support PHP constructs like "labels" as these are an essential part of my design). There is more information about all the research I did to inform my suite in the PDF paper included in the zip. That paper also decribes how ICAO and WMO are forcing abandonment of METAR in TAC format of traditional alphanumerical characters. They have agreed a new international XML style combined with GML, which is already in use by some nations (including part of USA) and is not made directly available to pilots, let alone the public at large. My decoder is not compatible with that that digital based format. So you only have until 2020 to try my script with the latest routine aviation weather reports whilst there are METAR reports still using TAC and available to the public!

From perfecting the decoder and writing the different web pages I have learnt a lot about PHP, especially about using it with HTML forms and the advantages of using global variables and breaking a script up into lots of sub-functions. My final suite represents a complex mixture of approaches to the design because I wanted to try all possibilities; and it may be in-efficient because I am self-taught for all the web languages and do not have the expertise to improve that aspect. Essentially, this was my project for July to November 2017. I have not tried to make my decoder work with PHP 7.2 because that is not compatible with my Windows Vista PC.

I have developed many web pages that call my decoder. None of these are in the attached zip of the suite as they were all created to aid my testing and may not suit you. One web page script is available at http://www.komokaweather.com/metar/metar_display.php?view=metar_display and the second of my two alternatives specially written for "PaulMy" is at http://www.komokaweather.com/metar/metar_display-alt.php. For his web pages I choose to show all three possible units for wind speed by combining three outputs from my suite, but you could choose just one unit of measure. A major aim of my suite is to produce lots of output, but leave it to the user to decide what to show! There are thousands and thousands of lines of code because the traditional alphanuneric character (TAC) format for METAR are designed for trained humans to read, not for computers. Indeed, the WMO states it is impossible to write a computer programme than can decode any more than the basic METAR and that probably accounts for why I have not found any other decoder as ambitous as mine. To ensure I can decode almost all worldwide METAR my Remarks Group decoding script covers the most widely used codes (I call them Common) plus those relating specifically to Australia, Canada, New Zealand (including Antarctica), Italy, and USA.


This decoder has been in continuous use at http://www.komokaweather.com/metar/metar_display.php since November 2017.
(You can compare the output from my decoder with the output for similar airports using Saratoga version at http://www.komokaweather.ca/wxmetar.php).
The top level script of my decoder can be found at http://www.komokaweather.com/metar/metar_display.php?view=decodeMETAR or in the zip.
The script at the next level down is at http://www.komokaweather.com/metar/metar_display.php?view=decodeMETAR_sub_funct or in the zip.

For USA I am decoding specifications in both Civil and Air Force handbooks as there are some differences, and although both books specify a sequence to be followed, in practice USA content in the Remarks Group has both inconsistent sequencing and a free format for much of the abbreviated codes; together those issues mean that my understanding of USA Remarks content is a continuing challenge for me and a significant amount of work remains before I can release an update for Remarks Group handling. I believe I have cracked all but one (inconsistency in recognizing pressure tendency segment) of the outstanding Canadian Remarks Group issues in my testing environment ready for next release (not yet in zip).
Meanwhile you cannot yet view the source on Paul's site to see if he uses the same version as in the zip, this is partly because there may sometimes be a little delay between my release of an update to remarks group processing and his adoption or I may release a script to him for testing in advance of updating the zip on this post.

I can guarantee that not everything that will ever be included in a METAR will be recognized by my decoder. However, I have probably gone further than is reasonable in covering a very wide range of possible METAR element combinations, including some misformed content. Remember, the TAC format for METAR that my suite addresses is not precisely specified and in some places almost any text may be be found. As I can't therefore cover all possibilities, from time to time there will be some text included that I have not anticipated, or an observer will input a METAR that does not obey the published guidance so I may not have anticipated that particular content. To help me any failures to understand any METAR content for Canada and USA are logged at http://www.komokaweather.com/metar/failed_to_decode_metar.log. If I can spot an easy obvious improvement for coping with something in that log, I issue an update (see zip below). Ideally, people elsewhere in the world would be producing similar logs to help me to check how well my decoder deals with their local METAR. I have tested my decoder against METAR issued by around 220 different aerodromes around the world, but I do not have the time to do lots of testing and a key point about METAR is what is issued at the time of my testing may be totally different to what is issued one hour/day/week/month later!

Read the pdf documentation for full background on my project, but essentially to use my suite everything bar the PDF goes into same directory on your web site. Then as explained in pdf, you call a function specifying a raw METAR as the first parameter, and the function returns an array from which you extract anything you want to see.

metar_decoding_suite(16Feb2018).zip

Feb 2018: EDITS to ZIP - For full version history, see end of each script as I have brought the version history right up to date now. Predominately recent changes are just format improvements to the outputs from the decoder scripts. EDIT ends.

29 Jan 2018 EDIT - Just a few more edits to last year's version of suite, principally for decoding of remark group for Canada based on its use in Komoka (as already highlighted), but also some corrections to other functions (see version history in files). One key improvement is to the output for pressure, sea level pressure is now clearly identified separated from any altimeter pressure. (That is one of the key advantages of my decoder if you use it to get a pressure reading to compare against your own personal weather station). EDIT ends.

30 Dec EDIT - Release update at end of 2017, consolidated the improvements and corrections that I have been doing since PaulMy started using my decoder prior to 8 November when I created this post, as his live running identified a few problems. I have had other priorities in my life since then, so can't do very much now. EDIT ends.

[DEC 2017 - Various edits that are just correcting my spelling errors and changing the paragraph order from that I used when I made this post originally; not releasing any updates yet]

November 2017 - Original post - original completion of my project with an update to my suite so it will cover more possible content and more PHP versions.
You do not have the required permissions to view the files attached to this post.
Last edited by sfws on Sun Feb 25, 2018 9:12 am, edited 16 times in total.

User avatar
PaulMy
Posts: 1625
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Fri Jan 12, 2018 3:17 pm

I have been using this decoder for some time now and this has been providing feedback on our local Metar conditions to help in overcoming some of the differences/variations by local airport Metar reporting. Sfws has put a tremendous amount of time and effort on this decoder and I find it very interesting to compare our area Metars http://www.komokaweather.com/metar/metar_display.php
Use the Click to Show/Hide to see more Metars...

Enjoy,
Paul

User avatar
PaulMy
Posts: 1625
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Sat Feb 17, 2018 2:47 am

For those following this metar decoder, sfws has updated the zip (16Feb2018) and this contains a PDF on the project. This helps to understand the complexity and the work and time involved with the decoder.

Enjoy,
Paul


Return to “Weather”

Who is online

Users browsing this forum: mcrossley and 5 guests