Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

Please read this post before posting

Latest Cumulus release v1.9.4 (build 1099) - Nov 28 2014
Latest Cumulus MX release - v3.0.0 build 3043 Jan 20 2017. See this post for download

Decoding METARS

Talk about the weather
User avatar
saratogaWX
Posts: 891
Joined: Wed May 06, 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows XP SP3
Location: Saratoga, CA, USA
Contact:

Re: Decoding METARS

Postby saratogaWX » Fri Sep 29, 2017 5:55 pm

Wow, thanks for the detailed report about the changes!

I'm interested in your PHP7 code if you're willing to share.

I will be updating the get-metar-conditions-inc.php script to accommodate the 2017 changes (and a couple of bug fixes that have emerged lately).

Thanks!

Best regards,
Ken

sfws
Posts: 416
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Customised mix of Fine Offset
Operating System: having problems with PC
Location: UK

Re: Decoding METARS

Postby sfws » Sat Sep 30, 2017 8:55 pm

saratogaWX wrote:I'm interested in your PHP7 code if you're willing to share.

Shared (September 2017 version) with Ken by PM. I have a new version, but the changes relate to coping better with just a few foreign METAR which use RMK group or push the boundaries a bit (e.g. a colour state of 'BLU+')!

Although I cannot guarantee my script will always work, I have tested my script with around a thousand METAR, most are UK sourced, some are from North Sea rigs, some from European aerodromes nearby, and other examples have been taken from a wide selection of countries in all the continents. To test the standard international format a few of my test METAR have been made-up. Unlike many scripts found on-line, my script does attempt to cope with many malformed METARs I have encountered, and it does decode not just the mandatory key parts of a METAR report (that is all that the current version of Ken's script can handle) but also the more rarely used conditional elements. There is a Group within the METAR definition for general remarks. As the allowed content is set by each country and often is abbreviated plain text (not always in English) I can only decode a selection of the possible content there.

Any Canadians familiar with their METAR content can help me as I do not yet have ability to translate your two letter codes for cloud and general weather, (in RMK Group) nor understand your 'BBBB' system with RR?, CC?, AA? or P?? that appears before the Surface Wind Group.

The previous most comprehensive internationally-focussed METAR decoding option in a freely available source code was written (2002 to 2007) in JavaScript by Manuel Heras and is available at "http://www.heras-gilsanz.com/manuel/METAR-Decoder.html". North American use of Altimeter readings in "in Hg" instead of QNH in "hPa" and prevailing visibility in "statute miles" instead of "metres" was added to this script by Ricardo Fernández. This script is widely used on many commercial web sites (JavaScript is processed by a browser so your browser can therefore display it, and allow it to be downloaded) but that script does less than my script as it does not cope with malformed METAR entry, nor Colour State codes, and it excludes any RMK group decoding. Those unable to use PHP might want to use that script instead.
Last edited by sfws on Mon Oct 23, 2017 7:15 pm, edited 1 time in total.

User avatar
PaulMy
Posts: 1698
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Mon Oct 02, 2017 11:12 pm

In trying to reply to PM I get
"Some users couldn’t be added as they have disabled private message receipt."

You can email me.

Paul

User avatar
PaulMy
Posts: 1698
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Mon Oct 02, 2017 11:48 pm

Hi sfws;
Any Canadians familiar with their METAR content can help me as I do not yet have ability to translate your two letter codes for cloud and general weather, (in RMK Group) nor understand your 'BBBB' system with RR?, CC?, AA? or P?? that appears before the Surface Wind Group.


I am not knowledgeable about METAR but a couple of years ago I started a spreadsheet by regularly copying in the local CYXU METAR (London, ON) and comparing my station's data to the METAR, especially pressure. At that time I also copied into my spreadsheet the METAR definitions that was linked at the CYXU site. I haven't updated this for about 18 months and now see that the METAR definitions are no longer linked at the CYXU station.

I can share my spreadsheet if that could be helpful.

I presume you have looked at https://ec.gc.ca/manobs/default.asp?lan ... 3a380ded-1
If there is something specific in this that is not clear I would gladly get in touch with an EC Meteorologist who I communicate with on occasion (CANWARN) and see if he has the information or where to get it.

Regards,
Paul

sfws
Posts: 416
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Customised mix of Fine Offset
Operating System: having problems with PC
Location: UK

Re: Decoding METARS

Postby sfws » Tue Oct 03, 2017 8:22 am

PaulMy wrote:I presume you have looked at https://ec.gc.ca/manobs/default.asp?lan ... 3a380ded-1

Paul

That is exactly what I was looking for, I had tried a search for 'METAR' on a different part of the EC site, but it gave me access to METAR reports, not the definition. I should have thought of adding 'publications' or something similar to my search.

Thank you very much. From a quick glance, that almost certainly contains everything I wanted. When I'm next able to spend time coding, that will allow me to really make my decoder the most comprehensive one available, accepting METAR from anywhere.

Since other people successfully send me (and reply to) PMs, and I have not changed any settings since I cannot account for the error you saw.

User avatar
PaulMy
Posts: 1698
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Tue Oct 03, 2017 1:23 pm

Hi sfws, and glad that will be helpful, and look forward to your finished work. Let me know if I can be of any further help.

Since other people successfully send me (and reply to) PMs, and I have not changed any settings since I cannot account for the error you saw.
Your PM was received but could not send a reply and that message came up. Tried several times and got the same message. Today I couldn't even find a PM button on your profile but does show up for other users. Here is what I get from my control panel when I enter your user name to add recipient.

Enjoy,
Paul
You do not have the required permissions to view the files attached to this post.

sfws
Posts: 416
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Customised mix of Fine Offset
Operating System: having problems with PC
Location: UK

Re: Decoding METARS

Postby sfws » Tue Oct 03, 2017 6:32 pm

PaulMy wrote:when I enter your user name to add recipient
Maybe what others have been doing is just hitting 'send reply' button (i.e. using button when my PM is displayed), rather than attempting to create a new message, maybe I just clicked the wrong place in the user panel sometime or maybe when Steve migrated to a new bulletin board version a while ago now some setting defaults changed! Although I have been contributing quite a bit recently, there was a period when I was focussed on more active outdoor activities. Anyway, I have reset all my user panel settings, so that should cure any PM sending difficulties.

PaulMy wrote: look forward to your finished work. Let me know if I can be of any further help.

I realise now that looking at other decoders or at places like Wipedia to see what codes exist, muddles me as the information there is often incomplete or out of date. That is why it is critical that I look at detailed reference books written by the appropriate office in each country. But each time I look at the script I am improving it and I remain confident that I will soon have the best possible script to parse METAR and decode almost everything that is not free text!

I've just added Canada's "Variable Visibility" type to my decoder, I made a few errors (for me it is the end of a busy day) while typing in the extra code, but when I corrected my carelessness that new addition works on the METAR quoted as an example and gives an equivalent explanation. It will be a while before I work through all the extras as I have some more busy days coming, but I do feel cheered up!

User avatar
PaulMy
Posts: 1698
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Tue Oct 03, 2017 8:06 pm

Success I think, PM sent.

Paul

sfws
Posts: 416
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Customised mix of Fine Offset
Operating System: having problems with PC
Location: UK

Re: Decoding METARS

Postby sfws » Wed Nov 08, 2017 3:42 pm

Some facts about my decoding suite of PHP scripts:
# - My scripts are now compatible with all PHP versions from 5.4.0 to 7.1.15
# - This project was essentially a challenge I set myself with the aim of learning much more about PHP (I am entirely self-taught)
#- Essentially, this was my major project for July to November 2017, PaulMy has been using the suite continuously since 7 November 2017, my suite was logging any initial failures to understand particular unanticipated METAR content for his choice of Canada and USA airports at
http://www.komokaweather.com/metar/encoded_metar.txt and more complicated difficulties were logged at http://www.komokaweather.com/metar/failed_to_decode_metar.log.
# - Most updates since this posting was originally made have been improvements based on something in those logs. I now have many other activities to fit in, so any recent updates will have been tested less well than the original script in November 2017.
# - The lack of any similar live running by anyone elsewhere means I must rely on my own testing to refine the script for airports outside Canada/USA. I can test my decoder against METAR issued by around 220 different aerodromes around the world, but I do not have the time to do lots of testing and a key point about METAR is what is issued at the time of my testing may be totally different to what is issued one hour/day/week/month later! I have tested against a couple of thousand issues, but that represents a drop in the ocean compared to the number actually issued since I started in July 2017.
# - The aim of my project was to output British English explanations of everything contained in any METAR issued by any airport anywhere in the world. I have not been able to find any other decoder that does not specialise in a sub-set of METAR content. Indeed, the WMO states it is impossible to write a computer programme than can decode any more than the basic METAR and that probably accounts for why I have not found any other decoder as ambitous as mine.
# - I can guarantee that not everything that will ever be included in a METAR will be recognized by my decoder. However, I have probably gone further than is reasonable in covering a very wide range of possible METAR element combinations, including much misformed content.
# - It remains possible that unexpected content might confuse my suite and could cause a crash, but I do believe I have anticipated as much content as I can. Remember, the TAC format for METAR that my suite addresses is not precisely specified and in some places almost any text may be found.
# - I deliberately experiment with a wide variety of php functions to try them out and the suite may not be using the most efficient design
# - I have learnt a lot about METAR as issued in every country in the world, see PDF paper that I have produced in zip.
# - That paper also describes how ICAO and WMO are forcing abandonment of METAR in TAC format of traditional alphanumerical characters I mentioned in earlier post
# - My suite decodes METARs still using the Traditional Alphanumeric Characters format, not those in the newer digital format, and it is possible that the TAC format will be abandoned by 2020 rendering my suite obsolete!
# - The outputs are in some cases, long phrases, because they are clear or precise explanations, in no cases do I output just the numbers. The explanatory text is fixed, so a user can in their own web page strip off unwanted parts of the text if they are able to write program code.
# - As just one example, when reporting air presures I differentiate between sea level pressure (SLP) and pressure at other altitudes, rival decoders just quote a pressure reading and you cannot tell whether that is SLP or an altimeter reading at a different level. It is difficult to decide which terms are too technical, because some people know more than others, but I am not aiming at meteorologists nor those who want just numbers, but at those who want enough to understand what the report is saying!
# - The expectation is that the suite will be used by those wanting to augment (or make comparisions) outputs from their personal weather stations with information from nearby airports, and I expect that users will choose to display only a subset of the information available in the output array, but hopefully all you might want is available.
# - For USA I am decoding specifications in both Civil and Air Force handbooks as there are some differences, and although both books specify a sequence to be followed, in practice USA content I have found in the Remarks Group in live METAR has both inconsistent sequencing and a free format for much of the abbreviated codes; I don't rely on any particular sequencing, but the free format aspect means that USA Remarks decoding, like that in other countries, may sometimes fail, although fortunately this happens very rarely.

For testing my suite, I have developed and revised many web pages that call my decoder. I don't know what design you as a user want, nor which parts of my output array you want to have displayed, so I don't provide any display facility in the zip. Read the sources for the two web pages I produced for 'PaulMy ' at http://www.komokaweather.com/metar/metar_display.php?sce=metar_display and http://www.komokaweather.com/metar/metar_display-alt.php?src=metar_display-alt. Remove the queryString (question mark onwards) to see the relevant page in action.
The PDF in the zip discusses all available outputs, but many will be seen on one or other of those web pages.

All the suite scripts, including those mentioned below as being available on Paul's site, and the two folders with images as used on his web pages, are available in the zip which includes the PDF paper that explains all the files in the zip and how to use them. If a particular file, or files, has/have been updated recently, the version on Paul's site and the latest version in the zip might have slight discrepancies.

metar (18Mar2018).zip

The top level script of my decoder can be read at http://www.komokaweather.com/metar/metar_display.php?viewSource=decodeMETAR.
The script at the next level down is read at http://www.komokaweather.com/metar/metar_display.php?sce=decodeMETAR_sub_funct.
The optional script for decoding the Remarks Group is at http://www.komokaweather.com/metar/metar_display.php?src=decodeMETAR_Rmk. It optionally calls further scripts for Canada and USA, although because these are not loaded at the start, sources can only be viewed once they are loaded, use either http://www.komokaweather.com/metar/decodeMETAR_Can_Rmk.php?viewSource=decodeMETAR_Can_Rmk or http://www.komokaweather.com/metar/decodeMETAR_USA_Rmk.php?viewSource=decodeMETAR_USA_Rmk.

Incidentally, to view PHP sources written by me, the queryString attribute can be 'sce', 'src', or 'viewSource' and although I vary between the three attributes in examples above, you can choose any one and use it in all places.

I've not been highlighting when I issue changes in this thread because only Paul is testing my suite in live running conditions, and there have been no extra downloads. If there is no more unanticipated content at his local airports, there will be no futher updates to my suite.
You do not have the required permissions to view the files attached to this post.
Last edited by sfws on Sun Mar 18, 2018 7:28 am, edited 44 times in total.

User avatar
PaulMy
Posts: 1698
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Fri Jan 12, 2018 3:17 pm

I have been using this decoder for some time now and this has been providing feedback on our local Metar conditions to help in overcoming some of the differences/variations by local airport Metar reporting. Sfws has put a tremendous amount of time and effort on this decoder and I find it very interesting to compare our area Metars http://www.komokaweather.com/metar/metar_display.php
Use the Click to Show/Hide to see more Metars...

Enjoy,
Paul

User avatar
PaulMy
Posts: 1698
Joined: Sun Sep 28, 2008 11:54 pm
Weather Station: Davis Vantage Pro 2 w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Decoding METARS

Postby PaulMy » Sat Feb 17, 2018 2:47 am

For those following this metar decoder, sfws has updated the zip (16Feb2018) and this contains a PDF on the project. This helps to understand the complexity and the work and time involved with the decoder.

Enjoy,
Paul


Return to “Weather”

Who is online

Users browsing this forum: No registered users and 1 guest