Page 1 of 2

WMR200 logger operation

Posted: Thu 05 May 2011 7:29 am
by steve
My WMR200 has arrived and I've started writing the code (the sensors are scattered around the office). One thing I hadn't appreciated about this device is that the logger has a vast capacity - over 27,000 entries. The station operates in a different way to all other stations which Cumulus supports. Other stations (those that have loggers) are continuously writing to the logger as well as sending live data. The WMR200, on the other hand, has two modes of operation; 'logging' and 'live'. As long as an application is reading live data, it doesn't bother logging it itself.

When an application starts up and starts downloading the stored logger data from a WMR200, the station sends live data along with the logger data, and the application is supposed to handle both sets of data. Unfortunately, Cumulus is simply not designed to work this way. While it's downloading logger data, it doesn't process live data, and there's no easy way that I can change it to do that (but possibly in Cumulus 2 I'll be able to). This does mean that even for existing stations, if the logger download takes a long time, you get a 'gap' in the Cumulus logs for the period it took to download the data.

For other stations, if this is an issue for anyone, you can always force Cumulus to go back and download the logger data for the 'gap' period. Because of the way the WMR200 works, this isn't an option. The problem is also bigger for the WMR200 because of the size of the logger. Apparently a 'full' logger can take 8 hours to download!

So, to come to the point, this is firstly a warning so that you know what's coming. Secondly, I'm interested to know how much of an issue this will be - WMR200 users, how do you (or will you) use the WMR200 logger? Will you tend to leave it for long periods without downloading the logger data? If you weren't planning to leave it for more than 24 hours, say, then even at the 1-minute interval it would only take about 20 minutes to download the data (hence a 20-minute 'gap' in the Cumulus logs), so it's less of an issue.

Re: WMR200 logger operation

Posted: Fri 06 May 2011 6:52 pm
by Tonky
Hi Steve,

I would normally download the logger information two times a day, the interval is on two minutes.
But when you would go away for the weekend or for summer holydays (2-3 weeks) it is a different situation.

But nice to see you work on it. :)

Re: WMR200 logger operation

Posted: Sat 07 May 2011 1:10 pm
by bigmac
Hi Steve,

Thanks for updating us. I'm just trying to understand the scope of the problem and how it relates to the existing way Cumulus is written. I'm guessing that data downloaded from the WMR200 is date and time stamped (although I don't actually know). Now I'm not suggesting you re-write Cumulus just for the sake of a few WMR200 users but is there no way to re-write a portion of code that can handle and re-order date and time stamped data (assuming all data is essentially stored as a simple database in Cumulus)? Or have I completely misunderstood the way Cumulus operates? :?

For me personally it probably wont be an issue as I have my WMR200 connected to a netbook 24/7 but obviously there are times when the netbook may get rebooted or possibly shut down for holiday periods. These are the times when data would be lost for me. But I'm not so anal about capturing every single millisecond of weather data ;)

Re: WMR200 logger operation

Posted: Sat 07 May 2011 1:54 pm
by steve
bigmac wrote:Now I'm not suggesting you re-write Cumulus just for the sake of a few WMR200 users but is there no way to re-write a portion of code that can handle and re-order date and time stamped data (assuming all data is essentially stored as a simple database in Cumulus)? Or have I completely misunderstood the way Cumulus operates?
It's not just a case of handling out-of-sequence data and getting it in the right order in the logs; that wouldn't be too hard in itself. It's the fact that Cumulus doesn't simply log the raw data as read from the station, it performs its own processing on the data to get the items that it needs for its logs. It has to work this way because it has to work with a number of different types of station which all supply data in different ways.

So it uses the same code and data structures to handle logger data as it does for live data, using a timestamp of 'now' for live data, and the logger timestamp for logger data, and there's just no way for it to handle data coming in simultaneously for two different periods in time. A quick reboot, or even switching off overnight (or for a couple of days) isn't an issue, as this would result in very little data being ignored, but processing 20,000 logger entries will definitely cause a large gap in the log.

I could argue that anyone who is really concerned about getting every last scrap of data should be running their PC 24/7 anyway ;)

Re: WMR200 logger operation

Posted: Sat 07 May 2011 2:25 pm
by bigmac
OK, I think I'm understanding it better now.
Could it be a case then that at startup Cumulus takes the live data feed and then a seperate program could fetch the historical data from the logger and present it back to Cumulus in a format that it can handle, i.e. similar to how other loggers operate and feed their data to Cumulus? This would then provide the historical data.

I'm probably being completely naive as to the amount of programming required to do this :) and have a complete lack of understanding of how the logger works. And there's also no reason why you should put so much work into solving this for us other than the fact I think you like a challenge!

Re: WMR200 logger operation

Posted: Sat 07 May 2011 2:55 pm
by steve
bigmac wrote:Could it be a case then that at startup Cumulus takes the live data feed and then a seperate program could fetch the historical data from the logger and present it back to Cumulus in a format that it can handle, i.e. similar to how other loggers operate and feed their data to Cumulus? This would then provide the historical data.
There's just no way that it could handle data arriving completely out of sequence with the way it is written, because of things like doing the end-of-day entry to dayfile.txt - it needs the full day's data to be able to do that. It has to catch up from where it left off before it can start handling live data.

It would have to do it the other way around to what you propose; storing away the live data while it was processing the logger data, and then process the stored live data as if it were logger data. And while it was doing that, it would have to store away the live data... etc. It would only be able to display some of the 'live' data while it was doing this as it wouldn't be able to calculate today's rainfall (for example) in every situation. And this would all require large changes to the existing code.

Re: WMR200 logger operation

Posted: Sat 07 May 2011 3:16 pm
by bigmac
Hmmm, OK I see the problem.
As I said, for me, not too much of an issue as I do run a 24/7 connection anyway.
Keep us informed of progress :D

Re: WMR200 logger operation

Posted: Sat 07 May 2011 4:17 pm
by steve
bigmac wrote:Keep us informed of progress :D
I currently have it reading wind, temperature and humidity from 'live' data. I need to finish the reading of the live data and then start on the logger stuff. Then we can see how big an issue it is.

Re: WMR200 logger operation

Posted: Sun 08 May 2011 8:56 pm
by gwheelo
Steve -

I am considering a WMR 200, perhaps, July when I visit the US - obviously a price issue between the price in Spain and the price in the US.

I would operate the WMR 200 on a 24 hr basis as I do now with my 968 so the logger issue should not be a problem for me.

Perhaps you might clear up a technical point on the WMR-200 for me. How is the wind bearing outputted. Is it as the 968 in one degree increments or is it in quadrants? Quadrants may be a problem for my Catalan wind naming routine as the winds are not based on the usual 16 quadrants. See: http://www.wheelocknet.net/cumulus/wind.html?gb.

George Wheelock

Re: WMR200 logger operation

Posted: Mon 09 May 2011 7:13 am
by steve
gwheelo wrote:Perhaps you might clear up a technical point on the WMR-200 for me. How is the wind bearing outputted. Is it as the 968 in one degree increments or is it in quadrants? Quadrants may be a problem for my Catalan wind naming routine as the winds are not based on the usual 16 quadrants.
It supplies 16 'directions', from 0 to 15 (N to NNW) and Cumulus converts these into degrees by multiplying by 22.5 (and rounding). So you would only ever see 0, 22, 45... etc

Re: WMR200 logger operation

Posted: Mon 09 May 2011 9:37 pm
by gwheelo
It supplies 16 'directions', from 0 to 15 (N to NNW)
Bummer - that is a real show stopper for me!

GW

Re: WMR200 logger operation

Posted: Sat 14 May 2011 2:30 pm
by ctccbc
Will the WMR100N be supported as well? I think the protocol for the WMR200 and WMR100N are pretty much the same.

Re: WMR200 logger operation

Posted: Sat 14 May 2011 4:28 pm
by steve
ctccbc wrote:Will the WMR100N be supported as well?
Hopefully, yes, but I haven't bought one of those, so it will be a little bit harder.
I think the protocol for the WMR200 and WMR100N are pretty much the same.
No, they're considerably different.

Re: WMR200 logger operation

Posted: Sun 15 May 2011 1:30 am
by TgT
Hm, there are two versions of WMR100, older - all-in-one setup
Image

and newer which uses same components/accessories/sensors as WMR200, OregonScientific protocol 3.0
Image
Works wirelessly ONLY with the WMR80, WMR90, WMR100 and WMR200 wireless weather station products using Protocol 3.0
:roll:

Re: WMR200 logger operation

Posted: Sun 15 May 2011 6:48 am
by steve
there are two versions of WMR100
As far as I am aware the protocol between the station and the PC is the same for the two types of WMR100, and is considerably different to that used by the WMR200.