Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Importing easyweather history

Discussion specific to Fine Offset and similar rebadged weather stations
J_K
Posts: 2
Joined: Mon 10 Nov 2008 1:16 pm

Importing easyweather history

Post by J_K »

Hi. First off I have to say this a brilliant piece of Software Steve. Well done for writing it and distributing it free. I will be donating soon.

I have a question that you may have guessed from the title of the post :D .

I live just outside Ottery St. Mary which is the place that had the freak (nothing new for Ottery) hail storm last week http://news.bbc.co.uk/1/hi/england/devon/7699112.stm and I've got some good data from the time of the storm but obviously, having just installed Cumulus, it's only showing the rainfall from the past hour or so.
I was wondering if there was a way of retrieving the history stored in the easyweather software to add to the Cumulus history??

I've also got a question about embedding the information on my companies websitre but I'll ask that in the appropriate forum.

Thanks again Steve.
Keep up the good work.
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Importing easyweather history

Post by steve »

There's no automated way of importing the data, but as the formats of the Easyweather and Cumulus files are both well-known, it shouldn't be too hard to convert one to the other (but I guess some programming of some sort would be involved). It's complicated by the fact that Cumulus has two files - the additional one being a daily summary.

Steve
User avatar
dane
Posts: 417
Joined: Wed 10 Sep 2008 2:15 pm
Weather Station: Rosenborg 68700
Operating System: Win10 Ult., 64-bit, RaspberryPi
Location: Gilleleje, Denmark

Re: Importing easyweather history

Post by dane »

J_K: is the data of interest to you still in your weather station unit?

If so, you can force Cumulus to read it and build all its data from the information in the unit.

First: backup all Cumulus data. :) (meaning: copy everything from the Cumulus\data folder to someplace else)
Then: edit the today.ini in the Cumulus\data folder to contain something like:
today.ini
that is: some date way back - and nothing else.
Delete everything else in the Cumulus\data folder (except maybe the log.xml file).
(You might as well delete everything in the Cumulus\Diags folder).

And then just start Cumulus.
Hopefully Cumulus will show a "Downloading information" window while it reads from the weather station.
There will be no progress bar for a while - but all of a sudden you should see numbers flashing and a progress bar. After a while Cumulus will settle down and continue normal operation.

Beware: when you shutdown Cumulus after this sort of "cold start" it may take an awful long time (and I mean several minutes....) before Cumulus has actually finished its file activity and closed.
Don't rush things - use Windows Task Manager to check that Cumulus has in fact stopped and gone away.
Ib
J_K
Posts: 2
Joined: Mon 10 Nov 2008 1:16 pm

Re: Importing easyweather history

Post by J_K »

Thanks Dane.
I have been editing the info from easyweather myself but it's a bit of a long winded process.

That should make things a lot easier!!
User avatar
dane
Posts: 417
Joined: Wed 10 Sep 2008 2:15 pm
Weather Station: Rosenborg 68700
Operating System: Win10 Ult., 64-bit, RaspberryPi
Location: Gilleleje, Denmark

Re: Importing easyweather history

Post by dane »

steve wrote:There's no automated way of importing the data, but as the formats of the Easyweather and Cumulus files are both well-known, it shouldn't be too hard to convert one to the other (but I guess some programming of some sort would be involved). It's complicated by the fact that Cumulus has two files - the additional one being a daily summary.

Steve
I had some old EasyWeather.dat data (from before my installation of Cumulus) that I wanted to try to import into Cumulus, so I threw together some code to do the conversion from EasyWeather.dat to Cumulus' two files: the dayfile.txt and the xxx08log.txt.
After the conversion I added the output to the Cumulus files - and the result seems to be OK: using Select-a-Graph I can see the old data. The code cannot update the "All time hi/low" data, naturally :)

So: if anybody has a need for this sort of conversion, I'll be happy to attempt to do it.
Ib
adrian5750
Posts: 43
Joined: Sat 15 Nov 2008 4:06 pm

Re: Importing easyweather history

Post by adrian5750 »

HI Dane

I have a file from easyweather with a couple of months of data in it -
if you had an easy way of transfaerring that into a format that Cumulus
could read then I'd be ever so grateful...

You can email me direct if you'd like - adrian@ambquality.co.uk

Many thanks
Adrian
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Importing easyweather history

Post by steve »

I must have missed your post, Ib, I didn't realise you had done this - sounds great.

Steve
User avatar
dane
Posts: 417
Joined: Wed 10 Sep 2008 2:15 pm
Weather Station: Rosenborg 68700
Operating System: Win10 Ult., 64-bit, RaspberryPi
Location: Gilleleje, Denmark

Re: Importing easyweather history

Post by dane »

steve wrote:I must have missed your post, Ib, I didn't realise you had done this - sounds great.

Steve
I wouldn't say I have done it....
I just had a go at Adrian's EasyWeather.dat file - only to realise that there are things like localisation (format of date, separation character, decimal point...) and selection of measurement units (mm or inches, m/s or mph, ...) to be prepared for. I begin to understand your difficulties catering for all our varieties :D
Ib
philcdav
Posts: 244
Joined: Tue 24 Jun 2008 9:43 pm
Weather Station: MyDEL WX2008 Mk2 Fine Offset
Operating System: XP and W7
Location: Maghull, nr Liverpool, UK
Contact:

Re: Importing easyweather history

Post by philcdav »

Hi guys.

I have been playing with EW data using CSVEd software. http://csved.sjfrancke.nl/index.html
It is VERY powerfull and Free!

Se also http://home.hccnet.nl/s.j.francke/software/software.htm

I have managed to mash all kinds of data with it but the most usefull has been to get EW and Cumulus data into Excell for data processing and be able to extract bits back into EW and C.

Sadly, i havent been able to get it all to work as required.

For example, i need help with rebuilding montly and yearly rainfall totalls which means rebuilding the cumulative data each time i add new data. But we will get there! ;)

But otherwise its all been a great experience.

Regards - Phil
Phil - G0DOR
andyf
Posts: 2
Joined: Thu 15 Jul 2010 12:08 pm
Weather Station: Fine Offset
Operating System: Windows XP SP3

Re: Importing easyweather history

Post by andyf »

dane wrote:J_K: is the data of interest to you still in your weather station unit?

If so, you can force Cumulus to read it and build all its data from the information in the unit.

First: backup all Cumulus data. :) (meaning: copy everything from the Cumulus\data folder to someplace else)
Then: edit the today.ini in the Cumulus\data folder to contain something like:
today.ini
that is: some date way back - and nothing else.
Delete everything else in the Cumulus\data folder (except maybe the log.xml file).
(You might as well delete everything in the Cumulus\Diags folder).

And then just start Cumulus.
Hopefully Cumulus will show a "Downloading information" window while it reads from the weather station.
There will be no progress bar for a while - but all of a sudden you should see numbers flashing and a progress bar. After a while Cumulus will settle down and continue normal operation.

Beware: when you shutdown Cumulus after this sort of "cold start" it may take an awful long time (and I mean several minutes....) before Cumulus has actually finished its file activity and closed.
Don't rush things - use Windows Task Manager to check that Cumulus has in fact stopped and gone away.
Hi

I'm a newcomer to Cumulus and whilst trying this method I got some "strange" results so I thought I'd share what I found.

The earliest data in the 1080 was 2/7/2010 17:15:16 (the last time I downloaded into Easyweather and cleared the 1080)

Thinking that the Timestamp in today.ini just needed to be before that I set it to 01/01/2010 00:00.

I connected the 1080 and ran Cumulus - after a very long wait the data from the 1080 was imported but I found Cumulus reporting maxima and minima in varying months BEFORE July 2010.

I cleared / reset the Cumulus data and tried again, this time with Timestamp set to 02/07/2010 17:10.

This time it ran much faster and only showed max / min info starting in July 2010 - a much more believable result.

I'm left wondering if Cumulus "made up" the data for January to June, or did it in fact find data in the 1080 that even it didn't show when cycling through its history (I'm note sure this is the case however - the 15 min interval I use gives about 40 days of data so so the memory couldn't have had anything earlier than about mid May.

Andy
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Importing easyweather history

Post by steve »

andyf wrote:I'm left wondering if Cumulus "made up" the data for January to June, or did it in fact find data in the 1080 that even it didn't show when cycling through its history (I'm note sure this is the case however - the 15 min interval I use gives about 40 days of data so so the memory couldn't have had anything earlier than about mid May.
The way it works is that it starts from the current entry in the logger and goes backwards using the interval in each logger entry, until it gets to the 'timestamp' date OR it has read all of the valid entries from the logger. I thought there were checks to make sure it doesn't keep wrapping around re-using entries (the entries don't have timestamps), but maybe not...

If you zip up the diags folder and attach it, I can see what happened. But it does sound like a bug...
Steve
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Importing easyweather history

Post by steve »

Actually, you don't need to upload the diags. I've looked at the code and if the memory is full it does indeed keep going round reusing entries until it gets back to the 'timestamp' date.
Steve
andyf
Posts: 2
Joined: Thu 15 Jul 2010 12:08 pm
Weather Station: Fine Offset
Operating System: Windows XP SP3

Re: Importing easyweather history

Post by andyf »

steve wrote:Actually, you don't need to upload the diags. I've looked at the code and if the memory is full it does indeed keep going round reusing entries until it gets back to the 'timestamp' date.
Thanks for the quick response

Thought I better clarify that in the case I describe the 1080 memory wasn't "full" (as in the bottom right circle being fully black) - it only had data for 2nd July to 15th July, 13 days when I usually get over 40 days before the bottom right indicator nears 100%.

However as the 1080 has been in use for over 2 years could it be that the old data from previous cycles was being returned as the 1080 had not re-used the slots for new data and the 1080 didn't recognise the old data as invalid?

If it helps I think I may have the diags etc from the 1st run as I only copied/renamed the folders rather than deleting.

I did have a suspicion about the data being re-read/reused but didn't analyse the cumulus files rigourously to prove it, but I'm also wondering if a side effect of the "endlessly going backwards" is that the max / min etc. data land up being recorded against the earliest dates hence my seeing data for between the Timestamp date and the 1st "real" data.

Andy
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Importing easyweather history

Post by steve »

When the memory is cleared, it gets set to all "FF" and Cumulus checks for this to know whether the entry is valid or not. It's the only way it can know if data is valid or not, as the entries aren't timestamped. So if your memory wasn't 'full' but had entries which weren't set to FF, then Cumulus is going to have problems when downloading old data if it's told to go back further than the earliest valid memory location, and I don't see that there's anything I can do about that. Normally, it's not a problem, of course.

Perhaps the station didn't reset the memory to FF correctly when you cleared it on the 2nd July. Did you have a particular reason for clearing the memory in the first place?

Perhaps you could upload the diags and I'll see what happened.
Steve
mywaynow
Posts: 1
Joined: Thu 03 Mar 2011 9:26 pm
Weather Station: Watson WH 1080
Operating System: Windows 7
Location: Bedford UK
Contact:

Re: Importing easyweather history

Post by mywaynow »

dane wrote:
steve wrote:There's no automated way of importing the data, but as the formats of the Easyweather and Cumulus files are both well-known, it shouldn't be too hard to convert one to the other (but I guess some programming of some sort would be involved). It's complicated by the fact that Cumulus has two files - the additional one being a daily summary.

Steve
I had some old EasyWeather.dat data (from before my installation of Cumulus) that I wanted to try to import into Cumulus, so I threw together some code to do the conversion from EasyWeather.dat to Cumulus' two files: the dayfile.txt and the xxx08log.txt.
After the conversion I added the output to the Cumulus files - and the result seems to be OK: using Select-a-Graph I can see the old data. The code cannot update the "All time hi/low" data, naturally :)

So: if anybody has a need for this sort of conversion, I'll be happy to attempt to do it.
Hi Dane,

I have jsut this evening installed Cumulus, having got feed up with Easy Weather, I already like it a lot compared to Easy Weather, I have data going back to the beginning of the year which I would like to import if possible? Trouble is, I am failry competent at handling Windows software but digging underneath the bonnet and into code is way outside of my capabilities, is there anything that you can do to help me? I can be contacted on david@cavanagh.cc Sadly I jsut downloaded all the data on my weather station to Easy Weather and deleted it off the station before deciding to give up with it and move to Cumulus! Otherwise I would have had most of the data still on the machine and probably wouldn't be too worried about what was missing.
Post Reply