Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

Please read this post before posting

Latest Cumulus release v1.9.4 (build 1099) - Nov 28 2014
Latest Cumulus MX release - v3.0.0 build 3043 Jan 20 2017. See this post for download

Cumulus - manually edit faulty data?

Discussion specific to Fine Offset and similar rebadged weather stations
IslandBoy77
Posts: 8
Joined: Wed Apr 04, 2012 10:38 pm
Weather Station: WS1081
Operating System: Vista Business 32-Bit
Location: Napier, New Zealand

Cumulus - manually edit faulty data?

Postby IslandBoy77 » Fri Sep 14, 2012 9:33 pm

Hi
I've been using the Cumulus software for my WS1081 for most of this year. I've found that either due to a program glitch or something to do with my weather station, Cumulus will periodically record wildly inaccurate data. A few minutes ago Cumulus showed an alert and new record, claiming the temp was -10. It would be nice if it were that cold (as it would mean it was snowing, which it does only once every 20 or so years here), but since it's 16 degrees I'm sure that reading is wrong. I also note the the pressure reading last night showed a sudden 0 rating, then back to normal.

Anyway, it's not that these glitches happen often, I just want to know how to get into the data file to manually edit the values to something approximating normal.

Advice please?

Thanks

User avatar
steve
Cumulus Author
Posts: 26655
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Cumulus - manually edit faulty data?

Postby steve » Fri Sep 14, 2012 10:06 pm

The data files are all in the Cumulus data folder, and the formats are described in the help and in the wiki. It's best to stop Cumulus before editing the files. There's a built-in editor for dayfile.txt.

You can use the calibration screen in Cumulus to set some 'spike removal' settings so that Cumulus can attempt to filter out any future strange readings from the station.
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development since I have retired from my day job

IslandBoy77
Posts: 8
Joined: Wed Apr 04, 2012 10:38 pm
Weather Station: WS1081
Operating System: Vista Business 32-Bit
Location: Napier, New Zealand

Re: Cumulus - manually edit faulty data?

Postby IslandBoy77 » Fri Sep 14, 2012 11:57 pm

Thanks for your reply Steve. I opened the monthly file into Excel, but I couldn't see the values for either the wrong pressure or the wrong temperature reflected in the data - it looks like only the hi's & low's file picked it up. What's the easiest way to reset the hi's & lows to their previous values? Thanks.

User avatar
steve
Cumulus Author
Posts: 26655
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Cumulus - manually edit faulty data?

Postby steve » Sat Sep 15, 2012 7:05 am

Have you checked dayfile.txt? They are likely to be in there even if they aren't in the monthly log files. Once you've corrected dayfile.txt, you can use the built in record editor(s) to scan dayfile.txt and rebuild the records from that.
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development since I have retired from my day job

IslandBoy77
Posts: 8
Joined: Wed Apr 04, 2012 10:38 pm
Weather Station: WS1081
Operating System: Vista Business 32-Bit
Location: Napier, New Zealand

Re: Cumulus - manually edit faulty data?

Postby IslandBoy77 » Sun Sep 16, 2012 9:49 pm

steve wrote:Have you checked dayfile.txt? They are likely to be in there even if they aren't in the monthly log files. Once you've corrected dayfile.txt, you can use the built in record editor(s) to scan dayfile.txt and rebuild the records from that.


Yes. I found the temperature discrepancy in the dayfile & made an arbitrary correction to approx what it should have been. The odd thing is that the Sep12log file, when opened in Excel (comma deliminated) doesn't show either the temperature aberration nor the 0 pressure reading late on the 14th. The dayfile doesn't show the pressure reading either, but using select a graph does. All very strange.

User avatar
steve
Cumulus Author
Posts: 26655
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Cumulus - manually edit faulty data?

Postby steve » Mon Sep 17, 2012 7:10 am

Select-A-Graph gets its data from the monthly log files.
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development since I have retired from my day job

rwilliam99
Posts: 42
Joined: Sun Apr 01, 2012 2:55 pm
Weather Station: WS-2080
Operating System: Windows 7
Location: Oregon City, OR

Re: Cumulus - manually edit faulty data?

Postby rwilliam99 » Mon Dec 17, 2012 1:21 am

I thought I would post my question here since it is related to editing faulty data. After my experiment with using EasyWeather instead of Cumulus to talk to the console, my rain totals are completely messed up. I've gone back through the data files and figured out the correct totals, however nothing I have tried has made it stick. I've edited all of the *.ini files where the monthly total was, and no matter what I do, as soon as Cumulus starts up, it changes it back to the value in error. I have not fixed the actual data files (the data was getting duplicated over and over in the EasyWeather.dat file it appears), but I did sort through it to figure out what the totals should be. So I have accurate totals for November and also accurate totals for December to date.

What do I need to do in order to fix this permanently? I hope I don't need to go back and fix the monthly .txt files. I'm hoping I can override the monthly and all time totals somewhere I'm not seeing it. I've tried editing the files manually, and also doing it through the Cumulus Toolbox editor and also using the editor within Cumulus (Edit/All time records, Monthly records, This month's records, This year's records). They all did the same thing. I made sure that Cumulus was not running when I edited them.

Any suggestions on how to fix these bad values for the high values and also fix the value for December?

Thanks

Rob

Matt.j5b
Posts: 424
Joined: Mon Nov 28, 2011 2:13 am
Weather Station: La Crosse WS 2306
Operating System: Windows 10 64 Bit
Location: Brisbane, Australia
Contact:

Re: Cumulus - manually edit faulty data?

Postby Matt.j5b » Mon Dec 17, 2012 1:56 am

To fix the monthly totals I think you need to edit the daily totals in the dayfile and then restart Cumulus. To edit the all time, this month, this year and monthly records either correct the log file for the affected month/s or the dayfile and use the editors in the edit menu to fetch the correct reading for the records or insert the correct figure manually into the editor. Editing the ini files for the records such as month.ini and year.ini I think is the same as using the editor. If this broke the all time records you can look at the alltimelog in the data folder to see what the previous record was.
Regards, Matt of Brisbane, Australia
Ferny Grove Weather
Cumulus MX testing

rwilliam99
Posts: 42
Joined: Sun Apr 01, 2012 2:55 pm
Weather Station: WS-2080
Operating System: Windows 7
Location: Oregon City, OR

Re: Cumulus - manually edit faulty data?

Postby rwilliam99 » Mon Dec 17, 2012 4:35 am

I tried both methods (using the editor and also editing the ini files manually when Cumulus is not running)t, but when Cumulus restarts, it "fetches" the prior all time high value (which is incorrect). I can't find that value in any of the data files, which leads me to believe it is somehow reprocessing the data files, which have the invalid data in them.

Matt.j5b
Posts: 424
Joined: Mon Nov 28, 2011 2:13 am
Weather Station: La Crosse WS 2306
Operating System: Windows 10 64 Bit
Location: Brisbane, Australia
Contact:

Re: Cumulus - manually edit faulty data?

Postby Matt.j5b » Mon Dec 17, 2012 4:59 am

So the problem is with incorrect all time records. Is that right? Unless you are editing the wrong file (the all time ini file is "alltime.ini") or when you used the editor you did not click copy as well as OK. I have used the all time editor in Cumulus with the spikes in the readings lots of times and once I removed them from the log file and dayfile, I clicked "fetch" and clicked "copy" for the incorrect fields and pressed "OK" and it updated correctly. After you correct the all time records using the editor and press OK the alltime.ini file is updated and as I understand that file stores the figures for the all time records. Full details of the records editor is in the help file under "The displays" and then click on "The Records (Highs and Lows) Editors".
Regards, Matt of Brisbane, Australia
Ferny Grove Weather
Cumulus MX testing

IslandBoy77
Posts: 8
Joined: Wed Apr 04, 2012 10:38 pm
Weather Station: WS1081
Operating System: Vista Business 32-Bit
Location: Napier, New Zealand

Re: Cumulus - manually edit faulty data?

Postby IslandBoy77 » Mon Dec 17, 2012 6:05 am

Hi Matt

You will probably end up doing what I'm thinking of doing: pull the battery from the transmitter. It appears that the culprit that keeps "resetting" any changes one makes to the various files is the rain gauge within the weather station transmitter itself. I've twice in the last 2 months had the sprinkler going on the front lawn and accidentally got water into the rain gauge. My plan is to make sure I have a backup of everything on 31/12/12, then pull the battery on the unit on 1/1/13, put the battery back in, and see what happens (after correcting my rain totals as per the notes I've made). I'm picking that all should be well, since one has to replace the batteries in the weather station transmitter itself from time to time anyway.

Matt.j5b
Posts: 424
Joined: Mon Nov 28, 2011 2:13 am
Weather Station: La Crosse WS 2306
Operating System: Windows 10 64 Bit
Location: Brisbane, Australia
Contact:

Re: Cumulus - manually edit faulty data?

Postby Matt.j5b » Mon Dec 17, 2012 6:23 am

My spikes I get generally occur in winter with the lower temperatures with the temperature and humidity readings. Last winter I most likely edited Cumulus of the spikes about 20 or 30 times and got about 40 to 50 spikes. Between July 27 and September 12 I got 33 spikes of which 17 needed editing as I run Cumulus off the logger when I started to keep a count of the spikes. I did try to reset the transmitter but nothing avoids the trouble, its the low temperatures that do it. I have not have much of false rain readings to correct as I tested the gauge before I used Cumulus and we get lots of rain except for recently.
Regards, Matt of Brisbane, Australia
Ferny Grove Weather
Cumulus MX testing

User avatar
steve
Cumulus Author
Posts: 26655
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Cumulus - manually edit faulty data?

Postby steve » Mon Dec 17, 2012 8:20 am

rwilliam99 wrote:I'm hoping I can override the monthly and all time totals somewhere I'm not seeing it.
As Matt says, the monthly totals are calculated from the daily totals in dayfile.txt, so you need to correct those and then your monthly totals will be correct (after a restart). You can then use the high/low editor(s) in Cumulus to regenerate the monthly high.
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development since I have retired from my day job

rwilliam99
Posts: 42
Joined: Sun Apr 01, 2012 2:55 pm
Weather Station: WS-2080
Operating System: Windows 7
Location: Oregon City, OR

Re: Cumulus - manually edit faulty data?

Postby rwilliam99 » Mon Dec 17, 2012 1:07 pm

That was the culprit. I did not edit the dayfile.txt. Once I did that and restarted, my totals are now accurate. I knew it had to be something simple....

THANK YOU!

Phil23
Posts: 111
Joined: Sat Jul 16, 2016 11:59 pm
Weather Station: Davis Vantage Pro2 Plus
Operating System: Win7 Pro
Location: Australia

Re: Cumulus - manually edit faulty data?

Postby Phil23 » Sat Nov 12, 2016 5:56 am

steve wrote:Have you checked dayfile.txt? They are likely to be in there even if they aren't in the monthly log files. Once you've corrected dayfile.txt, you can use the built in record editor(s) to scan dayfile.txt and rebuild the records from that.


Is it safe to do the above by transferring MX's data over to version 1 then back again?, or is that a big NO-NO?

Phil


Return to “Fine Offset/EasyWeather”

Who is online

Users browsing this forum: No registered users and 6 guests