Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

I/O Error 32 writing to log file

Discussion specific to Davis weather stations
RayProudfoot
Posts: 3373
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

I/O Error 32 writing to log file

Post by RayProudfoot »

Thanks David and Steve. Given the processing time 5 seconds seems okay.

Steve, I've found another instance of lost data. Between 23.30 last night and 17:00 today. I exited Cumulus and set the TimeStamp date/time to 19/08/2009 00:00:00, saved it and restarted Cumulus. The download window appeared and then disappeared without any data having been downloaded. This is using today's new build.

Did I follow the correct procedure? As I don't seem to be able to download old data from the logger can I trouble you to convert data for today from WeatherLink please?
Cheers,
Ray, Cheshire.

Image
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Option to send wind gust data to Wunderground...

Post by steve »

RayProudfoot wrote:Steve, I've found another instance of lost data. Between 23.30 last night and 17:00 today. I exited Cumulus and set the TimeStamp date/time to 19/08/2009 00:00:00, saved it and restarted Cumulus. The download window appeared and then disappeared without any data having been downloaded. This is using today's new build.

Did I follow the correct procedure?
Sounds right. Assuming that date format is correct for your system. Could you zip up your Diags folder and upload it, please, so I can see what went wrong?
Steve
RayProudfoot
Posts: 3373
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: Option to send wind gust data to Wunderground...

Post by RayProudfoot »

Thanks Steve. Here it is. I'm not sure why I have a spurious ~OG file there.
You do not have the required permissions to view the files attached to this post.
Cheers,
Ray, Cheshire.

Image
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Option to send wind gust data to Wunderground...

Post by steve »

The reason for the missing data is this:

2009-08-19 16:55:01 : 19/08/2009 16:55:00 Error writing to data file: C:\Program Files\Cumulus\data\Aug09log.txt. I/O error 32
2009-08-19 16:55:01 : 19/08/2009 16:55:00 Error writing to data file: C:\Program Files\Cumulus\data\Aug09log.txt. I/O error 103

This is the same problem you had before, I think? But I can't remember what the cause was...

Then, when asked for all the records since 19/08/2009 00:01:00, the station says there aren't any:

2009-08-19 17:56:23 : VP2: Downloading history from 19/08/2009 00:01:00
2009-08-19 17:56:25 : VP2: GetNumberOfArchiveRecords_V = 0
2009-08-19 17:56:25 : VP2: No history records to process

So that's why setting the timestamp back hasn't worked. It's a bit annoying that the station doesn't seem to want to co-operate, as that's by far the easiest way of getting the data back. Anyway, if you send me your Aug09log.txt and the section of exported Weatherlink file, I'll convert it for you. I think I need to package up the converter and make it available...

By the way, your station is on port 18? Is that an add-in serial card or something?
Steve
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Option to send wind gust data to Wunderground...

Post by steve »

I remembered what I concluded the cause of the previous occurrence was - a second copy of Cumulus running. And I think that's the case again here. You stopped Cumulus and then started it again shortly afterwards; then when Cumulus asked for the data from the logger, this happened:

2009-08-19 16:58:39 : VP2: Downloading history from 19/08/2009 16:55:00
2009-08-19 16:58:45 : VP2: GetNumberOfArchiveRecords_V = 2559
2009-08-19 16:58:45 : VP2: Number of history records = 2559
2009-08-19 16:58:45 : Logger entry zero: Y = 2009, M = 7, D = 24, H = 1, M = 0
2009-08-19 16:58:45 : VP2: Earliest timestamp 24/07/2009 01:00:00
2009-08-19 16:58:45 : VP2: Earliest timestamp is earlier than requested

You hit the 'feature' in the station where it decides to hand over its entire contents. Cumulus then had to sift through all this garbage and check whether there's actually anything new in there. This all gets logged, by a background thread - and there's lots and lots of it to log. You then stopped and started Cumulus again while the background thread was still running, so you end up with two copies of Cumulus running, and the failure to write to the log file occurs.

Did you not set the option to prevent multiple copies of Cumulus running?
Steve
RayProudfoot
Posts: 3373
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: Option to send wind gust data to Wunderground...

Post by RayProudfoot »

steve wrote:The reason for the missing data is this:

2009-08-19 16:55:01 : 19/08/2009 16:55:00 Error writing to data file: C:\Program Files\Cumulus\data\Aug09log.txt. I/O error 32
2009-08-19 16:55:01 : 19/08/2009 16:55:00 Error writing to data file: C:\Program Files\Cumulus\data\Aug09log.txt. I/O error 103

This is the same problem you had before, I think? But I can't remember what the cause was...
Two instances of Cumulus running as you point out in your second reply. But after closing Cumulus I opened Task Manager / Processes and made sure it had terminated before I started it again so I find this very odd.
steve wrote:Then, when asked for all the records since 19/08/2009 00:01:00, the station says there aren't any:

2009-08-19 17:56:23 : VP2: Downloading history from 19/08/2009 00:01:00
2009-08-19 17:56:25 : VP2: GetNumberOfArchiveRecords_V = 0
2009-08-19 17:56:25 : VP2: No history records to process

So that's why setting the timestamp back hasn't worked. It's a bit annoying that the station doesn't seem to want to co-operate, as that's by far the easiest way of getting the data back. Anyway, if you send me your Aug09log.txt and the section of exported Weatherlink file, I'll convert it for you. I think I need to package up the converter and make it available...
The file containg data from 23:45 last night to 16:45 today is attached. Thanks for doing this for me Steve. The converter sounds like a good idea.
steve wrote:By the way, your station is on port 18? Is that an add-in serial card or something?
I also run WeatherLink so I use VirtualVP. That's where port 18 comes from.
You do not have the required permissions to view the files attached to this post.
Cheers,
Ray, Cheshire.

Image
RayProudfoot
Posts: 3373
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: Option to send wind gust data to Wunderground...

Post by RayProudfoot »

steve wrote:I remembered what I concluded the cause of the previous occurrence was - a second copy of Cumulus running. And I think that's the case again here. You stopped Cumulus and then started it again shortly afterwards; then when Cumulus asked for the data from the logger, this happened:

2009-08-19 16:58:39 : VP2: Downloading history from 19/08/2009 16:55:00
2009-08-19 16:58:45 : VP2: GetNumberOfArchiveRecords_V = 2559
2009-08-19 16:58:45 : VP2: Number of history records = 2559
2009-08-19 16:58:45 : Logger entry zero: Y = 2009, M = 7, D = 24, H = 1, M = 0
2009-08-19 16:58:45 : VP2: Earliest timestamp 24/07/2009 01:00:00
2009-08-19 16:58:45 : VP2: Earliest timestamp is earlier than requested

You hit the 'feature' in the station where it decides to hand over its entire contents. Cumulus then had to sift through all this garbage and check whether there's actually anything new in there. This all gets logged, by a background thread - and there's lots and lots of it to log. You then stopped and started Cumulus again while the background thread was still running, so you end up with two copies of Cumulus running, and the failure to write to the log file occurs.

Did you not set the option to prevent multiple copies of Cumulus running?
Yes, I definitely set the 'Prevent two instances running' option so I find that strange. It looks as though Cumulus might still be running even though Task Manager / Processes shows no entry. Maybe I should leave it for a minute or so to make sure it really has stopped.

I'm not sure what that option is that hands over the entire contents. Give me a clue please. Is that a user-defined option?

I'm sorry to cause you all this grief. These two programs run on a netbook that isn't used for anything else so it shouldn't be causing these problems. But the next time I update Cumulus I'll definitely leave it a couple of minutes before I start it again after terminating it.
Cheers,
Ray, Cheshire.

Image
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Option to send wind gust data to Wunderground...

Post by steve »

RayProudfoot wrote: Yes, I definitely set the 'Prevent two instances running' option so I find that strange. It looks as though Cumulus might still be running even though Task Manager / Processes shows no entry.
It looks like the check that I do doesn't cover this case of the logger thread still running. I'll have a think about what I can do about this; maybe I can try to terminate the logger thread, or not log so much. I'm still not completely confident that I've identified the whole problem, though; it doesn't really explain why the log file is still open.
I'm not sure what that option is that hands over the entire contents. Give me a clue please. Is that a user-defined option?
No, it's not an option, it's a bug in the VP2 (which VirtualVP apparently replicates). If you ask it for the data since 'X', it occasionally gets confused and sends everything instead. Cumulus then has to process it all and pick out any new stuff. You may have noticed sometimes when starting Cumulus up that the 'downloading data' message is up for a long time, even though you wouldn't have expected much data to be downloaded.

One example of when this can occur is when you ask for the data since 'X' and 'X' doesn't correspond exactly to the timestamp of a record in the VP2 logger. This is one reason why I strongly recommend that people set the Cumulus log interval to the same as the VP2 logger interval, so the timestamps coincide. It may also explain why you got zero records back when you tried resetting the timestamp. the timestamp you set wasn't one that it knew about; although I'd have expected it to give everything rather than nothing. Maybe VirtualVP works differently.

Another common situation is starting Cumulus up shortly after closing it. It seems that if 'X' is very close to the current time (i.e. it's after the latest timestamp in the logger) then the VP2 can do its regurgitation trick.
I'm sorry to cause you all this grief. These two programs run on a netbook that isn't used for anything else so it shouldn't be causing these problems.
It's not your fault; it's either this logger thread that runs on after Cumulus closes, or a bug in Cumulus which only occurs under certain circumstances. But I restart Cumulus at least once a day, so you'd have thought I'd have hit it myself by now. I'll try to do some experimenting to see if I can reproduce it.

Processed file attached :)
You do not have the required permissions to view the files attached to this post.
Steve
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: I/O Error 32 writing to log file

Post by steve »

Ray, I've been looking at your diags files again, and I don't think my explanation is correct. It looks like you stopped Cumulus at about 16:56 yesterday, installed the new version, then started it at about 16:58? But the problem with writing to the log file started at about 23:35 the previous night, while Cumulus was just running normally. Can you think of anything you were doing on the PC at the time? (approx 23:35 on Tuesday night).
Steve
RayProudfoot
Posts: 3373
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: Option to send wind gust data to Wunderground...

Post by RayProudfoot »

steve wrote:
RayProudfoot wrote: Yes, I definitely set the 'Prevent two instances running' option so I find that strange. It looks as though Cumulus might still be running even though Task Manager / Processes shows no entry.
It looks like the check that I do doesn't cover this case of the logger thread still running. I'll have a think about what I can do about this; maybe I can try to terminate the logger thread, or not log so much. I'm still not completely confident that I've identified the whole problem, though; it doesn't really explain why the log file is still open.
Is the logger thread identifiable via Processes? If so I can ensure it has stopped before restarting Cumulus.

steve wrote:
RayProudfoot wrote:I'm not sure what that option is that hands over the entire contents. Give me a clue please. Is that a user-defined option?
No, it's not an option, it's a bug in the VP2 (which VirtualVP apparently replicates). If you ask it for the data since 'X', it occasionally gets confused and sends everything instead. Cumulus then has to process it all and pick out any new stuff. You may have noticed sometimes when starting Cumulus up that the 'downloading data' message is up for a long time, even though you wouldn't have expected much data to be downloaded.
I've never seen the progress bar in that downloading data show any progress. It appears empty and after a few seconds closes - still empty.
steve wrote:One example of when this can occur is when you ask for the data since 'X' and 'X' doesn't correspond exactly to the timestamp of a record in the VP2 logger. This is one reason why I strongly recommend that people set the Cumulus log interval to the same as the VP2 logger interval, so the timestamps coincide. It may also explain why you got zero records back when you tried resetting the timestamp. the timestamp you set wasn't one that it knew about; although I'd have expected it to give everything rather than nothing. Maybe VirtualVP works differently.

Another common situation is starting Cumulus up shortly after closing it. It seems that if 'X' is very close to the current time (i.e. it's after the latest timestamp in the logger) then the VP2 can do its regurgitation trick.
I have configured WeatherLink to log data every 15 minutes but Cumulus every 5. Is that okay? Regarding starting Cumulus again I now check via Task Manager / Processes that it has terminated before I restart it but the logger thread may still be running of course.
steve wrote:
RayProudfoot wrote:I'm sorry to cause you all this grief. These two programs run on a netbook that isn't used for anything else so it shouldn't be causing these problems.
It's not your fault; it's either this logger thread that runs on after Cumulus closes, or a bug in Cumulus which only occurs under certain circumstances. But I restart Cumulus at least once a day, so you'd have thought I'd have hit it myself by now. I'll try to do some experimenting to see if I can reproduce it.
I feel guilty because no-one else appears to be having these problems. I only restart Cumulus when either a) I load a new version or b) when Windows updates force a reboot. But I am also running WeatherLink and of course, VirtualVP but so are you I think? How long do you leave it between ending Cumulus and restarting it? And do you also stop W/Link & VirtualVP?

Thanks for the processed file. :)
Cheers,
Ray, Cheshire.

Image
RayProudfoot
Posts: 3373
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: I/O Error 32 writing to log file

Post by RayProudfoot »

steve wrote:Ray, I've been looking at your diags files again, and I don't think my explanation is correct. It looks like you stopped Cumulus at about 16:56 yesterday, installed the new version, then started it at about 16:58? But the problem with writing to the log file started at about 23:35 the previous night, while Cumulus was just running normally. Can you think of anything you were doing on the PC at the time? (approx 23:35 on Tuesday night).
Hi Steve,

I was also thinking the problem may have started the evening before with the logging ending after 23:35. The only thing I do with Cumulus at present is look at the data via the graphs - mainly via Select-A-Graph. Might it be possible that doing that somehow inteferes with the capturing of the data? I certainly didn't terminate and restart Cumulus that night but as you correctly point out I did update my version around 1700hrs yesterday.

I don't upload to a website at present, only to WU with Rapid Fire enabled. That was with WL until yesterday when I switched to doing it via Cumulus but that doesn't seem related to the problem as I've been doing that since May.
Cheers,
Ray, Cheshire.

Image
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Option to send wind gust data to Wunderground...

Post by steve »

RayProudfoot wrote:Is the logger thread identifiable via Processes? If so I can ensure it has stopped before restarting Cumulus.
In the task manager processes list, it appears to just show up as cumulus.exe, so I guess if you've made sure that's gone, then it's no longer running.
I've never seen the progress bar in that downloading data show any progress. It appears empty and after a few seconds closes - still empty.
This will be because you have Cumulus running all the time (apart from restarting it), so there's never any (real) data to process.
I have configured WeatherLink to log data every 15 minutes but Cumulus every 5. Is that okay?
Well, it's OK, but you do run the risk of hitting this bug in the VP2 where it downloads its entire contents. Cumulus should cope, though.
Regarding starting Cumulus again I now check via Task Manager / Processes that it has terminated before I restart it but the logger thread may still be running of course.
It seems that should be sufficient to confirm that the logger thread has stopped.
I feel guilty because no-one else appears to be having these problems.
I think that something happens on your system to prevent Cumulus writing to the log file (which may of course be down to a bug in Cumulus) and then Cumulus never recovers once this happens. I'm looking into changing the code which writes to the log file, so that hopefully once it occurs it does recover.
But I am also running WeatherLink and of course, VirtualVP but so are you I think? How long do you leave it between ending Cumulus and restarting it? And do you also stop W/Link & VirtualVP?
No, I don't have VirtualVP, and I don't even have WL installed now (I used to run it from time to time just to keep its logs up to date).
Steve
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: I/O Error 32 writing to log file

Post by steve »

RayProudfoot wrote:I was also thinking the problem may have started the evening before with the logging ending after 23:35. The only thing I do with Cumulus at present is look at the data via the graphs - mainly via Select-A-Graph. Might it be possible that doing that somehow inteferes with the capturing of the data?
Interesting - Select-A-Graph does read the log files - that's where it gets the data from to plot. It shouldn't interfere with the writing of the file, though, but that could well be the problem. I don't suppose you can remember if you were looking at Select-A-Graph at about 23:30 on Tuesday night?

I'll do some experimenting...
Steve
RayProudfoot
Posts: 3373
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: Option to send wind gust data to Wunderground...

Post by RayProudfoot »

steve wrote:
I have configured WeatherLink to log data every 15 minutes but Cumulus every 5. Is that okay?
Well, it's OK, but you do run the risk of hitting this bug in the VP2 where it downloads its entire contents. Cumulus should cope, though.
Should I change WL to 5 minute intervals too? IIRC there is an offset value in that program. If I set it to 0 then it will read data at the same time as Cumulus. As I use VirtualVP that shouldn't present a problem should it?
steve wrote:I think that something happens on your system to prevent Cumulus writing to the log file (which may of course be down to a bug in Cumulus) and then Cumulus never recovers once this happens. I'm looking into changing the code which writes to the log file, so that hopefully once it occurs it does recover.
Without suggesting wholesale changes maybe some indication on the main interface of the last time/date the log was written to? An audible alarm when more than 1 capture has been missed would be the icing on the cake.
steve wrote:No, I don't have VirtualVP, and I don't even have WL installed now (I used to run it from time to time just to keep its logs up to date).
I'm reluctant to uninstall WL because I would then lose the data and I do like some of its features.
Cheers,
Ray, Cheshire.

Image
RayProudfoot
Posts: 3373
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: I/O Error 32 writing to log file

Post by RayProudfoot »

steve wrote:
RayProudfoot wrote:I was also thinking the problem may have started the evening before with the logging ending after 23:35. The only thing I do with Cumulus at present is look at the data via the graphs - mainly via Select-A-Graph. Might it be possible that doing that somehow inteferes with the capturing of the data?
Interesting - Select-A-Graph does read the log files - that's where it gets the data from to plot. It shouldn't interfere with the writing of the file, though, but that could well be the problem. I don't suppose you can remember if you were looking at Select-A-Graph at about 23:30 on Tuesday night?

I'll do some experimenting...
Yes, I was using Select-A-Graph that night. Maybe it puts a lock on the log file preventing writes? Just a thought. Hope you can find the cause. :)
Cheers,
Ray, Cheshire.

Image
Post Reply