MX and thread usage

[duke]

Continued from this thread about half way down.

So this probably explains why Windows wasn't firing the timer. What I can't say is whether it was MX that was using up the threads, or something else; you would need to look at Task Manager, or Process Explorer or similar. You could keep an eye on it from time to time, to see if it's using up threads.

OK, so I've had a bit of time this week and had a chance to monitor Process Explorer, unfortunately due to a Windows update and an MX update it's only been running a few days. However, what I have noticed is that when MX is first started it uses about 11 threads but this gradually increases over time. MX has now been running for two days and at this time is averaging around 80 threads. That's an increase of thread usage by about 35 a day.

What I notice with other programs is that their thread count increases but then decreases back to at / near the original count. The thread count for MX does not appear to decrease by the increased amount.

And before we go any further, I know nothing about the technicalities of the thread count usage, I have just been monitoring them as per the original thread and just reporting my observations so I do not know if this is a issue with MX or not. I hope this is helpful.

Screen shot attached.

[steve]

Thanks. When I get chance I'll run it for a while in the Visual Studio debugger so I can see what the threads are being created for. Please keep an eye on it to see if it continues to increase. If you could determine what activity causes an increase, that would be useful - it will be ftp or something like that, at a guess.

[steve]

If I look at how many threads MX is using on my Pi after running for over 48 hours, it varies between 14 and 16.

[duke]

This morning the count was up to 91. I turned off all ftp and processing of files and confirmed this in the .ini. The count then dropped to 81. After 2 hours running like this the count was down to 70. I then reverted back and the count was back to the low 80's. This did not include the realtime files as that requires a restart to take efffect in the .ini. After a restart to re-establish the realtime the count is 15.

N.B. all figures are approximate as they tend to fluctuate to one place either side.

[BCJKiwi]

Running win7 pro on an older notebook.
Using the resource monitor built into Windows;
CumulusMX b3023 uses between 22 and 24 threads (Windows explorer uses 24!).
Total threads in the machine are in the mid 500 range.

[duke]

Running win7 pro on an older notebook.
Using the resource monitor built into Windows;
CumulusMX b3023 uses between 22 and 24 threads (Windows explorer uses 24!).
Total threads in the machine are in the mid 500 range.

Windows explorer on my machine is using between 50 and 80! Perhaps someone could take a look on a windows 8 machine???

Are the amount of threads in use affected by the speed / memory of the PC at all?

[steve]

Windows Explorer is currently using 78 threads on my Windows 8.1 system, according to Task Manager.

[duke]

I have found the cause of the high thread usage. In my 'extra files' tab I have 3 files which in the past I have processed and uploaded to my website, I currently do not use them and therefore have no boxes ticked - image below.

Capture2.PNG

In the diags I have noticed even though MX is supposedly doing nothing with these it causes an error:

Code: Select all

2015-04-11 19:55:00.442 Could not find a part of the path 'public_html/realtime/observetags.php'.
2015-04-11 19:55:00.443 Could not find a part of the path 'public_html/realtime/cumulustags_no_description_recent_array.php'.
2015-04-11 19:55:00.443 Could not find a part of the path 'public_html/realtime/realtime.txt'.

MX has now been running for about 12 hours with these entries deleted and the thread count has not been above 27 and also drops back to below 20 from time to time.

I also have another file which is currently unused, however this does not cause this situation, the only difference is that the 'remote' path is local and not on my web server - image below:

Capture1.PNG

I've also attached the diags from last night, I noticed that at the time I deleted the three files MX reported an error (at the and of the file). I restarted shortly after deleting the files so the error was cleared then.

[steve]

Thanks, I did wonder whether you were getting errors as part of the web tag processing or ftp upload. If you have extra files configured but 'ftp' is not ticked, you are telling it to copy them instead. The resulting (trapped) error must be causing a thread not to be deleted - I'll investigate.

[steve]

I don't think the invalid file copy is the cause. I added an entry with an invalid destination like yours, and I ran it for most of the day on Windows 8.1. It stayed at around 20 threads all day.

[duke]

That's odd, since I removed those entries MX is running around 25 threads. I'll take a look tomorrow night and if the thread count is still 'normal' I'll reinstate the invalid entry and see what happens.

[swyman18]

In my 'extra files' tab I have 3 files which in the past I have processed and uploaded to my website, I currently do not use them and therefore have no boxes ticked

If you have extra files configured but 'ftp' is not ticked, you are telling it to copy them instead.

This was something I recently learned, I was confusing the "Process" option with something like "Enabled". "Process" tells Cumulus to attempt to process the tags. If you have an entry and no boxes are ticked at all, Cumulus will still attempt to copy the file from the source to the destination. Useful if you want to merely copy a file to another location locally.

[duke]

.........I added an entry with an invalid destination like yours............

Was your invalid destination a 'local' one or 'remote' (public_html/.../.../somefile.txt)?

[steve]

It was "/public_html/crap.dat".

[duke]

Ok, just curious. For the record, MX is running low 20's for me now and been like that most of the day. I don't have time tonight but tomorrow night I'll reinstate the invalid path and let it run for a couple of hours.