Cumulus hangs at web site update

[archae86]

I've been running Cumulus on Windows posting data from a Davis Vantage Pro2 station since 2011. I've been running Cumulus version 1.9.4 build 1099 for many, many months. My setup failed abruptly at midnight October 2.

Since then, the Cumulus software stalls the moment it hits the next 5 minute "update web site" point.

While the most recent update timestamps on the various pages of my website:

https://pastoll.info/weather/index.htm

Suggest the original failure happened after about half of the pages had been uploaded, the way it fails now seems to happen before any attempted traffic at all. If I have the user interface showing on my desktop as a 5 minute interval approaches, the clock in the upper right counter just stops at either nn:nn:59 or nn:nn:00. Checking the application for activity as indicated by Process Explorer suggests complete stasis. Not only is new CPU time not accumulated, there is not even any change in Page Fault count, which normally increases by hundreds in each update interval if the user interface is in view on the desktop.

I tried turning off "Auto Update" under Web settings, and Cumulus ran successfully for over nine hours with no trouble detected. While I imagined I had turned off both the loading of my web page and my uploads to Weather Underground, in truth my Wunderground updates were running all that time. Successfully, I think.

https://www.wunderground.com/dashboard/pws/KNMALBUQ102

So my failure is in some sense specifically associated with starting to run the code involved in doing web updates. Unlike normal internet communication problems, I don't get an entry in the Error Log portion of the Status field in the desktop interface. I turned on Debug Log once, and the last few lines read:

0190.672 : 08:24:56.347 Load current data
0191.578 : 08:24:57.255 Data loaded
0191.578 : 08:24:57.255 Load current data
0192.484 : 08:24:58.164 Data loaded
0192.484 : 08:24:58.164 Load current data
0193.578 : 08:24:59.264 Data loaded
0193.578 : 08:24:59.264 Load current data
0194.484 : 10/5/2022 8:25:00 AM Minute timer 25
0194.484 : Adding LH entry, ts = 8:25:00 AM raintot = 10.7299995422363
0194.484 : 8:25:00 AM Creating WXNow File
0000.000 : **** Logging Disabled

0000.000 : **** Logging Disabled

After I started writing these notes, I decided to try a manual update by going to my session with over nine hours of success and triggering a manual web page update using File|Web Update.

As with my automatic 5-minute updates, the reaction was that the time-of-day indication in the user interface froze the moment I clicked the update button. Process Explorer indicates that Page Faults ceased entirely, and in other respects the user Interface seemed completely unresponsive. I accepted the offer from Windows to force a close of the program.

It is my hope to get my website updating again, without losing ten years of data. I seek advice on how I might reasonably proceed.

Possibly I should download a fresh install package for Cumulus.
Possibly I should initiate my long-intended transition to Cumulus MX, seeking the proper procedure for transitioning an existing Cumulus installation.

I'm eager for any advice. Meanwhile, I intend in the short term to run Cumulus with web page Auto Update disabled. I believe this means my data files should continue to accumulate station data, and my Wunderground data will be updated, but my Web page will remain as it froze at midnight October 2.

[PaulMy]

Hi,
If you suspect it is FTP related you could enable FTP Logging - Configuration > FTP Logging and then check C:\Cumulus\ftplog.txt
The Cumulus legacy (Cumulus1) ftplog is not as detailed as in current CumulusMX but it might give some indication. Possibly your webhost has made some FTP changes?

Possibly I should download a fresh install package for Cumulus.

Reinstalling does not usually fix things, but a computer reboot and a Cumulus restart may be good to try.
However, looks like you have not installed the fix to allow for showing data past the year 2020 https://cumuluswiki.org/a/Software#Cumulus_1 and you may want to do that if you continue with Cumulus1.

Possibly I should initiate my long-intended transition to Cumulus MX, seeking the proper procedure for transitioning an existing Cumulus installation.

It is relative quite simple. With a bit of caution and awareness you should be able to continue to use Cumulus1, and also install CumulusMX on your PC. Just stop one and start the other whenever you want. As noted in the Wiki you could copy your current C:\Cumulus\data and \Reports folders to the new C:\CumulusMX and also the C\Cumulus\cumulus.ini or Cumulus.ini settings file (for CumulusMX it must be saved with capital "C" i.e. Cumulus.ini )

Enjoy,
Paul

[freddie]

On the surface it sounds like there has been a change to the web host. Probably disabling FTP to force you to use SFTP or FTPS. Hosters do this as FTP is inherently insecure. As PaulMy suggests, switching on FTP logging will reveal more.

[RayProudfoot]

If the problems are related to your ISP and they cannot help consider a different hosting service.

I’ve been using HostPresto for a few years now and they’re excellent. I remain on Cumulus1 and don’t have any issues so I doubt the software is the problem.

https://hostpresto.com/

[archae86]

Cumulus ran fine overnight with both FTP logging and Debug log activated, and web settings auto update switched off.
After reading your responses I did a manual web update (File|web update).
Again I observed that the time of day display immediately stopped incrementing.
After using Windows to close the program in the unresponsive state, I looked in the Cumulus directory for recently time-stamped files
I had a nice big debug.log from the night of running, but again the last entries seem to contain no hint I can detect.
Where should I expect to find the result of the FTP logging--just additional entries in debug.log, or somewhere else?

Here are the last few lines from debug.log
4376.0156 : 07:50:59.497 Data loaded
4376.0156 : 07:50:59.497 Load current data
4376.0750 : 10/6/2022 7:51:00 AM Minute timer 51
4376.0750 : Adding LH entry, ts = 7:51:00 AM raintot = 11.1099996566772
4376.0750 : 7:51:00 AM Creating WXNow File

My suspicion is that it freezes before it even attempts any communication. I'll find my installation of Wireshark and try to remember how to get it to log actual comms.

I've been using A2 Hosting for the last four years. While I had an issue with their handling of traffic when I was using Starlink which differed from the behavior when I was using Xfinity (Comcast), it certainly did not give the result that Cumulus froze. But I'll try switching my service back to Xfinity later this morning to see if that makes any difference.

[PaulMy]

HI,

Where should I expect to find the result of the FTP logging--just additional entries in debug.log, or somewhere else?

As per my initial reply

enable FTP Logging - Configuration > FTP Logging and then check C:\Cumulus\ftplog.txt

Open the \Cumulus\ftplog.txt file with Notepad++ As mentioned the information is not as detailed in Cumulus1 as it would be in CumulusMX.

The ftplog.txt file can get quite large so make sure it gets unticked after a while.

Enjoy,
Paul

[archae86]

The ftplog.txt file can get quite large so make sure it gets unticked after a while.

Sorry. I was looking at file timestamps, and did not spot ftplog.txt since it has a file timestamp from 2017.

So running all night with ftp logging enabled but web update turned off added nothing to this file. (perhaps my updates to Wunderground don't use FTP?)
Clicking "web update" from the file menu both at the end of that long run, and after a couple of minutes of a new run also added nothing.

In the interim, I did try doing a fresh download of the installer (which was a byte-perfect match to the installer I had on my drive), and ran it. The installer apparently detected my existing install location, and advised me it would be installing there.

As one of you predicted, this action made no discernable difference.

I've not yet tried to re-learn how to do even the most basic monitoring with WireShark, but that comes next.

Oh, one other thing:
I fired up FileZilla. Using connection settings which I believe specified using FTP, I was able to connect to my A2 Hosting server with my old credentials, view directories, create a new test directory, and transfer the moon bitmap image from my Cumulus directory. Possibly this means my hosting service still accepts some FTP transfers.

In a few hours, I intend to try to relearn a bit of basic WireShark configuration technique, use it to watch a FileZilla transfer to my hosting service, then use it to monitor a new web update attempt from Cumulus.

Right now I'm going to take a breather, but would be most interested in any advice and comments. Thank you.

[archae86]

A little while ago I ran Wireshark while attempting web updates from Cumulus, both by clicking the manual update button while auto update was turned off, and by just letting a 5 minute boundary go by with auto turned on.

In both cases Wireshark showed me lines outside my net I presume were interactions with A2 Hosting. It colored some of them in ways indicating Wireshark thought them to be undesired or abnormal. Some of them were TCP. At my human level of time detection they appeared just at the moment that the web update attempt began.

So my previous guess posted here that the Cumulus program froze internally before even attempting Internet communication was wrong--and you folks who advised me that A2 Hosting was probably behaving in a different way than previously in response to the Cumulus behavior are very likely correct.

As I am unlikely to influence A2 hosting behavior, this leaves me wondering whether web updating on Cumulus MX is enough different from web updating on legacy Cumulus to give me hope that if I go through the transition effort it might reasonably be hoped to work? Or at least to have a more informative failure behavior?

If anyone here is adept at reading Wireshark captures, and is interested, I have a quite small pcapng file (12 kbytes) which shows monitoring during the auto update attempt which froze, which I'd be happy to share by means you prefer.

I currently suppose that legacy Cumulus is incompatible with my current web hosting service. So my main options are a new program or a new service. My first choice is a new program in the form of CumulusMX, so I intend to go looking at the various help available on this support site, paying particular attention to advice for people migrating an existing legacy Cumulus installation.

As it happens, I started down this road once before, as I can see an MX installation which on internal evidence I worked on in February 2017. I don't remember what stopped me back then, but my incentive this time is far higher, and anyway MX has progressed.

I remain grateful for advice and instruction already received, and am eager for more.

I do intend to try one last longshot in a few minutes. I've powered up my cable modem, and will switch my internet connection from Starlink to Xfinity, just in case that somehow matters. It actually did matter for some degraded Cumulus updating observed a bit over a year ago. That suddenly very much better a few days after I engaged in support interactions with both Starlink and A2 hosting, though neither one advised me that they had changed anything.

[archae86]

As intended, I did a trial using Xfinity as my internet provide, instead of Starlink. Cumulus still froze immediately that I commanded a web update. I have a Wireshark capture of that one also.

[freddie]

I can look at your Wireshark captures, if you attach them to a PM. I wouldn't attach them to a reply to this topic as this forum is public.

[water01]

As I am unlikely to influence A2 hosting behavior, this leaves me wondering whether web updating on Cumulus MX is enough different from web updating on legacy Cumulus to give me hope that if I go through the transition effort it might reasonably be hoped to work?

CumulusMX will undoubtably be able to overcome any changes to your hosts FTP rules. Cumulus 1 from memory was only capable of doing a standard FTP update whereas MX has the ability to do FTP, FTPS (SSL/TLS Explicit/Implicit) and SFTP (SSH) all using Passive, Active and EPSV modes.

That should cover any of your hosting service needs!!