Trouble Uploading NOAA Reports

[water01]

Sorry but the Wiki is wrong. Firstly, the entry should FTPlogging and secondly the settings are 0(zero) is off and 1(one) is on.

[Nossie]

Maybe that's from a previous release?

But yeah I have this in my config

IgnoreCertErrors=0
FTPlogging=1

still no dice with a log file though =( it used to log fine until I turned it off - how strange.

[freddie]

still no dice with a log file though =( it used to log fine until I turned it off - how strange.

Did you restart MX after changing the setting. I think that is a requirement.

[mcrossley]

Please use the program settings pages to set the logging on/off rather than directly editing Cumulus.ini, the scope for error is much less (like setting the wrong value, doing it whilst MX is running so it gets overwritten etc)

[MuntyWeather]

Hi Hans
While trying to load latest version of CUtils found that my CumulusMX file structure was corrupt in that the 'utils' directory was missing (?!) So corrected and now working again, but maps.txt seems to be missing.
Have a lot of sympathy with nossie, but my problem is slightly different. I've checked on IONOS and all the 2022 files are present, but when I put in year 2022, Oct and Nov are greyed out, however, when I put in year 2021 Oct and Nov show and moreover when I select Oct the report for Oct 2022 is comes up.
Regards Ted

[HansR]

Hi Ted,

While trying to load latest version of CUtils found that my CumulusMX file structure was corrupt in that the 'utils' directory was missing (?!) So corrected and now working again, but maps.txt seems to be missing.

I have no comment on this because I can't see on your computer. Please post your last logfile.
If you can run CumulusUtils again I would advise to do:

Code: Select all

bin/cumulusutils.exe website

to make sure everything is up to date and in working order. The website must reflect the version of CUtils AND the date of the run in the footer.

Have a lot of sympathy with nossie, but my problem is slightly different. I've checked on IONOS and all the 2022 files are present, but when I put in year 2022, Oct and Nov are greyed out, however, when I put in year 2021 Oct and Nov show and moreover when I select Oct the report for Oct 2022 is comes up.

They are greyed out because you did (or do) not run CumulusUtils on a daily basis. A run will guarantee the update of data and menu's of the different reports. The greyout means the menu only goes until september and inhibits progressing to October, it requires the daily refresh run to update. There is no realtime check of which files are present on the server.

So in summary: for you it is not the transfer of the files which is a problem but the daily run of CumulusUtils which apparently stopped for some reason which I cannot determine. So check on the daily run which should be like:

Code: Select all

bin/cumulusutils.exe thrifty website

Thrifty is optional but advised.

[HansR]

@sfws: I removed your post. Since you can't be communicated to by PM or email I give the reason here: there is already too much confusion and your post did not change that. I would appreciate you refrain from posting in this forum.

[MuntyWeather]

Hi Hans
I think the mysterious file corruption I mentioned stopped it doing the daily run of Cutils. I use external program facility in CMX. Have just done Website run, logfile attached.

Thanks and regards

[HansR]

Ted,

Aha... your problem may be greater than just these NOAA reports. While checking the NOAA context in the logfile (little info there, I'll change that) I check the resulting code on your server and that is OK, only the months disabled are indeed Oct, Nov, Dec while that should only be Dec.

This triggered me to look in the actual data using the YADR report and there I saw/see that the data acquisition apparently stopped on 12 September.

At the end of your logfile it shows a crash (recoverable) as it did before making the map. And that is where the problem lies.
It cannot make an SSL connection to make the map and later it cannot upload it to the server (intentional non-recoverable crash) so no updated files get uploaded to the server.

Most likely you reinstalled the RPi-OS at some point around September 12 and forgot to comply with NOTE 3 in the release post saying to do what is in this message (removing a disabled certificate). Don't ask why it is still in that list.

If you do the above and run CUtils again, my guess is it will be OK.
Please post the logfile again if it is not.

In addition, I would advise you to update the RPi OS:

Code: Select all

apt update
apt full-upgrade
apt autoremove
apt clean

and after that bring mono to 6.12 . To do so this procedure worked for me.

[MuntyWeather]

Hi Hans,
Rather long story: wrt certificates:/mozilla/DST_Root_CA_X3.crt was not on the list. Updated the system, deleted and reinstalled mono. CMX working. Ran Cutils website. Still stops while referencing maps.txt. NOAA reports the same. log attached .
Cautionary tale: this all started when I added a solid state drive to the system. I tried to get it to boot from the SSD but was unsuccessful. So now it boots from the microSD. CumulusMX and the data files are on the SSD.
Thanks and Regards, Ted

[HansR]

I attach maps.txt. Put that file on your utils directory, that should prevent the crash and at least it can continue and upload all necessary files. In the mean time, we hope somebody can tell us what is going on.

Apparently something inhibits to get the map from the map server and therefore it can't upload to your own server.

The actual error is below. If anybody can tell what actually would be the problem I would be much obliged. It is a simple GET with a HttpClient which works almost everywhere (but e.g. it does not on Win7 and on this RPi system). So what can be the cause? And better: what setting might prevent this?

Code: Select all

2022-11-10 15:13:08.565 Information GetUrlData Calling GetAsync
2022-11-10 15:13:08.791 Error GetUrlData : Exception - The SSL connection could not be established, see inner exception.
2022-11-10 15:13:08.795 Error GetUrlData: Inner Exception: System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
  at ./external/boringssl/ssl/handshake_client.c:1132
  at Mono.Btls.MonoBtlsContext.ProcessHandshake () [0x00064] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status, System.Boolean renegotiate) [0x00106] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 
  at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus,bool)
  at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 
  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x0012a] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 
   --- End of inner exception stack trace ---
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x00346] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 
  at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore (System.IO.Stream stream, System.Net.Security.SslClientAuthenticationOptions sslOptions, System.Threading.CancellationToken cancellationToken) [0x000d2] in <3588d5c507ec48a6b5d13a6d5ae64da2>:0 
2022-11-10 15:13:08.796 Error Fetch Map from server: Fail... empty map.

[HansR]

I made a test version that might make it possible to make the SSL connection and as such avoid the problem.
Attached you find 6.14.3 beta 1. If you wish you can replace your executable with this one and give it a try.
I am out of the loop for some days but if it does not work, just replace the exec with the old one again.

[PaulMy]

I don't know what's going on

The only reason I had reports working previously was because I had uploaded them manually on that date -

I can upload one manually and update it realtime through the extra web files as per the screenshot. (turned it off again)

@Hans

In CUtils the reports folder is given as relative to the webroot (!) so the parameter FTPDirectory becomes: ./Reports/
Closing slash is required.

./Reports/ is what I should have? but Extra web files is only public_html/Reports ?

Is there is no way to check this again until tonight?

Your NOAA Reports are current http://www.stewartonweather.com/Reports/NOAAMO1122.txt
is that from CMX Day End process success, or by your manual FTP?

Enjoy,
Paul

[Nossie]

Sorry Paul that I didn't update - just checked today while I was working and yes, you can imagine how happy I was to see another day added - no I didn't upload it manually it was all automagic

Thanks for everyones assistance

[PaulMy]

Great
You can fool with people... but you can't fool with code

Enjoy,
Paul