v3.28.1 bulid 3278

[alexvanuxem]

Hi,

the new build doesn't work for me.
some issue with the ftp upload :
2024-01-11 19:27:20 - RealtimeFTPLogin: Error connecting ftp - Timed out trying to connect!
2024-01-11 19:27:03 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:27:00 - RealtimeFTPLogin: Error connecting ftp - The underlying socket is null. Have you established a connection?
2024-01-11 19:26:43 - FTP[0]: Upload of wwwroot/Cumulus/realtime.txt failed
2024-01-11 19:26:09 - RealtimeFTPLogin: Error connecting ftp - The underlying socket is null. Have you established a connection?
2024-01-11 19:26:03 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:25:03 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:24:03 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:23:12 - RealtimeLocalCopy[78]: Error copying [web\realtimegauges.txt] to [web\realtimegauges.txt. Error = Kan geen toegang krijgen tot het bestand C:\CumulusMX\web\realtimegauges.txt omdat het wordt gebruikt door een ander proces.
2024-01-11 19:23:11 - RealtimeFTPLogin: Error connecting ftp - The underlying socket is null. Have you established a connection?
2024-01-11 19:23:03 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:22:03 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:21:42 - RealtimeFTPLogin: Error connecting ftp - The underlying socket is null. Have you established a connection?
2024-01-11 19:21:03 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:20:14 - RealtimeFTPLogin: Error connecting ftp - The underlying socket is null. Have you established a connection?
2024-01-11 19:20:04 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:19:03 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:18:33 - RealtimeFTPLogin: Error connecting ftp - Timed out trying to connect!
2024-01-11 19:18:04 - LocalCopy: Error copying moon image - Kan een gedeelte van het pad web\images\moon.png niet vinden.
2024-01-11 19:18:01 - RealtimeFTPLogin: Error connecting ftp - Timed out trying to connect!

the default website https://www.sint-katelijne-waver-meteo.be/ doesn't work anymore
nore does https://www.sint-katelijne-waver-meteo.be/Cumulus
nor does upload to weathercloud

when i revert back to the previous , magic, everything works

grtz
Alex

[ConligWX]

When you upgraded did you run the hash check to make sure all files were correct?

I'm using FTP and have had no issues at all.

[alexvanuxem]

No I didn't.

I never do. Can you quickly explain how to do it?

cheers a

[nossis]

Ah so I am not the only one. I got the same however I use PHP upload process. Found that it didn't play happy with uploading after upgrading to latest version. Logs complained a lot. I will have to see if I can set the logging level higher to see if we can learn anything more about it.
Ran hash check on the download and came up good.

Code: Select all

PS C:\Users\me> Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass

Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose
you to the security risks described in the about_Execution_Policies help topic at
https:/go.microsoft.com/fwlink/?LinkID=135170. Do you want to change the execution policy?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): y
PS C:\Users\me> cd\
PS C:\> cd .\temp\Download\CumulusMX\
PS C:\temp\Download\CumulusMX> .\MXutils\windows\CheckMD5HashFile.ps1 -HashFile .\hash_md5_3278.txt
All checksums are OK
PS C:\temp\Download\CumulusMX>

How to do it for the Windows users....
bring up PowerShell
run the command to allow it to run the not digitally signed PowerShell script (this only applies to the PowerShell session that you are running now)
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
Then navigate to the folder where your extracted CumulusMX install package is.
cd\ to take you back to C:\ drive
cd temp\Download\CumulusMX (I press tab key after typing 3 letters for it to show the folder, saves typing since I am lazy, it also put the . in front of the path when I did that)
Copy past the powershell script command as found in the file for me C:\temp\Download\CumulusMX\hash_md5_3278.txt, as I have shown.
If one gets as I did in lovely green letters "All checksums are OK" you know that your download is unaltered. Checksums match.
If you make a change to any of the files in the folder prior, then the checksums will not match. I did that, change one, just to see what would happen and the tool reports the file that it failed in. Cool.

[mcrossley]

The basic problem seems to be

Code: Select all

RealtimeFTPLogin: Error connecting ftp - Timed out trying to connect!

Which means either the FTP server is down or inaccessible. Or the FTP server details in MX are incorrect.

[alexvanuxem]

thx

checksums are ok!

grtz

a

[alexvanuxem]

and these are my logs,

issue with the ftp

[alexvanuxem]

@mcrossley

No, ftp settings didn't change.

It has something to do with the incremental upload of txt files.

Which are incremental?

when i rstart cumulus with the new build, the firts minuut is ok.

then it fails.

upload to weathercloud doesn't work either with the new build.

Grtz

[alexvanuxem]

@mcrossley

these are my extra web files

which are incremental?

ftp.png

[mcrossley]

[x-posted]
First you have your real time FTP upload interval set to 1 second! The files are taking about 3 seconds to complete the upload, meantime multiple other real time upload session start and abort because one is already in progress.

I think the best you could hope for is a real time interval of 5 seconds, you probably be better backing it off to 10 seconds.

You also seem to put all the files in the real time for some reason, even files which appear to have slow changing data like cloud base, and all-time records. Get all this moved to the interval update, real time should have the minimum amount of fast changing data in it.

From what I can see, the FTP detects an error and attempts to reconnect with the server, but because there are no pauses in upload requests it is getting, I do not think the reconnect works correctly and things go downhill from there.

Sorting out your upload strategy will help a great deal - it will also make interpreting the log file much simpler!

[After seeing your post]
I do not think this has nothing to do with incremental log files, you are not using that feature as the incremental column is not shown on screen grab. (I think you need to flush your browser cache)

Your screen shot shows what I thought, you are trying to upload everything at the real-time interval, this is madness - you think the moon image changes every second!

I attach what I think you should have set.

fm5wye55.png

Are you also uploading files using the standard upload settings? I'm just wondering why moon.png, realtime.txt, realtimegauges.txt, websitedata.json are being uploaded via Extra Files?

Also, if you created symbolic file links on the destination server for the files you are uploading multiple times, could you just upload them once?

[Cortmalaw]

@Mark
For those that you marked with X, is there any reason anyone would need them as realtime or process?
Maybe you should disable the option to request ...

[Nossie]

@Mark
For those that you marked with X, is there any reason anyone would need them as realtime or process?
Maybe you should disable the option to request ...

Problem is from my understanding is that you can add anything to be processed - which is great but as you have found also means the wrong settings - I don't believe this would be a good idea as it would cause problems with people that use custom files.

[mcrossley]

As Ian says, MX has no idea what the files contain or what they will be used for. Those options are where you tell MX what it needs to know about the files.

[HansR]

As Ian says, MX has no idea what the files contain or what they will be used for. Those options are where you tell MX what it needs to know about the files.

No, true.

But you could implement some heuristics: check file sizes, forbid realtime frequency < 5 secs, warn < 10 secs, no realtime and/or realtimegauges no logfile processing (dayfile, monthly logs etc...) etc... Everything alex does is not very rational, searching the edges. You could do something to prevent. On the other hand it is a users responsibility and rationality. They need to get to know the system don't they?

[freddie]

I agree with your last statement, Hans. It is the users responsibility, despite the fact that bad choices can be made. But that's true for many real life decisions.

I'm not in favour of any control over what can be set as the context is unpredictable, therefore it is impossible to formulate sensible rules that can be applied.

All of my readings (and some derived ones) are updated at a two second realtime interval, and my system copes fine with that cadence.