Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Dayfile.txt uploading to a web server via FTP?

Topics about the Beta trials up to Build 3043, the last build by Cumulus's founder Steve Loft. It was by this time way out of Beta but Steve wanted to keep it that way until he made a decision on his and Cumulus's future.

Moderator: mcrossley

Hobbyist
Posts: 83
Joined: Mon 30 Jun 2014 12:02 pm
Weather Station: Instromet Mk 3
Operating System: Raspberry Pi Zero & CumulusMX
Location: Kent

Dayfile.txt uploading to a web server via FTP?

Post by Hobbyist »

Hi,
I have spent some time trying to achieve this task and am failing so would like some advice.

I am setting up a system using CumulusMX with a Raspberry Pi.

All is working so far with a web server successfully displaying data.

On my existing set up using a PC/Cumulus 1 I have been using Toolbox to upload my DayFile.txt file to the server
every 24 hours to allow online data searching.

I can't seem to find a practical way to do this on the Raspberry Pi set up.

Has anyone been able to do this sort of task and if so how.

Many thanks

Dave
Karv
Posts: 40
Joined: Sun 16 Dec 2012 11:19 pm
Weather Station: WH-1080
Operating System: Windows XP / 7, Ubuntu, Centos
Location: South Gloucestershire

Re: Dayfile.txt uploading to a web server via FTP?

Post by Karv »

It should be pretty easy to achieve using either python or a shell script and Cron to schedule your job to do the upload. There shoudl be lots of examples if you google "FTP shell script" for example.
Hobbyist
Posts: 83
Joined: Mon 30 Jun 2014 12:02 pm
Weather Station: Instromet Mk 3
Operating System: Raspberry Pi Zero & CumulusMX
Location: Kent

Re: Dayfile.txt uploading to a web server via FTP?

Post by Hobbyist »

Thanks for that, I did try and write a script but it seemed that my Pi could not run FTP.

Googling found many folk suggesting using everything except FTP, for security reasons, so not much help.

I thought there would be a 'simple' way forward.

Dave
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Post by steve »

Googling for a linux ftp script returns a number of pages; this one, for example - http://www.stratigery.com/scripting.ftp.html

You could use a modified version of the example given there, invoked either by cron or using the Cumulus 'daily command'.

If you don't have the ftp client installed on your Pi: sudo apt-get install ftp

ncftp is an nicer alternative to ftp which makes scripting easier. sudo apt-get install ncftp
Steve
Karv
Posts: 40
Joined: Sun 16 Dec 2012 11:19 pm
Weather Station: WH-1080
Operating System: Windows XP / 7, Ubuntu, Centos
Location: South Gloucestershire

Re: Dayfile.txt uploading to a web server via FTP?

Post by Karv »

Your Pi should be able to run FTP, what error do you see if for example you type in "ftp" without the quotes and hit enter?
Hobbyist
Posts: 83
Joined: Mon 30 Jun 2014 12:02 pm
Weather Station: Instromet Mk 3
Operating System: Raspberry Pi Zero & CumulusMX
Location: Kent

Re: Dayfile.txt uploading to a web server via FTP?

Post by Hobbyist »

Many thanks Steve once again and Karv for your help.

I had, in my searches, come across that site and had tried to use that script but had failed.

They often say it can be the simpler things that trip you up and in this case, to a Linux and Raspberry Pi rookie, it was.

I has assumed that as my test weather station and CumulusMX were talking to each other and my web pages that FTP

was running on my PI. It Wasn't!

Having actually loaded it with your suggested link Steve and having spend a morning writing a script and getting paths

correct. It all is now working as it should.

I am still puzzled how CumulusMX was able to use an FTP link to my server but I couldn't without installing it but that

is why I am an amateur and you guys are the experts.

Thanks once again and I am learning from my mistakes.

Dave
Last edited by Hobbyist on Tue 13 Oct 2015 4:07 pm, edited 1 time in total.
User avatar
mcrossley
Posts: 12694
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Post by mcrossley »

Cumulus has its own built-in FTP routines, it doesn't rely on the client that you use from the command line.
Hobbyist
Posts: 83
Joined: Mon 30 Jun 2014 12:02 pm
Weather Station: Instromet Mk 3
Operating System: Raspberry Pi Zero & CumulusMX
Location: Kent

Re: Dayfile.txt uploading to a web server via FTP?

Post by Hobbyist »

Cheers Mark that makes total sense.

Dave
jank
Posts: 239
Joined: Sat 13 Jun 2015 5:57 pm
Weather Station: FineOffset WS3080
Operating System: Debian Buster - Raspbian - Raspb
Location: Germany - nearby Kassel

Re: Dayfile.txt uploading to a web server via FTP?

Post by jank »

Sorry, when I am late, answering to this topic, but you don't need a "real" FTP Program to upload files to an FTP Server.
Already installed on the Raspberry PI is: curl
Syntax to upload a file to an FTP Server is:

curl -m 60 -T /path_to/source_file ftp://"FTPHOST":"FTPPORT" "FTPDIR"/ --user "FTPUSER":"FTPPASS"


-T, --upload-file FILE Transfer FILE to destination
-m, --max-time SECONDS Maximum time allowed for the transfer

have fun
Jan
Do want a Managing Dashboard for CumulusMX on RaspberryPi? cumulusmx.sh
viewtopic.php?f=40&t=17907
Image
bigmac
Posts: 159
Joined: Wed 13 May 2009 9:35 am
Weather Station: Davis Vantage Pro 2 Wireless
Operating System: Raspbian wheezy
Location: Nr Ilkley, West Yorks, UK
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Post by bigmac »

As an extension of this original request, I was also running toolbox to copy the dayfile to my webserver. I have now created a script to upload it using cron from my raspberry pi, but is there a clever way to also update the MySQL server that I have running on my webserver with the contents of the dayfile?
I was using the "remote commands" option in Toolbox to run the ImportCumulusFile.php but there isn't an option to Cron on my webserver, other than to use one of the free utilities to do it.
Just wondering if I can use the "Daily Program" option to do this rather than utilising a third party utility? It is probably blindingly simple to many of you, but I can't fathom it (yet!).
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Post by steve »

MX has MySQL updating built in, including one for dayfile.txt, you just need to configure it with your MySQL details and enable it. Or am I misunderstanding what you want to achieve?
Steve
bigmac
Posts: 159
Joined: Wed 13 May 2009 9:35 am
Weather Station: Davis Vantage Pro 2 Wireless
Operating System: Raspbian wheezy
Location: Nr Ilkley, West Yorks, UK
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Post by bigmac »

It's probably my understanding of how this is meant to work Steve! I was playing around with it the other day (when I caused my issue with the port number hanging CumulusMX), but it didn't seem to do anything to the MySQL server that was running on my website.
To help my understanding, the MySQL that sits in CumulusMX, is that an alternative to running it on my webspace? If so, I don't have anywhere near the knowledge required to then get this integrated with my website, so was just looking for an easy way to continue updating the MySQL server that sits on my webspace.
I have for the moment configured one of the free Cronjob utilities to update my webspace MySQL server (seems to work fine), so don't worry too much about detailed instruction and spending time on helping me understand it, you have enough on your plate! I am quite happy to try and figure things out in my own time :D
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Post by steve »

MX acts as a client to update a table on your server. You have to give it the correct database and table names, and the schema has to match. I used the schema from http://wiki.sandaysoft.com/a/ImportCumulusFile as I thought it was the one most people were using already. There is more information in the announcement for build 3025, when I introduced it. Updates and any error messages are logged to the MXdiags files.

If you are using a different schema, you could do a 'custom' update instead of the default.
Steve
bigmac
Posts: 159
Joined: Wed 13 May 2009 9:35 am
Weather Station: Davis Vantage Pro 2 Wireless
Operating System: Raspbian wheezy
Location: Nr Ilkley, West Yorks, UK
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Post by bigmac »

Thanks Steve, very much appreciated. I am using the same schema, but I will re-read as I couldn't figure out how I could do this at a specified time, i.e. I upload the dayfile at 00:15 and then update the MySQL server a few minutes later at 00:30. These options only seemed to allow it to happen just before rollover, shouldn't it be after rollover, once the dayfile has been updated?
But that will be just my ignorance!
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Post by steve »

If I've said somewhere that it happens just before rollover, that's wrong and needs correcting. It does it when it's added a new entry to dayfile.txt, as part of the rollover.
Steve
Locked