Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

Please read this post before posting

Latest Cumulus release v1.9.4 (build 1099) - Nov 28 2014
Latest Cumulus MX release - v3.0.0 build 3043 Jan 20 2017. See this post for download

Dayfile.txt uploading to a web server via FTP?

Discussion of version 3 of Cumulus, which runs on Windows, Linux, and OS X. All Cumulus MX queries in here, please.
Hobbyist
Posts: 70
Joined: Mon Jun 30, 2014 12:02 pm
Weather Station: Instromet Mk 3
Operating System: Raspberry Pi3 & CumulusMX
Location: Kent

Dayfile.txt uploading to a web server via FTP?

Postby Hobbyist » Mon Oct 12, 2015 3:23 pm

Hi,
I have spent some time trying to achieve this task and am failing so would like some advice.

I am setting up a system using CumulusMX with a Raspberry Pi.

All is working so far with a web server successfully displaying data.

On my existing set up using a PC/Cumulus 1 I have been using Toolbox to upload my DayFile.txt file to the server
every 24 hours to allow online data searching.

I can't seem to find a practical way to do this on the Raspberry Pi set up.

Has anyone been able to do this sort of task and if so how.

Many thanks

Dave

Karv
Posts: 38
Joined: Sun Dec 16, 2012 11:19 pm
Weather Station: WH-1080
Operating System: Windows XP / 7, Ubuntu, Centos
Location: South Gloucestershire
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby Karv » Mon Oct 12, 2015 3:30 pm

It should be pretty easy to achieve using either python or a shell script and Cron to schedule your job to do the upload. There shoudl be lots of examples if you google "FTP shell script" for example.
Image

Hobbyist
Posts: 70
Joined: Mon Jun 30, 2014 12:02 pm
Weather Station: Instromet Mk 3
Operating System: Raspberry Pi3 & CumulusMX
Location: Kent

Re: Dayfile.txt uploading to a web server via FTP?

Postby Hobbyist » Mon Oct 12, 2015 3:33 pm

Thanks for that, I did try and write a script but it seemed that my Pi could not run FTP.

Googling found many folk suggesting using everything except FTP, for security reasons, so not much help.

I thought there would be a 'simple' way forward.

Dave

User avatar
steve
Cumulus Author
Posts: 26654
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby steve » Mon Oct 12, 2015 3:43 pm

Googling for a linux ftp script returns a number of pages; this one, for example - http://www.stratigery.com/scripting.ftp.html

You could use a modified version of the example given there, invoked either by cron or using the Cumulus 'daily command'.

If you don't have the ftp client installed on your Pi: sudo apt-get install ftp

ncftp is an nicer alternative to ftp which makes scripting easier. sudo apt-get install ncftp
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development since I have retired from my day job

Karv
Posts: 38
Joined: Sun Dec 16, 2012 11:19 pm
Weather Station: WH-1080
Operating System: Windows XP / 7, Ubuntu, Centos
Location: South Gloucestershire
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby Karv » Tue Oct 13, 2015 8:56 am

Your Pi should be able to run FTP, what error do you see if for example you type in "ftp" without the quotes and hit enter?
Image

Hobbyist
Posts: 70
Joined: Mon Jun 30, 2014 12:02 pm
Weather Station: Instromet Mk 3
Operating System: Raspberry Pi3 & CumulusMX
Location: Kent

Re: Dayfile.txt uploading to a web server via FTP?

Postby Hobbyist » Tue Oct 13, 2015 3:31 pm

Many thanks Steve once again and Karv for your help.

I had, in my searches, come across that site and had tried to use that script but had failed.

They often say it can be the simpler things that trip you up and in this case, to a Linux and Raspberry Pi rookie, it was.

I has assumed that as my test weather station and CumulusMX were talking to each other and my web pages that FTP

was running on my PI. It Wasn't!

Having actually loaded it with your suggested link Steve and having spend a morning writing a script and getting paths

correct. It all is now working as it should.

I am still puzzled how CumulusMX was able to use an FTP link to my server but I couldn't without installing it but that

is why I am an amateur and you guys are the experts.

Thanks once again and I am learning from my mistakes.

Dave
Last edited by Hobbyist on Tue Oct 13, 2015 4:07 pm, edited 1 time in total.

User avatar
mcrossley
Posts: 5192
Joined: Thu Jan 07, 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Stretch Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby mcrossley » Tue Oct 13, 2015 3:33 pm

Cumulus has its own built-in FTP routines, it doesn't rely on the client that you use from the command line.

Hobbyist
Posts: 70
Joined: Mon Jun 30, 2014 12:02 pm
Weather Station: Instromet Mk 3
Operating System: Raspberry Pi3 & CumulusMX
Location: Kent

Re: Dayfile.txt uploading to a web server via FTP?

Postby Hobbyist » Tue Oct 13, 2015 3:48 pm

Cheers Mark that makes total sense.

Dave

jank
Posts: 302
Joined: Sat Jun 13, 2015 5:57 pm
Weather Station: weatherstation
Operating System: Debian Wheezy
Location: Germany

Re: Dayfile.txt uploading to a web server via FTP?

Postby jank » Thu Oct 15, 2015 7:02 pm

Sorry, when I am late, answering to this topic, but you don't need a "real" FTP Program to upload files to an FTP Server.
Already installed on the Raspberry PI is: curl
Syntax to upload a file to an FTP Server is:

curl -m 60 -T /path_to/source_file ftp://"FTPHOST":"FTPPORT" "FTPDIR"/ --user "FTPUSER":"FTPPASS"


-T, --upload-file FILE Transfer FILE to destination
-m, --max-time SECONDS Maximum time allowed for the transfer

have fun
Jan
Want to see your Station values on SSH? Cumulusnow http://sandaysoft.com/forum/viewtopic.php?f=27&t=14062
Want a Start|Stop|Restart Script for CumulusMX on RaspberryPi? cumulusmx.sh http://sandaysoft.com/forum/viewtopic.php?f=27&t=13767

bigmac
Posts: 152
Joined: Wed May 13, 2009 9:35 am
Weather Station: Davis Vantage Pro 2 Wireless
Operating System: Raspbian wheezy
Location: Nr Ilkley, West Yorks, UK
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby bigmac » Sun Jan 03, 2016 3:56 pm

As an extension of this original request, I was also running toolbox to copy the dayfile to my webserver. I have now created a script to upload it using cron from my raspberry pi, but is there a clever way to also update the MySQL server that I have running on my webserver with the contents of the dayfile?
I was using the "remote commands" option in Toolbox to run the ImportCumulusFile.php but there isn't an option to Cron on my webserver, other than to use one of the free utilities to do it.
Just wondering if I can use the "Daily Program" option to do this rather than utilising a third party utility? It is probably blindingly simple to many of you, but I can't fathom it (yet!).

User avatar
steve
Cumulus Author
Posts: 26654
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby steve » Sun Jan 03, 2016 4:06 pm

MX has MySQL updating built in, including one for dayfile.txt, you just need to configure it with your MySQL details and enable it. Or am I misunderstanding what you want to achieve?
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development since I have retired from my day job

bigmac
Posts: 152
Joined: Wed May 13, 2009 9:35 am
Weather Station: Davis Vantage Pro 2 Wireless
Operating System: Raspbian wheezy
Location: Nr Ilkley, West Yorks, UK
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby bigmac » Sun Jan 03, 2016 4:39 pm

It's probably my understanding of how this is meant to work Steve! I was playing around with it the other day (when I caused my issue with the port number hanging CumulusMX), but it didn't seem to do anything to the MySQL server that was running on my website.
To help my understanding, the MySQL that sits in CumulusMX, is that an alternative to running it on my webspace? If so, I don't have anywhere near the knowledge required to then get this integrated with my website, so was just looking for an easy way to continue updating the MySQL server that sits on my webspace.
I have for the moment configured one of the free Cronjob utilities to update my webspace MySQL server (seems to work fine), so don't worry too much about detailed instruction and spending time on helping me understand it, you have enough on your plate! I am quite happy to try and figure things out in my own time :D

User avatar
steve
Cumulus Author
Posts: 26654
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby steve » Sun Jan 03, 2016 4:44 pm

MX acts as a client to update a table on your server. You have to give it the correct database and table names, and the schema has to match. I used the schema from http://wiki.sandaysoft.com/a/ImportCumulusFile as I thought it was the one most people were using already. There is more information in the announcement for build 3025, when I introduced it. Updates and any error messages are logged to the MXdiags files.

If you are using a different schema, you could do a 'custom' update instead of the default.
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development since I have retired from my day job

bigmac
Posts: 152
Joined: Wed May 13, 2009 9:35 am
Weather Station: Davis Vantage Pro 2 Wireless
Operating System: Raspbian wheezy
Location: Nr Ilkley, West Yorks, UK
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby bigmac » Sun Jan 03, 2016 5:28 pm

Thanks Steve, very much appreciated. I am using the same schema, but I will re-read as I couldn't figure out how I could do this at a specified time, i.e. I upload the dayfile at 00:15 and then update the MySQL server a few minutes later at 00:30. These options only seemed to allow it to happen just before rollover, shouldn't it be after rollover, once the dayfile has been updated?
But that will be just my ignorance!

User avatar
steve
Cumulus Author
Posts: 26654
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Dayfile.txt uploading to a web server via FTP?

Postby steve » Sun Jan 03, 2016 5:34 pm

If I've said somewhere that it happens just before rollover, that's wrong and needs correcting. It does it when it's added a new entry to dayfile.txt, as part of the rollover.
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development since I have retired from my day job


Return to “Cumulus MX”

Who is online

Users browsing this forum: No registered users and 2 guests