Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

Please read this post before posting

Latest Cumulus release v1.9.4 (build 1099) - Nov 28 2014
Latest Cumulus MX release - v3.0.0 build 3043 Jan 20 2017. See this post for download

Future data formats

Discussion and questions about Cumulus weather station software version 1. This section and its subforums are the main place to get help with Cumulus. Anything which is not specific to the type of weather station goes in here; for anything specific to a type of weather station, please use the appropriate subforum. Use the 'website development' section for any questions relating to creating or running a web site for Cumulus data. Discussion of the stations themselves in these sections is fine.

How should a future version of Cumulus store its data?

In a local SQL database
16
11%
In 'flat' text files like the current version
39
27%
In both simultaneously
90
62%
 
Total votes: 145

User avatar
steve
Cumulus Author
Posts: 26250
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: On tour in France
Contact:

Future data formats

Postby steve » Wed Sep 12, 2012 9:13 am

Purely hypothetical...
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

User avatar
mcrossley
Posts: 4979
Joined: Thu Jan 07, 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Jessie Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Future data formats

Postby mcrossley » Wed Sep 12, 2012 10:01 am

None of the above! Maybe :?

I am happy with flat files as you can import them into SQL fairly easily, but if you offer SQL as an option, then Cumulus should be able to use either a local OR remote database - maybe using a flat file as well, then it can buffer data if the remote db is unavailable ;)

User avatar
steve
Cumulus Author
Posts: 26250
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: On tour in France
Contact:

Re: Future data formats

Postby steve » Wed Sep 12, 2012 10:24 am

I considered a 'none of the above' option, but didn't want it getting out of hand. I was thinking that if it did have local SQL support, then there might be an option to maintain a remote database as well. But if that remote database was MySQL, we get into expensive licensing issues, unless the client was open source.
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

riordankj
Posts: 4
Joined: Sat Jul 21, 2012 3:06 am
Weather Station: Vantage Vue
Operating System: Windows 7-64bit
Location: Rockford, Illinois USA

Re: Future data formats

Postby riordankj » Thu Sep 13, 2012 12:55 am

I was very impressed with the screen shots of the now defunct version 2. I could never get it to run on my system so I couldn't see it real time. I understand that used SQL.

Spider-Vice
Posts: 207
Joined: Sat Sep 24, 2011 2:46 pm
Weather Station: PCE FWS-20 (Fine Offset)
Operating System: Windows 10

Re: Future data formats

Postby Spider-Vice » Thu Sep 13, 2012 8:32 pm

I'm blind when it comes to databases and benefits so I think the plain data is fine, but well, if you could maintain the database remotely, you could always suffer a hardware failure and keep your data safe.

RayProudfoot
Posts: 2404
Joined: Wed May 06, 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: Future data formats

Postby RayProudfoot » Thu Sep 13, 2012 9:36 pm

I voted for both options as eventually I would like to store data in a database for the power it gives the user. But those who don't wish to go down that route should not need to change hence the option to retain the existing system.
Cheers,
Ray, Cheshire.

Image

cats_five
Posts: 55
Joined: Sat Dec 26, 2009 4:24 pm
Weather Station: WH1081PC
Operating System: Win 7 x64 Pro
Location: N 56° 17' 29", W 3° 13' 39"

Re: Future data formats

Postby cats_five » Sun Oct 21, 2012 9:00 am

What about SQLite?

http://www.sqlite.org/
Pegase 101a
Pentax K7
Weather Station: Not currently available
Photography: http://www.fifescapes.co.uk
Cats: http://shunracats.wordpress.com

User avatar
steve
Cumulus Author
Posts: 26250
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: On tour in France
Contact:

Re: Future data formats

Postby steve » Sun Oct 21, 2012 9:30 am

SQLite is what I would use for the SQL database, yes, as I did in Cumulus 2.
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

cats_five
Posts: 55
Joined: Sat Dec 26, 2009 4:24 pm
Weather Station: WH1081PC
Operating System: Win 7 x64 Pro
Location: N 56° 17' 29", W 3° 13' 39"

Re: Future data formats

Postby cats_five » Sun Oct 21, 2012 11:00 am

I think ideally we would have a configuration choice of flat files or a database. Those of us that are sophisticated to want a database should also be able to extract the data from it as we require. However not sure if it would be easier from your PoV to do either flat files, or flat files plus a database so you only need one mechanism for generating the output from the saved data.
Pegase 101a
Pentax K7
Weather Station: Not currently available
Photography: http://www.fifescapes.co.uk
Cats: http://shunracats.wordpress.com

atb
Posts: 10
Joined: Sun Jan 03, 2010 1:58 pm
Weather Station: Fine Offset WH1080
Operating System: XP SP3
Location: Bredfield, Woodbridge, UK
Contact:

Re: Future data formats

Postby atb » Sun Oct 21, 2012 11:33 am

I can see the potential benefit in storing in SQL, however the simple text also provides useful access at times, even if via a spreadsheet.

My main concern in opting for both is whether that would place higher demand on the system, unless there could be an option to disable the SQL. My current setup relies on an old laptop running 24/7 as a dedicated "server", also servicing a USB Webcam (using Yawcam) that uploads via FTP every 10 minutes.

This old laptop usefully takes little power (15 watts on power meter, if I recall correctly), is 597MHZ Celeron with 512MB (max allowed!) running WinXPSP3 Pro (local maintenance via Remote Desktop). TaskManager indicates about 50 - 70% CPU load rising to 100% briefly when Cumulus or Webcam uploads (I try to set them interleaved!) and less than 300MB PF usage.
So, would a SQL or even both version require greater CPU or RAM?

User avatar
steve
Cumulus Author
Posts: 26250
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: On tour in France
Contact:

Re: Future data formats

Postby steve » Sun Oct 21, 2012 12:27 pm

My current thinking is that a future version would continue to use the existing file format and may offer an option to also add data to a local SQL database.
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

OliII
Posts: 54
Joined: Mon Mar 19, 2012 11:47 am
Weather Station: WMR 200 by Oregon Scientific
Operating System: WinXP SP3
Location: Belgium

Re: Future data formats

Postby OliII » Sat Nov 03, 2012 9:37 pm

I voted for both options.
Image
http://www.7331.be/meteo website based on the phpBB3 forum and template. WMR200 + UVN 800 - Cumulus 1.9.4 1062 / Windows XP Service Pack 3 build 2600 FR

uncle_bob
Posts: 459
Joined: Wed Aug 17, 2011 2:58 pm
Weather Station: WeatherDuino Pro2
Operating System: 2008
Location: Canberra

Re: Future data formats

Postby uncle_bob » Sat Nov 03, 2012 10:49 pm

I'm on the fence :)
Whatever option that could be taken would need a simple way of editing bad data (spikes).
It would also be great if Cumulus was OS independent so it could run on different platforms.

User avatar
steve
Cumulus Author
Posts: 26250
Joined: Mon Jun 02, 2008 6:49 pm
Weather Station: None
Operating System: None
Location: On tour in France
Contact:

Re: Future data formats

Postby steve » Sun Nov 04, 2012 11:27 am

uncle_bob wrote:It would also be great if Cumulus was OS independent so it could run on different platforms.
I'm working on it ;)
Steve
-----
Hosting available for Cumulus web sites. See http://sandaysoft.com/forum/viewtopic.php?f=2&t=11876

Please read the posts in the Announcements section about the current status of Cumulus development now that I have retired

dale
Posts: 8
Joined: Tue May 29, 2012 9:15 pm
Weather Station: WMR968
Operating System: Windows 7
Location: Glen Arbor, MI

Re: Future data formats

Postby dale » Fri Nov 16, 2012 2:10 pm

I probably would not use SQL so voted for flat file, but if SQL was there I'd really like the flat file to remain and option.


Return to “Cumulus 1”

Who is online

Users browsing this forum: No registered users and 7 guests