Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Future data formats

Discussion and questions about Cumulus weather station software version 1. This section is the main place to get help with Cumulus 1 software developed by Steve Loft that ceased development in November 2014.

How should a future version of Cumulus store its data?

In a local SQL database
17
11%
In 'flat' text files like the current version
41
27%
In both simultaneously
94
62%
 
Total votes: 152

peterh
Posts: 150
Joined: Fri 21 Dec 2012 1:08 pm
Weather Station: Alecto WS-5000 rebadged FO 3081
Operating System: Windows server 2008R2
Location: Nederland

Re: Future data formats

Post by peterh »

While I admire your desire to cater for the desires of your users, I think that, from a code maintenance point of view, you'd be better off ditching the flat-file-format storage thingy.
I mean, srsly. We are storing structured data here... and when you're storing structured data in a flat-file-system, you are essentially emulating a database anyway.

With Entity Framework (available to all of us, except for those who run on Win2k, which is dead anyway), you can support MySql (free), SQL Server, SQL Express (free, I think) and Obstacle, oops, sorry, I meant to say Oracle, officah... with all these engines using the same code. You'd have to write some code to facilitate the generation of storage models and mapping models for each database engine, but I have that code right here on the shelf, at least for SQL Server and Obstacle. I think that MySql would be capable of using the same storage models and mapping files as SQL Server... but if they don't, that's easy to solve.

I cannot see how anyone would have to worry about running MySql. If you only use it for Cumulus, the system load would be extremely small.

The main rationale for dumping the flat file system storage model is that it makes a huge difference in terms of software maintenance. Plus: using a database for storage is not going to make the software less stable, or slower. Less code to maintain means less moving parts, and thus more stability.

Last but not least:
If you give us something that is as good as Cumulus, and you are providing it as donationware, I think that justifies the fact that you, as a software architect, decide how you architect it.
An opinion should be the result of a thought process, not a substitution.
http://www.dnl-core.net/CothenWeather/
Karv
Posts: 40
Joined: Sun 16 Dec 2012 11:19 pm
Weather Station: WH-1080
Operating System: Windows XP / 7, Ubuntu, Centos
Location: South Gloucestershire

Re: Future data formats

Post by Karv »

What peterh said!

MySQL does for flat files what the petrol engine did for the horse and cart.
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Future data formats

Post by steve »

The last time I looked, the MySQL client was very expensive - in fact it's quite difficult/impossible to get an actual price from Oracle without setting up an account. Is there an alternative that isn't from Oracle that's free for closed source applications?

Note that if I do ever produce a 'new' version of Cumulus, I intend it to be cross-platform. And I will have to re-use a lot of my existing code, because I simply can't start from scratch again, so it's almost certain that the data storage will be as in my previous post. And the SQL option will almost certainly be SQLite. With possibly an option to upload to a remote MySQL server, if a free client is available.
Steve
peterh
Posts: 150
Joined: Fri 21 Dec 2012 1:08 pm
Weather Station: Alecto WS-5000 rebadged FO 3081
Operating System: Windows server 2008R2
Location: Nederland

Re: Future data formats

Post by peterh »

I beg thee pardon?

From the MySql website:
"MySQL Community Edition is the freely downloadable version of the world's most popular open source database."

http://www.mysql.com/products/community/
An opinion should be the result of a thought process, not a substitution.
http://www.dnl-core.net/CothenWeather/
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Future data formats

Post by steve »

And the sentence which follows the one you quoted?
Steve
peterh
Posts: 150
Joined: Fri 21 Dec 2012 1:08 pm
Weather Station: Alecto WS-5000 rebadged FO 3081
Operating System: Windows server 2008R2
Location: Nederland

Re: Future data formats

Post by peterh »

Looking at MySql versus SQLite, it looks like SQLite might be a better option if you're going to package a db engine.

The only issue with SQLite is that it doesn't support "database creation" for an Entity Framewodk CodeFirst approach, but that seems like a minor issue.

Personally, I would prefer an approach that allows me to either install SQLite, or use an existing SQL or MySql database engine. With EF, that's a piece of cake. Plus, EF makes coding against a database a lot easier.
An opinion should be the result of a thought process, not a substitution.
http://www.dnl-core.net/CothenWeather/
peterh
Posts: 150
Joined: Fri 21 Dec 2012 1:08 pm
Weather Station: Alecto WS-5000 rebadged FO 3081
Operating System: Windows server 2008R2
Location: Nederland

Re: Future data formats

Post by peterh »

steve wrote:And the sentence which follows the one you quoted?
You mean this one: "It is available under the GPL license and is supported by a huge and active community of open source developers."?

That, imho, is even better news. What do you think is wrong with it"?
An opinion should be the result of a thought process, not a substitution.
http://www.dnl-core.net/CothenWeather/
peterh
Posts: 150
Joined: Fri 21 Dec 2012 1:08 pm
Weather Station: Alecto WS-5000 rebadged FO 3081
Operating System: Windows server 2008R2
Location: Nederland

Re: Future data formats

Post by peterh »

Some info from StackOverflow:
Stackoverflow wrote:SQLite:

easier to setup
great for temporary (testing databases)
great for rapid development
great for embedding in an application
doesn't have user management
doesn't have many performance features
doesn't scale well.

MySQL:
far more difficult/complex to set up
better options for performance tuning
can scale well if tuned properly
can manage users, permissions, etc.
http://stackoverflow.com/questions/3630/sqlite-vs-mysql
An opinion should be the result of a thought process, not a substitution.
http://www.dnl-core.net/CothenWeather/
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Future data formats

Post by steve »

Cumulus isn't GPL, and it's unlikely that a future version would be either - sorry.

SQLite is a great database for this purpose. It's what Cumulus 2 uses/used.
Steve
peterh
Posts: 150
Joined: Fri 21 Dec 2012 1:08 pm
Weather Station: Alecto WS-5000 rebadged FO 3081
Operating System: Windows server 2008R2
Location: Nederland

Re: Future data formats

Post by peterh »

Aargh. I keep confusing GPL and LGPL... :-(

In that case, SQLite is the better option. But we had already learnt that this is the case anyway.
An opinion should be the result of a thought process, not a substitution.
http://www.dnl-core.net/CothenWeather/
BCJKiwi
Posts: 1255
Joined: Mon 09 Jul 2012 8:40 pm
Weather Station: Davis VP2 Cabled
Operating System: Windows 10 Pro
Location: Auckland, New Zealand
Contact:

Re: Future data formats

Post by BCJKiwi »

I'm no expert in this so perhaps I should not be posting, however,
Since the development of Cumulus would not in fact be altering in any way the actual mySQL code, and mySQL would not actually need to be distributed with Cumulus then I don't see the problem.

Steve, you could build the Cumulus system, distribute Cumulus and the end user could download and instal mySQL themselves. Then the GPL license would be between the end user and Cumulus would not have modified nor redistributed mySQL.

I have been thinking about this whole issue and am not entirely sure that Cumulus itself needs to have a SQL file store although I do think it would be better if it did.

The real benefit of a proper database comes when data extraction/reporting/manipulation is done. For the weather applications this is typically via a website.
So if the website ideal database schema is planned for, then the local files could mirror that structure but not necessarily be SQL. A schema in which different elements are in different files provides better long term flexibility and ease of ongoing maintenance.
peterh
Posts: 150
Joined: Fri 21 Dec 2012 1:08 pm
Weather Station: Alecto WS-5000 rebadged FO 3081
Operating System: Windows server 2008R2
Location: Nederland

Re: Future data formats

Post by peterh »

This would be true... but unfortunately for the majority of users, a database would need to be bundled in a default install. That, or a default install would use the flat-file system storage mechanism.
I can see my wife downloading and installing mySql (she manages a 150+ server park), but not my son-in-law.
An opinion should be the result of a thought process, not a substitution.
http://www.dnl-core.net/CothenWeather/
BCJKiwi
Posts: 1255
Joined: Mon 09 Jul 2012 8:40 pm
Weather Station: Davis VP2 Cabled
Operating System: Windows 10 Pro
Location: Auckland, New Zealand
Contact:

Re: Future data formats

Post by BCJKiwi »

peterh wrote:This would be true... but unfortunately for the majority of users, a database would need to be bundled in a default install. That, or a default install would use the flat-file system storage mechanism.
I can see my wife downloading and installing mySql (she manages a 150+ server park), but not my son-in-law.
Does the GPL actually 'forbid' distribution if mySQL itself is unmodified? Surely the schema and table(s) in the Cumulus database would not be considered part of the source covered by the GPL.
peterh
Posts: 150
Joined: Fri 21 Dec 2012 1:08 pm
Weather Station: Alecto WS-5000 rebadged FO 3081
Operating System: Windows server 2008R2
Location: Nederland

Re: Future data formats

Post by peterh »

I not the local license specialist, but if I remember correctly this time, the GPL says that if you distribute MySQL with your software, your software should be GPL as well.
LGPL doesn't have this restriction.
An opinion should be the result of a thought process, not a substitution.
http://www.dnl-core.net/CothenWeather/
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Future data formats

Post by steve »

In order to update a MySQL database, Cumulus would have to be linked with a MySQL client library, e.g. libmysql.dll as provided by Oracle. This DLL is dual-licensed, so any code using it has to be GPL (or FOSS) or have a commercial licence.
Steve
Post Reply