If you use Cumulus, please donate Image

Please read this post before posting

Latest Cumulus release v1.9.4 (build 1099) - Nov 28 2014
Latest Cumulus MX release - v3.0.0 build 3041 Aug 17 2016

Annual Data Summary - problem on year rollover

Discussion and support for 3rd-party (non-Sandaysoft) tools for Cumulus
User avatar
mcrossley
Posts: 4615
Joined: Thu Jan 07, 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Jessie Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Annual Data Summary - problem on year rollover

Postby mcrossley » Sun Jan 01, 2012 11:14 am

I notice that on the first day of the year, that the script looks for data for the current year - which does not exist yet. So we need to subtract one day from the date before getting the year.

Code: Select all

if (isset($_GET['year'])) {
   $tableYear = $_GET['year'];
} else {
   $tableYear = date("Y");
}

becomes

Code: Select all

if (isset($_GET['year'])) {
   $tableYear = $_GET['year'];
} else {
    $tableYear = date('Y', strtotime('-1 day'));
}


EDIT: Caveat, this only works for midnight rollover I guess - those using a 9am rollover will have a slightly more complex problem.

User avatar
beteljuice
Posts: 2429
Joined: Tue Dec 09, 2008 1:37 pm
Weather Station: None !
Operating System: XP SP3 - honest
Location: Dudley, West Midlands, UK

Re: Annual Data Summary - problem on year rollover

Postby beteljuice » Sun Jan 01, 2012 1:42 pm

In other words .... wait until tomorrow ;) :clap:
Image
......................Imagine, what you will KNOW tomorrow !

gemini06720
Posts: 1700
Joined: Mon Aug 10, 2009 10:16 pm
Weather Station: No weather station
Operating System: No operating system
Location: World...

Re: Annual Data Summary - problem on year rollover

Postby gemini06720 » Sun Jan 01, 2012 1:48 pm

Mark, how about adding a simple check for the first day of the first month of the year - if it is 01-01 then subtract one form the year, otherwise ignore:

Code: Select all

if (isset($_GET['year'])) { $tableYear = $_GET['year']; }
else
{
  if (date("n") == 1 and date("j") == 1)  { $tableYear = date("Y") - 1; }
  else { $tableYear = date("Y"); }
 }
I had to modify my script and that solve the lack of data for the first day of the year - the data for the first day of the year should (hopefully) be available at then end of the first day... ;)

User avatar
mcrossley
Posts: 4615
Joined: Thu Jan 07, 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Jessie Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Annual Data Summary - problem on year rollover

Postby mcrossley » Sun Jan 01, 2012 1:51 pm

Hmm, your solution still uses two date operations and is more complex to read :bash: :lol:

gemini06720
Posts: 1700
Joined: Mon Aug 10, 2009 10:16 pm
Weather Station: No weather station
Operating System: No operating system
Location: World...

Re: Annual Data Summary - problem on year rollover

Postby gemini06720 » Sun Jan 01, 2012 2:01 pm

Well, the 'hack' works, does it not... :mrgreen:

Have a great year Mark... :D ...and have another beer... :twisted:

RayProudfoot
Posts: 2274
Joined: Wed May 06, 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: Annual Data Summary - problem on year rollover

Postby RayProudfoot » Sun Jan 01, 2012 2:01 pm

Thanks Mark, file edited. :)
Cheers,
Ray, Cheshire.

Image

sfws
Posts: 289
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Mixed manufacturers
Operating System: off-line pc
Location: Not contributing now

Re: Annual Data Summary - problem on year rollover

Postby sfws » Sun Nov 04, 2012 5:08 pm

For a 9am winter rollover in the Northern Hemisphere, and a 9 or 10am summer-time rollover in the Southern Hemisphere, the dayfile.txt file at the last update before that time on 2nd January (calendar date) will still be the 31 December one (Meteorological date).
Here are 3 alternatives for finding correct default year around the time of year rollover (that also work at other times of year):

(a) The simplest solution, as the (PHP or jQuery) scripts produce a list of years from reading the dayfile, is don't read current date property of PHP or JavaScript at all, just default to latest year found as you build up the year selection list. A simple solution but it works at any time, including if Cumulus has not been running recently, so the dayfile.txt is not up to date.

(b) The <#metdate> web tag made available in (beta) version 1.9.3 build 1044
can be used instead of current date and will work for midnight rollover and in the 9 or 10 am rollover case. This returns the date that has just started processing (and will also work if Cumulus has not been running, or you have a problem with your station preventing Cumulus from updating). You still need to subtract 24 hours to obtain the year associated with the last dayfile.txt record. This involves modifying the HTML to pass the contents of the web tag across and I will post the required HTML and changes to the JavaScript version when I can.

(c) Final alternative - reading file last modified date. In PHP:

Code: Select all

$tableYear = date ('Y', filemtime('dayfile.txt'));

From another topic (Re: How to show image file date and time):
aduncan wrote:= X.getResponseHeader('Last-Modified');
gives a JavaScript equivalent way to read last file modifed date (where X is used for Ajax XMLHttpRequest). This can then replace 'new Date()' in

Code: Select all

if(urlParams['year']){drawYear=urlParams['year'];}else{var d=new Date();var curr_date=d.getDate();drawYear=d.getFullYear();}

NOTE: File modification date will be update of dayfile.txt at time of rollover:
steve wrote:when dayfile.txt is updated:
Cumulus uses a timer which triggers once a minute, on the minute. It checks each time whether the configured rollover hour has been reached (i.e. a new meteorological day is just starting), and if so, does the rollover, part of which involves writing an entry to dayfile.txt. The exact timing will depend on what the system is doing at the time. This assumes that Cumulus is actually running at the time. If it wasn't, and it's using logger data, then it does the rollover when it encounters an entry from the logger which takes it into the next meteorological day.

What this implies is that for a midnight rollover, file creation date will be either midnight (if Cumulus was running) or some time after (if rollover happened in catch up). In the running case, the date will still contain the year of 1 January at rollover, and at least one minute will need to be subtracted to get correct year of last dayfile.txt record. For 9 (or 10) am rollovers, subtract 9 (or 10) hours plus at least one minute to get correct year of last dayfile.txt record.
For a rollover happening in catch up, subtract the full 24 hours, regardless of rollover time, to get correct year of last dayfile.txt record. So in practice this method to cover all cases, requires the full 24 hours to be subtracted.
Last edited by sfws on Wed Nov 21, 2012 6:23 am, edited 4 times in total.

sfws
Posts: 289
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Mixed manufacturers
Operating System: off-line pc
Location: Not contributing now

Re: Annual Data Summary - problem on year rollover

Postby sfws » Mon Nov 05, 2012 10:26 am

I think this discussion would be better in third party tools.

As far as I could tell from searching the fora, this is only bug reported for David A Jamieson's excellent code. I found lots of implementations, but the only changes they made were to either change the parameters that could be selected and/or the colours in which values were shown according to magnitude.
Anyway, using JavaScript, here is my proposed fix, tested as best as I can (sorry I do not have access to PHP for testing that version as it is the one more often used):

HTML

Code: Select all

<script>var metDate=['<#metdate format="dd">','<#metdate format="mm">','<#metdate format="yyyy">',(parseInt("0"+"<#rollovertime>",10))];</script><!-- Pass current Meteorological date and rollover time to script to read dayfile, so knows what is default start up year -->
<script src="readDayfile.js" type="text/javascript"></script><!-- calls ECMA-262 Script to read the 'dayfile.txt' file -->
</body>
</html>

That is designed to work with midnight, 9am and 10am rollovers, I've deleted the script source directory as people may use different ones

JavaScript (as included in versions 1.0, 1.0a and 1.1 of David's code - those are only originals I have traced)

Code: Select all

if(urlParams['year']){drawYear=urlParams['year'];}else{var d=new Date();var curr_date=d.getDate();drawYear=d.getFullYear();}
This was original code that gives Year rollover problem on 1 Jan until rollover for 2 Jan as no new year data available

Revised JavaScript needed to be inserted with other variables at start of script (this does not affect loading time, that is incurred when the dayfile is processed)

Code: Select all

    var d=new Date(metDate[2],(metDate[1]-1),metDate[0], metDate[3]);  // metDate is an array containing the date and time for the last rollover time
   var todayMS = d.getTime();  // Milliseconds since midnight 1 January 1970
   d=new Date(todayMS -(1000*60*60*24)); // Now looking at one day ago, so okay
   var curr_year=d.getFullYear(); // default to calendar year for 1 day ago if parameters undefined on entry

Revise JavaScript to replace David's code quoted earlier (NB no additional IF testing)

Code: Select all

if(urlParams['year']){drawYear=urlParams['year'];}else{drawYear=curr_year;}

I will zip up my screen shots from testing and anything else useful and post later
Last edited by sfws on Thu Nov 15, 2012 7:28 pm, edited 2 times in total.

sfws
Posts: 289
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Mixed manufacturers
Operating System: off-line pc
Location: Not contributing now

Re: Annual Data Summary - problem on year rollover

Postby sfws » Tue Nov 06, 2012 8:01 am

Here are promised screen shots.
So all users of the annual daily summary, no excuse for you not implementing the bug fix before the end of 2012!
They reveal a bonus (as well as working for midnight, 9am or 10am rollover), the modifications proposed in my last posting will default to right year if the web page is being viewed in the new year, but has not been updated since the old year.
The HTML, CSS and Script I revised back in February produces a more informative caption, and adds a blank row after every 5 rows, as well as changes to parameters and value (and background) colouring.
You do not have the required permissions to view the files attached to this post.

sfws
Posts: 289
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Mixed manufacturers
Operating System: off-line pc
Location: Not contributing now

Re: Annual Data Summary - problem on year rollover

Postby sfws » Sat Nov 24, 2012 4:17 pm

sfws wrote:(a) The simplest solution, as the (PHP or jQuery) scripts produce a list of years from reading the dayfile, is don't read current date property of PHP or JavaScript at all, just default to latest year found as you build up the year selection list. A simple solution but it works at any time, including if Cumulus has not been running recently, so the dayfile.txt is not up to date.

I wrote that 20 days ago, but did not instantly know the correct code, but here is the relevant extract from jScript code I shall post in a new topic:

Code: Select all

$.get(dayfile,function(data) // Open dayfile.txt to read file contents and copy them to 'data'
         {  // begining of anonymous function
      var dRow=data.split("\n"); // split that 'data' into array with newline used as delimiter
      var last_record=dRow[dRow.length-2]
      wd_data=last_record.split(field_delimiter); // split row into fields
      cumulus_date=wd_data[0].split(date_delimiter);
      latest_year='20'+String(cumulus_date[2]);
      tableYear=latest_year;

Don't ask me why it is 'length-2'

sfws
Posts: 289
Joined: Fri Jul 27, 2012 11:29 am
Weather Station: Mixed manufacturers
Operating System: off-line pc
Location: Not contributing now

Re: Annual Data Summary - problem on year rollover

Postby sfws » Tue Jan 01, 2013 2:04 pm

Welcome to 2013 everybody. Today (New Year's Day for my timezone) only, you can see how the problem in this topic occurs.

The David Jamieson code displays a blank page on New Years Day:
DaJ.zip
but as the 2nd screen shot shows the updating stopped several weeks ago.

The replacement PHP code referenced from Wiki and from the navigation on Silver Acorn site are slightly different in behavoir:
Silver Acorn.zip
It is the URL parameters that vary (highlighted in relevant screen shot).

The alternative JavaScript version works on both 1 and 2 January as expected (Yes its 2 January already on the other side of the world) coping with local time differences:
sassfras.zip
and in UK is still showing 2012 by default:
Sfws.zip
as it is not yet rollover time on 2 January.
You do not have the required permissions to view the files attached to this post.
Last edited by sfws on Sat Apr 26, 2014 4:21 pm, edited 1 time in total.

BCJKiwi
Posts: 811
Joined: Mon Jul 09, 2012 8:40 pm
Weather Station: Davis VP2 Cabled with Solar
Operating System: Windows 7 SP1
Location: Auckland, New Zealand
Contact:

Re: Annual Data Summary - problem on year rollover

Postby BCJKiwi » Tue Jan 01, 2013 9:59 pm

@ swfs
I'm really not sure what the point of this post is.

The only difference I can see from the images you have posted relating to the Wiki version is that one shows the year selection box and the other does not.

However all data displays (or not) in the same manner.

Am I missing something?


Return to “3rd Party Tools”

Who is online

Users browsing this forum: No registered users and 1 guest