Page 14 of 15

Re: Top 10 Records Page

Posted: Sat 12 May 2018 8:35 pm
by sfws
My guess is that there is still an error, without seeing your source I cannot tell you what. Maybe you have created a file but the script does not exist in it. Has your script got "include ..." or "require ..." of another script that you do not have? Such an include/require would cause the PHP parser to fail, and it will not generate a HTML page, the error for this would appear in the location that your PHP ini file specifies, but would not be repeated on the browser page as the browser has not received any content.

From your description it sounds as if the failure is at the start and I believe it lies with a failure of the PHP parsing.

Maybe the encoding of the page does not match what is declared as the encoding. One advantage of notepad plus plus is that from its menus you can directly choose the language - i.e. tell the editor the page is "PHP", and choose the encoding i.e. select UTF-8-BOM.
Or if the <?php was missing the PHP parser would not find anything to recognise and would not produce a web page.


Top ten is a useful and popular summary page, but Mark's source is specific to his site with various extra scripts in the background; and the queries on it are quite complex, so there is lots of potential for problems further down the line. You might do better to start with a simpler retrieval script with a single query such as my "daily pick" (but that is not really intended to be for novices) or one of the others referenced from the wiki "Daily Summary" article, and once you have proved you can retrieve successfully, implement top ten.

If your top ten php file was loading properly, I would be able to read the source using http://kesis.info/Saa/Top10.php?view=sce, but as that is also blank, I'm pretty sure that either your process for uploading the script has failed or there is something fundamentally wrong with your script.
I'm off to bed now, but if you zip up your PHP file and attach it, somebody will take a look (but take out any passwords or other database access codes in a copy of the file and zip that copy).

Re: Top 10 Records Page

Posted: Sun 13 May 2018 5:02 am
by Mrc
Ok so here's the files.

Re: Top 10 Records Page

Posted: Sun 13 May 2018 8:00 am
by sfws
sfws wrote: Maybe you have created a file but the script does not exist in it. Maybe the encoding of the page does not match what is declared as the encoding.
The file you zipped uses UTF-8 encoding, but the file on your web server uses windows-1252 encoding. So there is a problem with the file on the web server. Firefox has an option (on a pc use shift, alt and 'I') called "View Page Info" that tells me this.
sfws wrote:If your top ten php file was loading properly, I would be able to read the source using http://kesis.info/Saa/Top10.php?view=sce, but as that is also blank, I'm pretty sure that ... your process for uploading the script has failed
The copy you zipped "TOP10copy.zip" looks as if it contains references to scripts you do not have, please make sure the script (without any reference to scripts you do not have) is loaded properly on your web server, with right encoding, by totally deleting the file off the server, then FTP the source file to the server again.

Re: Top 10 Records Page

Posted: Sun 13 May 2018 3:32 pm
by sfws
mcrossley wrote:To stop hi-jacking the other thread, I thought I 'd start a new one for the Top 10 page.

You can see my version here.

The source code is here.

This was inspired by Dans (captzero - brisbaneliveweather) pioneering work, though I developed my code independently.
Mark,
I've obviously looked at your code for my post this morning.
It may only be a very trivial point, but I note you use

Code: Select all

$mtime = microtime(false);
$mtime = explode(" ",$mtime);
$mtime = $mtime[1] + $mtime[0];
$starttime = $mtime;
when the same could be achieved in a single instruction:

Code: Select all

$starttime = microtime(true);
I have checked and this has been available since php4, so it is not an option that has only become possible recently.

I'm also trying to work out your changes in your version 1.5
// Ver 1.5 02/10/17 - Fix sql_mode=only_full_group_by errors
because I regressed back to an earlier MySQL version because I experienced those errors in several of my scripts and, just from reading the MySQL manual, did not find a way to eliminate them whilst keeping the functionality of my scripts. So the answer would be useful to me, and I was unable to pick it up by doing a version compare, as you only have the latest version on your site, and that is where your original post above links. I did find a
// Ver 1.1 23/11/15 - Removed least number of frost days and replaced with most consecutive frost days
// Ver 1.2 31/10/16 - Checks for no data in the frost records results
// Added Hemisphere variable, so frost records work south of the equator
// Ver 1.3 06/11/16 - Bit of tidying up of dataArrays - removed unused class name
on another site, but you must have a version control problem as that does not match your
// Ver 1.1 23/11/15 - Removed least number of frost days and replaced with most consecutive frost days
// Ver 1.2 31/10/16 - Checks for no data in the forst records results
// Ver 1.3 18/05/17 - Minor table header changes
// Ver 1.4 02/10/17 - Disable "sql_mode=only_full_group_by" if it is set (MySQL > 5.7.5)
// Ver 1.5 02/10/17 - Fix sql_mode=only_full_group_by errors
so I don't know if my file comparison between those versions is valid for the changes to your queries.

Re: Top 10 Records Page

Posted: Mon 14 May 2018 9:06 am
by mcrossley
sfws wrote: Mark,
I've obviously looked at your code for my post this morning.
It may only be a very trivial point, but I note you use

Code: Select all

$mtime = microtime(false);
$mtime = explode(" ",$mtime);
$mtime = $mtime[1] + $mtime[0];
$starttime = $mtime;
when the same could be achieved in a single instruction:

Code: Select all

$starttime = microtime(true);
I have checked and this has been available since php4, so it is not an option that has only become possible recently.
Ta, I'll use that!
sfws wrote: I'm also trying to work out your changes in your version 1.5
// Ver 1.5 02/10/17 - Fix sql_mode=only_full_group_by errors
because I regressed back to an earlier MySQL version because I experienced those errors in several of my scripts and, just from reading the MySQL manual, did not find a way to eliminate them whilst keeping the functionality of my scripts. So the answer would be useful to me, and I was unable to pick it up by doing a version compare, as you only have the latest version on your site, and that is where your original post above links. I did find a
// Ver 1.1 23/11/15 - Removed least number of frost days and replaced with most consecutive frost days
// Ver 1.2 31/10/16 - Checks for no data in the frost records results
// Added Hemisphere variable, so frost records work south of the equator
// Ver 1.3 06/11/16 - Bit of tidying up of dataArrays - removed unused class name
on another site, but you must have a version control problem as that does not match your
// Ver 1.1 23/11/15 - Removed least number of frost days and replaced with most consecutive frost days
// Ver 1.2 31/10/16 - Checks for no data in the forst records results
// Ver 1.3 18/05/17 - Minor table header changes
// Ver 1.4 02/10/17 - Disable "sql_mode=only_full_group_by" if it is set (MySQL > 5.7.5)
// Ver 1.5 02/10/17 - Fix sql_mode=only_full_group_by errors
so I don't know if my file comparison between those versions is valid for the changes to your queries.
Looks like those changes have been made by the owner of that site.

Unfortunately I do not have this script under a source control manager, so I'll have to go and look for old backups to do a compare for you...

Re: Top 10 Records Page

Posted: Mon 14 May 2018 9:17 am
by mcrossley
OK, found a copy of version 1.2!

The group by changes consist of where there is a function applied to the date - such as...

Code: Select all

SELECT MAX(LogDate) AS date_to,
...
then previously I had the order by on the date column name...

Code: Select all

ORDER BY cnt DESC, LogDate ASC
I changed it to use the value returned in the query...

Code: Select all

ORDER BY cnt DESC, date_to ASC

Re: Top 10 Records Page

Posted: Mon 14 May 2018 10:20 am
by sfws
mcrossley wrote:OK, found a copy of version 1.2!

The group by changes consist of where there is a function applied to the date
Aha, that is the insight I needed - Group By needs to be applied to the query output, not the input column!
I know there is some reference to the GROUP BY problem earlier in the thread, but it does not explain that subtle detail.
Thank you, Mark.
mcrossley wrote:Looks like those changes have been made by the owner of that site.
I could ask him, but a) he does not have any knowledge of PHP, and b) he is in Northern hemisphere, so I am sure he will deny all. I may be wrong, but your posting of 30 Oct 2016 in this thread does suggest that you were considering the frost query logic, so I still suspect it was a change you made, and subsequently lost.

Re: Top 10 Records Page

Posted: Mon 14 May 2018 11:13 am
by mcrossley
sfws wrote: Aha, that is the insight I needed - Group By needs to be applied to the query output, not the input column!
I know there is some reference to the GROUP BY problem earlier in the thread, but it does not explain that subtle detail.
Thank you, Mark.
No problem.
sfws wrote:
mcrossley wrote:Looks like those changes have been made by the owner of that site.
I could ask him, but a) he does not have any knowledge of PHP, and b) he is in Northern hemisphere, so I am sure he will deny all. I may be wrong, but your posting of 30 Oct 2016 in this thread does suggest that you were considering the frost query logic, so I still suspect it was a change you made, and subsequently lost.
Oh dear, that will teach me to not put everything under source control. Hmm, maybe I'll try running GitLab on my NAS...
Meanwhile can you send me a link to that source and I'll merge the changes back in to my current version! :bash:

Re: Top 10 Records Page

Posted: Mon 14 May 2018 9:20 pm
by mcrossley
Mrc wrote:Ok so here's the files.
OK, my source code *is* specific to my web site, it has "includes" for files that I use to keep a standard look and feel to every page, plus an include for the MySQL login details. You'll need to comment all those out and provide your own page header/footer HTML, plus include your own MySQL login details.

The source is probably not runnable as-is on another web site, it requires a basic knowledge of PHP to edit.

Maybe one day I'll provide a generic version that anyone could use "out of the box" so to speak, but until then it is up to anyone wanting to use this to put a bit of work in themselves ;)

Re: Top 10 Records Page

Posted: Tue 15 May 2018 12:17 pm
by sfws
mcrossley wrote:Meanwhile can you send me a link to that source
Just to record that Mark and I resolved that by PM, where we also discussed site-specific and generic issues, as mentioned in his last post.

Edit - unrelated to that, but perhaps relevant to this thread, when David Jamieson was writing articles for the Cumulus Wiki he wrote something about using scripts (JavaScript and PHP). I built the current article http://wiki.sandaysoft.com/a/PHP up from his initial contribution, whilst still trying to keep it simple. If you are just venturing into PHP for your web pages and you are using Cumulus, then take a look as there are some simple code examples that might inspire you.

Re: Top 10 Records Page

Posted: Tue 15 May 2018 7:44 pm
by mcrossley
OK, I have updated the script to version 2.0. Wow! A whole integer update you say! :lol:

Yes, the script has been restructured so that it only outputs the actual data, making it easier for you to incorporate into your own styled pages.

Refer back to the first post on this thread, I have updated the links there, and also provided a very simple generic page that uses the script without any formatting.

Version 2.0 also has the changes I put into 1.6 a day or so ago...
- the "lost" hemisphere option is restored
- optimised microtime() code
- adds a $frostStart variable to specify when you want your first frost 'season' to start. This was previously hardcoded and many of you have just carried it across to your pages and not noticed :groan: :lol:

Thanks to sfws for the nudge to get this script updated :clap:

Re: Top 10 Records Page

Posted: Tue 05 Jun 2018 4:18 pm
by Mrc
Well, finally i got it working also :D Atleast almost, the max temperatures are wrong. There is data from 2015, 2016 and 2017 but there should be data only from this year because last month was so hot. Any idea what's wrong?

Re: Top 10 Records Page

Posted: Tue 05 Jun 2018 10:40 pm
by mcrossley
Mrc wrote:Well, finally i got it working also :D Atleast almost, the max temperatures are wrong. There is data from 2015, 2016 and 2017 but there should be data only from this year because last month was so hot. Any idea what's wrong?
Well, it just displays what is in your database, so you maybe have a data problem. Have a look at your data files for the dates in 2015 etc, and see what the source data shows.

Re: Top 10 Records Page

Posted: Tue 03 Jul 2018 9:50 pm
by stewartwlewis
Having finally decided to move my Cumulus data into SQL I found the top10 thread when trying to sort out some data import issues.
What a resource Mark – fantastic thanks.

Reason for this post is I had a bit of trouble getting it all to work and the answer was actually quite simple in the end so others might find this useful.

When running the script I kept getting the following error:-
“ERROR - Bad Select Statement (4) -'springweather.Dayfile.LogDate' isn't in GROUP BY”
springweather is my database name.
This suggested there was something wrong with the dry/wet day in month returns. Commenting them out and everything else worked and looked great.
Checking the dry and wet day data didn’t show any errors.
Getting the script to print out the SQL query being run and running it in phpmysql produced a perfect top 10 result for wet and dry – no errors what so ever!
So the query was working fine just not when run in the script.
Reading the forum thread completely indicated that a group by method had been causing problems and disabled to make it compatible with version 5.7.5 or later with the code at line . 130 headed “#### For testing SQL/99 compatability ####” (not my spelling honest).

Commented out this section and everything worked perfectly.

So if you are running mysql 5.6.39 (my GoDaddy version) or probably anything before 5.7.5 you need to comment out the section below the above header at line 130.

Mark, if you ever make any updates and my conclusions are correct, it might be worth expanding on the comments at line 130.

Great work, thanks again.

Stewart

Re: Top 10 Records Page

Posted: Wed 04 Jul 2018 8:06 pm
by mcrossley
Hi Stewart,

Sorry, you had some issues. I'd forgotten that I'd posted the source from my test site. I had already made some updates to the script, including commenting out that section!

If you follow the links in the first post of this thread again, you should find the test page now actually displays some data, and the source of the include script has been updated.