Sandaysoft

Support forum for Cumulus weather station software
It is currently Wed May 22, 2013 2:19 am
Please click here before posting. Help me to help you!
Useful Links: Cumulus FAQ • Enhancement requests • Wiki (documentation)
Please put your approximate location into your profile
Add your web site to the Cumulus user map
Vantage Pro2 users with firmware 3.00 should upgrade to fw 3.12 and Cumulus 1.9.4

All times are UTC




Post new topic Reply to topic  [ 24 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Wed Feb 01, 2012 9:02 am 
Offline
User avatar

Joined: Thu Jan 07, 2010 9:44 pm
Posts: 2512
Location: Wilmslow, Cheshire, UK
Weather Station: Davis VP2
Operating System: XP SP3, Win 7
RayProudfoot wrote:
Hi Mark,

I've copied gauges-ss.htm from the server to my hard drive and replaced the location, lat, lon and altitude tags with actual values
Why would you do that Ray? (probably because my comment above implied it :roll: ) You already have a 'processed' version of the page on your web server, all you need to do is remove the tick in Cumulus for 'FTP'. Then if the template gets updated again, all you need to do is re-enable FTP, once the updated page has uploaded to your server stop it FTP'ing again :D
RayProudfoot wrote:
but after I save it and upload it to the server the gauges remain at zero and no updating takes place.

For now I've reinstated the processing of gauges-ssT.htm but if you could offer any pointers as to why it didn't work I'd appreciate it.
No idea without taking a look at it, my gauge page is static and manually maintained (as it is a little different from the sample page I supply in the download). But as I outlined above, you don't actually need to edit the file at all.

_________________
Mark
Wilmslow Astro Weather


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Wed Feb 01, 2012 5:12 pm 
Offline

Joined: Wed May 06, 2009 6:29 pm
Posts: 1708
Location: Cheadle Hulme, Cheshire, England
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
mcrossley wrote:
Why would you do that Ray? (probably because my comment above implied it :roll: ) You already have a 'processed' version of the page on your web server, all you need to do is remove the tick in Cumulus for 'FTP'. Then if the template gets updated again, all you need to do is re-enable FTP, once the updated page has uploaded to your server stop it FTP'ing again :D

The solution was so obvious I completely missed it! I've now unchecked FTP for that file and I shoudl know if it's okay in the next few minutes. Your second response is now redundant after doing the above.

To Tau Bootis (Mark), thanks for your suggestion but the simpler one is to do as Mark in Wilmslow suggests. Thanks all the same.

I've asked my web host a few times if there are any tools available to me so I can see how my monthly allowance is doing but haven't had an answer. Are any available or do you actually have to be the host before you can analyse the throughput? Having made these changes I'd like to know if they've made any difference without bothering my web host.

_________________
Cheers,
Ray, Cheshire.

Image


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 12:59 am 
Offline
User avatar

Joined: Sun Jun 13, 2010 11:44 pm
Posts: 101
Location: Audley Staffordshire UK
Weather Station: WH1081
Operating System: Windows XP Home SP3
Quote:
I've asked my web host a few times if there are any tools available to me so I can see how my monthly allowance is doing but haven't had an answer. Are any available or do you actually have to be the host before you can analyse the throughput? Having made these changes I'd like to know if they've made any difference without bothering my web host.


Try NetWorx Its free and should be just what you need.

http://www.v3.co.uk/articles/download_t ... 384#thanks

_________________
Mark

Image


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 6:36 am 
Offline

Joined: Tue Aug 03, 2010 6:23 am
Posts: 484
Location: Ferntree Gully, VIC, Oz
Weather Station: WM918
Operating System: Win Server 2008 R2
Ray

You could also cut down on uploading most of the files except the Cumulus web tags file by re-writing them to include the cumuluswebtags.php file and then only displaying that data when the page is loaded.

This way your thismonthyear.htm becomes thismonthyear.php, noaa.php etc etc all exist on your web site but do NOT get transferred ever again (well, until you change the layout).

If you do this it can save you around 7.67 Mb a day or roughly 230 Mb a month just on those files (not sure how the figures were arrived at in mrcrossley's post - 154 Mb a week isn't 2.3 Gb per month.

In fact if you trim cumuluswebtags down (ask your yourself 'Do you gfeell lucky..'....errr no wrong thread - 'Do I need all the alarm tags" you can even cut that down quite a bit - remove comments and you can trim it down to about a 1/3 of the original size.

_________________
Punctuation is the difference between 'Let's eat, grandma' and 'Let's eat grandma'


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 6:41 am 
Offline

Joined: Tue Aug 03, 2010 6:23 am
Posts: 484
Location: Ferntree Gully, VIC, Oz
Weather Station: WM918
Operating System: Win Server 2008 R2
Another thing to look at (although you haven't mentioned this as being your problem but it can, and does hit some people very hard) is whether you let robots onto your site.

Googlebot and (think its) yahoo crawler (might be MSN, can't recall off hand) can chew through bandwidth liek youj wouldn't believe.

You can usually see through the stats available via your cPanel (or whatever your host uses) how much they are using - if anything over a couple of Mb a week you really should look at restricting them - on one data intensive site we ran googlebot alone was accounting for over 400 Mb per month of bandwidth - and it all comes of your alloted allowance!

Now, because we restrict ALL bots, we are lucky to see any bot or crawler go ver 1 or 2 Mb per month....If they don;t obey the robots.txt file - we simply ban the IP completely.

_________________
Punctuation is the difference between 'Let's eat, grandma' and 'Let's eat grandma'


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 11:46 am 
Offline

Joined: Wed May 06, 2009 6:29 pm
Posts: 1708
Location: Cheadle Hulme, Cheshire, England
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
[quote="Tau Bootis]Try NetWorx Its free and should be just what you need.

http://www.v3.co.uk/articles/download_t ... 384#thanks[/quote]

Thanks Mark. It appears to monitor all your router traffic. Can it be configured to measure only the traffic to and from my webhosting servers? I don't need to know how much I've used surfing the web.

_________________
Cheers,
Ray, Cheshire.

Image


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 11:54 am 
Offline

Joined: Wed May 06, 2009 6:29 pm
Posts: 1708
Location: Cheadle Hulme, Cheshire, England
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
serowe wrote:
Ray

You could also cut down on uploading most of the files except the Cumulus web tags file by re-writing them to include the cumuluswebtags.php file and then only displaying that data when the page is loaded.

It's an excellent suggestion and I'd love to do it. But being clueless about php would probably result in me completely messing up my pages. I do intend learning PHP when I retire but for now the pages need to stay as they are.

Quote:
If you do this it can save you around 7.67 Mb a day or roughly 230 Mb a month just on those files (not sure how the figures were arrived at in mrcrossley's post - 154 Mb a week isn't 2.3 Gb per month.

My monthly allowance of 3Gb is quite modest and my hoster will soon be ending the service he provides. So I need to look elsewhere and he has suggested one alternative that has a much higher monthly allowance in which case the problem goes away. In fact it probably has already as reducing my update time to 15mins will result in a 1.6Gb usage per month.

Quote:
In fact if you trim cumuluswebtags down (ask your yourself 'Do you gfeell lucky..'....errr no wrong thread - 'Do I need all the alarm tags" you can even cut that down quite a bit - remove comments and you can trim it down to about a 1/3 of the original size.

I'd have to be very lucky not to break something. I just kept adding extras to my Cumulus setup probably not realising the effect on bandwidth. I'm definitely a lean 'n mean person but I need the skills first.

_________________
Cheers,
Ray, Cheshire.

Image


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 11:59 am 
Offline

Joined: Wed May 06, 2009 6:29 pm
Posts: 1708
Location: Cheadle Hulme, Cheshire, England
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
serowe wrote:
Another thing to look at (although you haven't mentioned this as being your problem but it can, and does hit some people very hard) is whether you let robots onto your site.

Googlebot and (think its) yahoo crawler (might be MSN, can't recall off hand) can chew through bandwidth liek youj wouldn't believe.

You can usually see through the stats available via your cPanel (or whatever your host uses) how much they are using - if anything over a couple of Mb a week you really should look at restricting them - on one data intensive site we ran googlebot alone was accounting for over 400 Mb per month of bandwidth - and it all comes of your alloted allowance!

Now, because we restrict ALL bots, we are lucky to see any bot or crawler go ver 1 or 2 Mb per month....If they don;t obey the robots.txt file - we simply ban the IP completely.


I don't have any control panel as such. I use FileZilla for uploading files to the server. The stats provided to me showed requests amounted to 0.5Gb per month and no mention was made of excessive robot activity. Presumably if you have a contract with a commercial organisation you get a cPanel tool that allows you to interrogate the server which I cannot currently do.

_________________
Cheers,
Ray, Cheshire.

Image


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 12:06 pm 
Offline

Joined: Tue Aug 03, 2010 6:23 am
Posts: 484
Location: Ferntree Gully, VIC, Oz
Weather Station: WM918
Operating System: Win Server 2008 R2
Converting the files to PHP is not as hard as it appears. The beauty of it would be that once done, you don't have to touch them again. So don;t go putting yourself down thinking you can't do it. If you know HTML enough to get pages up and running, PHP will be quite easy.

As for trimming down cumuluswebtags - two simple rules really:

Anything after a // can be deleted (including the //) as these are purely comments and not needed for processing (you would be surprised how bloated that file is (no people, not knocking it - in line documentation is ALWAYS needed but should be removed for a production file like this)

and

The tags are placed in sections - one of the latter sections (from memory it has a heading) is for alarms. If you aren't showing the alarms on your web site they can be removed without causing a problem.

It is always handy to have a local web server running - XAMPP or WAMP - then you can try all these changes on your local machine before placing them in the Cumulus data directories ready for FTP'ing to your site.

_________________
Punctuation is the difference between 'Let's eat, grandma' and 'Let's eat grandma'


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 12:51 pm 
Offline
User avatar

Joined: Wed Oct 21, 2009 11:19 am
Posts: 280
Location: Bayswater, Australia
Weather Station: La Crosse WS-2355
Operating System: Windoze 7
Ray,
A excellent piece of free network monitoring software is Wireshark, available here: http://www.wireshark.org/
Filters can be set up to create reports from all network activity down to just those packets to/from a single ip.
Takes a bit of work to get set up the way you want, but the results are worth it.
It'll work on single or multiple ethernet ports, the ideal setup being a dedicated port to a managed switch, where you could monitor any port on the switch via a second ethernet port on your pc, but single port operation is all that's needed to identify bandwidth usage issues.


Cheers,

_________________
Graeme.
http://weather.gktnet.com/cumulus/index.htm
Image


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 5:07 pm 
Offline

Joined: Wed May 06, 2009 6:29 pm
Posts: 1708
Location: Cheadle Hulme, Cheshire, England
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
serowe wrote:
Converting the files to PHP is not as hard as it appears. The beauty of it would be that once done, you don't have to touch them again. So don;t go putting yourself down thinking you can't do it. If you know HTML enough to get pages up and running, PHP will be quite easy.

I'm sure I could eventually learn what to do. My main problem is lack of time.

Quote:
As for trimming down cumuluswebtags - two simple rules really:

Anything after a // can be deleted (including the //) as these are purely comments and not needed for processing (you would be surprised how bloated that file is (no people, not knocking it - in line documentation is ALWAYS needed but should be removed for a production file like this)

and

The tags are placed in sections - one of the latter sections (from memory it has a heading) is for alarms. If you aren't showing the alarms on your web site they can be removed without causing a problem.

I do use record lights on the all-time records page if we're talking about the same thing. The only other alarm I have enabled is the wind exceeding 30mph which plays a wav on my netbook. Nothing uploaded for that one.

Quote:
It is always handy to have a local web server running - XAMPP or WAMP - then you can try all these changes on your local machine before placing them in the Cumulus data directories ready for FTP'ing to your site.

That would be very helpful. I shall investigate, thanks.

_________________
Cheers,
Ray, Cheshire.

Image


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Thu Feb 02, 2012 5:18 pm 
Offline

Joined: Wed May 06, 2009 6:29 pm
Posts: 1708
Location: Cheadle Hulme, Cheshire, England
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Hi Graeme,

Thanks for the link. I watched one of the videos but it was pretty technical for a non-network guru. One question. Where would you run this software from? Logically I would expect it to be installed on the root of my webhosting area so it can monitor all inbound and outbound traffic.

Is that the case or am I just showing my lack of understanding on how you tell it only to read data to and from my account either via login parameters I supply or a http address?

Sorry to sound like a thicko but this is way different to what I expected. :oops:

_________________
Cheers,
Ray, Cheshire.

Image


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Fri Feb 03, 2012 10:00 pm 
Offline
User avatar

Joined: Sun Jun 13, 2010 11:44 pm
Posts: 101
Location: Audley Staffordshire UK
Weather Station: WH1081
Operating System: Windows XP Home SP3
Quote:
Can it be configured to measure only the traffic to and from my webhosting servers?


Hi Ray
I have only just started to use this one my self but it does not appear to do a breakdown as you require, it does lots or other stuff though :)
If you are looking for another provider soon, I am on Plusnet and they give an excellent service and are very competitive on price.

Regards

_________________
Mark

Image


Top
 Profile  
 
 Post subject: Re: Excessive monthly upload figure but what's causing it?
PostPosted: Sat Feb 04, 2012 9:42 am 
Offline

Joined: Wed May 06, 2009 6:29 pm
Posts: 1708
Location: Cheadle Hulme, Cheshire, England
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Hi Mark,

I'm still not getting notified of new posts so apologies if I'm late to any.

I'm trialling networkx which does give me an indication of the amount uploaded from my netbook so that's a starter. The fact that my web hoster was able to separate uploads from http downloads suggests you have to run the software on the server itself.

I'll put PlusNet on the shortlist, thanks.

_________________
Cheers,
Ray, Cheshire.

Image


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 24 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: Bing [Bot], gemini06720, Google [Bot], rci136 and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group