Import data Error

Paul C · **Posted:** Thu Sep 17, 2009 8:24 pm

I successfully imported the data going back to 2007, took about 50seconds. Unfortunatley the data displayed on Cumulus2 is now very wrong indeed. It would appear my data is corrupted ?

Dont despair Steve, its only me !

As you can see some impressive readings. It then hung and I had to end programme.

Update - restarted programme and most of the odd data has gone, but to the right its saying no data series and the programme has hung again. CPU usage at 85-90%, falls back to 30% after programme is terminated.

steve · **Posted:** Thu Sep 17, 2009 8:48 pm

Looks like some sort of communication problem between Cumulus and Virtual VP. It may be that the serial port settings I am using aren't suitable. But I'm using the same settings as Cumulus 1, I think, and in any case it's working for others with Virtual VP. I think Virtual VP has some sort of monitor, where you can see the communication between it and the software?

gemini06720 · **Posted:** Fri Sep 18, 2009 3:40 am

Steve.

The monitor, as you call it, would be VPLive (also by SoftWx).

Although not really a monitoring program as such, VPLive connects to any of the ports of VirtualVP (either COM or TCP/IP), reads all the data picked up by VirtualVP and displays it on its screen - it is not a weather program but more of a "kind' of VirtualVP monitor screen.

Here is a description of VPLive:

"The purpose of this program is to provide additional capabilities for the Davis VantagePro and VantagePro2 weather stations. This program is not intended to be a full featured weather software program, nor be a replacement for such programs. It is intended as a useful add-on to those programs, particularly WeatherLink, and also as a demonstration of the techniques used, so they may be incorporated into other weather software and provide enhanced support for VantagePro users."

And also "...The program may be used free of charge, but my only be distributed by SoftWx...".

I find it quite useful...

Here is the link to the Web page: http://www.softwx.com/weather/vplive.html

steve · **Posted:** Fri Sep 18, 2009 7:28 am

gemini06720 wrote:

The monitor, as you call it, would be VPLive (also by SoftWx).

But I thought Virtual VP had a panel where it displayed the communication between it and the programs using it? A 'protocol' trace? I had a quick look at the VPLive page, but it just seems to display the readings from the station?

Paul C · **Posted:** Fri Sep 18, 2009 8:00 am

Whats the best way to start again ?

Uninstall or delete a folder with the data in.

I think I would prefer to have Cumulus2 running for a while on its on with just new data, maybe the best way to evaluate it.

gemini06720 · **Posted:** Fri Sep 18, 2009 9:13 am

Steve.

You are using VirtualVP and thus you must know about the information displayed on the VirtualVP screen...

Indeed, VirtualVP displays on the main 'Status' screen its communication data between itself and the Vantage Pro station as well as between itself and the different virtual ports. But that is just telling me very little...

On the other hand, VPLive tells me exactly which weather data is being received and transmitted by VirtualVP and that displayed information can be compared to what I see on the screen of any weather software connected to VirtualVP...

steve · **Posted:** Fri Sep 18, 2009 10:08 am

Paul C wrote:

Whats the best way to start again ?

Uninstall or delete a folder with the data in.

To start again, you'll need to uninstall, then delete the database (the file is called cumulus.s3db, and the path is on the first line of the trace.txt file), then reinstall.

steve · **Posted:** Fri Sep 18, 2009 10:13 am

gemini06720 wrote:

You are using VirtualVP and thus you must know about the information displayed on the VirtualVP screen...

No... I don't think I am. Why do you think I am?

Quote:

Indeed, VirtualVP displays on the main 'Status' screen its communication data between itself and the Vantage Pro station as well as between itself and the different virtual ports. But that is just telling me very little...

On the other hand, VPLive tells me exactly which weather data is being received and transmitted by VirtualVP and that displayed information can be compared to what I see on the screen of any weather software connected to VirtualVP...

The problem appeared to be that the communication between Cumulus and Virtual VP was being corrupted in some way. I was hoping that the display in Virtual VP which actually showed the data between Virtual VP and Cumulus would help to resolve this.

gemini06720 · **Posted:** Fri Sep 18, 2009 11:30 am

Steve.

Quote:

No... I don't think I am. Why do you think I am?

Something you wrote in another message (probably an answer) made me think that you were using VirtualVP... Sorry for the confusion... :oops:

The main 'Status' screen of VirtualVP provides the following:
- VirtualVP Uptime - 9 hrs 45 mins 35 secs;
- Last connect - Yesterday 19:27:12;
- Last disconnect - Never;
- Last to VP - 3 secs ago;
- Last from VP - 3 secs ago;
- Commands to VP - 14,609;
- Replies from VP - 13,629;
- Bytes to VP - 132,274;
- Bytes from VP - 1,458,331.

Also on that main 'Status' screen are displayed the name of the virtual ports as well as the time they were 'last heard' and the time they were 'Last Sent To'.

From the information provided on the main 'Status' screen of VirtualVP, it is not possible to know/tell if the data between VirtualVP and Cumulus (or, for that matter, any other programs connected to VirtualVP virtual ports) has been corrupted.

Thus my suggestion to use VPLive ... which will read the data downloaded by VirtualVP (from the Vantage Pro weather station) and display it on its screen - if the user can see the weather information on the VPLive screen, then the data is good.

Other than that, there is no way (to my knowledge, and someone can correct me) to find out if the data between VirtualVP and Cumulus has been corrupted...

steve · **Posted:** Fri Sep 18, 2009 11:56 am

gemini06720 wrote:

Other than that, there is no way (to my knowledge, and someone can correct me) to find out if the data between VirtualVP and Cumulus has been corrupted...

Ah, OK. Maybe Virtual VP has changed - I did look at it many years ago when I was first adding VP support to Cumulus 1, and back then I'm pretty sure it had a screen where it showed the commands received from the program, and the replies it sent back.

steve · **Posted:** Fri Sep 18, 2009 4:09 pm

I bought a copy of Virtual VP, so now I can see what I was talking about - it's the 'activity' tab.

I decided it was silly not have a copy of Virtual VP, if only for validation purposes, as so many people use it. And I'm using some of Steve's data calculation functions in Cumulus 1, anyway, so he deserves it just for that. He also has some .net stuff that looks like it might be useful...

RayProudfoot · **Posted:** Fri Sep 18, 2009 6:21 pm

Hi Steve,

I'm piggy-backing on this message as I too have data import issues. I downloaded the file you provided and launched C2 again. It processed the data files created by Cumulus but after looking at the data via the option provided I have a gap in the data imported which would probably explain the incorrect high values for temp, wind etc.

Here's what the data editor is showing. Nothing for July, August or the first week in September has been imported. Any ideas why?

Attachment:

Cumulus2_Data.JPG

steve · **Posted:** Fri Sep 18, 2009 6:58 pm

The import code doesn't handle errors in the data files very well; I'm trying to address this in the next build. After the first error, it probably gets stuck and can't import any more. Your last entry before the gap - 30/06/09 22:55 - is probably the last entry in the June file (because the database times are UTC) so you may have a problem at the beginning of your July log. I think the September data has come from the VP2 logger rather than the import.

If you can't see any problems in your July file, zip up your data files and I'll try them. I'd also need your cumulus.ini file (with passwords removed), or at least the 'unit' parts, as the import code needs that to know what units your old data is in.

RayProudfoot · **Posted:** Fri Sep 18, 2009 10:44 pm

Hi Steve,

If you remember I had problems back in July with Cumulus not logging any data for a fe weeks but I sent you WL data and you kindly converted it for me. Other than the logging period changing to 15 mins for that period there's no difference to it that I can see.

Since then I have changed to a 5-minute logging time. All my data files are attached since the station went operational on 10 May 2009 together with my cumulus.ini with passwords removed.

I'm away from Sat noon until Sunday evening so no rush getting back to me.

Thanks!

gemini06720 · **Posted:** Sat Sep 19, 2009 2:54 am

Steve.

I think you registration of VirtualVP is a good thing - you will now be able to run consecutively both version of Cumulus and VPLive...

So that other can benefit from our discussion about VirtualVP, I have installed a screen capture software and I have taken a few shots/images of VirtualVP:

This is the main or status screen:

This is the activity screen:

There is a third screen called the 'alert' screen but I have never (yet) seen anything on it.

Finally, this is the VPLive screen:

	It is currently Fri May 24, 2013 3:48 am
Please click here before posting. Help me to help you!
Useful Links: Cumulus FAQ • Enhancement requests • Wiki (documentation)
Please put your approximate location into your profile
Add your web site to the Cumulus user map
Vantage Pro2 users with firmware 3.00 should upgrade to fw 3.12 and Cumulus 1.9.4
Latest Cumulus release v1.9.3 (build 1059) - Feb 09 2013 Latest Cumulus beta v1.9.4 build 1077 - May 23 2013 --> Latest Cumulus Realtime v1.0.0.5 - Sep 05 2009

Sandaysoft

Import data Error

Who is online