Right, Steve's data worked fine, and I'll post again when I've done something interesting with it!
But another member PM'd me with more data (he's not replied to whether I can use it publicly yet, so I won't post links here until he does).
It's 5 years of data, at 1 minute resolution. I quickly found the limits on Google Fusion Tables and Excel - just over 1 million rows PER IMPORT.
But it seems to suggest you can MERGE tables and make them bigger.
So I batched it up into years - made 5 folders called 2008,2009,etc and copy the months for each year into there.
Open cmd in windows - copy *.txt 2008.csv etc.
Now we need to get the standard date and time into column A, but spreadsheets don't seem to get on well with standard concatenate for date/time.
So, you need: =INT(B2)+MOD(C2,1) - this assumes you've copied in the header row and are starting at row two.
Copy down to the end of the sheet. Save as CSV.
Then I hit another well-hidden limit in Fusion Tables - 250Mb per user across all fusion tables, despite my having bought extra storage (25Gb per year for £5 across all services including Drive is still a bargain though!).
So, I used another account to import the last table, then a third, fresh account with no fusion data at all to do the merge, having shared the other tables.
It was a Google apps account which said that fusion tables were not enabled, despite the fact it says it can be enabled.
Turns out Fusion tables
CAN'T be enabled for apps accounts, yet:
So I created brand new "standard" account to test - I managed to import one more table, then started to merge with one other table (so 2008 and 2009 are merged).
It seemed to go OK, but when I went to sort by date, it said:
Failed to create table.OR
Could not fetch exact count. Try reloading the page.This latter message also appears when sorting by date, descending.
As does
"Could not fetch data. Try reloading the page."When I go back to the docs list and click that table I mentioned above, after about a minute of working, it says:
Not Found
Error 404it also seems to have created another identically named table with a different ID which only has 59,158 rows. This is well short of any of the single tables.
I've posted all this in the fusion help group - and it's very useful having this large data set, precisely so I could see what problems I might run into in the future.
One way round it would be to somehow average out the 1 minute resolution into a 1 hour timeframe average, but given that there aren't always 60 samples per hour, this is beyond my current spreadsheet knowledge.
And besides, the way Excel has been struggling with this data even on my 3Ghz quad core 4Gb RAM SSD PC, it's been generating enough heat and fan activity to keep my little office nice and warm on these cold summer days!
I'll post again when I get anything useful or interesting.
Login 
Register
Enhancement requests
Search
