By Grant Fisher, chief information officer, Pecan Street

I spend a lot of time talking about Pecan Street’s data. It is, after all, at the heart of our effort to create more reliable energy transmission, larger and more efficient distributed energy resources, and better energy monitoring equipment.

One of the most common questions we hear is “how much data do you have?”. There are a few ways to answer that. We can say we collect 10 million records per-day, per-house. Or, we can say the database is many terabytes large. Or that we have the world’s largest collection of energy, water, and gas data for use by academic researchers.

However, it isn’t until recently that I think Pecan Street has crossed into the Big Data realm. With the acquisition of a few new project servers, we have surpassed one petabyte of data storage availability at Pecan Street.

To put that in perspective, 1 petabyte is equal to 1,000 terabytes, or 1,000,000 gigabytes. 1 PB is equivalent to over 1.5 million CD-ROMs, or is more than 4,000 digital photos per day, over your entire life (you can get more comparisons here).

Why do we need this much storage? Recently, we were awarded a grant from the Alfred P. Sloan Foundation to increase the collection capabilities of the measurement equipment installed in the homes in our research network. Instead of capturing data each minute, the upgrade will allow us to capture data every second. This means that we’ll get about 300 times more data per home. We’ll collect more data in one day than we have in the last 10 years!

I’m sure that in 20 years or 30 years, cell phones and other residential devices will have petabytes of data storage. But right now, a petabyte is a big deal to us.

For more information on the data we store, how we store it, and other data related topics, stay tuned.