How on Earth are we going to continue to create data at this speed and still figure out how to store it all? We’ll need unbelievable data centers and we are going to need some technological breakthroughs so we don’t break the bank. We cannot expect all of this to burden one company; Google. Let’s talk about just how much data we are producing and how things are getting out of control.
First, I’d like to point out that this isn’t a new problem. We saw it happening at the Patent Office, Library of Congress, and it’s been getting progressively worse. You see, in the 1965 Year Book accompanying the World Book Encyclopedia there was an essay titled; “Harrison Brown On Science – Science Faces a Crisis in Handling the Flow of Data it Generates; Can it Meet the Challenge?” Well, what has changed since then? Nothing except we collect more data than ever before now, and we are still having challenges storing it all aren’t we, even with the Moore’s Law effect on computer science. Consider if you will the following challenges:
1. Storing All the NASA Telescope Data and Imagery
2. Storing All of the NSA Data Collection
3. Storing All the Drone UAV footage
4. Storing Every Living Human’s DNA for Medical Reasons
5. Storing All of the Content Put Online
You think this is only a problem in the US? Think again, how’d you like to be China right now with their giant nationwide intranet system (aka: Great Firewall of China)?
Now then, the question posed in the titled of this article; “What is the amount of data we can store in an infinite universe?” Might I suggest the answer: An infinite amount. Yes, this issue of Data Storage really does get into a bit of Science Philosophy but think about it for a moment. And mind you, I am not the only one considering this. DARPA put out a solicitation for revolutionary breakthrough concepts in big data, see the article in Washington Technology on February 24, 2014 titled by Frank Konkel; “$45M up for grabs in DARPA big data push.”
There is no reason to collect all this data if you cannot use it or search through it for relevance. So, we need a way to put that data to good use, and that means we need to store it and have a way to quickly go through it, regardless of the ever growing pile of accumulated data. It’s a simply stated problem, although the answer may not be nearly as easy as you’ve been led to believe. Consider that IBMs Watson that won Jeopardy only went through 200 million pages, today we are talking trillions of pages of information online and that information is everywhere, just like all the information of every particle in the Universe.
We’ve essentially created a whole new world of data as an overlay of the natural world, one we don’t even fully comprehend, nevertheless, we will solve this challenge because we must and we will build systems to do it, control the data, and use that data, and we will figure out a way to continue to grow the amount of data toward infinity in our ever expanding infinite universe, which one cannot debate because even if the universe were not expanding, our creation of this new virtual realm within it is – and yes, it is a whole new world waiting for us, if we dare to take it. Let’s do it.