Invitation Digital Tech Blog

Building Scalable & Responsive Architecture


Digital hoarders

With the proliferation of digital devices that are all capturing 0s and 1s, storing them to memory or duplicating them over the wire to a data center on the other side of the world isn’t it time we thought about a clean up?

All of the devices that are now storing data; CCTV; mobile phones, email; digitally connected buildings; even your light bulbs are helping to create this tsunami of digital matter. We take photos, post to social media sites and generally create this matter at an ever increasing rate, all of us, all of the time. Just by reading this there are another few thousand bytes stored on your device.

If every time you finished a box of cereal you duplicated it 4 times then stuffed some of them into various cupboards and sent a couple to friends you would soon be buying a bigger house just to keep empty cereal packets in, and losing friends. It sounds like the actions of a mad man but are we not doing this every day of our lives?

Of the roughly 700M iPhones that have been sold, let’s assume that there is an average capacity of 32Gb, in reality this will probably be a bit larger. That is 22400000 terabytes of storage just on iPhones worldwide before we count Android and the couple of Windows phones. If each of these phones had 500Mb of photos and each of those was 2Mb and of that there were 10% of bad ones then there would be 17500000000 photos of the inside of people’s pockets or the dinner that they posted on Instagram last night. There are all the cereal packets right there sitting in people’s pockets, never being looked at.

In ‘Do Androids Dream Of Electric Sheep’ Philip K. Dick refers to kipple, it’s the stuff that accumulates without human intervention, piles of abandoned matter that gathers but still remains festering over time. I have drawers at home that we refer to as kipple drawers, they’re the ones with some marbles, a few elastic bands, the broken foot from the toaster etc. Nobody has any idea where any of it came from, they just fill themselves over time like the entropy of a functioning household. We all have them, since the drawer was invented people have had them and they contain 80% junk, kipple. That concept has transpired to the digital world, the drawer is your hard drive and the kipple is your junk mail.

In Q3 of 2013 over 100 billion spam emails were sent every day this increase 98% between december 2012 and March 2013, imagine what the levels are today. I checked a few that I received recently and at about 2k each that is 100TB of junk mail spinning round the globe accumulating like drifts in people’s homes every 24 hours. A lot of this is now getting stored on cloud servers as more people use Gmail and other webmail provisions. According to what if? it is very roughly calculated that Google store 10-15 exabytes of data, that is 10-15 million terabytes. If this was 50% 1s and each 1 was an electron in memory it would weigh about 0.000015g, about half a poppy seed. If it was punch cards it would cover New England 3 miles deep. How much of this is pictures of kittens?

But not all big data is bad, genomic research, predicting weather patterns and seismic activity, recommendation engines and medical tests such as MRI scans. These all require huge amounts of data to be kept indefinitely, this is fine and has to be encouraged. However how many Petabytes of data are there of scans of perfectly healthy people never to be reviewed, never to be looked at again. How many X-Rays of legs with nothing wrong with them are there, why are we so obsessed with keeping everything?

There is a cure though, one that we all can do. walk away and don’t look back!