Object storage is the future to solving all of our data needs

Object storage is a new storage paradigm that is seeing growing interest as it addresses the challenge of efficiently storing massive volumes of unstructured data. Big Unstructured Data.

Depending on which analyst firm you talk to, you will hear storage growth predictions that vary between a factor 30 and 40 for the next decades. That means we will all be storing 30-40 times as much digital data ten years from now, compared to today. At the same time companies will only invest an additional 50 percent in personnel to manage their storage infrastructures, meaning that the average storage operator will have to manage 15-20 times as much storage in the next ten years. This calls for storage platforms that require little management and where scale-out capabilities are virtually unlimited.

Billions of users, always online

Major cloud application providers like Google, Facebook and Twitter have deployed object storage to meet the requirements of their fast-growing user base: billions of users are storing trillions of objects in infrastructures that were designed to scale infinitely and perform with the lowest latency.

Network Attached Storage (NAS) or Storage Area Networks (SAN)?

The biggest problem with those systems is scalability: NAS does not really scale as a single system, at least not when talking about petabyte-size environments. SANs are already pretty complex when deployed with a file system layer on top. Scaling-out makes the problem a lot worse.

Object storage is essentially just a different way of storing, organising and accessing data on disks; it is a storage infrastructure to store objects, which are essentially files with lots of metadata added to them. The backend architecture of an object storage platform is designed in such a way that all the storage nodes are presented as one single pool; there is no file system hierarchy. The architecture of the platform, and new data protection schemes (vs. RAID, the de-facto data protection scheme for SAN and NAS) allow this pool to scale virtually unlimited while keeping the system simple to manage.

Users access object storage through applications that will typically use a REST API, an internet protocol, optimised for online applications. This makes object storage ideal for all online, Cloud environments. When objects are stored, an identifier is created to locate the object in the pool. Applications can quickly retrieve the right data for the users through the object identifier or by querying the metadata – information about objects like the name, when it was created, or by who.  This is a lot faster than trying to locate a file through a file system.

Ultimately as data explodes, more and more companies will be forced to look at their storage and invest in sustainable, long-term solutions.  At DataDirect Networks (DDN), we believe that object storage will play a major role in addressing those problems. 

Object storage solutions are designed to scale beyond Petabytes as one single system, as well as optimise overall TCO without compromising performance or durability. This results in a platform for a variety of storage cloud solutions, including online collaboration, active archives, cloud backup and worldwide data distribution.

So at the next industry get together, when people start talking about trying to manage the exploding growth of unstructured data, feel free to walk them through how object storage can address their problems.  It’s a smart solution to a growing problem.