Challenge: Cost-Effectively Adapt to Growth in Unstructured Data
Organisations are experiencing dramatic growth in unstructured data, driven by forces such as social media, online video, user-uploaded content, gaming and software-as-a-service applications. This object-oriented information is typically written, read and deleted but never modified, unlike conventional data files that are frequently revised or updated. The shift to mobile and tablet computing compounds the storage challenge.
Not surprisingly, traditional storage systems fall short when confronted with the object-based storage archetypes of today, which are characterised by data that needs to be instantly accessible from any device over HTTP and stored forever. It is simply no longer sufficient to build storage silos utilising non-web protocols that are tied to specific applications. While many public cloud storage services have risen to meet these new storage needs, not every organisation can - or should - use public cloud storage.
For many companies, the challenge is how to deploy a storage solution that can cost-effectively meet their needs in this modern object-based data environment. Solutions based on the conventional storage stack are inherently complex and inefficient, relying on decades-old layers of software and hardware stacked together to provide a data path between documents and rich media residing in a file system atop a hardware layer based on block storage. Such solutions offer relatively poor scalability, their expansion typically entailing both increased cost and complication as well as diminished performance and reliability.
This approach is not only inefficient, it is also unnecessary. The majority of today’s mass scale applications do not need either file semantics or a file system to optimise space management on a device. Modern applications only need object semantics (e.g., write the whole thing, read the whole thing, delete the whole thing, refer to it by a handle chosen by the client and cluster manager), not where data resides on a given device.
Ideally, organisations would be able to deploy their own private cloud solutions, leveraging the architectural scalability, accessibility and reliability of major public cloud computing platforms (e.g., Amazon Web Services or AWS) leveraging standard hardware, but in a form that is simpler and more affordable to configure, manage and scale out than traditional storage.
Solution: Private Cloud Storage Powered by OpenStack Swift
SwiftStack private cloud storage provides object storage for today’s applications in a scale-out architecture that overcomes the limitations of traditional storage systems. In addition to addressing scalability, this software-defined solution lowers the cost of storage distributed across multiple datacentres, allowing end users to affordably respond to user and data growth.
First used by service providers and enterprises with SaaS and web/mobile applications, software-defined storage (SDS) solutions leveraging standard server hardware and standard drive technology dramatically lower the cost of supporting hardware; open-source SDS solutions also lower the price of software, providing an end solution that is 1/4th to 1/5th the cost of public cloud or traditional on-premise storage arrays.
Like other software-defined storage solutions, SwiftStack differs from traditional storage by running on standard server hardware. Powered by OpenStack Swift, SwiftStack is an open solution with a vibrant development community and no lock-in. Also unique among open solutions is its exclusive focus on object storage, optimizing for superior scalability, and made easy via the SwiftStack controller (see Figure 1, below).
Figure 1: SwiftStack Architecture Overview
OpenStack Swift can scale from a few nodes and a handful of drives to thousands of machines with hundreds of petabytes of storage. Swift is designed to be horizontally scalable - there is no single point-of-failure, significantly increasing reliability. Swift is also ideal for storing and serving content to a multitude of concurrent users, which further differentiates it from other storage systems.
Managed through a browser-based controller, SwiftStack is easy to operate from multiple levels of access. The dashboard interface removes the heavy lifting from configuration, organisation, authentication, and load balancing. Regular alerts, reports and system stats keep operators constantly updated on their storage needs. In short, the power of public cloud storage in your own datacentre, managed as a product not a project.
For Application Developers:
- Data stored and served directly over HTTP.
- Access to storage in minutes, not days.
- One multi-tenant storage system for all your apps.
- Focus on app development, not infrastructure plumbing.
- A rich ecosystem of tools and libraries.
For IT Operations Teams:
- Use low-cost, industry-standard servers and drives.
- Manage more data and use cases with ease.
- Enable new applications quickly.
- Has highly durable architecture with no single-point of failure.
- Doesn’t require vendor lock-in.
Complementary Technologies: Seagate Kinetic Open Storage Platform and SwiftStack Private Cloud Storage
Modern applications need storage to scale out horizontally because more and more user data is being concentrated on fewer and fewer distributed systems. Software-defined storage meets this need for horizontal scalability, enabling additional amounts of storage capacity by adding servers in a cluster.
Utilising ubiquitous technologies like HTTP and Ethernet, it’s possible to create storage systems that span not just one rack, not just one networking switch, not just one data centre - but that can extend over the Internet (or a large-scale private corporate network). And now with the Seagate Kinetic Open Storage platform, those same protocols are available all the way down to the drive.
The Kinetic platform ushers in a new era to define the new unit of measure for storage. The platform includes the following components that are radically different from the existing generation of disk drives:
- The hard drives speak Ethernet rather than SATA, SAS or fibre channel. This enables the HDDs to talk directly to other devices and other components in the system, rather than going through intermediary devices, controllers or other compute nodes.
- The key/value storage interface enables clients to communicate objects to the devices, rather than blocks. This reflects the fact that the units of storage and the system are not bytes on disk - they are objects. Scale-out storage systems use objects because objects enable systems to scale - and with Seagate Kinetic-ready HDDs (available in 2014), objects are available at the drive level.
- By speaking over Ethernet, these devices can be part of a platform that easily scales out to virtually any size. Simple networking infrastructure can route requests across an enormous number of devices.
The composition of storage systems that take advantage of Kinetic drives will be radically simplified. The storage nodes will simply be an enclosure for the drives and an everyday Ethernet switch. As a result, these enclosures will reduce system total cost of ownership (TCO) The enclosures will also consume less electricity and enable greater storage density, further reducing TCO.
Compared with the conventional storage stack described earlier, the Seagate Kinetic stack is much simpler and more direct (see Figure 2, below):
Figure 2. Seagate Kinetic Open Storage Platform Stack
The Seagate Kinetic Open Storage platform model significantly improves storage efficiency, cost-effectiveness and TCO:
- Superfluous layers of legacy software and hardware are removed.
- Need for traditional storage server tier is eliminated.
- Storage can truly be disaggregated from compute.
- Racks can be denser.
- Number of fans is minimised.
- Data traffic leverages existing data centre transit fabric (Ethernet).
- Data centre operational management is simplified, with reduced costs and risks.
Simply put, scale-out is easier, more cost-effective and unconstrained by the limitations of legacy architectures and infrastructure. By enabling scale independence at the component level, the entire storage system can be built with just enough external compute resources to satisfy the demands of the workload.
Today’s software-defined approach - with compute and storage together in the same node - is already changing storage architectures. The Seagate Kinetic platform takes it to the next level. Having the storage device speak in keys and values means that there is less impedance mismatch between OpenStack Swift (which is native object) and traditional drives (which speak blocks). This results in more efficiency and greater utilisation.
To learn more about SwiftStack Private Cloud Object Storage or register for a live demonstration, visit www.swiftstack.com or the Seagate Kinetic Open Storage Platform.