Posts Tagged ‘SDS’

Goodbye Data Migration: ScaleIO Software-Defined Block Storage

Jason Brown

Consultant Product Marketing Manager at Dell EMC
Data migration is a big IT headache – but could software-defined storage be the remedy? Let’s see how Dell EMC ScaleIO software-defined block storage makes data migration blues a thing of the past.


Traditionally, data migration during a hardware ‘tech refresh’ is a costly and painful process that also puts your data at risk – but it’s unfortunately been a requirement with traditional SAN arrays on a 3-5 year repeating cycle.

There are three major categories of pain that need to be considered in regard to a traditional data migration:

  • Human impact
  • Technology challenges
  • Financial costs

The pain of data migration to your people

First consider the human impact of data migration. There’s the stress caused to the storage admin or team – and the owners of the applications and data that are hosted on the storage. A data migration is very disruptive and can typically be a year-long process from start to finish.

For any datacenter-class array, the planning for a data migration is a months-long exercise that involves going to every single affected application owner, and scheduling a window to migrate their application and planning for the potential downtime. The storage admin then needs to build a calendar with the schedule for each individual app migration. This negotiation process can be a very painful and politically charged process – and causes a lot of stress on the admin and the organization.

Then, the execution of the migration takes place, usually over many months, with the migration team using nights and weekends to perform the migration during lowest impact hours. During the actual execution of the migration, the storage admin will face even more pain, such as:

  • 24/7 monitoring and daily reporting
  • Angst and stress of the migration process
  • Constant demands from application teams during the process
  • Time away from other job activities (and potentially family) due to the migration.

The technology challenges of data migration

Another key category of pain is the technology aspect of data migration. How will the migration be executed? Using a specialist migration tool or application? Conducted by professional services or storage team? What processes will be used to decrease risk and minimize downtime?

There are also big risks associated with data migration. What if you don’t complete your data migration within the allotted time? What if the migration fails due to a data corruption during the movement? There’s no 100% guarantee of success. Organizations that are particularly risk-averse will move all production off that system first – but this kind of pre-work beforehand leads to more costs.

The economic and financial costs of data migration

A study by Wikibon in 2014 estimated the cost of a storage migration project to be approximately 17% of the total cost of ownership for a storage array, including initial cost and maintenance. The typical cost of a data migration could therefore amount to hundreds of thousands of dollars for a datacenter-class storage array.

The expense of moving to a new storage system will include not only the so-called ‘hard costs’:

  • Purchase and installation cost of the new storage array itself.
  • Cost of specialized migration software and services if using them.
  • Overtime costs for admins executing and monitoring the migration.

Additionally there are the ‘soft costs’ involved:

  • Economic impact of lost revenue from application downtime.
  • Typical loss at least a year of serviceable time on the storage array due to the year-long migration process.
  • Any costs associated with risk / downtime / migration issues.

A future without data migration?

All these kinds of painful issues are top-of-mind for admins when it comes to data migration. Getting rid of migrations is about getting rid of all these hard and soft costs. It also releases value by giving your IT people a big chunk of their lives back, and allowing them to focus on value-added projects – instead of another year of moving data around.

One of the attractive promises made for new software-defined storage (SDS) is its elimination of the dreaded process of data migration. So can SDS really remove data migration from the storage management equation?

Eliminating data migrations – with Dell EMC ScaleIO

With Dell EMC ScaleIO software-defined block storage you really can transform your approach. In a similar way that VMware virtualized the compute operations of the datacenter 15 years ago, ScaleIO today abstracts and pools local storage resources in each server, and automates provisioning of capacity and performance back to applications on those servers.

Your shared pool of ScaleIO storage is physically dissociated into distributed software components running on industry-standard x86 server hardware. This software-defined approach helps make data movement between storage painless and automatic.

To extend the analogy with virtualization, think about how an admin does a ‘slow roll’ when upgrading a compute cluster. There is no wholesale VM migration or replacement of the cluster – it’s an ongoing perpetual process, performed a server at a time as needed, and requiring no downtime or impact to the hosted applications. There are also specific tools in place (from the hypervisor) to execute and automate this as a seamless and transparent background process.

Guess what? ScaleIO achieves a very similar transformation for storage!

 

A seamless, perpetual process

Of course, data movement still happens within ScaleIO – but we like to think it’s nothing like ‘data migration’ as you’ve previously known it. And much of the time, you won’t even notice it’s happening.

The processes of balancing in a new node and evicting an old node are all seamless to your applications. When a new server node is added to the cluster, an automatic data rebalance occurs, spreading some data from each existing server node across to it. Similarly, when the storage admin decides to ultimately decommission a server node, it is removed by issuing a command that automatically redistributes its data across the rest of the servers in the cluster. This is all completely transparent to your applications.

With ScaleIO, maintaining and upgrading storage becomes a painless process – rather than the old-style painful epic project. This is basically the model of storage management pioneered by web-scale companies like Facebook, Google, and Amazon. They had to overcome the challenges of their huge data environments, running mission critical apps that cannot afford any downtime, and where wholesale migrations are not feasible. They achieved this by moving from arrays to home-grown software-defined infrastructure running on top of many x86 servers. ScaleIO is an enterprise-grade, commercially supported, and standards-based solution that follows similar operational principles.

It’s automatic, it’s programmatic – it’s ScaleIO greased lightning!

Organizations that are making the best use of software-defined technologies like ScaleIO tend to be highly automated in their IT. They are already using software automation tools for deployment of servers, OSs and applications. With ScaleIO, storage provisioning and deployment is automated in the same framework – programmatically.

You can simply decide when it’s time to expand your storage and do it in granular steps, one x86 server at a time. Similarly, when a server is old and no longer desired, you can decide to evict it from the cluster, one server at a time, on-demand. ScaleIO enables the concept of a perpetual rolling upgrade, so you maintain optimum efficiency and performance across all your storage – without the headaches of traditional data migration.

With ScaleIO, it doesn’t really matter what specific kind of server is attached to the storage cluster – whether it’s VMware, Hyper-V, bare-metal Linux, or whatever. There’s no need to worry about operating any particular environment. ScaleIO takes a platform-agnostic, heterogeneous approach. It enables the same processes, regardless of your OS or applications.

Say goodbye to data migration challenges with ScaleIO

We think you might enjoy a world without data migration headaches. So why not explore the advantages of Dell EMC ScaleIO software-defined block storage for yourself?

Dell EMC ScaleIO software is available to download and try for free.

Learn more about Dell EMC ScaleIO advantages for your datacenter.

Dell & EMC Showcase Their Synergies with the All-Flash ScaleIO Ready Node

Jason Brown

Consultant Product Marketing Manager at Dell EMC

scaleioimage1In case you missed it, on September 15th we announced the Dell EMC ScaleIO Ready Node. This announcement highlights months of collaboration between Dell and EMC to combine the best of both worlds – Dell PowerEdge servers and Dell EMC ScaleIO. The ScaleIO Ready Node brings All-Flash capabilities to Software-Defined Storage to enable customers to transform their data centers, making the path to the modern data center easier with Dell EMC:

There are tons of specs and details about the ScaleIO Ready Node which I won’t rehash here. You can check out the data sheet, spec sheet, and FAQ to get all the details you need. What I’d like to highlight are two key points regarding this announcement:

  1. Bringing best of breed EMC software-defined storage and Dell server hardware together
  2. Optimizing All-Flash with SDS

The first point is really important. There’s a reason why Dell spent 500 gazillion dollars on EMC… oh, it was only $67 billion you say? Peanuts then! But seriously, there are obviously a lot of synergies and opportunities between Dell and EMC, and the ScaleIO Ready Node is one of the first examples. Dell builds best-of-breed servers with its PowerEdge line. EMC is a leader in software-defined storage (SDS), with ScaleIO as its block storage offering. And guess what? ScaleIO runs on x86-based industry standard servers! Bringing PowerEdge servers and ScaleIO together was a no-brainer. It’s like peanut butter and jelly. Or mac and cheese. Or even “peas and carrots”*.

But, it’s not as simple or straightforward as you think. A ton of thought and work went into the planning and R&D processes associated with the ScaleIO Ready Node. Yeah, we’ve loaded ScaleIO onto a variety of Dell PowerEdge servers. But it doesn’t stop there. We’ve introduced a system that is pre-configured, pre-validated, and optimized for running ScaleIO. Plus, it comes with “one throat to choke” for procurement and support: Dell EMC.

I can’t emphasize how important that is. When I talk to customers, they get SDS, they understand there can be significant 3-5-year TCO savings, and they absolutely love the performance, scalability, and flexibility of ScaleIO. But, when the rubber meets the road, a majority of customers are not going to buy ScaleIO software and then procure <insert your favorite brand> servers from another vendor (but if they do, I hear Dell has good stuff). So, we’ve simplified the process and enabled faster time-to market by using Dell EMC’s supply chain and services so customers can hit the ground running – while preserving the flexibility which is a huge differentiator for ScaleIO. See what two ScaleIO veterans have to say about this:

The second point is the formal introduction of ScaleIO into the All-Flash (AF) arena. Yeah I know, every product out there has AF capabilities, and yeah, flash is becoming commoditized, and yeah, you could run AF ScaleIO clusters before the ScaleIO Ready Node. Regardless, AF is the way of the future and one of the foundations of the modern data center. So, we’re combining two key foundations to transform your data center – All-Flash and Software-Defined Storage – into a single platform to make it much easier for customers to start their journey to the modern data center.

What’s important about the AF ScaleIO Ready Node is how we optimize flash with SDS. ScaleIO’s architecture is unique and is the key behind unlocking the power of AF. All of the SSD drives within the All-Flash ScaleIO Ready Node work in parallel, eliminating any and all bottlenecks. Each node scaleioimage2participates in IOPS and there’s no cache to get in the way – for reads or writes. The ability to take full advantage of the aggregate performance of all SSDs makes it possible for performance to scale linearly as you add more servers.

Customers have the ability to migrate Tier 1 and Tier 2 application workloads with high-performance requirements to All-Flash ScaleIO Ready Nodes without missing a beat! Check out Dell EMC’s All-Flash page for more details, and if you want to see some guy talking about the All-Flash ScaleIO Ready Node, click here.

We’re extremely excited about the release of the ScaleIO Ready Node. It’s awesome to be one of the first products to be released by Dell Technologies that highlights the synergies between Dell and EMC. With this collaboration, we’re able to bring peace of mind to customers and provide unique product capabilities now and in the future. Please visit the Dell EMC ScaleIO Ready Node page to learn more! #GoBigWinBig

*If you’ve never seen Forrest Gump, go watch it. Now!

ECS for OpenStack is a ‘swifter’ route to enterprise-grade persistent storage

Nisha Narasimhan

Nisha Narasimhan

Principal Product Manager at EMC Emerging Technologies Division

A vast majority of our enterprise customers continue to ask us about using EMC Elastic Cloud Storage (ECS) with their OpenStack infrastructure – and how easy it would be to use ECS as a drop-in replacement for OpenStack’s object storage, Swift. The good news is that, with the release of ECS 2.2.1, ECS integration for OpenStack just got easier – as I’ll explain.

OpenStack Swift is great to get started with OpenStack. However, as you move out of proof-of-concept (POC) into larger production environments, the platform weaknesses swiftly show up – keeping costs under control while maintaining compliance- typical enterprise grade features in a persistent storage system at scale is very difficult.

OpenStack Swift has gaps to address to be truly enterprise-ready

OpenStack Swift falls short of being a true enterprise-grade production-ready solution. It is not simple to scale out to thousands of nodes, has limited erasure coding support, and performance for both large and small data objects is not on par with commercial object storage offerings.

Further, as you grow across sites you will require ‘strong’ data consistency – to ensure that changes to a dataset in one location are immediately reflected in the copies of the dataset in other locations. OpenStack Swift provides only ‘eventual’ data consistency – which can leave your applications struggling to handle synchronization across multiple sites.

OpenStack Swift purely focuses on accessing objects via the Swift protocol leaving out other protocols in use by applications today. For example, if you want to analyze your data using Hadoop, you will first need to move your data from OpenStack into a separate HDFS cluster – which will cost your time, resources and datacenter space. Similarly NFS access to OpenStack Swift can only be achieved via external NFS Gateways.

An enterprise-grade object storage for OpenStack

ECS is a cloud-scale object storage platform. It is an attractive drop-in replacement for OpenStack Swift – with its ability to handle massive scale, production-grade performance for objects (large or small), built-in metadata search capabilities, multi-protocol access on the same data (object via Swift API or S3, NFS and HDFS for in-place Hadoop analytics), multi-tenancy, and active-active geo-distribution with strong consistency.

With the release of ECS 2.2.1, featuring ‘ECS Swift’, you can now easily provision and use ECS object storage from OpenStack.

Seamlessly integrate ECS into your OpenStack environment

In OpenStack deployments, identity and access management is handled by OpenStack Keystone. To enable integration with OpenStack, ECS 2.2.1 offers native support for interop with Keystone v3.0.

Out of the box, ECS can now register itself with Keystone, and use Keystone to authorize ECS Swift service requests. This integration also enables Keystone to be the ‘single source of truth’ with respect to user identities, eliminating the need for duplication of identities in ECS.

The behind-the-scenes interaction between ECS and OpenStack Keystone works as follows: if you are an OpenStack user, when you authenticate to Keystone, it will present digital credentials explaining who you are. Keystone verifies your identity and presents you with a Keystone token. You then make a request to the ECS Swift service with this token. To serve the request, ECS forwards the token to Keystone for token authorization, and ensures that you are authorized to perform the requested action, prior to rendering the request.

 

ECS Intergration with OpenStack

Keystone integration allows ECS to be a drop in replacement for OpenStack Swift. But our plans don’t stop here! With upcoming releases we will continue to enhance ECS for tighter integration with OpenStack (support for Manila File Services and support for OpenStack metadata search) – so keep watching this space.

EMC ECS software is available to download now and try free.

Learn more about object storage with EMC ECS.

Breakfast with ECS: Most Wanted Cloud Storage Feature Series – Part 4: Enterprise Class

Diana Gao

Senior Product Marketing Manager at EMC² ECS

Welcome to another edition of Breakfast with ECS, a series where we take a look at issues related to cloud storage and ECS (Elastic Cloud Storage), EMC’s cloud-scale object storage platform.

Hello folks!

Welcome back to ECS feature series!Breakfast with ECS Enterprise

In the previous blog of this series, we discussed ECS’ smart capabilities. In this blog, we’ll discuss how ECS is built to serve the needs of enterprises.

When selecting a cloud storage vendor, enterprises have many questions: Is this cloud built for scale? Is it secure? Is it able to handle the different business applications needed for long term business growth? Will it simplify my operational management? We at EMC were thinking these exact things when we introduced ECS.  How can ECS improve your data privacy, manageability and operation efficiency?

Watch the video below and find out the answers.

ECS is enterprise-grade yet has incredibly low storage overhead, and is capable of storing nearly 4 PB of data in a single rack. Stay tuned for the next blog discussing more about ECS’ economical capabilities. Yes! You don’t want to miss it.

Additional resources:

Breakfast with ECS: Most Wanted Cloud Storage Feature Series

Diana Gao

Senior Product Marketing Manager at EMC² ECS

Welcome to another edition of Breakfast with ECS, a series where we take a look at issues related to cloud storage and ECS (Elastic Cloud Storage), EMC’s cloud-scale object storage platform.

Social media, mobile, cloud and big data are here to stay. Each has brought new opportunities to create more intimate, immediate relationships withElastic Cloud Storage customers, deliver better experiences, enhance business value, and gain competitive advantage. The principal asset that makes all this possible is the insight that comes from data – data that is growing exponentially and without structure.

Burdened by traditional storage systems, IT organizations have struggled to keep up with this explosion of data. This has led to adoption of public-cloud storage platforms like AWS S3, driven by compelling economic advantages over traditional SAN and NAS storage systems. However, public cloud storage platforms involve navigating some fundamental tradeoffs in the areas of data residency, compliance with local laws and regulations, as well as unforeseen costs.

With EMC’s Elastic Cloud Storage (ECS), you don’t have to balance these tradeoffs. It is one storage system that is able to balance the needs between capacity (flat budget for growing data), capability (more demands imposed by applications) and perceived economical alternatives in the public cloud.

Watch the video below and learn more about why Elastic Cloud Storage (ECS) is one storage solution that can cater to all of your needs.

With ECS, you’ll enjoy these key benefits:

  • Multi-purpose platform
  • Smart storage
  • Enterprise class
  • Superior economics

What features make ECS such an awesome platform? Stay tuned for more blogs.

Learn more about ECS and try ECS for free.

Demystifying Software-Defined Storage and Hyperconvergence

David Noy

VP Product Management, Emerging Technologies Division at EMC

If you read the storage news these days you simply can’t miss a story around hyper-converged storage or yet another vendor looking to release a software version of its platform. If you believe Gartner, by 2019 about 70% of existing storage array products will become available in “software-only” versions. The information industry is quickly waking up to the fact that the thing that turns a cheap white box server into a branded product that commands high margins is the software. Increasingly, end users are looking to standardize on low cost servers in order to reduce operational costs and obtain better purchasing leverage to get better pricing. Some web scale customers do this to the extreme and from that the Open Compute Project was born.

To participate in this market,Computer applications different strategies have emerged by data storage technology companies and the borders between software-defined, hyper-converged, and commodity hardware have gotten blurred.

Before I delve into what’s out there, let’s define terms. Software-defined storage has been around for a long time. A software-defined storage solution provides a hardware agnostic solution to data management and provisioning based on storage virtualization. Said more plainly, software-defined storage takes a bunch of disks and processors and turns them into the functional equivalent of a storage appliance. This could be object, block or file based storage. Hyperconverged refers to the ability to run both your software-defined storage services (a virtualized storage appliance) and applications on the same servers. This could be a cluster of servers where direct attached hard disks and flash drives are virtualized and made available to applications running (potentially virtualized) on the same physical infrastructure.

“Commodity hardware” refers to servers that are built from commonly interchangeable, standards based, high volume components. Data storage companies are bringing all of these aspects together to build low cost, highly customizable alternatives to the legacy storage architectures of the past.

In EMC’s portfolio there are several unique and powerful software-defined storage offerings for object, block and (soon) file based storage. For today, I am focusing on the EMC® ScaleIO® product which enables a “Software-defined Scale-out SAN” by  virtualizing servers with DAS to provide block storage for applications running either hyper-converged or on separate sets of servers dedicated to storage and applications (“two-layer” approach). The EMC ScaleIO product was designed from day one to be a software-defined storage offering that takes any server hardware and pools its storage in scale-out fashion. What does it mean to be scale-out? Scale-out (as opposed to scale-up) means that the design center for the product is optimized to incrementally add capacity and compute. Scale-out storage products allow end users to start small, often times with only a few nodes (another term for servers) and incrementally grow as their business demands increase.

One of the advantages that EMC ScaleIO has over some of the other approaches to software-defined block storage is that it was designed for scale, performance, and flexibility out of the gate. ScaleIO is first and foremost a software product. As such, it can be easily applied to a wide variety of commodity servers allowing customers to avoid vendor lock-in, maximize their existing server vendor relationships, and pick and choose the storage media that meets their performance requirements. The ScaleIO product was also designed exclusively as a high performance block storage virtualization product, so it does not have to suffer from the performance overhead that comes with trying to take-on “multiple storage personalities”, which I will explain later. Finally, the ScaleIO team recognized the importance of platform choice and implemented support for a wide range of hypervisors and operating systems including integration with cloud management products like OpenStack.

Why the SDS Approach for Hyperconverged InfrastructureServers Conquers All
With the recent shift in thinking towards taking advantage of commoditization and convergence, many vendors are now competing in the hyper-converged storage market. There are several approaches they have taken: an appliance model, a layered model, or a hypervisor model.

Appliance Model:
The first approach, where vendors have taken an appliance model to the solution, has had moderate success. However, in an effort to rush to market, these solutions have made rigid assumptions around hardware choices and rules. These rules help when you are trying to force a quick solution into a new market, but ultimately they lead to pain for the end users. Rigid rules around how to grow your hyper-converged appliances, which components you have to use, flash to spinning disk ratios, and other “non-solutions” to engineering rather than customer problems are forcing these product vendors to rethink their approach. Many of them are now looking at how to take their embedded software and reengineer it to run on a wider variety of hardware vendor platforms. Ultimately, what they are finding is that what customers really want is the software bits and not the vendor lock-in. Unfortunately, systems designed to take advantage of hardware choice shortcuts aren’t so easily repurposed for a hardware vendor neutral world. Fortunately, EMC ScaleIO was built as a software product from inception. This means it can easily be adapted to hardware delivered solutions later, but will never have to worry about struggling to become a software-only product.

Layer Model:
The second approach is to take a layered model to building software-defined block storage services on top of object storage architecture. Now there is nothing wrong with using abstractions in any systems design – abstractions help to simplify things. The problem comes when you have a system that is designed to optimize around the underlying abstraction and not the service layered on top. It’s really hard to do a good job of building one data paradigm on top of another when the two are optimized for totally different parameters. For example, a block storage system should be optimized around maximum uptime, minimal resource utilization, and maximum performance even if it means taking advantage of more expensive media like flash for persistence. On the other hand, an object file system should be optimized for billions or even trillions of objects, geographic dispersion of data, and low cost relatively static data.  Layering a block storage system optimized for uptime and performance on top of a system optimized for object sprawl and low cost seem at odds with one another! That’s exactly what we see in practice; software-defined block storage built on object stores tend to be slow, consume a lot of resources, and require a lot of care and feeding into the underlying storage paradigm to keep operational. These offerings have been successful primarily because their business model is a freemium model that allows end-users to download and use the product without a support contract. The performance penalties and reliability issues have certainly not played in their favor. In order to make sure that end users have choices other than the current cumbersome freemium offerings, this summer EMC ScaleIO will be releasing the first “Free and Frictionless” versions of its product, designed to give anyone the ability to download and operate a software-defined SAN storage cluster, for an unlimited time and capacity, for non-production workloads.

Hyperconverged Model:
Finally, hypervisor vendors (of which there are only a few) have also jumped on the commodity bandwagon. The advantage of these solutions is that they are generally built into the hypervisor platform, which means that if you have the hypervisor platform deployed then you have a block storage virtualization product ready to go. Hypervisor clusters of servers tend to be small though and so while this can provide a quick and easy way to get going with block storage, they tend not to be a scalable, high performance, solution and as with solutions designed for a specific hardware platform, come with a level of rigidity. End-users that have a mix of Windows and Linux platforms, or may be looking to take advantage of less expensive virtualization platforms like KVM and OpenStack will find themselves limited by solutions that are built into a single vendor’s hypervisor. Once again, EMC ScaleIO addresses the needs of these end-users looking for choice of platforms, high performance, and massive scale while in some cases plugging directly into the hypervisor for optimal performance. While EMC ScaleIO can be deployed in conjunction with hypervisor platforms in a hyper-converged fashion, it is different from the hypervisor vendor solutions in that you aren’t forced to run hyper-converged. You can choose to deploy your storage servers and your virtualized application servers separately if that’s what suits your organization.

It’s no surprise given the rapid growth of the software-defined, commodity storage market that every large vendor and many more startups are introducing or tailoring their products for this new world. But the approach matters. Products designed with hardware constraints early on will have a real challenge trying to disentangle themselves from the assumptions that they have made. Products built with dual personalities that attempt to imitate one storage type on top of another will find themselves optimized for one thing while trying to deliver another, leaving end-users dissatisfied. And finally, hypervisor-based solutions, while simple to set up and integrated into the hypervisor, may work for some small deployments but will lack the flexibility and scale of a true software defined storage solution for the enterprise. Fortunately for end-users, the EMC ScaleIO software block storage solution avoids these limitations since it was born and raised in the software defined world.

Follow Dell EMC

Categories

Archives

Connect with us on Twitter