Posts Tagged ‘data migration’

Goodbye Data Migration: ScaleIO Software-Defined Block Storage

Jason Brown

Consultant Product Marketing Manager at Dell EMC
Data migration is a big IT headache – but could software-defined storage be the remedy? Let’s see how Dell EMC ScaleIO software-defined block storage makes data migration blues a thing of the past.

Traditionally, data migration during a hardware ‘tech refresh’ is a costly and painful process that also puts your data at risk – but it’s unfortunately been a requirement with traditional SAN arrays on a 3-5 year repeating cycle.

There are three major categories of pain that need to be considered in regard to a traditional data migration:

  • Human impact
  • Technology challenges
  • Financial costs

The pain of data migration to your people

First consider the human impact of data migration. There’s the stress caused to the storage admin or team – and the owners of the applications and data that are hosted on the storage. A data migration is very disruptive and can typically be a year-long process from start to finish.

For any datacenter-class array, the planning for a data migration is a months-long exercise that involves going to every single affected application owner, and scheduling a window to migrate their application and planning for the potential downtime. The storage admin then needs to build a calendar with the schedule for each individual app migration. This negotiation process can be a very painful and politically charged process – and causes a lot of stress on the admin and the organization.

Then, the execution of the migration takes place, usually over many months, with the migration team using nights and weekends to perform the migration during lowest impact hours. During the actual execution of the migration, the storage admin will face even more pain, such as:

  • 24/7 monitoring and daily reporting
  • Angst and stress of the migration process
  • Constant demands from application teams during the process
  • Time away from other job activities (and potentially family) due to the migration.

The technology challenges of data migration

Another key category of pain is the technology aspect of data migration. How will the migration be executed? Using a specialist migration tool or application? Conducted by professional services or storage team? What processes will be used to decrease risk and minimize downtime?

There are also big risks associated with data migration. What if you don’t complete your data migration within the allotted time? What if the migration fails due to a data corruption during the movement? There’s no 100% guarantee of success. Organizations that are particularly risk-averse will move all production off that system first – but this kind of pre-work beforehand leads to more costs.

The economic and financial costs of data migration

A study by Wikibon in 2014 estimated the cost of a storage migration project to be approximately 17% of the total cost of ownership for a storage array, including initial cost and maintenance. The typical cost of a data migration could therefore amount to hundreds of thousands of dollars for a datacenter-class storage array.

The expense of moving to a new storage system will include not only the so-called ‘hard costs’:

  • Purchase and installation cost of the new storage array itself.
  • Cost of specialized migration software and services if using them.
  • Overtime costs for admins executing and monitoring the migration.

Additionally there are the ‘soft costs’ involved:

  • Economic impact of lost revenue from application downtime.
  • Typical loss at least a year of serviceable time on the storage array due to the year-long migration process.
  • Any costs associated with risk / downtime / migration issues.

A future without data migration?

All these kinds of painful issues are top-of-mind for admins when it comes to data migration. Getting rid of migrations is about getting rid of all these hard and soft costs. It also releases value by giving your IT people a big chunk of their lives back, and allowing them to focus on value-added projects – instead of another year of moving data around.

One of the attractive promises made for new software-defined storage (SDS) is its elimination of the dreaded process of data migration. So can SDS really remove data migration from the storage management equation?

Eliminating data migrations – with Dell EMC ScaleIO

With Dell EMC ScaleIO software-defined block storage you really can transform your approach. In a similar way that VMware virtualized the compute operations of the datacenter 15 years ago, ScaleIO today abstracts and pools local storage resources in each server, and automates provisioning of capacity and performance back to applications on those servers.

Your shared pool of ScaleIO storage is physically dissociated into distributed software components running on industry-standard x86 server hardware. This software-defined approach helps make data movement between storage painless and automatic.

To extend the analogy with virtualization, think about how an admin does a ‘slow roll’ when upgrading a compute cluster. There is no wholesale VM migration or replacement of the cluster – it’s an ongoing perpetual process, performed a server at a time as needed, and requiring no downtime or impact to the hosted applications. There are also specific tools in place (from the hypervisor) to execute and automate this as a seamless and transparent background process.

Guess what? ScaleIO achieves a very similar transformation for storage!


A seamless, perpetual process

Of course, data movement still happens within ScaleIO – but we like to think it’s nothing like ‘data migration’ as you’ve previously known it. And much of the time, you won’t even notice it’s happening.

The processes of balancing in a new node and evicting an old node are all seamless to your applications. When a new server node is added to the cluster, an automatic data rebalance occurs, spreading some data from each existing server node across to it. Similarly, when the storage admin decides to ultimately decommission a server node, it is removed by issuing a command that automatically redistributes its data across the rest of the servers in the cluster. This is all completely transparent to your applications.

With ScaleIO, maintaining and upgrading storage becomes a painless process – rather than the old-style painful epic project. This is basically the model of storage management pioneered by web-scale companies like Facebook, Google, and Amazon. They had to overcome the challenges of their huge data environments, running mission critical apps that cannot afford any downtime, and where wholesale migrations are not feasible. They achieved this by moving from arrays to home-grown software-defined infrastructure running on top of many x86 servers. ScaleIO is an enterprise-grade, commercially supported, and standards-based solution that follows similar operational principles.

It’s automatic, it’s programmatic – it’s ScaleIO greased lightning!

Organizations that are making the best use of software-defined technologies like ScaleIO tend to be highly automated in their IT. They are already using software automation tools for deployment of servers, OSs and applications. With ScaleIO, storage provisioning and deployment is automated in the same framework – programmatically.

You can simply decide when it’s time to expand your storage and do it in granular steps, one x86 server at a time. Similarly, when a server is old and no longer desired, you can decide to evict it from the cluster, one server at a time, on-demand. ScaleIO enables the concept of a perpetual rolling upgrade, so you maintain optimum efficiency and performance across all your storage – without the headaches of traditional data migration.

With ScaleIO, it doesn’t really matter what specific kind of server is attached to the storage cluster – whether it’s VMware, Hyper-V, bare-metal Linux, or whatever. There’s no need to worry about operating any particular environment. ScaleIO takes a platform-agnostic, heterogeneous approach. It enables the same processes, regardless of your OS or applications.

Say goodbye to data migration challenges with ScaleIO

We think you might enjoy a world without data migration headaches. So why not explore the advantages of Dell EMC ScaleIO software-defined block storage for yourself?

Dell EMC ScaleIO software is available to download and try for free.

Learn more about Dell EMC ScaleIO advantages for your datacenter.

Breakfast with ECS: ECS and Centera – Optimizing the Data Archive

Welcome to another edition of Breakfast with ECS, a series where we take a look at issues related to cloud storage and ECS (Elastic Cloud Storage), EMC’s cloud-scale storage platform.

Breakfast wtih ECSWhen Centera was introduced in 2002, it promised long term storage of a company’s archives with an architecture that completely separated the application access method from the storage infrastructure.  From the beginning, EMC has always promised to maintain the Centera data, the APIs that are used to access it, and the applications that have integrated to those APIs, with the full knowledge that the lifecycle of the Centera data would exceed the lifecycle of any individual HW or SW platform.  The Centera SW architecture has fulfilled this promise across multiple hardware generations and across multiple disk drive variants, and with ECS 2.2, the promise is being extended even across SW architectures.

Why a new architecture?

The state of the industry has changed dramatically from when Centera was originally designed and built.  Scale-out, distributed storage accessed via an IP network is no longer a novel concept, and increases in network speeds, CPU power, disk density, and memory sizes mean that the Centera model of a small number of disks per node is no longer economically feasible.

ECS is a new software architecture which was designed from the ground up to meet the needs of the modern data center, and with the ECS 2.2 release, it not only has full support for the Centera API, it has built in capabilities to automatically index and migrate content from a Centera cluster to the ECS, all in a non-disruptive fashion, maintaining full application access throughout the process.

But ECS does far more than just support the Centera use cases, ECS adds additional capabilities and efficiencies to the storage of Centera data and can even apply these to enhance the value of existing Centera data, as well as newly written data.  These capabilities include:

  • Erasure coding protection for all data, including clips, in addition to a greater variety of choice in the protection scheme to best meet the needs of the individual situation. ECS can protect data with 20% or 33% overhead and can apply this protection equally for large and small objects, all the way down to 4 node systems, for far better storage efficiencies than Centera.
  • Protection against multiple disk failures. While Centera offered protection only against single disk failures, ECS protects against the simultaneous failure of 2 disks (20% overhead) or 4 disks (33% overhead).
  • With the introduction of Data at Rest Encryption in ECS 2.2, ECS can automatically and transparently encrypt data before storing it, adding an additional level of protection for your most sensitive data.
  • ECS provides site failure protection for less than 2x the original data size in a three or more site environment, unlike Centera which would need minimally 2.3x (4x if using mirroring) data storage for a similar level of protection.
  • ECS offers far denser configurations, with up to 60 drives per node, for far more economical data storage than Centera.

In addition, ECS preserves Centera’s compliance capabilities, including fixed retention periods and retention classes, allowing all Basic, GE, and CE+ data from Centera to be safely migrated to ECS.

Read the Top Reasons Handout on why you should choose EMC Elastic Cloud Storage (ECS) for archiving.

How do I move from Centera to ECS?

There are two ways to start leveraging the benefits of ECS for Centera customers – either by leveraging an external migration tool, of which there are several choices available now, or by leveraging the built-in transformation capabilities in ECS 2.2.

The transformation capabilities of ECS 2.2 fully automate the migration process – the Centera is added to the ECS cluster and the application directs its reads and writes to the ECS – the ECS enumerates the Centera, creates an ECS-based index for that content, migrates the content, then performs a final reconciliation to validate that all data has been correctly migrated.  The entire process is fully automated, driven by ECS, and transparent to the application.

Of course, ECS is capable of supporting workloads far different than the typical Centera workload.  ECS is suited for modern S3 or Swift based applications, for analytics workloads, for web-based application storage, for Internet of Things applications, and much more.  ECS 2.2 brings the benefits of a modern software architecture to the existing Centera data while simultaneously supporting modern application workloads and allowing a single software/hardware environment to satisfy the full set of scalable storage needs.

Datadobi and EMC: Fast, Reliable Data Migrations

Michael Jack

Global Sales Director at Datadobi

Latest posts by Michael Jack (see all)

*The following is a guest post by Michael Jack, Global Sales Director at Datadobi.

Over the past 13 years, EMC Centera has – and continues to be – the most reliable object storage and compliant archiving platform in the industry.  While EMC continues to sell and support Centera, customers are also looking at migration strategies to gain new cloud scale features and automation services.

That’s where Datadobi can help.  Datadobi is an EMC Technology Connect business partner that offers DobiMiner, our migration software that migrates Centera customers to other storage platforms. In this blog, we’ll discuss Datadobi’s unparalleled migration experience, and we’ll share considerations and strategies on how to migrate Centera data to EMC Elastic Cloud Storage (ECS).Datadobi_blog_image

Centera is one of the most stable platforms in EMC’s storage platform portfolio providing 6 nines of availability. It also has an excellent feature set including the coveted Compliance Edition Plus model used by financial institutions around the world to ensure the immutability of their data. Centera created a new paradigm in the storage industry and set the standard for Object Storage that no other vendor has yet to achieve.

While many use cases for Centera remain strong, the move to next generation distributed cloud, mobility, and Big Data applications are driving many Centera customers to take advantage of the next generation object storage. And with many companies having experienced the cost, risk, and time associated with moving from one platform to another, they are rightly asking ‘how do we move all that data without all that pain?’

So how do you move CAS data? Many EMC customers have experience with NAS migrations but have no idea how to move CAS data. To answer that question, let’s take a brief look at how CAS works. CAS data sits in a flat file space without a classic file structure and accessing it requires having the key known as the Content Address or CA on Centera. When an application writes data (a user file) to a Centera, Centera creates a Content Descriptor file (CDF) and places the CA of the user file in the CDF. It then creates a CA for the CDF and passes that to the application, which stores this in its database. When the application needs to retrieve the file again, it passes the CA of the CDF to the Centera, which looks in the CDF to find the CA of the user file. All pretty complex stuff which is what makes Centera such a great product.

To migrate a CAS file to another platform (be it CAS, NAS, RESTful, OpenStack, etc.) you can’t use a tool such as Robocopy because you have to use the CA to locate the file. You can only achieve this by using:

  • The application that wrote the data or
  • A specialized migration software such as DobiMiner

You may think migrating the data using the application that wrote the data would be the easiest way; however, this is not the case. Applications migrate data by re-reading it from the Centera and then re-writing it to the new platform. Most applications are not designed for high-speed read and write. As a result, migrations done in this way tend to be very lengthy affairs resulting in additional expenses such as professional services and extended support contracts for the old platform.

DobiMiner takes a different approach – it quickly collects the list of data to be migrated directly from the Centera by parsing the CDFs information to the DobiMiner instance. Because the CDFs contain the CA of the user file and the file sizes the entire scope of the migration is completely understood. This allows the migration team to make informed decisions and have a predictable finish date before actually migrating the data. Once the scope is agreed, the data can be rapidly pumped to the new platform at Terabytes a day.

Once the data has been copied to the new platform, DobiMiner reads back each individual file and compares it with the same file on the old platform to validate its authenticity before declaring it fully migrated.

Mining all the CDFs also enables you to have a complete understanding of your data before migrating. For example, you’ll know whether the data is still recognized by the application, has the data expired but not been deleted, and are the replicated environments in sync? All this information enables you to migrate only the data (and all the data) with true business value ensuring the best ROI on the target platform.

CAS-to-non-CAS – a slightly different kettle of fish

Where all of the above relates to both CAS-to-CAS migrations (such as Centera-to-Centera and Centera-to-ECS) and CAS-to-non-CAS migrations (such as Centera-to-NAS or a RESTful interface), moving data from a CAS to a non-CAS platform can involve additional complexity.

Applications write data to a Centera in one of two ways:

  • Through a file-system gateway or
  • Natively through the Centera API

Some applications use a file-system gateway (such as the Centera Universal Archive) to write the data to the Centera and it is the file-system gateway that stores the CAs not the application. The application database stores a file path making migrating that data to a non-CAS platform straightforward. The file path known to the application can be duplicated on the new platform making the migration transparent to the application.

On the other hand, applications using the API store the list of CAs in their database and therefore require an additional step in the migration process. Firstly, a file path naming scheme must be agreed for the new platform. A combination of fields in the CDF can be used for this or DobiMiner can create a path itself. Secondly, the list of CAs in the application database must be replaced with the new file path so that the application can access the files after the migration. This database update can occur in one of a number of ways:

  • Fully automated – the migration software automatically connects to the application database and overwrites the CA with the new file path.
  • Handshake – the migration software creates a file that maps the old CA to the new file path and this mapping is manually used by the application vendor to update the application database.

Unfortunately, some application vendors do not allow either method and only allow migrations through the application itself (making for a lengthy migration).

In conclusion, migrating Centera to another platform can seem like a pretty daunting process – it really isn’t. With DobiMiner, most of the difficult tasks are automated to ensure a simple, low risk, predictable, and fast migration.

Avoid the pain. Take advantage of Datadobi and our partners’ expertise to perform your next migration and let your team focus on your business.

Data Migrations: Seven10 and EMC Revolutionize the Industry Paradigm

Bobby Moulton

President & CEO of Seven10 Storage Software

Latest posts by Bobby Moulton (see all)

*The following is a guest blog post by Bobby Moulton, President & CEO of Seven10 Storage Software, a leading developer of cloud-based information lifecycle management (ILM) and data migration software.

An assertive headline is essential for a bold undertakingMigrate to cloud that forever changes how data is moved from old storage to new.  A few years ago, Seven10 set out to transform how users, vendors, and application providers consider file and storage migrations.  It started with a customer challenge: move critical data off proprietary hardware over to new storage without interrupting the patient care process – and resulted in the Storfirst simple, trusted,  data migration platform.

Seven10 searched the industry and was surprised at the lack of innovation.  Where was the automation?  Where was the vision?  Where was the hands-off, ‘we make it so easy you can do it yourself’ innovation?  It seemed that some were busy developing next-generation SaaS, Big Data, IoT, or cloud based offerings because they weren’t working on a data migration solution.

Seven10 Storfirst was Born.
So Seven10 stepped up to the plate.  We focused on customer-driven migrations that were highly automated, supremely reliable, and ridiculously cost effective.  We tossed the PS-lead blue-print and created a new 100% software-driven model.

From day one, Seven10’s Storfirst software seamlessly transitions data from the widest range of legacy storage environments, including EMC Centera, NetApp StorageGrid, HP MAS, IBM GMAS, Oracle SAMFS – as well as any existing NAS platform, cloud gateway or file system.  In addition to data migration capabilities, Storfirst is the only solution offering a standard SMB/CIFS or NFS presentation layer for immediate access into EMC platforms such as ECS.

Why Migrate Data to EMC ECS?
EMC’s Elastic Cloud Storage (ECS) software-defined cloud storage platform combines the cost advantages of commodity infrastructure with the reliability, availability and serviceability of traditional storage arrays.  ECS delivers protocol support for Object and HDFS – all within a single storage platform. Seven10’s Storfirst Gateway allows EMC customers to quickly decommission legacy storage devices while simultaneously modernizing their infrastructure with the adoption of ECS.

How Seven10 Storfirst Gateway Works:
Seven10 offers migration PLUS go-forward data management – all without breaking the bank or interrupting day-to-day operations.  Seven10 changed the paradigm from a resource intensive, PS-led effort, to a repeatable, software-driven five-step migration process:

1. Inventory – Storfirst “ingests” existing file system as read-only and configures new storage or storage tiers under a single managed share.

2. Sync – While providing uninterrupted access to legacy data and writing to new storage, Storfirst copies all legacy data onto new storage or storage tiers.

3. Verify – Using an MD5 hashing algorithm for data verification, Storfirst delivers zero risk of data loss migration.

4. Audit – Storfirst provides a detailed logging capability in order to conduct a file-by-file comparison on the new storage to ensure all data has been copied without any loss.

5. Decommission – Once the migration is complete, the application communicates to the new storage platform while the legacy storage is decommissioned and removed from the environment.

Thanks to the longstanding Technology Connect Select Partnership of EMC and Seven10, organizations retire and/or refresh their storage architecture with software-driven, secure data migrations that guarantee zero data loss.  Storfirst meets compliance regulations by enforcing policies such as: encryption, file-locking, lifespan, auditing, and tiering.  Industries from healthcare to financial services and from manufacturing to government now have the answer to the data migration challenge.

Seven10 and EMC Ease Customer Stress with Safe, Trusted, Proven Migration Solutions

For Allegiance Health, Storfirst seamlessly migrates critical files to ECS.  Due to long-term reliability concerns with the existing NetApp StorageGRID, Allegiance selected Storfirst to migrate millions of electronic records off NetApp and over to ECS.  This all-in-one solution includes optimum storage with a built-in migration path and a 100% auditable transition to ECS – all while delivering Allegiance uninterrupted access to their patient files.

“The combined offering from EMC and Seven10 provides Allegiance Health with an easy and safe migration solution for moving and protecting our critical patient data.  Seven10’s Storfirst migration and management software is very robust, allowing us to quickly and easily adopt the EMC cloud storage platform,” said Allegiance Health’s Information Systems Vice President and Chief Information Officer Aaron Wootton.

It’s clear, the question is not if companies migrate their data, but rather how they complete the migration.  Understanding the features, advantages and benefits of the options is essential.  Through a well-defined, proven, best-of-breed technology partnership with real-world applications, Seven10 and EMC redefine the industry paradigm.



Connect with us on Twitter