Why Purpose-Built Storage Still Rules Over “Unified Storage”: How ScaleIO Spanks Ceph on Performance

Randy Bias

Randy Bias

VP Technology at EMC

Collectively it’s clear that we’ve all had it with the cost of storage, particularly the cost to maintain and operate storage systems.  The problem is that data requirements, both in terms of capacity and IOPS are exploding and growing exponentially, while the cost of storage operations and management is growing proportionally to those data needs.  Historically the biggest culprit is “storage sprawl” where we have pairs of arrays throughout the datacenter, each of which has to be managed individually.  Silo after silo, requiring specialized training, its own HA and resiliency, monitoring, and so on.  It’s for this reason that many turned to so-called “unified storage.”  This, unfortunately, is a terrible idea for larger deployments and those running production systems.

Let me explain.

The Storage Unicorn & a Rational Solution
We all want what we can’t have: a single globally distributed, unified storage system, that is infinitely scalable, easy to manage, replicated between datacenters and serves block devices, file systems, and object, all without hiccups.  Bonus points if you throw in tape as well!  This is the Storage Unicorn.  No such beast exists and never will.  Even before the EMC acquisition of Cloudscaling I was talking about these issues in my white paper: The Case for Tiered Storage in Private Clouds.

The nut of that white paper is that tier-1, mission critical storage, is optimized for IOPS, while tier-3 is optimized for long term durability.  Think of flash vs. tape.  These are not the same technologies nor can they serve the same purpose or use cases.

A multi-purpose tool is great in a pinch, but if you need to do real work, you need a purpose-built tool:
Multi Purpose Purpose Built

The issue remains though; how do we reduce storage management costs and manage the scaling of storage in a rational manner?  There is no doubt, for example, that the multi-purpose tool is lower overhead.  It’s simply less things to manage. That is a pro and a con.

The challenge then is to walk away from the Unicorn.  You can’t have a single storage system to solve all of your woes.  However, you probably don’t have to live with tens or hundreds of storage systems either. Continue reading

3 Ways Scalable Technology is Enabling Media & Broadcast Companies

Yasir Yousuff

Yasir Yousuff

Sr. Director, Global Geo Marketing at EMC Emerging Technologies Division

We want great content, 3 ways scalable techand we want it now. We are a generation used to instant gratification, and technology has only served to amplify the need to be constantly connected on the go. This has created a unique broadcasting market dynamic in many developing countries – particularly in the Asia Pacific—the era of the Connected Consumer. According to International Data Corporation (IDC) in an article from OnScreen Asia, mobility is crucial to the lives of many consumers and the survival of businesses in Asia Pacific region, due to the combination of a fast growing economy and a lack of fixed infrastructure in developing countries. Broadcast companies must adapt to suit changing consumer preferences or risk becoming irrelevant.

How can businesses leverage mobile device adoption and utilize it to engage a fragmented audience?

Future-proofing—Staying Three Steps Ahead
Multi-platform content delivery is an industry trend broadcast companies are starting to embrace as a new way to engage consumers, monetizing those engagements, and better managing the user experience. This is to be expected especially with the rise of on-demand video platforms, consumer mobility, and the proliferation of smartphones and tablets. New media services deliver web content to audiences through live streaming, as well as on-demand services via PCs and mobile devices.

This converged development of traditional and new media services has led to transmedia storytelling—the development of stories across multiple forms of media to deliver unique pieces of content across each channel. These pieces of sequential content work together to form a bigger picture, serving the purpose of reaching a wider target market. Therefore, the move to IP-based infrastructure becomes increasingly important as broadcast technology buyers continue to look for ways to make broadcast operational environments more efficient.

Broadcast companies, such as Zhejiang Radio and Television Group (ZRTG) in China, are focusing on enhancing the customer experience using intelligent selection to enhance content delivery. Selecting IP storage with EMC Isilon was a forward-thinking decision ZRTG made because it met the long-term performance requirements to access and edit media assets for high quality broadcast content. According to Luo Leiyi, Sector Chief at ZRTG, “One of the most important features of the EMC Isilon solution is the modular scalability, because as a broadcaster we need storage capacity to be quickly added as our programming and content services expand across multiple formats.”

Going Harder, Better, Faster, Stronger
In recent times, efficiency has been a key motivation of broadcast technology purchasing. Increased operational efficiency and cost savings have proven to be considerably more significant than cutting-edge technology, according to Devoncraft’s 2014 Big Broadcast Survey. We have seen this with production companies moving to file-based or tapeless workflows—choosing to entirely digitize their content, consolidate silos, as well as streamline processes. This centralized data management and retrieval lowers migration workloads between core online facilities and secondary storage. To put it simply, content is created faster and at a lower cost, thanks to improved management and application efficiency.

Post-production firm, Adnet Global prides itself on same-day delivery for most of their services with EMC Isilon. To achieve this speed, large file and dataset transfers are spread over the EMC Isilon nodes, enabling massive gigabit per second throughput and the exchange of files within a predictable time. Nivas Patil, Senior Manager of Technology at Adnet Global explains, “The files for production are immediately available, which means we have more time to complete each job. Due to the performance, scalability, and high availability of the EMC Isilon storage, we have definitely increased our productivity.”

Time is Money
We all know time is money, and this is even more pertinent in the fast paced broadcast industry. As rich-media file sizes continue to increase and deadlines get shorter, production companies need to evolve the workflow to decrease time-to-market as well as reduce media production costs. There can be no time for downtime, and high systems availability is crucial. Downtime can be very expensive when you are Rising Sun Pictures (RSP), an Australia-based visual effects company which has worked on high profile feature films such as Gravity and X-Men: Days of Future Past.

Mark Day, Head of Systems at RSP comments, “If a single node goes offline, there’s an automatic failover to other nodes in the Isilon cluster. Every piece of work we do is complex. Our software is demanding on storage because we produce more than a terabyte of data per hour. But we’re very confident that our valuable creative work and intellectual property is protected by Isilon.”

 

Live Long and Store Data: From Star Trek to the Internet of Things

Robert "Bobski" Masson

Robert "Bobski" Masson

Product Marketing Manager at EMC

The Internet of Things [IoT] is likely to be the Internet’s next big tidal wave and has the potential to impact how we live and interact with the world around us.

Hype, aggrandizement or timing aside, the market will be significant.  Forecasts for Internet of Things vary widely; with IDC’s Worldwide Internet of Things Forecast, 2015–2020 projecting an $8.9 trillion dollar market by 2020 while Gartner is more conservative at a $1.7 trillion dollar market and Bain projecting $1.2 trillion by 2017.  Truthfully like all new waves, IoT will probably not resemble the descriptions of today’s market analysts but the impact will be felt in how we prepare for the tidal wave.

The big questions in this whole Internet of Things puzzle revolve around data collection and storage.  How are we going to interact with the data and collect it and how is the data going to be stored in a meaningful way to make sense of all the data points.

Do you know what this is?

TricorderThis is an image of a Tricorder from the fictional television series Star Trek.  There were primarily three distinct versions used throughout the series but in general these multifunction hand-held devices were used for sensor scanning, data analysis, and recording data.   Captain Kirk or Spock could walk up to just about anything or anybody and with a push of a button get detailed information about the “thing” that was in front of them: medical and health information, engineering details of the ships computer system and whether the creature in front of them was friend or foe.

Today your iPhone or android device in your pocket is your Tricorder.  Think about that.  Today you have apps that monitor your health, track your workout, and control the heat in your house.  Soon you will be able to walk up to just about anything or anyone and with a push of button get detailed information on the spot.  Imagine walking into a restaurant and having the ability to learn about the ingredients in your meal, where the food was grown and how it arrived at the restaurant.  We are talking about the ability to do analytics on data in real-time and this will fundamentally change the way we interact and collect data.

The iPhone “Tricorder” answers the first question around interaction and data collection but what about the data storage?

If you follow along with the Star Trek theme there was a concept of “Ultimate Computer” in one of the episodes.  The ultimate computer system was built to handle all the ships functions without human assistance by gathering information through sensors and analyzing data to make decisions.   The computer system stores data regardless of whether the information came from legacy systems, Tricorder or even from new foreign systems.  It grew and learned through interactions.   It was essentially a data lake that collected the information and allowed for real-time analytics to make better decisions.  Think about that for our case today.

Fast forward to 2015 and the ultimate computer system is called EMC ECS, the scale-out object platform that is the foundation to our IoT data lake story.  EMC solves the question of “how is the data going to be stored in such a way that we can make sense of it?” by providing a geo-distributed data lake approach to collecting IoT information—perhaps billions of objects— regardless of where the data comes from. The geo-distributed data lake approach allows for Cloud apps, mobile devices, and analytics all to reside and coexist with legacy systems.

Just like in the case of Star Trek, you will have a system in place that will grow with you no matter what new connectivity the Internet of Things will bring.  The EMC data lake will grow and learn with you. ECS IOT

So here’s to jumping into the deep end of the data lake with EMC and embracing the new wave coming from the Internet of Things.

Live Long & Store Data

Star Trek Data

IT Operations Management Insights From the United Airlines, NYSE and WSJ Outages

Brian Lett

Brian Lett

Senior Product Marketing Manager

In rapid-fire succession on 8 July 2015, United Airlines, the New York Stock Exchange (NYSE), and the Wall Street Journal (WSJ) website experienced extremely high-profile outages.

The culprits? For United Airlines and the IT OperationsNYSE, it was the network (routers and gateways, respectively); for the WSJ, it was the combination of insufficient capacity for unanticipated demand, and a failure of server connections (a 504 error).

Key factors in these events were network devices, outages, downtime, service impact, configurations, and capacity –the realm of monitoring and management of IT infrastructure used to deliver applications and services. Yet most of what I’ve read sensationalized the nonexistent cybersecurity angle. So I’m highlighting some key IT operations management insights to take away from incidents such as these:

Any assumption that could impact ongoing business and IT operations should be considered wrong. Felix Unger said it best: Never assume.

The United Airlines router outage impacted applications and services that ultimately grounded all planes. An IT operator easily could have assumed in such a scenario that he or she would see a relevant alert generated by a faulty device. But what if the bad router effectively imploded before getting a chance to deliver its “suicide note”? Then the operator has to know to look for events that should have been generated but weren’t. In my experience, the kind of out-of-the-box thinking needed to immediately deduce this type of problem is extremely rare – especially in a major-outage scenario, in which significant pressure exists to solve the problem ASAP, and everyone presumes all relevant data is there, but it just hasn’t (yet) been properly analyzed into something insightful. Continue reading

Closing the Gap between the 4K Dream & Reality

Charles Sevior

Charles Sevior

Chief Technology Officer at EMC Emerging Technologies Division

Television has come a long way since video formats such as NTSC, PAL, VHS and Betamax – the transition of digital video from standard definition (SD) to high definition (HD), and now most recently Ultra HD or 4K.

What is 4K and what does it mean for professional broadcasters?

With quadruple the 4K Broadcastingresolution of HD (1920 x 1080 pixels), the first noticeable benefit of 4K (3840 x 2160 pixels) is the improved picture sharpness. However, this brings new challenges to the broadcast workflow such as increased storage capacity, processing power, delivery bandwidth, and rendering time. Broadcast and production companies are on the hunt for solutions to seamlessly transition from HD to 4K. For the 4K dream to become a reality, an industry-wide shift in consumer, manufacturer, and content creator habits has to happen. Continue reading

New Enhancements to EMC Isilon Data Lake Foundation

Suresh Sathyamurthy

Suresh Sathyamurthy

Sr. Director, Product Marketing & Communications

Today, EMC® is pleased to announce the availability of new EMC Isilon® products that further extend the advantage that the industry-leading Isilon scale-out data lake and NAS storage platform provides to enterprise customers across a broad range of industry segments and public sector agencies. The new EMC Isilon products include:

  • EMC Isilon OneFS 7.2.1: This new version of the EMC Isilon OneFS® operating system provides new security options including U.S. Department of Defense Security and Technical Implementation Guide (STIG) hardening, FIPS 140-2 validated OpenSSL and PIV/CAC support for Smart Card Authentication. OneFS 7.2.1 also offers new options to increase flexibility and simplify management including improved support for IPv6 networking; enhanced node compatibility between older and newer generation of Isilon storage platforms; and rolling upgrades from OneFS 7.1.x and OneFS 7.2.
  • EMC Isilon X210: The new Isilon X210 is a platform refresh of the Isilon X200 and provides increased capacity, performance and flexibility. The Isilon X210 offers a new 4TB HDD option that expand the maximum node capacity by 33% from 36 TB to 48 TB in a highly efficient 2U platform. Based on SPECsfs testing, the Isilon X210 with OneFS 7.2.1 delivers a performance increase of more than 30% over the Isilon X200. To increase data center flexibility, the Isilon X210 uses QDR InfiniBand networking to allow cable lengths between nodes of up to 100 meters. And to further simplify management and improve flexibility in expanding existing Isilon storage environments, the Isilon X210 offers node equivalence with the Isilon X200. The highly versatile Isilon X210 is an ideal scale-out NAS storage platform for a wide range of applications and workloads including file shares, home directories and big data analytics.
  • EMC Isilon NL410: The new Isilon NL410 is a platform refresh of the Isilon NL400 and provides increased flexibility and simplicity. The NL410 includes QDR InfiniBand networking to allow cable lengths between nodes of up to 100 meters. The Isilon NL410 also offers node equivalence with the Isilon NL400 to provide greater flexibility in expanding existing Isilon storage environments. The EMC Isilon NL410 is designed to provide cost-effective, highly scalable nearline storage and is an ideal solution for active archiving.
  • EMC Isilon InsightIQ 3.2: This new version of Isilon InsightIQ software provides powerful performance monitoring and reporting tools to help organizations optimize the performance of their Isilon storage systems. New features now available with Isilon InsightIQ 3.2 include improved analytics and pool-centric reporting options to provide granular performance data filtered by node tiers or node pools.

Here is a snapshot of the latest portfolio of EMC Isilon storage platforms, including the new Isilon X210 and NL410:

EMC Isilon Scale-Out NAS Product Family

Isilon Product FamilyWith these new products, EMC Isilon scale-out Data Lake further extends its leadership as an ideal platform to store, manage and protect unstructured data efficiently while supporting a wide range of applications and workloads. Key advantages of EMC Isilon include:

  • Simplified management: Single file system, single volume, global namespace
  • Massively scalable: Scales from 16 TB to over 50 PB in a single cluster
  • Unmatched efficiency: Over 80% storage utilization with automated tiering and data deduplication options
  • Enterprise data protection: Efficient backup and disaster recovery, and N+1 thru N+4 redundancy
  • Robust security and compliance options: RBAC, Access Zones,  WORM data security, File System Auditing, Data At Rest Encryption with SEDs, STIG hardening, CAC/PIV Smartcard authentication, FIPS OpenSSL support
  • Operational flexibility: Multi-protocol support including NFS, SMB, HTTP, FTP and HDFS, Object and Cloud computing including OpenStack Swift

To learn more how EMC Isilon scale-out storage solutions can benefit your organization, contact your EMC sales representative or authorized reseller.  Also be sure to see our solutions in the EMC Store at https://store.emc.com/isilon.

ScaleIO: When More is Better…and Less is More!

Rodger Burkley

Rodger Burkley

Principal Product Marketing Manager at EMC

By now, you’ve no doubt heard how Software Defined Storage (SDS) and hyper-converged server SANs are reshaping the storage industry (and use case) landscape.  Market acceptance continues to grow at an accelerated pace.  There are now many established and new vendors offering hyper-converged storage appliances and SDS systems.   EMC’s own ScaleIO SDS platform continues to score mounting recognition, kudos and accolades with our growing list of customers.Hyperconverged  ScaleIO has even won-over storage techies and IT Management who were initially skeptical that ScaleIO would actually completely deliver on its value proposition and capabilities.

Why all the buzz and hype?  Customers – and the IT datacenter and storage market — get it and are embracing this new, “disruptive” technology by introducing it to their enterprise data centers or using it to create hyper-scalable virtualized infrastructures for cloud applications.  The promise and appeal of installing software on individual commodity host application servers to create a virtual storage pool (i.e., “server SAN”) from each participating server’s excess direct attached storage (DAS) without requiring additional specialized storage/fabric hardware is alluring…and almost too good to be true.  High Availability, commodity based hardware, linear performance and storage capacity scalability, “SAN-less” data storage, lower TCO, elasticity and flexibility are just a few compelling arguments in favor of hyper-converged architectures in general and SDS system in particular. Continue reading

Why you want a Witness to Disaster Recovery

Bob Williamsen

Bob Williamsen

Sr. Business Development Manager at EMC

How to protect data and be prepared to survive a disaster is a big topic to cover. Obviously data protection has a lot of components—ranging from hardware capabilities, algorithms used by the system to protect data and ensure integrity, and concerns like backup and replication strategies. In this note let’s mercifully contain this discussion to what is arguably one of the most important topics – surviving a disaster with minimal disruption to your business.

Disaster Recovery, DR, is a necessary consideration when designing robust enterprise solutions. By definition, DR always implies physical distance between the systems involved in the design. Isilon supports data replication over distance and the components of an Isilon DR solution include:

  1. Application Servers
  2. LAN connecting the application servers to the Isilon storage
  3. Isilon cluster at the production data center
  4. Isilon cluster at the DR data center
  5. A WAN connecting the two data centers
  6. Isilon SyncIQ for data replication

 

Ok, now the data is automatically replicated—but what happens in the event of a DR fail-over?  It requires manual intervention or scripts to make the data on the DR site available to users and applications. What is missing is a way to orchestrate and automate the fail-over process so that the business magically recovers from the loss of the primary site.

Continue reading

EMC {code} shares how to deploy ECS with Five Ways of Docker

Kendrick Coleman

Kendrick Coleman

Developer Advocate at EMC

We’ve been in a lot of conversations about DevOps recently, with customers, partners, community members and EMC teams. Whether or not you believe in the buzz around DevOps, there is definitively a wave of new and open tools, proclogo-emc-codeesses, and operational models being used in IT. These are all trends that can’t be ignored, and we’re continuously working to make sure EMC’s product strategy adapts with these changes.

EMC {code} is a developer evangelism team within EMC. We contribute to major open source projects, create our own tools and projects (also entirely in the open– in fact– you can find them all on GitHub, here), and engage with developer communities. Several of our team members have worked with EMC SDS products over the past few years, and we’ve enjoyed seeing the progress made to make our SDS tools readily accessible for developers and IT/ops alike.

ECS has long been a developer-friendly product, with universal protocols like object and HFDS, and a software-only download option. The recently announced ECS 2.0 is chock full of updates that are making the lives of dev/ops teams easier—including enhanced geo-caching, multi-tenancy capabilities, monitoring and reporting, and the ability to automatically and rapidly failover and recover from outages.

ECS 2.0 only enhances the experience for developers with a containerized download, free for test-and-dev environments (non-production use today). This has been another milestone in providing free and frictionless access to EMC software (along with the open-sourced CoprHD and free download of ScaleIO).

Our team works with Docker containers on a regular basis, and when ECS was announced, we rolled up our sleeves to see exactly how we could integrate Docker tools into the ECS experience.

In addition to the containerized download, we wanted to explore ways of deploying ECS with broader Docker technology. Deploying multi-node ECS is fast and easy with Docker tools. Using Docker Machine, you can deploy Ubuntu hosts that will be a part of a Docker Swarm cluster. With Docker Compose, you can deploy ECS from Docker Hub to a Docker Engine container on each host in the Docker Swarm cluster. If you are interested in how to do this, you can check out the GitHub repo or watch this quick video.

With this automated process, everything will be complete and configured in the span of 10-15 minutes. To get all of the details, please visit the EMC {code} blog post on the topic, or send us a tweet with your feedback to @EMCcode.

A New Blueprint for Success: EMC VSPEX with Isilon

Carl Washington

Carl Washington

Sr. Business Development Manager at EMC

It’s been two years since EMC’s VCE and Isilon teams joined forces to begin offering converged infrastructure (CI) solutions that combine VCE Vblock Systems with Isilon scale-out NAS in support of a range of applications and workloads. And we couldn’t be more excited about the results. It’s extremely gratifying to hear about the many customers who are now realizing the significant advantages that these CI solutions deliver: accelerated time to value, management simplicity, increased efficiency, single point of contact for support and the knowledge that they are getting a tested and proven solution.

Enablement VSPEX Proven Infrastructures are modular, virtualized solutions validated by EMC and delivered by EMC VSPEX partners. They include virtualization, server, network, storage, and backup layers.

Enablement VSPEX Proven Infrastructures are modular, virtualized solutions validated by EMC and delivered by EMC VSPEX partners. They include virtualization, server, network, storage, and backup layers.

Building on this success, EMC converged infrastructure solutions that incorporate EMC Isilon scale-out NAS have recently been extended to include integration with EMC’s VSPEX program. As a result, Isilon storage is now featured in a new VSPEX reference architecture:  EMC VSPEX Proven Infrastructure with Isilon Scale-Out NAS.

VSPEX Proven Infrastructure with EMC Isilon combines the simplicity, efficiency, and flexibility of VSPEX coupled with Isilon to deliver improved storage utilization, IT productivity, and increased end-user productivity. This new VSPEX reference architecture provides a blueprint for building a proven converged infrastructure that can support a number of use cases that most medium to large organizations are likely to need including:

  • File Shares
  • Home Directories for End-User Computing
  • Active Archiving

The VSPEX reference architecture also includes the necessary details for building a proven converged infrastructure for businesses in the Media & Entertainment sector.

With this new VSPEX and Isilon reference architecture, organizations can gain the following advantages:

  • Application-specific workload sizing, design and implementation guidance
  • Accelerated time to value
  • Leverage of VSPEX ecosystem partners for application flexibility and support

Built for virtualized environments, VSPEX enables faster deployment, more simplicity, greater choice, higher efficiency, and lower risk. VSPEX with Isilon can be added to existing VSPEX environments or be deployed as a stand-alone solution to consolidate storage to lower costs and optimize today’s 2nd Platform applications while providing a foundation for next-generation 3rd Platform applications. This is also a great first step in building a highly versatile data lake foundation to store, manage, and protect Big Data.

Testing and validation, outlined in a VSPEX Proven Infrastructure Guide, coupled with the VSPEX Sizing Tool, simplifies solution sizing and eliminates the risk and expense of over-provisioning. VSPEX with Isilon further simplifies and refines the sizing process by leveraging Isilon’s Predictive Performance Planning (P3) sizing tool functionality to enable efficient solution sizing for a broad range of file data workloads.

Of course, all these solutions are based on Isilon storage which provides a number of important benefits:

  • Simple management – with Isilon’s single file system, single volume architecture
  • Massive scalability – easily expands up to 50 PB in a single cluster
  • Operational flexibility – multi-protocol support for a wide range of applications and workloads on one platform
  • Unmatched efficiency – with over 80% storage utilization
  • Robust data protection and security options

By leveraging the advantages of EMC VSPEX Proven Infrastructure with EMC Isilon scale-out NAS, organizations do indeed have a new blueprint for success!

For more information on how this solution may benefit your organization, I suggest that you contact your authorized EMC reseller or local representative.