Closing the Gap between the 4K Dream & Reality

Charles Sevior

Charles Sevior

Chief Technology Officer at EMC Emerging Technologies Division

Television has come a long way since video formats such as NTSC, PAL, VHS and Betamax – the transition of digital video from standard definition (SD) to high definition (HD), and now most recently Ultra HD or 4K.

What is 4K and what does it mean for professional broadcasters?

With quadruple the 4K Broadcastingresolution of HD (1920 x 1080 pixels), the first noticeable benefit of 4K (3840 x 2160 pixels) is the improved picture sharpness. However, this brings new challenges to the broadcast workflow such as increased storage capacity, processing power, delivery bandwidth, and rendering time. Broadcast and production companies are on the hunt for solutions to seamlessly transition from HD to 4K. For the 4K dream to become a reality, an industry-wide shift in consumer, manufacturer, and content creator habits has to happen. Continue reading

New Enhancements to EMC Isilon Data Lake Foundation

Suresh Sathyamurthy

Suresh Sathyamurthy

Sr. Director, Product Marketing & Communications

Today, EMC® is pleased to announce the availability of new EMC Isilon® products that further extend the advantage that the industry-leading Isilon scale-out data lake and NAS storage platform provides to enterprise customers across a broad range of industry segments and public sector agencies. The new EMC Isilon products include:

  • EMC Isilon OneFS 7.2.1: This new version of the EMC Isilon OneFS® operating system provides new security options including U.S. Department of Defense Security and Technical Implementation Guide (STIG) hardening, FIPS 140-2 validated OpenSSL and PIV/CAC support for Smart Card Authentication. OneFS 7.2.1 also offers new options to increase flexibility and simplify management including improved support for IPv6 networking; enhanced node compatibility between older and newer generation of Isilon storage platforms; and rolling upgrades from OneFS 7.1.x and OneFS 7.2.
  • EMC Isilon X210: The new Isilon X210 is a platform refresh of the Isilon X200 and provides increased capacity, performance and flexibility. The Isilon X210 offers a new 4TB HDD option that expand the maximum node capacity by 33% from 36 TB to 48 TB in a highly efficient 2U platform. Based on SPECsfs testing, the Isilon X210 with OneFS 7.2.1 delivers a performance increase of more than 30% over the Isilon X200. To increase data center flexibility, the Isilon X210 uses QDR InfiniBand networking to allow cable lengths between nodes of up to 100 meters. And to further simplify management and improve flexibility in expanding existing Isilon storage environments, the Isilon X210 offers node equivalence with the Isilon X200. The highly versatile Isilon X210 is an ideal scale-out NAS storage platform for a wide range of applications and workloads including file shares, home directories and big data analytics.
  • EMC Isilon NL410: The new Isilon NL410 is a platform refresh of the Isilon NL400 and provides increased flexibility and simplicity. The NL410 includes QDR InfiniBand networking to allow cable lengths between nodes of up to 100 meters. The Isilon NL410 also offers node equivalence with the Isilon NL400 to provide greater flexibility in expanding existing Isilon storage environments. The EMC Isilon NL410 is designed to provide cost-effective, highly scalable nearline storage and is an ideal solution for active archiving.
  • EMC Isilon InsightIQ 3.2: This new version of Isilon InsightIQ software provides powerful performance monitoring and reporting tools to help organizations optimize the performance of their Isilon storage systems. New features now available with Isilon InsightIQ 3.2 include improved analytics and pool-centric reporting options to provide granular performance data filtered by node tiers or node pools.

Here is a snapshot of the latest portfolio of EMC Isilon storage platforms, including the new Isilon X210 and NL410:

EMC Isilon Scale-Out NAS Product Family

Isilon Product FamilyWith these new products, EMC Isilon scale-out Data Lake further extends its leadership as an ideal platform to store, manage and protect unstructured data efficiently while supporting a wide range of applications and workloads. Key advantages of EMC Isilon include:

  • Simplified management: Single file system, single volume, global namespace
  • Massively scalable: Scales from 16 TB to over 50 PB in a single cluster
  • Unmatched efficiency: Over 80% storage utilization with automated tiering and data deduplication options
  • Enterprise data protection: Efficient backup and disaster recovery, and N+1 thru N+4 redundancy
  • Robust security and compliance options: RBAC, Access Zones,  WORM data security, File System Auditing, Data At Rest Encryption with SEDs, STIG hardening, CAC/PIV Smartcard authentication, FIPS OpenSSL support
  • Operational flexibility: Multi-protocol support including NFS, SMB, HTTP, FTP and HDFS, Object and Cloud computing including OpenStack Swift

To learn more how EMC Isilon scale-out storage solutions can benefit your organization, contact your EMC sales representative or authorized reseller.  Also be sure to see our solutions in the EMC Store at https://store.emc.com/isilon.

ScaleIO: When More is Better…and Less is More!

Rodger Burkley

Rodger Burkley

Principal Product Marketing Manager at EMC

By now, you’ve no doubt heard how Software Defined Storage (SDS) and hyper-converged server SANs are reshaping the storage industry (and use case) landscape.  Market acceptance continues to grow at an accelerated pace.  There are now many established and new vendors offering hyper-converged storage appliances and SDS systems.   EMC’s own ScaleIO SDS platform continues to score mounting recognition, kudos and accolades with our growing list of customers.Hyperconverged  ScaleIO has even won-over storage techies and IT Management who were initially skeptical that ScaleIO would actually completely deliver on its value proposition and capabilities.

Why all the buzz and hype?  Customers – and the IT datacenter and storage market — get it and are embracing this new, “disruptive” technology by introducing it to their enterprise data centers or using it to create hyper-scalable virtualized infrastructures for cloud applications.  The promise and appeal of installing software on individual commodity host application servers to create a virtual storage pool (i.e., “server SAN”) from each participating server’s excess direct attached storage (DAS) without requiring additional specialized storage/fabric hardware is alluring…and almost too good to be true.  High Availability, commodity based hardware, linear performance and storage capacity scalability, “SAN-less” data storage, lower TCO, elasticity and flexibility are just a few compelling arguments in favor of hyper-converged architectures in general and SDS system in particular. Continue reading

Why you want a Witness to Disaster Recovery

Bob Williamsen

Bob Williamsen

Sr. Business Development Manager at EMC

How to protect data and be prepared to survive a disaster is a big topic to cover. Obviously data protection has a lot of components—ranging from hardware capabilities, algorithms used by the system to protect data and ensure integrity, and concerns like backup and replication strategies. In this note let’s mercifully contain this discussion to what is arguably one of the most important topics – surviving a disaster with minimal disruption to your business.

Disaster Recovery, DR, is a necessary consideration when designing robust enterprise solutions. By definition, DR always implies physical distance between the systems involved in the design. Isilon supports data replication over distance and the components of an Isilon DR solution include:

  1. Application Servers
  2. LAN connecting the application servers to the Isilon storage
  3. Isilon cluster at the production data center
  4. Isilon cluster at the DR data center
  5. A WAN connecting the two data centers
  6. Isilon SyncIQ for data replication

 

Ok, now the data is automatically replicated—but what happens in the event of a DR fail-over?  It requires manual intervention or scripts to make the data on the DR site available to users and applications. What is missing is a way to orchestrate and automate the fail-over process so that the business magically recovers from the loss of the primary site.

Continue reading

EMC {code} shares how to deploy ECS with Five Ways of Docker

Kendrick Coleman

Kendrick Coleman

Developer Advocate at EMC

We’ve been in a lot of conversations about DevOps recently, with customers, partners, community members and EMC teams. Whether or not you believe in the buzz around DevOps, there is definitively a wave of new and open tools, proclogo-emc-codeesses, and operational models being used in IT. These are all trends that can’t be ignored, and we’re continuously working to make sure EMC’s product strategy adapts with these changes.

EMC {code} is a developer evangelism team within EMC. We contribute to major open source projects, create our own tools and projects (also entirely in the open– in fact– you can find them all on GitHub, here), and engage with developer communities. Several of our team members have worked with EMC SDS products over the past few years, and we’ve enjoyed seeing the progress made to make our SDS tools readily accessible for developers and IT/ops alike.

ECS has long been a developer-friendly product, with universal protocols like object and HFDS, and a software-only download option. The recently announced ECS 2.0 is chock full of updates that are making the lives of dev/ops teams easier—including enhanced geo-caching, multi-tenancy capabilities, monitoring and reporting, and the ability to automatically and rapidly failover and recover from outages.

ECS 2.0 only enhances the experience for developers with a containerized download, free for test-and-dev environments (non-production use today). This has been another milestone in providing free and frictionless access to EMC software (along with the open-sourced CoprHD and free download of ScaleIO).

Our team works with Docker containers on a regular basis, and when ECS was announced, we rolled up our sleeves to see exactly how we could integrate Docker tools into the ECS experience.

In addition to the containerized download, we wanted to explore ways of deploying ECS with broader Docker technology. Deploying multi-node ECS is fast and easy with Docker tools. Using Docker Machine, you can deploy Ubuntu hosts that will be a part of a Docker Swarm cluster. With Docker Compose, you can deploy ECS from Docker Hub to a Docker Engine container on each host in the Docker Swarm cluster. If you are interested in how to do this, you can check out the GitHub repo or watch this quick video.

With this automated process, everything will be complete and configured in the span of 10-15 minutes. To get all of the details, please visit the EMC {code} blog post on the topic, or send us a tweet with your feedback to @EMCcode.

A New Blueprint for Success: EMC VSPEX with Isilon

Carl Washington

Carl Washington

Sr. Business Development Manager at EMC

It’s been two years since EMC’s VCE and Isilon teams joined forces to begin offering converged infrastructure (CI) solutions that combine VCE Vblock Systems with Isilon scale-out NAS in support of a range of applications and workloads. And we couldn’t be more excited about the results. It’s extremely gratifying to hear about the many customers who are now realizing the significant advantages that these CI solutions deliver: accelerated time to value, management simplicity, increased efficiency, single point of contact for support and the knowledge that they are getting a tested and proven solution.

Enablement VSPEX Proven Infrastructures are modular, virtualized solutions validated by EMC and delivered by EMC VSPEX partners. They include virtualization, server, network, storage, and backup layers.

Enablement VSPEX Proven Infrastructures are modular, virtualized solutions validated by EMC and delivered by EMC VSPEX partners. They include virtualization, server, network, storage, and backup layers.

Building on this success, EMC converged infrastructure solutions that incorporate EMC Isilon scale-out NAS have recently been extended to include integration with EMC’s VSPEX program. As a result, Isilon storage is now featured in a new VSPEX reference architecture:  EMC VSPEX Proven Infrastructure with Isilon Scale-Out NAS.

VSPEX Proven Infrastructure with EMC Isilon combines the simplicity, efficiency, and flexibility of VSPEX coupled with Isilon to deliver improved storage utilization, IT productivity, and increased end-user productivity. This new VSPEX reference architecture provides a blueprint for building a proven converged infrastructure that can support a number of use cases that most medium to large organizations are likely to need including:

  • File Shares
  • Home Directories for End-User Computing
  • Active Archiving

The VSPEX reference architecture also includes the necessary details for building a proven converged infrastructure for businesses in the Media & Entertainment sector.

With this new VSPEX and Isilon reference architecture, organizations can gain the following advantages:

  • Application-specific workload sizing, design and implementation guidance
  • Accelerated time to value
  • Leverage of VSPEX ecosystem partners for application flexibility and support

Built for virtualized environments, VSPEX enables faster deployment, more simplicity, greater choice, higher efficiency, and lower risk. VSPEX with Isilon can be added to existing VSPEX environments or be deployed as a stand-alone solution to consolidate storage to lower costs and optimize today’s 2nd Platform applications while providing a foundation for next-generation 3rd Platform applications. This is also a great first step in building a highly versatile data lake foundation to store, manage, and protect Big Data.

Testing and validation, outlined in a VSPEX Proven Infrastructure Guide, coupled with the VSPEX Sizing Tool, simplifies solution sizing and eliminates the risk and expense of over-provisioning. VSPEX with Isilon further simplifies and refines the sizing process by leveraging Isilon’s Predictive Performance Planning (P3) sizing tool functionality to enable efficient solution sizing for a broad range of file data workloads.

Of course, all these solutions are based on Isilon storage which provides a number of important benefits:

  • Simple management – with Isilon’s single file system, single volume architecture
  • Massive scalability – easily expands up to 50 PB in a single cluster
  • Operational flexibility – multi-protocol support for a wide range of applications and workloads on one platform
  • Unmatched efficiency – with over 80% storage utilization
  • Robust data protection and security options

By leveraging the advantages of EMC VSPEX Proven Infrastructure with EMC Isilon scale-out NAS, organizations do indeed have a new blueprint for success!

For more information on how this solution may benefit your organization, I suggest that you contact your authorized EMC reseller or local representative.

Configuring ECS: Start Uploading Files in 5 Minutes!

Chris Kanthan

Chris Kanthan

Senior Technical Marketing Manager at EMC

Many times after an ECS installation, the question that arises is, “Now, what do we do with this thing?” It’s understandable because an object-storage appliance is not like a SAN or a NAS array. You cannot just walk back to your cube and start archiving old documents to the ECS.

At least, not yet. ECS will have NFS support and some other tools in the near future that will enable just that.

But for now, a quick way to demo access to ECS is to use a freeware such as the “S3 Browser.”

To summarize the two-step process: Go through the ECS portal (GUI) to create a Namespace, and then use the S3 Browser to access ECS.

Step 1: Configure ECS 2.0 PortalECS GUI

There are a few easy steps to accomplish this.

  1. First, create a Storage Pool in the ECS portal by choosing the ECS nodes and giving that collection a name.
  2. Next, from the Virtual Data Center (VDC) tab on the left
      • Click on “Get VDC Access Key”. Copy the displayed key
      • Click on “New Virtual Data Center” and create a VDC using the above key and the IP addresses of all the ECS nodes.
  3. Then create a Replication Group using the newly created VDC and the Storage Pool.
  4. Now, create a Namespace using “root” and the Replication Group. Namespace is analogous to a “Tenant.” This is where buckets will be created.
      • At this point, you have an option of moving to the next step or creating a bucket. If you create a bucket, you will see that bucket after configuring the S3 browser. To create a bucket, go to the “Buckets” tab, choose the Namespace and the Replication Group that you just created, and give the bucket a name.
  5. Finally, go to Users, click on “Object Users” à “New Object users”; choose the Namespace for “root” and click on “Next to add Passwords”. In that next page, generate an S3 secret key for root and copy it to a text editor (to be later used within S3 Browser).

You’re done with ECS portal.

Step 2: Configure S3 Browser and Access ECS

Now, open the S3 Browser. Go to “Accounts” on the top left and choose “Add New Account”. There are four things that need to be filled out or chosen:

  1. Type in an Account name
  2. Under “Storage Type”, Choose S3 Compatible Storage
  3. In “REST Endpoint”, type in the IP Address of one of the ECS nodes along with the port of 9020 (9021 is for secure https). The format is <IP_address>:<port_number>. Note that, by default, S3 Browser uses http, and if you want to use https, go to Tools -> Options -> Connection and check the top option to use secure connection.
  4. Under “Access Key Id”, put in “root”
  5. Copy and paste the secret key from Step 1-E into the field for “Secret Access Key”.
  6. Save Changes and you should be good to go.
  7. Now you can click on “New bucket” in the main page and upload files to that bucket from your local drive. If you had created a bucket from within the ECS portal at the end of Step 1-D, you will now see the bucket listed here as well.

Here is an image of the configuration of S3 Browser with some annotations:Configuration

Hope you found that easy! Of course, after this demo, customers would use their real enterprise applications to access ECS using REST API and using different users, namespaces, secret keys etc., but the fundamentals are the same.

Introducing ECS 2.0

Vikram Bhambri

Vikram Bhambri

Vice President, Product Management, Emerging Technology Division at EMC

ECS (Elastic Cloud Storage) provides a hyperscale, geo-distributed, software-defined cloud storage platform that is perfect for unstructured data, Big Data applications and Data Lakes. Deployed as a software-only solution or as a turnkey appliance, ECS offers all the cost advantages of public cloud running on commodity infrastructure while featuring enterprise reliability, availability and serviceability.

The new version of ECS headlines many new features and enhancements. Here is a “Top Ten” list of the most significant improvements and changes in ECS, version 2.0:

1. Built-in Element ManagementECS Infographic
Any storage array requires strong element management capabilities to manage the infrastructure and its lifecycle. Until last release of ECS, the element management functionality of ECS was delivered via ViPR Controller. This meant that each installment of ECS would require a minimum of 3 additional VMs to run the ECS Appliance.

In ECS 2.0, all of the element management functionality is part of the ECS software itself and there is no requirement for additional ViPR Controller to manage ECS. The user interface for managing ECS is on each node and the administrator can hit any of the ECS nodes to get access to it. The improved interface is easy and simple to use with lot of enhanced functionality.

For customers who want a single pane of glass for management of their storage infrastructure, ECS will be able to plug into ViPR Controller like any other array that ViPR Controller manages and provide one single northbound interface for administrators for storage management.

2. Better UI, simplified management and operations
One of the fundamental changes in 2.0 release is the introduction of self-service UI and management built into the ECS software itself. ECS offers a simpler and more intuitive UI that allows administrators to easy deploy and manage their Appliance’s lifecycle.

In addition, based on the feedback from 1.x customers, a lot of the terminologies have been simplified, better-named or even removed. For example, a “tenant” is now the same as a “namespace”, so you don’t need two different variables. ECS 2.0 has simplified the multi-tenancy model and number of roles involved for managing the ECS. All of these operations managed by the UI are consuming a standard set of REST APIs that can be consumed directly if the customer chooses to use another user experience for managing their storage environment.

3. Improved Monitoring and Diagnosis in ECS UI

The new UI has lots of new information that will assist in monitoring, diagnosis and performance analysis, with drill downs and charts to make the analysis easier to visualize.

To start with, administrator can get information on

  • Capacity utilization for disks,nodes and storage pools.
  • Granular information on bandwidth, IOPs, latency and network utilization in different categories such as erasure coding, geo-caching, metadata, user data etc.

Other enhancements include detailed reports on replication groups

  • Bandwidth for ingress/egress
  • Progress of replication, details about chunks that are being cached and replicated etc.

In case of site failure or disaster, Administrators are able to get details on duration of recovery and amount of data to be recovered.

All this information will be stored for 7 days. For a longer period of data, the Customer can use ViPR SRM  (with a new Solution Pack) or their own software that accesses ECS via REST API as all of the monitoring and diagnostics data is available through REST APIs.

4. Rack-level Awareness for better HA
ECS 2.0 brings rack-level awareness to the software so that when data has to be
distributed across disks and nodes, ECS software can spread the data across different racks for increased high-availability and redundancy. Each disk, node, rack and DCs are considered fault domains in ECS and data is distributed in a way to maximize the data availability.

5. Geo-replication: Unsealed chunk replication means better RPO
For geo-replication cases before ECS 2.0, ECS software wrote an object to chunks (of size 128MB), it waited for the chunk to get filled up, and then replicated the chunk to a remote site in an asynchronous process. Although this strategy is more efficient, the drawback is that if an entire site or a rack goes down, there could be many chunks with less than 128MB of data that have not been replicated. To reduce the risk of data loss, the software now starts streaming the data to remote Data center and the replication process kicks off as soon as a chunk receives new data. This feature will help with an improved RPO.

6. Geo-caching for better performance in multi-sites & Overall performance Improvements
In a multi-site environment, the data were always accessed from the primary site where it was originally written. This meant that every time customers in other sites accessed the data, it involved cost for WAN bandwidth as well as slower performance caused by WAN latency.

ECS 2.0 solves the above problem by the use of geo-caching the data at the secondary site on the local disk so that customers can access the data locally without a WAN transfer. This is more applicable to scenarios where the number of sites is greater than 2.

EMC 2.0 has stepped up the game in object performance when compared to earlier object platforms. Performance comparing ECS Appliance to ATMOS:

*Small Objects: write 6X faster – read 2X faster
*Large Objects: write 5X faster – read 9X faster

7. Temporary Site Failover
Temporary site failures like network drops are pretty common in data centers. ECS 2.0 has smart temporary site failures and failback features that allow applications to access its data even when the primary site is unavailable or unreachable. The delta writes can go to the secondary site. ECS software will also automatically re-sync the sites and reconcile the data when all the sites are operational and connected to each other again. Any conflicts that arise, get resolved leveraging the algorithms built into the software layer.

8. Metering and Auditing
One of the common requirements for running a large-scale multi-tenant distributed storage environment is to have very detailed metering. ECS 2.0 provides key statistics for individual buckets and tenants. This includes capacity, object count, objects created, objects deleted and bandwidth consumption (inbound as well as outbound bandwidth). The design and implementation is done in a way that it will satisfy the requirements of large scale services providers either in a single managed customer or a multi-tenant shared environment.

The new software also enables auditing for buckets, which allows administrators to view activities regarding creation, update, and deletion of buckets, and any changes in bucket ownership. This is especially important for environments that have to be governed by specific regulations. The events can be accessed through the UI or the REST API.

9. Quotas
ECS 2.0 software now allows administrators to set soft or hard quotas for buckets and tenants. This allows for administrators to set guard rails on consumption by application thus creating a sandbox without impacting other application users.

Alerts are raised and writes can be blocked after the set limit if the administrator chooses so. The administrator can also set up policies that can lock a specific bucket or a user if in case their application workload is causing other tenants to be impacted.

10. Free & Frictionless Download of ECS Software
Today, EMC announced the “Free and Frictionless” version of ECS. There is now a free download of ECS software for development and testing purposes with unlimited capacity and perpetual licensing.  ECS made a bet on Docker even before it was GA and every single ECS Appliance ships with Docker in it. Now, we are making the same software containerized for broader audience. This allows developers, partners and customers to use and develop for ECS, while enjoying access to a large developer community. ECS is downloaded as a docker container, and installation can be done manually or automated through Puppet or Vagrant to run the software on one or more VMs/ bare metal servers. Stay tuned for information in EMC Pulse blog, the ECS Twitter account, and the EMC Community Network.

That’s ECS 2.0 in a nutshell. Of course, there will be more blog posts and white papers coming out soon as well. Customers, partners and developers have helped shaped the 2.0 version. We will continue to solicit more feedback on additional features should be added to the platform. If you have any request for features, please let us know and we will incorporate that in our roadmap.

3 Big Market Opportunities For Video Surveillance – 1 File System

Steve Lewis

Steve Lewis

Sr. Manager, Video Surveillance Global Business Development at EMC

According to IHS annual market research, EMC is the leader in enterprise video surveillance storage and has been for the past 4 years.  The video surveillance market continues to see robust growth with more cameras being installed today than ever before, as organizations, businesses and governments desire to deter, protect, prosecute and predict with their surveillance systems.

Over the past few years, the use of Isilon network attached storage (NAS) arrays has grown rapidly in video surveillance markets—largely because of changing industry trends:

  • A transition from standard definition (SD) cameras to high definition (HD) surveillance cameras.
  • End users moving from an appliance model (DVR) to an IP/IT model (COTS) utilizing best of breed IT components in the overall surveillance solution.
  • The need to efficiently and cost effectively store massive amounts of video surveillance data so that it can be shared by multiple stakeholders for various purposes.

In this blog we’ll examine the key growth markets which drive video surveillance, and what makes Isilon such a good fit in them.  So let’s roll…

Casinos
CasinoHigher resolution cameras and higher frame rates give casino owners better weapons to fight fraud and “beat the cheats.”  However, it’s not just about better video—it’s also about continuous operation.

Gaming commission regulations require a casino’s surveillance solution be operational at all times, and if not, the casino must shut-down. An unreliable surveillance storage system can mean hundreds of thousands of dollars in lost revenue every hour.  Isilon can handle these requirements with its built in auto failover capabilities and rock solid reliability.

Furthermore, casino’s business operations are looking for new ways to gain insights into their patrons’ patterns.  The relatively new science of video analytics can show casino managers not only when and where patrons played, but also how they played. That data can be correlated with “player’s cards” data to determine insights into behavior and develop targeted marketing and promotional programs.  EMC Isilon’s analytics-ready architecture accelerates analytics and helps provide casino management with timely and more accurate information.

Law enforcement
PoliceBody cameras are a hot topic these days—given the recent incidents in Ferguson, NYC, Baltimore and South Carolina. The benefit of body cams as a tool for law enforcement has been proven in independent studies to reduce complaints, use of force, and provide evidentiary video.  Body cams use SD cards for local storage and that video is transferred to a PC or Server after each officer’s shift (8-12 hours). The average body camera with a HD resolution (720p) conservatively generates 1.2GB per hour—or ~10GBs per officer per shift. A police force with 1000 officers can generate more than 300TB every month.

While the police department is the one buying the body cameras, it is the District Attorney who decides the data storage policies and practices. Combine the video resolution with the mandated retention period (which in some jurisdictions is for the life of the accused), and police departments are looking at very large storage requirements.

A storage solution that can handle mixed workloads and the many forms of digital media evidence (interview room video/audio, photographs, DNA, etc.), in addition to body cameras, is essential.  The storage solution must store the data efficiently and cost-effectively, and be able to run queries from analytics without having to make multiple copies of the data sets. Isilon has all of these capabilities—enabling law enforcement to limit the cost of storing data and derive more “actionable intelligence” from disparate sources. The ability to predict crime is no longer in the realm of Minority Report, it is a reality.

Safer Stadiums
Stadium
More new stadiums are being built to attract a new generation of “digitally connected” patrons, not only for sporting events but also for concerts and festivals. Stadiums are also being developed as integrated shopping malls for residents in and around the stadium property/complex.

Cameras are everywhere… in and out of the stadium – from the parking lots to the concession stands to the seats. High definition/multi-megapixel cameras are a necessity to cover the entire stadium and capture every fan in their seat…literally.  If a fight breaks out in the stands, that level of detail is needed to prosecute effectively.

There are ~12,000 stadiums in the world and 2,900 stadiums in the USA alone. Out of those 2,900, 225 stadiums in the US have the seating capacity over 20,000. With retention periods growing (If a criminal assault or theft occurs the video must be retained as evidence for the life of the crime or until the crime is expunged), that is a lot of cameras and plenty of storage requirements. For example, check out how MetLife Stadium used Isilon to reduce the amount of time (and cost) of managing their storage from five hours a week to five minutes a week.

In conclusion, the convergence from analog (CCTV) surveillance solutions that were based on appliances and internal storage is well underway. Isilon is the ideal storage platform for video surveillance because of its scale out design and proven ability to handle thousands of surveillance cameras without compromising performance or data integrity—enabling surveillance operators to manage the situation, not the storage.

 

Datadobi and EMC: Fast, Reliable Data Migrations

Michael Jack

Michael Jack

Global Sales Director at Datadobi

*The following is a guest post by Michael Jack, Global Sales Director at Datadobi.

Over the past 13 years, EMC Centera has – and continues to be – the most reliable object storage and compliant archiving platform in the industry.  While EMC continues to sell and support Centera, customers are also looking at migration strategies to gain new cloud scale features and automation services.

That’s where Datadobi can help.  Datadobi is an EMC Technology Connect business partner that offers DobiMiner, our migration software that migrates Centera customers to other storage platforms. In this blog, we’ll discuss Datadobi’s unparalleled migration experience, and we’ll share considerations and strategies on how to migrate Centera data to EMC Elastic Cloud Storage (ECS).Datadobi_blog_image

Centera is one of the most stable platforms in EMC’s storage platform portfolio providing 6 nines of availability. It also has an excellent feature set including the coveted Compliance Edition Plus model used by financial institutions around the world to ensure the immutability of their data. Centera created a new paradigm in the storage industry and set the standard for Object Storage that no other vendor has yet to achieve.

While many use cases for Centera remain strong, the move to next generation distributed cloud, mobility, and Big Data applications are driving many Centera customers to take advantage of the next generation object storage. And with many companies having experienced the cost, risk, and time associated with moving from one platform to another, they are rightly asking ‘how do we move all that data without all that pain?’

So how do you move CAS data? Many EMC customers have experience with NAS migrations but have no idea how to move CAS data. To answer that question, let’s take a brief look at how CAS works. CAS data sits in a flat file space without a classic file structure and accessing it requires having the key known as the Content Address or CA on Centera. When an application writes data (a user file) to a Centera, Centera creates a Content Descriptor file (CDF) and places the CA of the user file in the CDF. It then creates a CA for the CDF and passes that to the application, which stores this in its database. When the application needs to retrieve the file again, it passes the CA of the CDF to the Centera, which looks in the CDF to find the CA of the user file. All pretty complex stuff which is what makes Centera such a great product.

To migrate a CAS file to another platform (be it CAS, NAS, RESTful, OpenStack, etc.) you can’t use a tool such as Robocopy because you have to use the CA to locate the file. You can only achieve this by using:

  • The application that wrote the data or
  • A specialized migration software such as DobiMiner

You may think migrating the data using the application that wrote the data would be the easiest way; however, this is not the case. Applications migrate data by re-reading it from the Centera and then re-writing it to the new platform. Most applications are not designed for high-speed read and write. As a result, migrations done in this way tend to be very lengthy affairs resulting in additional expenses such as professional services and extended support contracts for the old platform.

DobiMiner takes a different approach – it quickly collects the list of data to be migrated directly from the Centera by parsing the CDFs information to the DobiMiner instance. Because the CDFs contain the CA of the user file and the file sizes the entire scope of the migration is completely understood. This allows the migration team to make informed decisions and have a predictable finish date before actually migrating the data. Once the scope is agreed, the data can be rapidly pumped to the new platform at Terabytes a day.

Once the data has been copied to the new platform, DobiMiner reads back each individual file and compares it with the same file on the old platform to validate its authenticity before declaring it fully migrated.

Mining all the CDFs also enables you to have a complete understanding of your data before migrating. For example, you’ll know whether the data is still recognized by the application, has the data expired but not been deleted, and are the replicated environments in sync? All this information enables you to migrate only the data (and all the data) with true business value ensuring the best ROI on the target platform.

CAS-to-non-CAS – a slightly different kettle of fish

Where all of the above relates to both CAS-to-CAS migrations (such as Centera-to-Centera and Centera-to-ECS) and CAS-to-non-CAS migrations (such as Centera-to-NAS or a RESTful interface), moving data from a CAS to a non-CAS platform can involve additional complexity.

Applications write data to a Centera in one of two ways:

  • Through a file-system gateway or
  • Natively through the Centera API

Some applications use a file-system gateway (such as the Centera Universal Archive) to write the data to the Centera and it is the file-system gateway that stores the CAs not the application. The application database stores a file path making migrating that data to a non-CAS platform straightforward. The file path known to the application can be duplicated on the new platform making the migration transparent to the application.

On the other hand, applications using the API store the list of CAs in their database and therefore require an additional step in the migration process. Firstly, a file path naming scheme must be agreed for the new platform. A combination of fields in the CDF can be used for this or DobiMiner can create a path itself. Secondly, the list of CAs in the application database must be replaced with the new file path so that the application can access the files after the migration. This database update can occur in one of a number of ways:

  • Fully automated – the migration software automatically connects to the application database and overwrites the CA with the new file path.
  • Handshake – the migration software creates a file that maps the old CA to the new file path and this mapping is manually used by the application vendor to update the application database.

Unfortunately, some application vendors do not allow either method and only allow migrations through the application itself (making for a lengthy migration).

In conclusion, migrating Centera to another platform can seem like a pretty daunting process – it really isn’t. With DobiMiner, most of the difficult tasks are automated to ensure a simple, low risk, predictable, and fast migration.

Avoid the pain. Take advantage of Datadobi and our partners’ expertise to perform your next migration and let your team focus on your business.