Posts Tagged ‘ECS’

Buckets, Apps & Digital Exhaust…All in a Day’s Work For a Dell EMC Splunk Ninja

Cory Minton

Principal Systems Engineer at EMC

Grab your hoodies, your witty black t-shirts, and maybe your capes…it’s time for another exciting Splunk .conf, the annual Splunk User Conference taking place this week at the Walt Disney Swan and Dolphin Resort.  All of us at EMC are excited to be sponsoring .conf for the third year in a row, and this year our presence will be bigger and better than ever before. Dell EMC is hosting two technical sessions this year, we’ll have more than 20 of the Dell EMC Splunk Ninjas running around learning and a large booth in the partner pavilion demonstrating our technology solutions.. For all the details, check out our .conf16 site.

This year marks the beginning of a great relationship between two awesome tech businesses: Dell EMC and Splunk.  We joined forces through a formal strategic alliance that started in February.  This alliance enables Dell EMC and its partners to sell Splunk’s industry leading platform. And, it allows Dell EMC unique access to Splunk technical resources for solution design, testing, and validation.  Most importantly, it creates a framework for these two technology powerhouses to collaborate more effectively for customer success.

Why Dell EMC for Splunk?


When we talk about customer success, we mean it in two distinct ways: deploying Splunk on Dell EMC platforms and, using Splunk to derive value from Dell EMC infrastructure.

First, we believe success is deploying Splunk on a flexible infrastructure that not only helps Splunk run fast and efficiently, but also one that can scale easily as the usage of Splunk evolves in a customer organization.  We believe that converged and hyper-converged technologies powered by Dell EMC’s robust portfolio of storage technologies delivers on this vision and provide additional enterprise capabilities:

  • Cost effective & Optimized Storage – Dell EMC delivers optimized and efficient storage by aligning the right storage to Splunk’s hot, warm, and cold data long retention and varying performance requirements.
  • Flexible & Scale-Out capacity consumption model – Scale-out infrastructure to meet capacity and compute requirements independently or as a single, converged platform as per your data growth.
  • Data Reduction & other data Powerful Enterprise Capabilities – including secure encryption, compression & deduplication of indexes, and fast, efficient zero-overhead copies for protection.
    Bottom-less cold bucket – Scale-Out storage platforms, whether on premise or in the cloud, obviates the need for a frozen bucket by providing a PB-scale cold bucket solution, simplifying data management and making data always searchable.

splunk2

 

Splunk and Dell EMC engineering teams have engaged in a strategic collaboration to ensure that all Dell EMC platforms have been validated by Splunk to “meet or exceed Splunk’s published reference server hardware” guidelines.  The Splunk team takes this validation process very seriously and customers can rest assured that if they are considering infrastructure for your Splunk deployment, we have done extensive testing. Whether you are looking at hyper-converged solutions like VXRail or VxRack, converged solutions like VBlock systems, or just storage from EMC like ScaleIO, XtremIO, VNX, Unity, Isilon, or ECS, you can be confident that the work has been done by both Splunk and Dell EMC to make sure it runs well.

Secondly, we believe Splunk is an incredibly powerful platform for capturing and deriving value from machine data.  As it turns out, Dell EMC products spin off a massive amount of “digital exhaust” that can be captured easily and used to drive operational intelligence in IT.  Dell EMC has made massive investments over the last few years to build apps for our platforms and make them available in Splunkbase for free.  We’ve built apps for XtremIO, Isilon  and VNX and expect to have many more in the works.  These apps make it simple to ingest data from Dell EMC platforms, and we offer useful, pre-built reports and dashboards to make monitoring these assets simple.  And it doesn’t stop there…once the data is extracted from your Dell EMC platforms, the underlying searches powering our reports or just the indexes themselves can be used in investigations across the entire IT service stack.  One of my favorite things to hear from our customers is the exciting ways they use the apps beyond just simple reporting and I hope to hear many more stories this year at .conf2016.

Dell EMC Splunk Ninjas And Our Top Ten List

 

splunk_ninja_resizeThe Dell EMC Splunk Ninja team are at the show in their Dell EMC blue Ninja Hunt shirts. The Dell EMC Splunk Ninja team is a group of more than 40 systems engineers from across Dell EMC who have been trained the same way Splunk trains its own systems engineers.  The Ninjas hold certifications ranging from SE1 all the way to SE3, we’ve got skills across not only using Splunk, but administering and architecting it at scale.  This is a global team not only available to talk to you at .conf, but also available in the field to have direct conversations with you when you head back to the office.

 

 

Our wise, passionate and zany Ninja team recently pulled together a list of Top Ten Best Practices for Splunk on Dell EMC. This list has been amassed based on years of lab testing and real world customer experience. You may say ‘Duh’ to some, but others may surprise you.

Happy Splunking!

Enterprise Strategy Group on ECS: Meet data growth challenges with software-defined object storage

“This new era of massive content storage environments requires a new storage architecture, object storage.” That’s the view of Scott Sinclair, Storage Analyst at Enterprise Strategy Group (ESG) in its October 2015 whitepaper: “EMC Elastic Cloud Storage Offers Resilient Scalability for the New Generation of Workloads”. Let’s explore why ESG believes object storage is an essential enterprise technology.

Identifying The Challenges Of Data Growth

Managing today’s ever-growing quantities of (mostly unstructured) data is a constant challenge for most enterprises – a fact that ESG underlines in its whitepaper. ESG asked 373 IT decision-makers: “What are your organization’s biggest challenges in terms of its storage environment?

emcecs_1

The rapid growth of data was cited as a top challenge by 26% of IT decision-makers, which ESG says should not be a surprise. However, what may be more surprising is that all the other top challenges – including increased hardware costs, data protection costs, and staffing costs – can also be considered symptoms of data growth.

Why The Industry Is Moving To Object Storage

ESG says that object-based storage offers a strong solution to meet the multiple challenges created by data growth. It represents an evolution in the ability to store unstructured data, with near limitless scalability. Object storage also offers automatic geo-dispersed data protection and the ability to leverage commodity hardware, making it an ideal storage environment for a new generation of IT workloads.

To discover more about what attracts IT decision-makers to object storage, ESG also asked: “Which factors are responsible for your organization’s initial deployment or consideration of object storage technology?”

emcecs_2

As ESG reveals, across the wide variety of potential use cases for object storage, organizations are turning to the technology to help control TCO in the wake of rising digital content levels. But of those potential use cases, more organizations are looking to object storage as the foundation for the next generation of modern and cloud-based workloads.

Evaluating ECS Object Storage Advantages

ESG says that EMC’s object storage solution Elastic Cloud Storage (ECS) delivers key features for object storage, such as:

  • Software-defined architecture leveraging commodity components
  • A strongly consistent, global scale-out namespace
  • In-place Hadoop analytics with HDFS support
  • Support for S3 and OpenStack APIs
  • Combined erasure-coded and replication-based protection
  • Built-in journaling, snapshots, and versioning
  • Multi-tenant architecture
  • Flexible deployment models
  • On-premises data security

Today’s Demanding Workloads Are Ripe For ECS

As ESG comments: The next generation of workloads is looking to be even more dependent on larger levels of digital content than ever before, and therefore needs the next generation of storage.” Highlighted workload areas where ECS can help improve TCO and ROI for organizations include:

  • Foundation for a cloud storage solution: ECS supports your deployment of a cloud infrastructure with its ability to dynamically allocate resources – and to report on how they are being leveraged. Its potential for nearly infinite scalability along with its ability to leverage commodity hardware help keep cloud infrastructure costs low.
  • Data lake for business analytics: ECS offers the ability to pool all your data into a single globally distributed repository for business analytics to derive new value, while its multi-protocol access reduces the need to move data back and forth for analysis.
  • Global content repository: Offering a single scalable, resilient, and cost-effective pool that can serve multiple application and content types, ECS enables all the content within the repository to be globally accessible by web, mobile, and cloud applications – at up to 65% lower cost than public cloud.
  • Universal object storage platform for Internet of Things (IoT) with in-place analytics: The geo-distributed strong consistency of ECS provides the ability to collect and store IoT sensor data in locations closer to the actual “things,” which improves performance and saves cost.
  • Modern application development: The scale and automatic geographic accessibility of ECS can provide significant benefits for developing your modern applications, especially those that require access to a large pool of read-only content.
  • Cold storage archive: ECS-based archiving controls the growth of unstructured data by migrating “cold” data off of high-performing and more expensive storage – while allowing for the data to remain online and accessible.

The Next-Generation Object Storage Platform

As ESG concludes: “Object storage is designed for and ideally suited for large content storage. EMC’s ECS solution, ultimately, offers a foundation upon which IT organizations can build out the next generation of applications and workloads. The next-generation datacenter will need a next-generation storage solution, like ECS.”

Read the full ESG Whitepaper for yourself: ”EMC Elastic Cloud Storage Offers Resilient Scalability for the New Generation of Workloads”, October 2015.

Learn more about object storage with EMC ECS.

EMC ECS software is available to download now and try free.

 

Breakfast with ECS: Files Can’t Live in the Cloud? This Myth is BUSTED!

Welcome to another edition of Breakfast with ECS, a series where we take a look at issues related to cloud storage and ECS (Elastic Cloud Storage), EMC’s cloud-scale storage platform.

The trends towards increasing digitization of content and towards cloud based storage have been driving a rapid increase in the use of object storage throughout the IT industry.  However, while it may seem that all applications are using Web-accessible REST interfaces on top of cloud based object storage, in reality, while new applications are largely being designed with this model, file based access models remain critical for a large proportion of the existing IT workflows.

Given the shift in the IT industry towards object based storage, why is file access still important?  There are several reasons for this, but they boil down to two fundamental reasons:

  1. There exists a wealth of applications, both commercial and home-grown, that rely on file access, as it has been the dominant access paradigm for the past decade.
  2. It is not cost effective to update all of these applications and their workflows to use an object protocol. The data set managed by the application may not benefit from an object storage platform, or the file access semantics may be so deeply embedded in the application that the application would need a near rewrite to disentangle it from the file protocols.

What are the options?

The easiest option is to use a file-system protocol with an application that was designed with file access as its access paradigm.

ECS - Beauty FL_resizedECS has supported file access natively since its inception, originally via its HDFS access method, and most recently via the NFS access method.  While HDFS lacks certain features of true file system interfaces, the NFS access method has full support for applications and NFS clients are a standard part of any OS platform, thus making NFS the logical choice for file based application access.

Via NFS, applications gain access to the many benefits of ECS, including its scale-out performance, the ability to massively multi-thread reads and writes, the industry leading storage efficiencies, and the ability to support multi-protocol access, e.g. ingesting data from a legacy application via NFS while also supporting data access over S3 for newer, mobile application clients and thus supporting next generation workloads at a fraction of the cost of rearchitecting the complete application.

Read the NFS on ECS Overview and Performance White Paper for a high level summary of version 3 of NFS with ECS.

An alternative is to use a gateway or tiering solution to provide file access, such as CIFS-ECS, Isilon CloudPools, or third-party products like Panzura or Seven10.  However, if ECS supports direct file-system access, why would an external gateway ever be useful?  There are several reasons why this might make sense:

  • An external solution will typically support a broader range of protocols, including things like CIFS, NFSv4, FTP, or other protocols that may be needed in the application environment.
  • The application may be running in an environment where the access to the ECS is over a slow WAN link. A gateway will typically cache files locally, thereby shielding the applications from WAN limitations or outages while preserving the storage benefits of ECS.
  • A gateway may implement features like compression, thereby either reducing WAN traffic to the ECS, thus providing direct cost savings on WAN transfer fees, or encryption, thus providing an additional level of security for the data transfers.
  • While HTTP ports are typically open across corporate or data center firewalls, network ports for NAS (NFS, CIFS) protocols are normally blocked for external traffic. Some environments, therefore, may not allow direct file access to an ECS which is not in the local data center, though a gateway which provides file services locally and accesses ECS over HTTP would satisfy the corporate network policies.

So what’s the right answer?

The there is no one right answer; instead, the correct answer will depend on the specifics of the environment and of the characteristics of the application.

  • How close is the application to the ECS? File system protocols work well over LANs and less well over WANs.  For applications that are near the ECS, a gateway is an unnecessary additional hop on the data path, though 3d Kugel mit Fragezeichen im Labyrinthgateways can give an application the experience of LAN local traffic even for a remote ECS.
  • What are the application characteristics? For an application that makes many small changes to an individual file or a small set of files, a gateway can consolidate multiple such changes into a single write to ECS.  For applications that more generally write new files or update existing files with relatively large updates (e.g. rewriting a PowerPoint presentation), a gateway may not provide much benefit.
  • What is the future of the application? If the desire is to change the application architecture to a more modern paradigm, then files on ECS written via the file interface will continue to be accessible later as the application code is changed to use S3 or Swift.  Gateways, on the other hand, often write data to ECS in a proprietary format, thereby making the transition to direct ECS access via REST protocols more difficult.

As should be clear, there is no one right answer for all applications.  The flexibility of ECS, however, allows for some applications to use direct NFS access to ECS while other applications use a gateway, based on the characteristics of the individual applications.

If existing file based workflows were the reason for not investigating the benefits of an ECS object based solution, then rest assured that an ECS solution can address your file storage needs while still providing the many benefits of the industry’s premier object storage platform.

Want more ECS? Visit us at www.emc.com/ecs or try the latest version of ECS for FREE for non-production use by visiting www.emc.com/getecs.

Breakfast with ECS: The Swiss Army Knife of Cloud Solutions

Corey O'Connor

Senior Product Marketing Manager at Dell EMC² ETD

Welcome to another edition of Breakfast with ECS, a series where we take a look at issues related to cloud storage and ECS (Elastic Cloud Storage), EMC’s cloud-scale storage platform.

ECS Cloud Enabling ToolsA Swiss army knife is a multi-layered tool equipped with a variety of attachments that can serve up many different functions. When first introduced in the late 1880s, it revolutionized the way soldiers performed their daily tasks – anything from disassembling service rifles to opening up canned rations in the field.  Fast forward to 2016, the use of the Swiss army knife may have changed quite a bit but the initial concept of consolidating various components into a single multi-purpose tool has certainly influenced organizations and industries across the world.

EMC’s Elastic Cloud Storage (ECS) is without question the Swiss army knife for cloud solutions.  ECS revolutionizes storage management by consolidating varied workloads for object, file, and HDFS into a single, unified system.  You can manage both traditional and ‘next-gen’ or ‘cloud-native’ applications on a platform that spans geographies and acts as a single logical resource.  Just like a Swiss army knife, ECS can maximize capacity by packing a lot into a tiny space.  ECS Appliance can squeeze in sixty 8TB drives into a standard 4U DAE with up to 4PB of storage in single rack, for a highly dense platform with a very economical data center footprint.

(more…)

Breakfast with ECS: Doubling Down on Docker

Corey O'Connor

Senior Product Marketing Manager at Dell EMC² ETD

Welcome to another edition of Breakfast with ECS, a series where we take a look at issues related to cloud storage and ECS (Elastic Cloud Storage), EMC’s cloud-scale storage platform.

Unless you’ve been living under a rock I’m sure you’ve heard of Docker at this point. If you haven’t, it’s time to dust yourself off and understand that Docker containers will wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. Genius right? Docker_1

The usage of containers has been around quite some time now, but the extra juice worth squeezing came from Docker’s ability to provide total isolation of resources to package and automate applications more effectively than ever before.  Docker provides system administrators and developers the ability to package any kind of software with all its dependencies into a container. Simply put, this resource efficiency standardizes each container and promotes massive scalability – this plugs in very nicely for cloud-scale, geo-distributed systems such as EMC’s Elastic Cloud Storage (ECS). In the early stages of product development, EMC took an early bet on Docker containers and it certainly has proved to payoff.

(more…)

Follow Dell EMC

Categories

Archives

Connect with us on Twitter