Atmos rising…(from Maui)

by dave on November 10, 2008


Today is the official GA of EMC‘s latest-n-greatest product, Atmos.  For all intents and purposes, you probably have heard of Atmos under a different guise:  Maui.  In any case, I’ll be taking a look at some of the features and functionalities of Atmos as well as potential integration points for your business.  Finally, if you’re really interested, I’ll discuss the hardware underpinnings of Atmos which point towards its ability to scale to multiple-petabytes of information storage.  It will probably be easiest to “unpack” the press release, but lets get some of the foundational information out of the way.

What Atmos IS NOT

Invariably, there are going to be comparisons between Atmos and other Tier One storage offerings from NetApp, HP, Isilon, IBM, etc.  You might see comparisons between an Isilon OneFS system, for example, and Atmos, pointing out the inherent NAS-like capabilities of both (to the detriment of the Atmos, to be honest).  However, Atmos is NOT a NAS appliance.  It’s not a clustered NAS appliance and its decidedly not designed to ever act as a Tier 0, 1, or 2 storage device. It’s not a Symmetrix or a Clariion and it’s not geared towards the same workloads.  It’s not a Centera and it’s not a Celerra.   So, this begs the question, then…what IS Atmos?

What Atmos IS

Atmos is a policy-based software system designed and optimized for cloud-based computing and storage.  In English:  Atmos is the software that uses “rules” and meta-data in conjunction with YOUR operating environment to intelligently move, manage, and distribute data both geographically and locally. Atmos also scales to multiple petabytes of information.  Consider it the foundations for a “data anywhere” storage design.

Diving in…

Let’s take a look at the fundemental architecture of Atmos (exclusive of hardware):

Atmos Architecture Overview

Atmos Architecture Overview

The beauty of Atmos’ design is that it is truly designed to accomodate your applications and environment, both from an infrastructure standpoint (IP-based) as well as a code-base standpoint (REST/SOAP integration).  Starting from the upper left hand corner, let’s unpack each of the core components of Atmos.

Core Services:

Core services can be be defined as the components that are inherent to the ongoing functionality and operation of the Atmos solution.  Since Atmos relies heavily on metadata and objects (storage function), it’s no small wonder that most of the core functionality is clustered around metadata and object services.  You have your MDS (MetaData Server) and MDLS (Metadata Location Service) mappings as well as RMS (Resource Management Service) and SS (Storage Server) operational capabilities.  Overarching those functions is the CL (Client Services) and JS (Job Services) application layers that provide contact points with both API-related calls (SOAP/REST, for example) as well as the underlying PM (Policy Manager) and SM (System Manager) functional layers.  Confused yet? 😉

Think of it this way: If you’ve ever used Netflix before, this illustration will probably seem familiar.  When you log into Netflix, you’re logging into a web-based application that integrates with a meta-data server.  You click on the 5th star next to “Kung Fu Panda” and the system applies a meta-data tag on the deliverable according to your profile.  Using that meta-data tag as a method of “screening,” Netflix then can determine recommendations for additional movies to be seen based on common criteria (meta-data tags example: comedy, animation, panda bear, martial arts, 5 star rating, etc. etc.).  To break this into Atmos parlance, the Metadata server and Policy Manager will be the engine(s) working in the backend to allocate, tag, and notify the Storage Server to commit your data to physical disk.  Pretty simple, huh?

Meta-data and Objects:  What? No filesystem?

Just to diverge from the illustration above for a second, you’ll note the repeated use of meta-data and objects when referring to Atmos.  The key here is that Atmos doesn’t provide a filesystem (as it were); rather it uses meta-data and objects for annotations and storage, making retrievals infinitely more easy than typical file system approaches.  Additionally, when it comes to replication (an EMC product without innate replication facilities is…well, inhuman), the policy server can act on the objects and utilize the meta-data to determine whether or not synchronous or asynchronous replication is appropriate. Pretty nifty, eh?

Connectivity and Clients…

Moving to the top of the architecture image above, you can see that Atmos is well connected. Using IP connectivity is the foundation to this.  No expensive fibre channel connectivity here and, no iSCSI.  This is truly NAS-like functionality.  For Linux/Unix systems, you can create a IFS/NFS share or mount point and the same goes for Windows using either Samba or CIFS.  The more interesting connectivity comes with the SOAP and REST integration.  Again, relating this to the Netflix story above, the interactivity of Atmos with Web 2.0 application sets belies its ultimate functionality and power.  This truly puts Atmos as the halo product of “cloud aware storage.”  It’s feasible to see Atmos as a backend layer to something like Amazon‘s Elastic Computing (EC2/S3) or, even better, Microsoft Azure, the newly announced distributed computing OS/framework from Microsoft.

Policy, Policy, Policy.  It’s all about the Policy

What truly sets Atmos apart from others in the market place is the policy engine.  Like I stated above, Atmos is more about “software” than hardware.  At its heart, it relies on meta-data to inform and enforce the policy rules set by the developers, storage admins, etc.  What’s beneficial about this approach is the integration points with the customer’s software (whether that be organic or templated).  For example, one of the customers that I worked with this year was developing a software framework to use meta-data to move assets based on particular user’s permissions and geography.  Coupling their framework with Atmos’ policy engine meant that they could cut development time dramatically since Atmos could automate the asset management and control based on the information from the driving application.

Policy and replication are usually never considered together.  Sure, you’ll have a disaster recovery policy and procedure that dictates that replication must be used (appliance, asynchronous, or synchronous) but not HOW it should be used.  Atmos reverses that trend and notes that replication is a truly necessary way of propagating data to multiple geographies, staging areas as well as data protection.

Closing Thoughts for Today

Fundamentally, Atmos is a large departure from the “bigger and faster” mantra that EMC usually follows. Not only does Atmos promise storage on average of $2/Gb, it moves EMC firmly into the Web 2.0 (and beyond) storage age while keeping things immeasurably more simple than competitors.  In my next article on Atmos, I’ll get into the hardware that forms the platform on which Atmos can fun, but, until then, enjoy the wide variety of perspectives on the launch!

Reblog this post [with Zemanta]
Share
  • Pingback: Cloud Optimized Storage: Atmos | Info Exponential()

  • Pingback: EMC Atmos Versus VMware VDC-OS: Will The Real Cloud Strategy Please Stand Up? - Stephen Foskett, Pack Rat()

  • Dave,

    Thanks for the added detail! It’s starting to look more like a storage virtualization layer (super DFS?), but I guess the fact that it looks like everything (and tastes like chicken!) makes it seem more innovative to me.

    One thing that jumps out at me: It manages objects, and object = file = pretty big, right? So is it schlepping massive media files all over the place all the time? Or can it move chunks of objects/files?

    Thanks,
    Stephen

  • Dave,

    Thanks for the added detail! It’s starting to look more like a storage virtualization layer (super DFS?), but I guess the fact that it looks like everything (and tastes like chicken!) makes it seem more innovative to me.

    One thing that jumps out at me: It manages objects, and object = file = pretty big, right? So is it schlepping massive media files all over the place all the time? Or can it move chunks of objects/files?

    Thanks,
    Stephen

  • Dave,

    Thanks for the added detail! It’s starting to look more like a storage virtualization layer (super DFS?), but I guess the fact that it looks like everything (and tastes like chicken!) makes it seem more innovative to me.

    One thing that jumps out at me: It manages objects, and object = file = pretty big, right? So is it schlepping massive media files all over the place all the time? Or can it move chunks of objects/files?

    Thanks,
    Stephen

  • Pingback: Atmos | Craig's Musings()