Future Storage Systems: Part 4: Operating System – Conceptual Overview

by dave on October 13, 2008

In the previous Future Storage System articles, we’ve covered the basic hardware foundation for what I envision to be a powerful future-oriented storage solution for the commercial midrange.  However, as you’re probably aware, hardware is meaningless without software to provide the operational capabilities that are needed to mange information.  In this article, I will focus on a general design for an extensible software layer (an OS) that will provide future-oriented capability expansion as well as robust analytics, capabilities, and integration with business continuity principles.  As always, please reference the diagram below.

Future Storage System - Operating System - Conceptual

Future Storage System - Operating System - Conceptual

I will review this diagram starting from the upper right hand corner and will drill down on each bullet point made.  Again, as these ideas are further developed, I’ll probably move to a more classic diagram via Visio or something similar, but, for now this will have to suffice. 😉 You should see my whiteboard right now. 😉


When discussing the core of the operating system, I have to be somewhat ambiguous, namely, the underlying kernel technology will dictate the inherent flexibility of the design. I will be honest in that I believe a hypervisor-based model (a la VMWare, Xen, et al.) will enable more features and flexibility than a bare-metal OS with a direct kernel/HAL relationship.  Additionally, as we move further into the features and functionality of this FSS OS, this hypervisor building model will become extremely important.  I’ll reserve details on that for another post.

Getting back to the subject at hand, I’ve divide the “Core” section into 4 basic sections: Drivers, I/O Engine, Hypervisor (?), and Management (Base). Let’s discuss the roles of each of these.

  • Drivers.  This particular aspect of the Core OS deals with how the hardware interfaces with the OS itself.  This can be either a generic driver model (a la VMWare) or can be manufacturer specific and developed/maintained (a la Hyper-V).  There are advantages to both approaches;  with generic driver models, you get away from havoc-causing revisions to driver stacks or resource models that can (and are) introduced on a regular basis.  By standardizing on a generic driver model, you can also set specific boundaries around drivers for QA purposes and keep your overall OS implentation “clean.”  With specific driver models, you reap the benefits of core optimizations made by the manufacturer. This adds to the overall performance envelope of the underlying hardware technology as it relates to the OS layer. It obviously introduces some level of risk to the OS in that driver refreshes could theoretically “break” existing connections or the OS itself, but this could be mitigated by extensive modeling and alpha/beta programs.
  • I/O Engine. Again, talking in theoretics here, the I/O engine encompasses the basic storage functions for the OS.  This could be broken out into Data Protection Algorithms (i.e RAID) and Encode/Decode functions for passing data to/from cache and disk.
  • Hypervisor.  As stated in the start of this section, the model I’m embracing is a hypervisor-based one that allows for functionality and features to be “plugged in” through software and hardware (don’t forget Torrenza!!!).
  • Management (Base).  This management function relates to the capabilities of the FSS operator to accomplish basic tasks through a GUI or CLI interface.  It would involve RAID/Storage group creation, host assignment, registration & configuration, and other basic tasks that are typically accomplished.  As noted in the hardware overiew of Part 2, there would be a dedicated management port (GigE) designed to handle interaction.


When discussing the extended section of the OS, I’m referring to features that extend the native capabilities of the OS as installed.  This section deals with building out the OS to cover such features as Thin (virtual) Provisioning, System Analytics, Quality of Services features, and DeDuplication.  I’ve kept this list to four major categories but I do assume that there could be other technologies that emerge that will add to this list.

  • Thin (Virtual) Provisioning. Call it “thin,” call it “virtual,” it doesn’t matter.  Companies (and people) that get caught up in semantics really have better things to do with their time (like actually implementing hardware FC, not software emulation) ;).  Essentially, Thin Provisioning allows the Future Storage System to manage information growth by only allocating storage that is needed versus what is wanted.  By spoofing the storage size to the attached hosts, you’re able to better handle growth and storage utilization (important when you’re already dealing with usable capacities below 50%).
  • System Analytics. Included in the Core OS is a base level of analytics, equivalent to the SysInfo command you could run on a typical Microsoft or VMWare OS.  By including this from the start, most customers would be able to report on their hardware/software frameworks and determine if there were any failures.  Beyond this basic level of reporting, there often arises the need to determine what is happening from a hardware standpoint, i.e. Processor load, RAID Group response time, etc.  The System Analytics extension would provide in-depth reporting facilities to “fine tune” particular aspects of the FSS.
  • Quality of Service. Similar in approach to DRS-enabled clusters within the VMWare environment (where you can assign resource “shares” across several different slices of physical hardware), the QoS features of the FSS would allow RAID Group or LUN level performancing based on policies (covered in the “Specialized” Section below) in addition to potentially handing vNIC (IP, iSCSI, FCoE?) or vPORT (FC) splitting and throttling for bandwidth.  Current storage system implement similar features at the LUN level or RAID Group level only; the extension of QoS to the port level on the storage system itself is rather unique.
  • Deduplication.  There’s no doubt about it; whether we’re talking about NetApp‘s ASIS or EMC’s Avamar, data deduplication is a feature that is here to stay.  The affects of deduplication may not be obvious at first but, over the lifespan of your data, it will enable quicker backups and replication in addition to increasing storage efficiency on your storage system.  Not much else to say about this particular technology for now, but, suffice it to say, as an extension to the Core OS, it can be a powerful tool.


Outside of the Extended features that could be “plugged in” to the Core framework, I determined that there might be a few more specialized tasks that would require extra integration from either 3rd party resources or just additional developmental work beyond the norm.  In this particular category, I’ve put In-Band/Out-of-Band (Data at Rest) Data Encryption and Policies (Spindown, Power Management).  Let’s dive into these for a moment.

  • Data Encryption.  With compliance laws (PCI, HIPAA, DoD, et al) becoming more and more de rigueur in storage and communication, it almost seems that Data Encryption should be a Core component versus an add on.  Nevertheless, it’s important to recognize that you need to be able to accomplish Data Encryption (and Decryption) on resting data and on data that is being processed for storage.  If anything, this is a perfect use case for Torrenza pluggables. 😉  Standardizing encrytion on the FSS also means that you’ll need to have some level of “hooks” to the Core engine.
  • Policies.  Ah, Policy.  Everyone hates the word but doesn’t understand the importance of policy until they run into a policy-less situation.  In any case, the FSS policy add-on would be more akin to a scheduling engine for job processing, encrypt/decrypt, deduplication, tiering (storage tiering), etc.  It could assume responsibilities for power management as well, similar to the Dynamic Power Management (DPM) functions of VMWare. Whether this stays as an add on or moves to Core remains to be seen.

Business Continuity:

Business Continuity, like Data Deduplication, is a hot topic these days.  In my mind, Business Continuity encompasses not only Disaster Recovery metrics but also data management characteristics. To that end, I’ve broken this section into 3 distinct pieces: Remote replication, local replication, and tiering.

  • Remote Replication. As the term states, this section is devoted to moving data from source FSS to target FSS.  The goal is ultimately data protection over any distance outside of the physical geography of the source array (i.e. New York City to New Jersey).  The mechanism for this replication would be either synchronous (acknowledged write commits from source/target arrays) or asynchronous (delayed ack from target array), depending on RPO/RTO SLAs.
  • Local Replication. Local replication I view as being within the data center, NOC, or campus.  While the physical location might be removed somewhat, the communication would stay on dedicated high speed links (dark fibre, fibre connection, multi-fabric SAN, etc.).  Replication under these auspices would more than likely err on the side of synchronous but could be asynchronous as well.
  • Tiering.  Before anyone objects, there is a method to my madness.  Typically, storage tiering would be slapped under the Extended or Specialized categories based on my diagram above.  However, think of it this way.  Moving low priority data from primary to secondary storage is a form of BC, isn’t it?  You’re moving, managing, and making available data under the auspices of a different SLA, potentially to different subsystems (i.e. Clariion to Centera), with different connectivity.  What if the Future Storage System could move data AND the communication channel from higher speed SSDs or mechanical disks on FCoE or IB to slower SATA drives using vNICs at GigE bandwidth levels?  That type of manipulation would obviously require some enormous computingcomputing capabilities (as well as potentially buffering to SSDs or high speed cache), but the end result could be completely managed data SLAs.

Closing Thoughts:

A lot to chew through on this article and I hope you’ll point out the flaws in my thinking.  For the next part, I probably will dive into the Core of the OS a bit more and see what technical underpinnings would work.



Reblog this post [with Zemanta]