Posts tagged as:

google

Micro-burst: Master Node Topology

by dave on August 12, 2009


Going along with the Cloud theme that I’m fortunate enough to be a part of, I’ve decided to use the “micro-burst” moniker to section off quick n’ dirty posts on a variety of cloud subjects that I don’t have time to dive into fully.  With that in mind, let’s get on with the show.

Master Node Topologies: Google File Systems

Today’s topic is based on the link here from The Register.  What I find fascinating is that Google has been able to manage their growth using a single master node topology for their filesystem.  To the article’s point, a single master node offers a single point of failure especially from a chunklet processing and scheduling standpoint.  Bandwidth would also be constrained seeing as how meta would have to pass through and be processed by a single  entity.  Since I’m unaware of the underlying hardware and scalability of their processing complex for this (though I’ve read through the articles that have attempted to explain it), these processing issues could reasonably be remedied by more powerful system hardware and/or software refinement.

It’s exciting to see that Google has thus far been able to move their GFS platform forward and embrace a horizontal scale-out mechanism for the revision 2 product.  Good luck to them as they continue to move their company forward!

Why a Master Node Plurality Makes Sense

When designing for any sort of scale-out filesystem (or what I’d consider a horizontally scalable file system), it makes sense to include the ability to scale the master node (or scheduler node) complexes.  The obvious reason behind this is filesystem growth, to be sure, but as metadata processing becomes increasingly complex (i.e. more FS abilities driven by custom meta), the need to ingest data at the same or higher rate as originally specified becomes critical.  By having a more robust front-end driven by more powerful master nodes with synchronous metadata indexes (or siloed masters with individual meta dbs), you can maintain latency (time to disk or time to commit) SLAs without completely crushing your cloud’s ability to service I/O operations in general. (see image below for conceptual diagram)

Multiple Masternode File system

Multiple Masternode File system

Hopefully my musings on this subject make sense.  Let me know if you have any questions!

cheers,

Dave

Related articles by Zemanta
Reblog this post [with Zemanta]

{ 2 comments }

Cloud Personality Types…

by dave on February 20, 2009


In the mental health world, a lot of talk is made about various pathologies, their presentations, as well as tying them to certain personality types that pre-dispose them to affliction.  That’s a mouthful to process through, but the same type of examination can be applied to those who engage in “cloud oriented behaviour.”  Here is my DSM of cloud personality types.

The “Cloud Idiot” – (cloudiot) – This is the person who thinks they know more about the cloud than anyone else. They’re constantly on the prowl for the “what is …?” questions on social media platforms and provide blustery responses with vapid data validation.  Oftentimes, these folks are proven wrong in a rather humiliating and public fashion.

The “Cloud Chaser” – (cloudparazzi) – This is the person who is “All cloud, all the time” and is looking for the next “big thing” in the the cloud “atmosphere.”  Usually found trolling for data at Geva Perry‘s blog or accosting CTOs, VPs, etc. on social media.

The “Cloud Antagonist” – (cloudagonist) – This is the commiserate cloud “hater.”  This person loves DAS storage, SANs, divided fabrics and can be found extoling the virtues of direct server management via commandline and a collection of USB sticks.

The “Cloud Masochist” – (cloudochist) – This person, as opposed to the cloudagonist, is ALL in the cloud.  Using S3, GoGrid, SnapLogic, et al. as his storage and processing via EC2, the cloudochist is “putting the hurt on” those naysayers who think the cloud is bad for business.  Incidentally, the cloudochist gets unpaid vacation when the backend services go down.

The “Cloud Evangelist” – (cloudgelist) – This person extols the virtues of the cloud and it’s capabilities without ever having used any of the services. Ignorance, while blissful, doesn’t pay the bills. However, certain cloudgelists have noted that they use the product (whether it’s their own, who knows?) and it does indeed lessen wrinkles and promotes good colon health…wait, that’s something else entirely! UPDATE:  @CXI mentioned that the cloudgelist can use the cloud and still extol its virtues. Far be it from me to exclude anyone. definition revised/extended.

The “Cloud Manic” – (cloudpolar) – This person alternatively loves AND hates the cloud and all it stands for within a given week. Can usually be found smack talking a vendor about their primary storage platform in the cloud while loving on a PaaS vendor for “connecting the dots.” Rinse and repeat.

[click to continue…]

{ 13 comments }

Atmos rising…(from Maui)

by dave on November 10, 2008


Today is the official GA of EMC‘s latest-n-greatest product, Atmos.  For all intents and purposes, you probably have heard of Atmos under a different guise:  Maui.  In any case, I’ll be taking a look at some of the features and functionalities of Atmos as well as potential integration points for your business.  Finally, if you’re really interested, I’ll discuss the hardware underpinnings of Atmos which point towards its ability to scale to multiple-petabytes of information storage.  It will probably be easiest to “unpack” the press release, but lets get some of the foundational information out of the way.

[click to continue…]

{ 4 comments }

Search Term Reviews: October 15th Edition

October 15, 2008

It’s been awhile since I last did a review of what people are searching for (July 30th was the last time…wow) so, let’s see what’s new.
Search Term #1: EMC NX4
Not really suprised here.  Honestly, take the Celerra NS-20, cut the price signficantly, allow blended SAS/SATA drive trays,

<br />