Apache Hadoop Operations at Scale

Hadoop Summit Content Curation

Although the Hadoop Summit San Jose 2014 has come and gone, the invaluable content—keynotes, sessions, and tracks—is available here. I’ve selected a few sessions below for Hadoop system administrators and dev-ops, curating them under a general Hadoop operations theme.

Dev-ops engineers and system administrators know best that ease of operations and deployments can make or break a large Hadoop production cluster, which is why they care about all of the following:

• how rapidly they can create or replicate a cluster;
• how efficiently they can manage or monitor at scale;
• how easily and programmatically they can extend or customize their operational scripts; and
• how accurately they can foresee, forestall, or forecast resource starvation or capacity stipulation.

Not having so many desirable outcomes can deprive them of their sleep. In the case of Hadoop’s large-scale cluster operations and management, where the Enterprise Hadoop ecosystem comprises of both traditional and modern infrastructure components, the operational tasks can be herculean.  As @DevOps_Borat sanguinely satirizes:

The good news is that people at the helm—at the nerve center of operations—shared their best practices on how they address and manage theses complex challenges at the Hadoop Summit. Here are a few:

Hadoop Operations at Scale

I cherry picked these few tracks that best addressed those topics, but you can always peruse through all the tracks on the schedule’s session description along any time slot, on any day, that piques your curiosity.

For example, when you hover and click on a session description, a popup will display in which you can either elect to watch the video or view the slides.

What’s Next?

In the next blog, I’ll curate content on data access and management, in particular, the role YARN plays as an architectural anchor and the center of Modern Data Architecture (MDA).

The post Apache Hadoop Operations at Scale appeared first on Hortonworks.