Hadoop Cluster - Bookmarks and help URLs

1.5. I have a new node I want to add to a running Hadoop cluster; how do I start services on just one node?

This also applies to the case where a machine has crashed and rebooted, etc, and you need to get it to rejoin the cluster. You do not need to shutdown and/or restart the entire cluster in this case.

First, add the new node's DNS name to the conf/slaves file on the master node.

Fine tuning Apache Hadoop Security Settings

Apache Hadoop is equipped with a robust and scalable security infrastructure. These notes are intended to cluster administrators fine-tune the security settings of their clusters.

Quality of Protection:

Security infrastructure for Hadoop RPC uses Java SASL APIs. Quality of Protection (QOP) settings can be used to enable encryption for Hadoop RPC protocols.

Java SASL provides following QOP settings:

Best Practices Selecting Hadoop Hardware

Excerpts of this article are copyright there respective owners: The original article can be found at : Hortonworks

Apache Hadoop worker node hardware @ Yahoo!, a lot of nodes with 6*2TB SATA drives, 24GB RAM and 8 cores in a dual socket configuration. This has proven to be a pretty good configuration. This year, I’ve seen systems with 12*2TB SATA drives, 48GB RAM and 8 cores in a dual socket configurations. We will see a move to 3TB drives this year.

Setting up Apache Hadoop on RHEL6/CentOS 6

Setting up Apache Hadoop on RHEL6/CentOS 6 is simple wiith the recent availability of RPMs for Apache Hadoop it makes it much simpler to setup a basic Hadoop cluster. This will allow you to focus on how to use the features instead of having to learn how they were implemented.

These instructions DO NOT Hadoop settings to make Hadoop fast but it will get you running a Hadoop culster fast. We will leave Hadoop optimization for another day.

Subscribe to RSS - hadoop