Mapreduce Performance Evaluation through Benchmarking and Stress Testing On Multi-Node Hadoop Cluster

Over the past few years, many data based applications have been developed using the powerful tools provided by the World Wide Web. In today’s world an organization may generate petabytes of data each second. To gain insights from this vast pool of data, applications based on data-mining and web indexing such as artificial neural networks, decision trees etc., play an important role. Such tools are tremendously helping social networking sites such as Facebook and Twitter to get meaningful information from their data. Hadoop is an open source distributed batch processing infrastructure. It works by distributing the amount of processing required by a task across a number of machines, thereby balancing the load depending on the node. Hadoop, in a very short time has gained huge popularity amongst many IT firms. Infact, many big players such as IBM, Google, Quora and Yahoo are using numerous applications based on Hadoop infrastructure.