It is a Java based framework that is used for storing and further process of the big data. All the data were stored in the less expensive commodity and they are working on the servers which are known as clusters.
It is a Java based framework that is used for storing and further process of the big data. All the data were stored in the less expensive commodity and they are working on the servers which are known as clusters. In the Hadoop framework it contains distributed file system which enables the processing and does not allow to make any fault. The framework has been developed by Doug Cutting and Michael J. Cafarella. Big Data Hadoop Training uses the MapReduce programming for making the storage faster than before and the getting the information quickly from the end of the framework. Hadoop framework is currently operated by the Apache Software Foundation and the license of the software company is Apache License 2.0.
From last some years the processing power of the different application servers has been increased than ever, previously the database were not working properly due to the internet issues since the internet was not able to provide the high speed data to the users. In early days the database was lagging due to the limited capacity and the slow internet. But now everything is changing and the internet speed has increased than expected and the framework Hadoop had generated lots of big data and proceed them further. The framework has changed the database completely from some of the past years and it is beneficial for the both consumers and the programmers.
Methods by which Hadoop changed the early traditional databases
1. Storage capacity: As it is was one of the major issues in the early days, the programmer had decided to solve it at first and then they will move towards any of the features. Hadoop had started using the distributed file system that is known as the Hadoop Distributed File System (HDFS). The HDFS changes the huge data into the small codes and after it the data is saved into the clusters of the servers. As the developer had taken care of the money regarding also they had built it in a very simple configuration and it is economical affordable for everyone and the data had grown largely.
2. More power: In the early database the speed was slow because of the capacity of the database and it was very difficult to extract and store the data more quickly. The data was very much big in earlier date and it was becoming difficult to submit the programmes and codes in short period of time. Hadoop framework is using the MapReduce functional programming model to perform the parallel processing data codes. Whenever any of the code is send to the database then, it is split and then send across different servers. After all those codes are saved by the programmer and then it is sent back to application.
Benefits of Hadoop
1. Resilience- The data is stored in any of the replicated in the other nodes of the cluster. It denies all the tolerance. If any case the nodes fails downs every backup is available in the database. Every code is having a duplicate of it.
2. Scalability- The data system in the early stage have a limited storage. It has now become so much convertible so that any program can be stored easily. Now the users has been rise so much, the data can be easily accessible through the framework.
3. Low cost- As Hadoop has been declared as a free framework so the cost of the framework is very less and it can be afford by all the programmers. It has been made by keeping in mind the economical factor.
BIG DATA
It is a collection of different datasets that can’t be processed through the old computing techniques. Big data is not a tool in the framework it is a complete set of book in which it contains the techniques and the frameworks. It involves the data which is produced by the different devices and the different application.
Interview Questions and Answers
Q1. What are the different Hadoop configuration files?
Ans- There are different Hadoop configuration which are enlisted in the following
a. Hadoop-env.sh
b. mapred-site.xml
c. core-site.xml
d. yarn-site.xml
e. Master and Slaves
Q2. What are the three modes in which Hadoop can run?
Ans- There are three modes in the Hadoop can be run-
1. Standalone mode: It is the default mode. It uses the system local system and single Java process to run the Hadoop services.
2. Pseudo-distributed mode: This mode use the node to Hadoop development to execute the Hadoop services.
3. Fully- distributed mode: It is used to run the Hadoop Master and the other services.
Q3. Why HDFS is fault tolerant?
Ans- HDfs is tolerant because it duplicates the data on the different DataNodes. The block of data is duplicated on the three Datanodes.
Q4. What are the two types of metadata that a NameNode server holds?
Ans- There are two different types of the Name nodes server holds are:
Metadata in Disk- This contains the edit log and the ESimage.
Metadata in RAM- This contain the information about nodes.
Q5. If you have an input file of 350mb, how many input splits would HDFS create and what would be the size of each input split?
Ans- Each block in HDFS is divided into 128 MB. The size of all the blocks, except the last block will be 128 MB. For an input file of 350 MB, there are three input splits in total. The size of each spilt is 128 MB, 128MB and 94 MB.
1897