Components of HDFS

HDFS, it mainly consists of the following components:

  • NameNode:
  • It is the master server, which runs on the Master Node in this configuration.
  • It is responsible for maintaining the metadata (data about the locations of file blocks across the cluster, ownership rights, etc.) of the different files present in the cluster.
  • It is also responsible for assigning work to the slave nodes as well as executing file system namespace operations such as opening and renaming files, other types of modifications, etc.
  • Secondary NameNode:
  • It is responsible for keeping the metadata of the NameNode.
  • The NameNode then uses this metadata to update its own metadata.
  • The metadata present in the Secondary NameNode is used for implementing(note that this is different from itself acting as the NameNode) a new NameNode in case of failure of the current NameNode.
  • Standby NameNode:
  • This is another component of HDFS and is responsible for providing fault tolerance against the problem of a single point of failure (SPOF) related to the NameNode.
  • It provides automatic failover in case an active NameNode fails.
  • DataNodes:
  • They are the slave nodes.
  • They are responsible for actually storing and processing data of the Hadoop cluster.
  • They provide access to data files when requested.
  • They also send heartbeat messages to the NameNode regularly to indicate they are alive.
  • HDFS Client:
  • It acts as an intermediate command-line tool for interacting with HDFS.
  • It has access to the Java libraries that are needed for the various applications in HDFS.
  • When accessing HDFS, it is the HDFS Client that interacts with the NameNode and the DataNodes to get the job done.

--

--

--

Tech enthusiastic, life explorer, single, motivator, blogger, writer, software engineer

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

My learning experience in 2020

The Recruit’s Experience

3 reasons why I went framework agnostic and why you should do that too

February 2022 — Verus Monthly Recap

The Silo Dance

Scraping Skoob’s Bookshelf

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
TechGuy

TechGuy

Tech enthusiastic, life explorer, single, motivator, blogger, writer, software engineer

More from Medium

Number of Good Pairs | Python Solution | LeetCode Easy Problem #1512

Role of Stack and Queue in Problem Solving

Finding Work: Devotion and work

How to remove namespaces in XML? Here is how!!