Hadoop Essentials: Decoding HDFS and YARN

Hadoop is like a HelloFresh kit. Just as HelloFresh provides pre-planned ingredients for a meal, Hadoop offers essential components and guidelines for processing and storing large datasets in a scalable, distributed system. It includes four key components: HDFS for data storage, MapReduce for data processing, YARN for managing resources and scheduling tasks, and Hadoop Common, which provides shared resources like libraries and utilities. This makes Hadoop a powerful tool for building scalable distributed systems for big data.

In this first part, we dive into the first two components: HDFS and YARN.

You can see my interactive slides here

OR static slides below:

Hadoop Visualized - Part 1: HDFS & YARN

S0 - Hadoop Overview

S0 - Hadoop Architecture

S1 - HDFS section intro

S1 - HDPF's NameNode & DataNode

S1 - How HDFS works?

S1 - What ensure HDFS's data availability & fault-tolerance?

S1 - Replication & Placement Strategies in HDFS

S1 - Summary - HDFS for data storage

S2 - YARN section intro

S2 - What is a YARN container?

S2 - What are the benefits of using YARN containers?

S2 - How YARN works?

S2 - Summary - YARN for resource management and job scheduling

Follow The Data Cherry