BigData Practical Analytics

Introduction to Hadoop

• Hadoop Distributed File System
• Comparing Hadoop & SQL
• Industries using Hadoop
• Data Locality
• Hadoop Architecture
• Map Reduce & HDFS

Hadoop Distributed File System (HDFS)

• HDFS Design & Concepts
• Blocks, Name nodes and Data nodes
• HDFS High-Availability and HDFS Federation
• Hadoop DFS The Command-Line Interface
• Basic File System Operations
• Anatomy of File Read, File Write
• Block Placement Policy and Modes
• More detailed explanation about Configuration files
• Metadata, FS image, Edit log, Secondary Name Node and Safe Mode
• How to add New Data Node dynamically, decommission a Data Node dynamically (Without stopping cluster)
• FSCK Utility. (Block report)
• How to override default configuration at system level and Programming level

Map Reduce

• Map Reduce Functional Programming Basics
• Map and Reduce Basics
• How Map Reduce Works
• Anatomy of a Map Reduce Job Run
• Shuffling and Sorting
• Splits, Record reader, Partition, Types of partitions & Combiner
• Distributed Cache and Hadoop Streaming (Python, Ruby and R)
• YARN
• Sequential Files and Map Files
• Map side Join with distributed Cache

Map Reduce Programming – Java Programming

• Hands on “Word Count” in Map Reduce in standalone and Pseudo Distribution Mode
• Write some Map Reduce programs to solve some real world problems

Spark Basics

• What is Apache Spark?
• Spark Installation
• Spark Configuration
• Spark Context
• Using Spark Shell
• Resilient Distributed Datasets (RDDs) – Features, Partitions, Tuning Parallelism
• Functional Programming with Spark

Working with RDDs

• RDD Operations – Transformations and Actions
• Types of RDDs
• Key-Value Pair RDDs – Transformations and Actions
• MapReduce and Pair RDD Operations
• Serialization

Spark on a cluster

• Overview
• A Spark Standalone Cluster
• The Spark Standalone Web UI
• Executors & Cluster Manager
• Spark on YARN Framework

Writing Spark Applications

• Spark Applications vs. Spark Shell
• Creating the Spark Context
• Configuring Spark Properties
• Building and Running a Spark Application
• Logging
• Spark Job Anatomy

دردوره هاي Non-Accredit شركت كنندگان در دوره هاي آموزشي ازسوي مرجع اعتبار دهی IT HOUSE گواهي حضوردردوره ي آموزشي دريافت خواهند نمود.