System Components

Bigtable — System Modules

The core building blocks that make Bigtable scalable, reliable, and fast.

Overview

Bigtable is composed of five main modules working together to provide distributed, large-scale structured storage:

  • Master Server — Coordinates tablet assignments and handles metadata.
  • Tablet Servers — Store and serve data; each handles a subset of tablets.
  • GFS (Google File System) — Stores commit logs and SSTable files.
  • Chubby — Provides distributed locking and master election.
  • Client Library — Interfaces directly with tablet servers for read/write operations.

Master Module

The Master controls the overall system but stays out of the data path to avoid bottlenecks. It assigns tablets, monitors server health, and handles tablet splits and recovery.

  • Manages metadata and tablet distribution.
  • Detects and reassigns tablets from failed servers.
  • Coordinates load balancing across the cluster.

Tablet Server Module

Tablet Servers manage tablets, which are continuous ranges of rows from a table. They serve client read/write requests directly and handle compactions and splits automatically.

  • Each tablet stored as commit log + SSTables + memtable.
  • Handles user queries and updates efficiently.
  • Splits tablets when they grow too large.

GFS Module

The Google File System provides the underlying file storage for Bigtable. It ensures durability and fault tolerance for all persistent data.

  • Stores immutable SSTable files and commit logs.
  • Automatically replicates data across multiple machines.
  • Facilitates fast recovery from hardware failures.

Chubby Module

Chubby is a distributed lock service that Bigtable uses for coordination, master election, and metadata storage.

  • Ensures only one active master at a time.
  • Stores critical configuration and tablet location metadata.
  • Provides lightweight synchronization for servers.

Client Library Module

Clients interact directly with tablet servers to perform reads and writes, bypassing the master for efficiency.

  • Caches tablet location data for quick access.
  • Automatically recovers from stale tablet mappings.
  • Uses efficient retry and backoff strategies for reliability.

Bigtable API

The Bigtable API provides a simple interface for applications to read, write, and manage data stored in tables.

  • Create/Delete Tables — Define new tables and column families or remove unused ones.
  • Read/Write Operations — Access rows using Get or Scan; modify data with Put and Delete.
  • Atomic Row Mutations — All operations on a single row are atomic.
  • Filters — Select only specific columns, versions, or values during reads.
  • Timestamps & Versions — Retrieve or write different versions of a cell by timestamp.

Bigtable’s API is intentionally simple — no joins or complex queries — focusing instead on scalable, high-throughput operations across massive datasets.

Summary Table

Module Role In Data Path Manages State Scalable
Master Coordinates & monitors tablet servers No Yes Yes
Tablet Server Serves data and manages tablets Yes Yes Yes
GFS Stores commit logs and SSTables Yes (I/O) Yes Yes
Chubby Lock service & metadata coordination No Yes Yes
Client Library Interfaces with tablet servers Yes No Yes