Understanding Docker Technologoy

This article will talk about docker, one of the most popular containerisation technologies that every developer should know and master.

Understanding Docker and Docker Compose: A Comprehensive Guide

1. What is Docker?

Docker is a platform that enables developers to package applications and their dependencies into lightweight, portable containers. Think of a container as a standardized unit that includes everything needed to run a piece of software: code, runtime, system tools, libraries, and settings.

The Core Problem Docker Solves: Have you ever heard developers say “it works on my machine”? Docker eliminates this problem by ensuring that applications run identically across different environments—whether on your laptop, a colleague’s workstation, or a production server.

Key Concepts in Docker

• Images: Read-only templates that contain the application and its dependencies
• Containers: Running instances of images—lightweight, isolated processes
• Dockerfile: A text file with instructions to build a Docker image
• Docker Engine: The runtime that executes and manages containers

2. What is Docker Compose?

Docker Compose is a tool for defining and running multi-container Docker applications. While Docker handles individual containers, Docker Compose orchestrates multiple containers that work together.

2.1 Why Docker Compose?

Let me ask you this: What happens when your application needs multiple services? For example, a web application might need:

• A web server (Node.js, Python, etc.)
• A database (PostgreSQL, MongoDB)
• A cache layer (Redis)
• A reverse proxy (Nginx)

Running each service individually with separate docker run commands becomes tedious and error-prone. Docker Compose solves this by letting you define all services in a single YAML file.

3. Key Differences

Aspect	Docker	Docker Compose
Purpose	Manages individual containers	Orchestrates multiple containers
Configuration	Command-line or Dockerfile	YAML configuration file
Use Case	Single-container applications	Multi-container applications
Networking	Manual network setup	Automatic network creation
Startup	One container at a time	All services with one command

4. Benefits of Docker Technology

4.1. Consistency Across Environments

Eliminates the “works on my machine” problem by ensuring identical environments everywhere.

4.2. Isolation

Each container runs independently, preventing conflicts between different applications or versions.

4.3. Efficiency

Containers share the host OS kernel, making them more lightweight than traditional virtual machines.

4.4. Rapid Deployment

Start, stop, and rebuild applications in seconds rather than minutes.

4.5. Scalability

Easily replicate containers to handle increased load.

4.6. Version Control

Docker images can be versioned, making rollbacks simple and reliable.

5. Practical Example: Dockerizing a Full-Stack Application

Let me walk you through dockerizing a real-world application: a Node.js/Express API with a PostgreSQL database and Redis for caching.

5.1 Project Structure

my-app/
├── backend/
│   ├── src/
│   │   └── server.js
│   ├── package.json
│   └── Dockerfile
├── docker-compose.yml
└── .env

5.2 Steps

Step 1: Create the Backend Application

backend/src/server.js

const express = require('express');
const { Pool } = require('pg');
const redis = require('redis');

const app = express();
const port = process.env.PORT || 3000;

// PostgreSQL connection
const pool = new Pool({
  host: process.env.DB_HOST,
  port: process.env.DB_PORT,
  database: process.env.DB_NAME,
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
});

// Redis connection
const redisClient = redis.createClient({
  socket: {
    host: process.env.REDIS_HOST,
    port: process.env.REDIS_PORT,
  }
});

redisClient.connect();

app.use(express.json());

// Health check endpoint
app.get('/health', async (req, res) => {
  try {
    await pool.query('SELECT NOW()');
    await redisClient.ping();
    res.json({ status: 'healthy', timestamp: new Date() });
  } catch (error) {
    res.status(500).json({ status: 'unhealthy', error: error.message });
  }
});

// Example endpoint with caching
app.get('/users', async (req, res) => {
  try {
    // Check cache first
    const cachedUsers = await redisClient.get('users');
    if (cachedUsers) {
      return res.json({ source: 'cache', data: JSON.parse(cachedUsers) });
    }

    // Query database
    const result = await pool.query('SELECT * FROM users LIMIT 10');
    
    // Cache the result
    await redisClient.setEx('users', 60, JSON.stringify(result.rows));
    
    res.json({ source: 'database', data: result.rows });
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

backend/package.json

{
  "name": "backend",
  "version": "1.0.0",
  "main": "src/server.js",
  "scripts": {
    "start": "node src/server.js"
  },
  "dependencies": {
    "express": "^4.18.2",
    "pg": "^8.11.0",
    "redis": "^4.6.7"
  }
}

Step 2: Create the Dockerfile

backend/Dockerfile

# Use official Node.js runtime as base image
FROM node:18-alpine

# Set working directory in container
WORKDIR /app

# Copy package files
COPY package*.json ./

# Install dependencies
RUN npm install --production

# Copy application code
COPY src ./src

# Expose the port the app runs on
EXPOSE 3000

# Command to run the application
CMD ["npm", "start"]

**Note: **Each comment means what each instruction does. Understanding the layered nature of Docker images is crucial—each instruction creates a new layer.

Why copy package.json separately? This is a Docker best practice called “layer caching.” Docker caches each step. If your code changes but dependencies don’t, Docker skips reinstalling everything. This makes rebuilds MUCH faster.

Step 3: Create the Docker Compose Configuration

docker-compose.yml

version: '3.8'

services:
  # Backend API Service
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    container_name: my-app-backend
    ports:
      - "3000:3000"
    environment:
      - PORT=3000
      - DB_HOST=postgres
      - DB_PORT=5432
      - DB_NAME=myappdb
      - DB_USER=admin
      - DB_PASSWORD=secretpassword
      - REDIS_HOST=redis
      - REDIS_PORT=6379
    depends_on:
      - postgres
      - redis
    networks:
      - app-network
    volumes:
      - ./backend/src:/app/src  # Hot reload for development
    restart: unless-stopped

  # PostgreSQL Database
  postgres:
    image: postgres:15-alpine
    container_name: my-app-postgres
    environment:
      - POSTGRES_DB=myappdb
      - POSTGRES_USER=admin
      - POSTGRES_PASSWORD=secretpassword
    ports:
      - "5432:5432"
    volumes:
      - postgres-data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    networks:
      - app-network
    restart: unless-stopped

  # Redis Cache
  redis:
    image: redis:7-alpine
    container_name: my-app-redis
    ports:
      - "6379:6379"
    networks:
      - app-network
    restart: unless-stopped

# Named volumes for data persistence
volumes:
  postgres-data:

# Network for service communication
networks:
  app-network:
    driver: bridge

Step 4: Create Database Initialization Script

init.sql

-- Create users table
CREATE TABLE IF NOT EXISTS users (
    id SERIAL PRIMARY KEY,
    username VARCHAR(50) UNIQUE NOT NULL,
    email VARCHAR(100) UNIQUE NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Insert sample data
INSERT INTO users (username, email) VALUES
    ('john_doe', 'john@example.com'),
    ('jane_smith', 'jane@example.com'),
    ('bob_wilson', 'bob@example.com')
ON CONFLICT (username) DO NOTHING;

Step 5: Running the Application

Here’s where Docker Compose shows its power. Instead of running multiple docker run commands, you execute just one:

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f

# Check service status
docker-compose ps

# Stop all services
docker-compose down

# Rebuild and restart
docker-compose up -d --build

5.3 Understanding the Docker Compose Configuration

Let me break down what’s happening here—can you identify why we use depends_on? It ensures services start in the correct order, though it doesn’t wait for a service to be “ready,” only “started.”

Key Features Demonstrated:

1. Service Definition: Three services (backend, postgres, redis) working together
2. Networking: Automatic network creation allows services to communicate using service names
3. Volume Mounting: Data persistence for PostgreSQL and hot-reload for development
4. Environment Variables: Configuration without hardcoding values
5. Port Mapping: Exposing services to the host machine
6. Dependencies: Ensuring proper startup order

5.4 Testing the Application

# Check health
curl http://localhost:3000/health

# Get users (first call hits database)
curl http://localhost:3000/users

# Get users again (second call hits cache)
curl http://localhost:3000/users

6. Advanced Docker Compose Features

6.1 Environment Files

Create a .env file for sensitive data:

DB_PASSWORD=secretpassword
POSTGRES_PASSWORD=secretpassword

Reference in docker-compose.yml:

environment:
  - DB_PASSWORD=${DB_PASSWORD}

6.2 Multiple Compose Files

# Development
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

# Production
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up

7. Best Practices of Docker Compose

1. Use .dockerignore: Exclude unnecessary files from the build context

2. Multi-stage builds: Reduce image size by separating build and runtime dependencies

   1. Single-Stage Build (Inefficient)

      dockerfile

      FROM node:18
      WORKDIR /app
      COPY package*.json ./
      RUN npm install           # Installs ALL dependencies (including dev tools)
      COPY . .
      RUN npm run build         # Builds your app
      CMD ["npm", "start"]
      
      # Problem: Final image includes build tools, dev dependencies, source code
      # Image size: ~500MB

      Multi-Stage Build (Efficient)

      dockerfile

      # Stage 1: BUILD - Use full Node.js with build tools
      FROM node:18 AS builder
      WORKDIR /app
      COPY package*.json ./
      RUN npm install                    # All dependencies
      COPY . .
      RUN npm run build                  # Creates /app/dist
      
      # Stage 2: RUN - Use lightweight Node.js, copy only what's needed
      FROM node:18-alpine               # Smaller base image
      WORKDIR /app
      COPY package*.json ./
      RUN npm install --production      # Only production dependencies
      COPY --from=builder /app/dist ./dist   # Copy ONLY the built files
      
      CMD ["node", "dist/server.js"]
      
      # Result: Final image only has production code and dependencies
      # Image size: ~150MB

      Key insight: The final image only contains what’s in the LAST stage. Everything from earlier stages is discarded.

3. Don’t run as root: Create non-root users in containers for security

4. Use specific image tags: Avoid latest for reproducibility

5. Health checks: Implement health checks for better orchestration

6. Named volumes: Use named volumes for important data