Docker & Docker Swarm: 18-Day Production Masterclass 🐳☸️

Day 1 Docker Architecture & Core Fundamentals

Understanding how Docker packages and executes applications starts with its six core architectural components:

• Docker Host: The physical or virtual Linux server where Docker is installed and containers execute.
• Docker Image: Think of an image like an .exe or .iso file. It is a read-only blueprint containing application code, runtime libraries (bin/lib), and dependencies.
• Docker Container: A lightweight, isolated, and portable running instance of a Docker image.
• Docker Client: The command-line interface tool (docker command) that accepts user instructions and forwards them to the daemon.
• Docker Daemon: The background engine running on the Docker Host that listens to client requests, builds images, runs containers, and manages system resources.
• Docker Registry: A storage and distribution repository for Docker images. Registries can be Public (Docker Hub, default and open to everyone) or Private (used by organizations to store internal images securely).

How the Architecture Works Together & Initial Commands

When you run docker run nginx, the Docker Client sends the instruction to the Docker Daemon. The Daemon pulls the nginx image from the Registry (if not already cached locally) and launches a running Container.

# Pull official images from Docker Hub (Public Registry)
docker pull nginx
docker pull mongo

# Run an Nginx web server container
docker run nginx

# Delete a single specific image from the Docker Host
docker rmi nginx

# Full system cleanup: delete all unused containers, networks, and images
docker system prune -af

Day 2 Linux Installation, Container Lifecycle & Port Mapping

Install the latest stable versions of Docker CLI, Docker Engine, and their dependencies on any Linux server using the official convenience script:

# 1. Download the official Docker installation script
curl -fsSL https://get.docker.com -o get-docker.sh

# 2. Execute the installation script
sh get-docker.sh

# 3. Verify installed Docker version
docker --version

# 4. Switch to root account for administrative operations
sudo su - root

Working with Containers & Port Mapping (-p vs. -P)

• Foreground vs. Detached Mode (-d): Running docker run nginx streams live logs in the foreground and locks your terminal prompt. Adding -d runs the container in the background and prints its Container ID.
• Custom Container Naming (--name): If you omit --name, Docker assigns a random name automatically.
• Manual Port Mapping (-p): Syntax is -p <HOST_PORT>:<CONTAINER_PORT> (e.g., -p 8888:80). Note that two containers cannot bind to the exact same host port simultaneously.
• Random Port Mapping (-P): Capital -P automatically maps the container's internal port to an available high host port above 30000.

# List all locally downloaded images
docker images

# Run Nginx in detached mode with a custom name and manual port mapping (8888 -> 80)
docker run --name mynginx -d -p 8888:80 nginx

# Inspect running containers (ID, Image, Command, Created, Status, Ports, Names)
docker container ls

# Run current Jenkins LTS (note: 'jenkins' is deprecated; always use 'jenkins/jenkins')
docker pull jenkins/jenkins
docker run --name jenkins123 -d -p 9090:8080 jenkins/jenkins

# Pull and run Apache TomEE with automatic high-port mapping (-P maps > 30000)
docker run --name tomcat -d -P tomee

# Run TomEE on explicit host port 8080
docker run --name tomcat1 -d -p 8080:8080 tomee

# Force remove all running and stopped containers at once
docker rm -f $(docker ps -aq)

Day 3 Interactive OS Containers, SQL Databases & Container Linking

Containers can run interactive Linux distributions, stateful SQL databases, or interconnected multi-tier applications:

1. Operating Systems as Interactive Containers (-it)

# Launch an interactive Ubuntu bash terminal (-it)
docker run --name u1 -it ubuntu
# Note: Prompt changes to root@0ffb094e529e:/#
# Press Ctrl+P followed by Ctrl+Q to detach without stopping, or type 'exit'

# Launch an interactive CentOS terminal and clean up
docker run --name c1 -it centos
docker rm -f c1 u1

2. SQL Database as a Container (-e & docker exec)

Database images like MySQL require an environment variable (-e MYSQL_ROOT_PASSWORD) at launch; without it, the container will fail to initialize. To run SQL queries inside the running container, use docker exec -it:

# List both running and stopped containers
docker ps -a

# Start MySQL with mandatory root password environment variable
docker run --name mydb -d -e MYSQL_ROOT_PASSWORD=secretpass mysql:latest

# Open an interactive bash terminal inside the running MySQL container
docker exec -it mydb bash

# Log into the MySQL CLI inside the container
mysql -u root -p
# Inside MySQL prompt:
# USE mysql;
# EXIT;

3. Multi-Container Communication via --link

Five methods enable container-to-container communication: 1. --link (legacy CLI linking), 2. Docker Compose, 3. Docker Networks, 4. Python scripting, and 5. Ansible Playbooks.

# Example 1: Linking two BusyBox containers with an alias
docker run --name c1 -it busybox
# Detach using Ctrl+P, Ctrl+Q

docker run --name c2 -it --link c1:mybusybox busybox
# Inside c2 terminal, ping c1 by container name or alias:
ping c1
ping mybusybox
docker rm -f c1 c2

# Example 2: Linking WordPress Frontend to MySQL Database
docker run --name mydb -d -e MYSQL_ROOT_PASSWORD=secretpass mysql:latest
docker run --name mywordpress -d -p 8888:80 --link mydb:mysql wordpress

Day 4 Multi-Container Real-World Stacks (CI/CD, LAMP & Postgres)

Deploy three enterprise multi-container architectures and verify container links in the low-level JSON configuration using docker inspect:

Scenario 1: Jenkins CI/CD Server Linked with QA & Production TomEE

# 1. Run Jenkins CI/CD server on host port 5050
docker run --name myjenkins -d -p 5050:8080 jenkins/jenkins

# 2. Run QA TomEE server on port 6060 linked to Jenkins
docker run --name qaserver -d -p 6060:8080 --link myjenkins:jenkins tomee

# 3. Verify the container link inside the JSON output (check the 'Links' section)
docker inspect qaserver

# 4. Run Production TomEE server on port 7070 linked to Jenkins
docker run --name prodserver -d -p 7070:8080 --link myjenkins:jenkins tomee

Scenario 2: Full LAMP Stack (Linux, Apache, MySQL, PHP)

# Step 1: Run the MySQL database container
docker run --name lamp-database -d -e MYSQL_ROOT_PASSWORD=hariprasad mysql

# Step 2: Run the Apache web server container linked to MySQL
docker run --name lamp-apache -d -p 8080:80 --link lamp-database:mysql httpd

# Step 3: Run the PHP + Apache application container linked to both MySQL and Apache
docker run --name lamp-php -d --link lamp-database:mysql --link lamp-apache:httpd php:7.2-apache

# Inspect the PHP container link metadata
docker inspect lamp-php

Scenario 3: PostgreSQL Database with Adminer Web GUI Client

# Step 1: Start PostgreSQL with custom password, user, and database name
docker run --name mydb -d \
  -e POSTGRES_PASSWORD=hariprasad \
  -e POSTGRES_USER=myuser \
  -e POSTGRES_DB=ydb \
  postgres

# Step 2: Start Adminer graphical web interface on port 9090 linked to PostgreSQL
docker run --name myadminer -d -p 9090:8080 --link mydb:postgres adminer

Day 5 Automated Cross-Browser Testing with Selenium Grid

Selenium Grid enables automated cross-browser and cross-platform testing using a Hub-and-Node architecture. As a DevOps engineer, you deploy the central selenium/hub container and attach browser debug nodes (Chrome and Firefox) that expose VNC ports so QA engineers can watch browser tests live via VNC Viewer.

# 1. Start the Selenium Hub server on port 4444
docker run --name hub -d -p 4444:4444 selenium/hub
docker container ls

# 2. Start a Chrome Debug Node linked to the Hub (VNC port 5901 -> 5900)
docker run --name chrome -d -p 5901:5900 --link hub:selenium selenium/node-chrome-debug

# 3. Start a Firefox Debug Node linked to the Hub (VNC port 5902 -> 5900)
docker run --name firefox -d -p 5902:5900 --link hub:selenium selenium/node-firefox-debug

# 4. Verify running containers and connect via VNC Viewer to :5901 or 5902
docker container ls

Day 6 Multi-Container Orchestration with Docker Compose

Docker Compose uses declarative YAML files (docker-compose.yml) to launch, stop, and tear down multi-container stacks with a single command.

1. WordPress + MySQL & LAMP Stack in Docker Compose

# === Example A: WordPress + MySQL (docker-compose.yml) ===
---
version: "3.8"
services:
  mydb:
    image: mysql:5
    environment:
      MYSQL_ROOT_PASSWORD: hariprasad
  mywordpress:
    image: wordpress
    ports:
      - "8888:80"
    links:
      - mydb:mysql

# === Example B: LAMP Stack (hari1.yml) ===
---
version: "3.8"
services:
  mydb:
    image: mysql
    environment:
      - MYSQL_ROOT_PASSWORD=hariprasad
  webserver:
    image: httpd
    ports:
      - "8080:80"
    links:
      - mydb:mysql
  phpserver:
    image: php:7.2-apache
    links:
      - webserver:httpd
      - mydb:mysql

2. Jenkins CI/CD (with depends_on) & Selenium Grid (with container_name)

By default, Docker Compose prefixes container names with the current directory name (e.g., tmp-chrome-1). You can set explicit names using container_name and enforce service startup order using depends_on:

# === Example C: Jenkins CI/CD with depends_on (harii.yml) ===
---
version: '3.8'
services:
  frontendjenkins:
    image: jenkins/jenkins
    ports:
      - "8080:8080"
  qaserver:
    image: tomee
    ports:
      - "7070:8080"
    depends_on:
      - frontendjenkins
  paserver:
    image: tomee
    ports:
      - "6060:8080"
    depends_on:
      - frontendjenkins

# === Example D: Selenium Grid with Custom Container Names ===
---
version: "3.8"
services:
  hub:
    image: selenium/hub
    ports:
      - "4444:4444"
    container_name: hub
  chrome:
    image: selenium/node-chrome-debug
    ports:
      - "5901:5900"
    links:
      - hub:selenium
    container_name: chrome
  firefox:
    image: selenium/node-firefox-debug
    ports:
      - "5902:5900"
    links:
      - hub:selenium
    container_name: firefox

3. Essential Docker Compose CLI Commands

# Launch all services in detached background mode
docker compose up -d

# List running containers and active Compose projects
docker container ls
docker compose ls

# Stop and restart all Compose containers without deleting them
docker compose stop
docker compose start

# Shut down and delete containers and networks
docker compose down

# Run or tear down a custom-named YAML file using -f
docker compose -f harii.yml up -d
docker compose -f harii.yml down

Day 7 Persistent Storage with Docker Volumes & Comparison Matrix

Containers are ephemeral by default—data written inside a container disappears when the container is removed. Docker Volumes act like an external USB drive attached to your container, providing persistent storage across three main approaches:

1. Simple Docker Volumes (Host Bind Mounts)

# Create /data directory on the Docker Host and mount it into Ubuntu
mkdir /data
docker run --name u1 -it -v /data:/data ubuntu

# Inside the container:
cd /data
touch file1 file2
ls
exit

# Inspect the container's JSON 'Mounts' section, then delete the container
docker inspect u1 | grep Mounts
docker rm -f u1

# Verify that file1 and file2 still persist on the Docker Host
ls /data

2. Sharable Docker Volumes (--volumes-from)

# Start container c1 with a volume and create files f1, f2
docker run --name c1 -it -v /data:/data centos
# Inside c1: cd /data && touch f1 f2
# Detach via Ctrl+P, Ctrl+Q

# Start container c2 sharing c1's volume and create f3, f4
docker run --name c2 -it --volumes-from c1 centos
# Inside c2: cd /data && touch f3 f4
# Detach via Ctrl+P, Ctrl+Q

# Start container c3 sharing c2's volume and create f5, f6
docker run --name c3 -it --volumes-from c2 centos
# Inside c3: cd /data && touch f5 f6 && ls (all files f1-f6 are visible!)

# Cleanup containers and unused volumes
docker rm -f c1 c2 c3
docker volume prune

3. Named Volume Containers & Persisting TomEE Configs

# Create, list, and inspect a Docker-managed Named Volume
docker volume create myvolume
docker volume ls
docker volume inspect myvolume

# Mount the named volume into /tmp inside an Ubuntu container
docker run --name u1 -it -v myvolume:/tmp ubuntu

# Real-World Lab: Persisting custom tomcat-users.xml in a TomEE container
docker volume create newvolume
# Place your modified tomcat-users.xml inside the volume:
# 
#   
# 

docker run --name t1 -d -P -v newvolume:/tmp tomee
docker exec -it t1 bash
cp /tmp/tomcat-users.xml /usr/local/tomee/conf/

Docker Volumes Comparison Matrix

Day 8 Custom Image Snapshots (docker commit) & The 15 Dockerfile Keywords

To preserve installed software (like Git and Maven) even after a container is deleted, you can create a custom image either manually via docker commit or automatically via a Dockerfile.

Method 1: Creating an Image Snapshot with docker commit

# 1. Start an interactive Ubuntu container and install Git & Maven
docker run --name u1 -it ubuntu
apt-get update
apt-get install -y git maven
git --version
mvn --version
exit

# 2. Save a snapshot of container u1 as a new custom image 'myubuntu'
docker commit u1 myubuntu

# 3. Delete old container and launch a new one from your custom image
docker rm -f u1
docker run --name u1 -it myubuntu

Method 2: Automating Image Builds with a Dockerfile (113 Bytes vs. 454 MB)

While a compiled custom image may take 454 MB, the Dockerfile text file that builds it is only 113 bytes, remains preserved even if images are deleted, and can be tracked in Git version control:

FROM ubuntu
LABEL maintainer="cloudsinfo"
RUN apt-get update
RUN apt-get install -y git
RUN apt-get install -y maven

# Build the image (note the trailing dot '.' representing the current directory):
# docker build -t myubuntu .
# docker images

All 15 Dockerfile Keywords Explained

Day 9 Dockerfile Layer Caching, COPY/ADD & Docker-in-Docker (DinD)

Every instruction in a Dockerfile is executed and stored in the Docker build cache. When you add a new instruction (such as RUN apt-get install -y tree), Docker reuses the cached layers for earlier steps and only executes the new instruction. To force a fresh build without cached layers, pass --no-cache:

# Force a fresh Dockerfile build ignoring all cached layers
docker build --no-cache -t myubuntu .

Building Custom Ansible, Jenkins WAR (ADD), and Custom Port (EXPOSE) Images

# === 1. Custom Ansible Image ===
FROM ubuntu
LABEL maintainer="cloudsinfo"
RUN apt-get update && \
    apt-get install -y software-properties-common && \
    apt-get install -y ansible
# Build & Verify: docker build -t ansible . && docker run --name a1 -it ansible ansible --version

# === 2. Downloading Remote Jenkins WAR using ADD ===
FROM ubuntu
RUN apt-get update
ADD https://get.jenkins.io/war-stable/2.528.3/jenkins.war /

# === 3. Customizing Exposed Ports ===
FROM nginx
LABEL maintainer="cicdclouds"
EXPOSE 90

Can a Docker Container Run Docker? (DinD vs. DoD)

Yes! Running Docker inside a container is common in CI/CD pipelines (such as Jenkins or GitLab runners). However, simply copying and running get-docker.sh inside an Ubuntu container is not enough—it will fail with permission errors unless you use one of two approaches:

• 1. Docker-in-Docker (DinD): Runs a completely isolated child Docker daemon inside the container using the --privileged flag.
• 2. Docker-outside-of-Docker (DoD): Mounts the host's Docker socket (/var/run/docker.sock) so the container communicates directly with the host's Docker daemon.

# Build an Ubuntu image with Docker installed inside
# COPY get-docker.sh /
# RUN sh /get-docker.sh
docker build -t myubuntu-docker .

# Approach 1: Run with Privileged Mode (Docker-in-Docker / DinD)
docker run --name u1 -it --privileged myubuntu-docker

# Approach 2: Mount the Host's Docker Socket (Docker-outside-of-Docker / DoD)
docker run --name u2 -it -v /var/run/docker.sock:/var/run/docker.sock myubuntu-docker

Day 10 Container Permissions (USER root) & Default Processes (ENTRYPOINT)

When you exec into the official jenkins/jenkins container (docker exec -it j1 bash) and try to run apt-get update or sudo apt-get update, it fails because the container runs as a restricted jenkins user. Switching to USER root inside a custom Dockerfile resolves this immediately:

FROM jenkins/jenkins
LABEL maintainer="cloudsinfo"

USER root
RUN apt-get update && apt-get install -y git maven

# Build and verify:
# docker build -t myjenkins .
# docker run --name j2 -d -P myjenkins
# docker exec -it j2 bash
# whoami && git --version && mvn --version

Understanding Default Container Processes & ENTRYPOINT

Every container stays running only as long as its default PID 1 process remains active. For example, Ubuntu and CentOS default to /bin/bash, Nginx runs docker-entrypoint.sh, and TomEE runs catalina.sh. If the default process exits, the container stops immediately.

# Package a self-starting Jenkins WAR server on Ubuntu + OpenJDK 11
FROM ubuntu
LABEL maintainer="cloudsinfo"
RUN apt-get update && apt-get install -y openjdk-11-jdk
ADD https://get.jenkins.io/war-stable/2.401.1/jenkins.war /
ENTRYPOINT ["java", "-jar", "jenkins.war"]

# Build and run the self-starting Jenkins container
docker build -t myubuntu-jenkins .
docker run --name u1 -it -P myubuntu-jenkins
# Detach with Ctrl+P, Ctrl+Q

# Override ENTRYPOINT at runtime if you need an interactive bash shell:
docker run -it --entrypoint /bin/bash myubuntu-jenkins

Day 11 Foreground Daemon Execution (CMD vs. ENTRYPOINT) & Network Drivers

When packaging a custom web server like Nginx on top of Ubuntu, running Nginx in the foreground with daemon off; keeps the container alive. Compare how ENTRYPOINT and CMD handle runtime command overrides:

• ENTRYPOINT: Specifies what application to run (fixed command).
• CMD: Specifies how to run it (default command or arguments that can be replaced at the end of docker run).

# Dockerfile using CMD so default command can be overridden at runtime
FROM ubuntu
LABEL maintainer="cloudsinfo"
RUN apt-get update && apt-get install -y nginx
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

# 1. Build and run with default CMD (starts Nginx web server)
docker build -t myubuntu .
docker run --name u2 -d -P myubuntu

# 2. Override CMD at runtime to execute 'date', 'ls -la', or 'bash'
docker run --name n2 -it -P myubuntu date
docker run --name n3 -it -P myubuntu ls -la
docker run --name n4 -it -P myubuntu bash

The 4 Docker Network Drivers

1. Bridge Network (bridge): The default network when running containers on a single Docker Host. Containers on the same bridge network communicate with each other.
2. Host Network (host): Binds the container directly to the Docker Host's network stack without isolation.
3. None / Null Network (null): Completely isolates a container with no external network connectivity.
4. Overlay Network (overlay): Connects containers running across multiple servers in a Docker Swarm cluster.

Day 12 Custom Bridge Networks, Subnets & Image Registries

Create isolated custom bridge networks, dynamically bridge a container across multiple networks using docker network connect, and configure custom CIDR subnets:

# 1. List existing networks and create two custom bridge networks
docker network ls
docker network create --driver bridge cloudsinfo1
docker network create cloudsinfo2

# 2. Run c1 and c2 on cloudsinfo1 (they can ping each other by name)
docker run --name c1 -itd --network cloudsinfo1 busybox
docker run --name c2 -itd --network cloudsinfo1 busybox

# 3. Run c3 on cloudsinfo2 (isolated from c1 and c2)
docker run --name c3 -itd --network cloudsinfo2 busybox

# 4. Dynamically connect c2 to cloudsinfo2 (c2 can now communicate with both c1 and c3!)
docker network connect cloudsinfo2 c2

# 5. Create a custom bridge network with a specific CIDR subnet
docker network create --driver bridge --subnet 10.0.0.0/28 cloudsinfo_sub
docker run --name n1 -d -P --network cloudsinfo_sub nginx
docker inspect n1

Pushing to Docker Hub & Hosting a Local Private Registry (Port 5000)

# === 1. Push Custom Image to Docker Hub ===
docker login
docker build -t username/nginx16 .
docker push username/nginx16
docker pull username/nginx16

# === 2. Run a Self-Hosted Local Registry Container on Port 5000 ===
docker run --name lr -d -p 5000:5000 registry

# Pull Alpine, tag it for localhost:5000, and push to your local registry
docker pull alpine
docker tag alpine localhost:5000/alpine
docker push localhost:5000/alpine

Day 13 AWS Elastic Container Registry (ECR) & Advanced Docker Compose

To store enterprise images securely in AWS Elastic Container Registry (ECR) without hardcoding credentials:

1. Create an EC2 instance as your Docker Host.
2. In AWS IAM, create an IAM Role for EC2 with ECR administrative permissions and attach it to your EC2 instance (Actions → Security → Modify IAM Role).
3. Create a Private or Public repository in the AWS ECR console and install the AWS CLI on your EC2 host.
4. Click View push commands in AWS ECR and paste the authentication, build, tag, and push commands into your SSH terminal.

Advanced Docker Compose: External Networks, Multi-Network Isolation, Volumes & Build

# === Scenario 1: Attach Compose to a Pre-Existing External Network ===
# First run: docker network create --driver bridge --subnet 10.0.0.0/26 cloudsinfo
---
version: "3.8"
services:
  mydb:
    image: postgres
    environment:
      POSTGRES_PASSWORD: cloudsinfo
      POSTGRES_USER: myuser
      POSTGRES_DB: mydb
  myadminer:
    image: adminer
    ports:
      - "8989:8080"
networks:
  default:
    external:
      name: cloudsinfo

# === Scenario 2: Isolate Services Across 3 Custom Networks ===
---
version: '3.8'
services:
  myjenkins:
    image: jenkins/jenkins
    ports:
      - "5050:8080"
    networks:
      - hari
  qaserver:
    image: tomcat
    ports:
      - "6060:8080"
    networks:
      - prasad
  prodserver:
    image: tomcat
    ports:
      - "7070:8080"
    networks:
      - amgoth
networks:
  hari: {}
  prasad: {}
  amgoth: {}

# === Scenario 3: Persistent Named Volumes & Inline 'build: .' in Compose ===
# Note: 'docker compose down' removes containers but preserves named volumes!
---
version: "3.8"
services:
  mydb:
    image: mysql:5
    environment:
      MYSQL_ROOT_PASSWORD: cloudsinfo
    volumes:
      - db:/var/lib/mysql
  mywordpress:
    image: wordpress
    ports:
      - "8888:80"
    volumes:
      - wordpress:/var/www/html
  myjenkins:
    build: .
    ports:
      - "6060:8080"
volumes:
  db:
  wordpress:

Day 14 Container Orchestration & Docker Swarm Setup on AWS EC2

While Dockerfiles and Docker Compose serve single-host development environments, production applications require strict SLAs powered by container orchestration (Docker Swarm, Kubernetes, OpenShift, ECS):

• Load Balancing: Automatic traffic routing across container replicas.
• Scaling: Scaling replicas in or out based on traffic demand.
• Rolling Updates: Zero-downtime image version upgrades.
• High Availability (Self-Healing): Automatically rescheduling failed containers when a container or VM crashes.

Setting Up a 3-Node Docker Swarm Cluster on AWS

Provision 3 EC2 instances (manager, worker-01, worker-02) and open these inbound ports in your AWS Security Group: 22 (SSH), 2377 (Swarm Management), 7946 (Node Communication), 4789 (Overlay Network), and 8080 (Web Service).

# === Phase 1: Install Docker & Set Hostnames on All 3 Nodes ===
sudo yum update -y
sudo amazon-linux-extras install docker -y
sudo service docker start
sudo usermod -a -G docker ec2-user

# Set unique hostnames and reboot ('sudo reboot' or 'init 6')
sudo hostnamectl set-hostname manager     # On Manager
sudo hostnamectl set-hostname worker-01   # On Worker 1
sudo hostnamectl set-hostname worker-02   # On Worker 2

# === Phase 2: Initialize Swarm on the Manager Node ===
docker swarm init --advertise-addr 

# === Phase 3: Join Both Workers Using the Generated Token ===
docker swarm join --token  :2377

# === Phase 4: Verify Nodes & Deploy Load-Balanced Replicated Services (On Manager) ===
docker node ls
docker service create --name webserver -p 8080:80 --replicas 5 nginx
docker service create --name mydb -e MYSQL_ROOT_PASSWORD=cloudsinfo --replicas 3 mysql

# Check service status and see which worker nodes are running each replica
docker service ls
docker service ps webserver
docker service ps mydb

Day 15 Swarm Service Scaling, Rolling Updates & Node Lifecycle Management

Manage service scaling, perform zero-downtime rolling updates and rollbacks, and control node availability across your Swarm cluster:

# === 1. Dynamic Service Scaling ===
docker service create --name myhttpd -p 8080:80 --replicas 4 httpd
docker service ps myhttpd
docker service scale myhttpd=8
docker service scale myhttpd=2
docker service rm myhttpd

# === 2. Zero-Downtime Rolling Updates & Rollbacks ===
docker service create --name myredis --replicas 3 redis:3
docker service update --image redis:4 myredis

# Filter out historical/shutdown tasks to view only active replicas
docker service ps myredis | grep -v "\_"

# Roll back to the previous image version (redis:3)
docker service update --rollback myredis

Complete Docker Swarm Node Lifecycle Commands

# 1. List all nodes in the swarm
docker node ls

# 2. Drain a worker node for maintenance (evacuates containers & stops new scheduling)
docker node update --availability drain worker1

# 3. Reactivate a worker node to accept tasks again
docker node update --availability active worker1

# 4. Permanently remove a node (run 'leave' on the worker node, then 'rm' on Manager)
docker swarm leave
docker node rm worker2

# 5. Retrieve join tokens on the Manager node to rejoin workers or additional managers
docker swarm join-token worker
docker swarm join-token manager

# 6. Promote a worker to Manager or demote a Manager to Worker
docker node promote worker1
docker node demote worker2

Day 16 Multi-Manager RAFT Quorum Math, Overlay Networks & Global Services

If a single Swarm Manager crashes, the cluster becomes headless. To achieve Manager High Availability, deploy multiple managers using the RAFT consensus quorum formula: (N / 2) + 1. Always use an odd number of managers (3, 5, or 7) because even numbers add sync overhead without increasing fault tolerance:

Custom Overlay Networks & Global Services (DaemonSets)

# 1. Create custom multi-host Overlay networks
docker network create --driver overlay cloudsinfo1
docker network create --driver overlay cloudsinfo2

# 2. Deploy a service directly onto a custom overlay network
docker service create --name webserver -p 8080:80 --replicas 4 --network cloudsinfo1 httpd
docker service inspect webserver --pretty

# 3. Dynamically attach or detach an existing service to a custom overlay network
docker service create --name appserver -p 9090:8080 --replicas 4 tomee
docker service update --network-add cloudsinfo1 appserver
docker service update --network-rm cloudsinfo1 appserver

# 4. Global Service (DaemonSet): Runs exactly 1 container on EVERY node (ideal for log agents)
docker service create --name mynginx -p 8888:80 --mode global nginx
docker service ps mynginx

Day 17 Enterprise Docker Stack, Placement Constraints, Limits & Secrets

Docker Stack brings declarative Compose YAML files to a multi-server Docker Swarm cluster, adding support for replica counts, node placement constraints, CPU/Memory limits, and encrypted Docker Secrets.

1. Stack with Node Placement Constraints & Hardware Resource Limits

# === stack3.yml: Pinning Services to Specific Nodes ===
version: '3.8'
services:
  myjenkins:
    image: jenkins/jenkins
    ports:
      - "5050:8080"
    deploy:
      replicas: 2
      placement:
        constraints:
          - node.hostname == Manager
  qaserver:
    image: tomcat
    ports:
      - "6060:8080"
    deploy:
      replicas: 3
      placement:
        constraints:
          - node.hostname == worker1
  prodserver:
    image: tomcat
    ports:
      - "7070:8080"
    deploy:
      replicas: 4
      placement:
        constraints:
          - node.hostname == worker2

# === stack4.yml: Preventing Hardware Exhaustion with CPU & Memory Limits ===
---
version: '3.8'
services:
  hub:
    image: selenium/hub
    ports:
      - "4444:4444"
    deploy:
      replicas: 2
      resources:
        limits:
          cpus: "0.1"
          memory: "300M"
  chrome:
    image: selenium/node-chrome-debug
    ports:
      - "5901:5900"
    deploy:
      replicas: 2
      resources:
        limits:
          cpus: "0.01"
          memory: "150M"
  firefox:
    image: selenium/node-firefox-debug
    ports:
      - "5902:5900"
    deploy:
      replicas: 2
      resources:
        limits:
          cpus: "0.01"
          memory: "150M"

2. Protecting Sensitive Credentials with Docker Secrets

Never expose plain-text database passwords in YAML files. Create encrypted secrets on the Swarm Manager so they mount securely inside containers at /run/secrets/<secret_name>:

# 1. Create encrypted secrets from standard input
echo "cloudsinfo" | docker secret create pg_password -
echo "myuser"     | docker secret create pg_user -
echo "mydb"       | docker secret create pg_db -
docker secret ls

# 2. Reference secrets inside stack5.yml
version: '3.8'
services:
  mydb:
    image: postgres
    deploy:
      replicas: 3
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/pg_password
      POSTGRES_USER_FILE: /run/secrets/pg_user
      POSTGRES_DB_FILE: /run/secrets/pg_db
    secrets:
      - pg_password
      - pg_user
      - pg_db
  myadminer:
    image: adminer
    ports:
      - "8080:8080"

secrets:
  pg_password:
    external: true
  pg_user:
    external: true
  pg_db:
    external: true

# 3. Deploy the stack and verify mounted secrets inside the container
docker stack deploy -c stack5.yml myapp
docker stack ls
docker stack ps myapp
docker exec -it  ls /run/secrets/

Day 18 Real-World Docker Swarm Troubleshooting & Interview Guide

In production environments and technical interviews, the most critical skill is diagnosing containers when they fail to start. Master the three most common Docker Swarm failure states:

1. The "PENDING" State

• Symptom: Running docker service ps shows tasks stuck at Pending.
• What It Means: Swarm cannot find any node in the cluster that meets the service's requirements.
• Common Causes: Insufficient free CPU/RAM for resource limits, unmatched placement constraints (e.g., node.hostname or node labels), or all worker nodes are Down / Drained.
• The Fix: Inspect node resources with docker node inspect <node-id> and verify service placement constraints.

2. The "REJECTED" State

• Symptom: The task status immediately reports Rejected.
• What It Means: The Swarm Manager assigned the container to a Worker node, but the Docker Engine on that node refused to start it.
• Common Causes: Missing host bind-mount path on the worker node, host port conflict, or failure to pull from a private registry due to missing authentication.
• The Fix: Run docker service ps --no-trunc <service-name>. The --no-trunc flag displays the full untruncated error message explaining why the node rejected the task.

3. The "SHUTDOWN" Crash Loop

• Symptom: Tasks start briefly, transition to Shutdown or Failed, and Swarm endlessly spawns replacements.
• What It Means: The container starts properly, but the application inside crashes immediately.
• Common Causes: Application code crash, missing required environment variable (such as MYSQL_ROOT_PASSWORD), or a failing HEALTHCHECK probe.
• The Fix: Inspect application logs with docker service logs -f <service-name> and check the task exit code using docker inspect <task-id>.

The 4-Step Interview "Detective" Diagnostic Workflow

# Step 1: Check if service replicas are missing (e.g., 0/3 running)
docker service ls

# Step 2: Inspect task status and print the FULL untruncated error message
docker service ps --no-trunc 

# Step 3: Stream live application logs to identify crashes or missing variables
docker service logs -f 

# Step 4: Verify cluster node availability and status
docker node ls