Dockerising Your IoT Stack

A practical guide to containerising your open‑source IoT backend for scalable, reproducible, and maintainable deployments.

---

Introduction

Deploying an IoT backend involves juggling multiple services: an MQTT broker, a database, a web server, a data processing worker, and often a frontend application. Managing these components manually across different environments is a recipe for configuration drift, deployment headaches, and hard-to-debug production issues.

Docker solves these problems by packaging each service—and all its dependencies—into lightweight, isolated containers. With a single docker-compose up command, your entire stack springs to life, identical in development, staging, and production. This article walks you through containerising a typical open‑source IoT backend, from writing Dockerfiles to orchestrating multi‑service stacks with Docker Compose.

---

1. Why Containerise Your IoT Backend?

Containerisation brings several compelling benefits to IoT deployments:

Benefit	Description
Reproducibility	The same container image runs identically anywhere—your laptop, a staging server, or production
Isolation	Each service runs in its own environment with its own dependencies, avoiding version conflicts
Resource Efficiency	Containers share the host kernel and are far lighter than virtual machines
Scalability	Services can be scaled independently (e.g., more workers during peak loads)
Simplified Updates	Roll out new versions by rebuilding and restarting containers with zero or minimal downtime
Configuration Management	Environment variables keep configuration out of the image, enabling the same image across environments

Key Insight: Containers are especially valuable for IoT because they enable edge deployments—you can run the same containerised stack on a Raspberry Pi gateway, a cloud VM, or an on‑premise server with minimal changes.

---

2. The Typical IoT Stack

Before diving into Docker, let’s define the components of a standard open‑source IoT backend:

┌─────────────────────────────────────────────────────────────┐
│                    IoT Devices (ESP32/STM32)                │
│                         │                                   │
│                         ▼                                   │
│              ┌─────────────────────┐                        │
│              │   MQTT Broker       │  ← Mosquitto / EMQX    │
│              │   (Port 1883)       │                        │
│              └─────────────────────┘                        │
│                         │                                   │
│                         ▼                                   │
│              ┌─────────────────────┐                        │
│              │   MQTT Worker       │  ← Node.js / Python    │
│              │   (Subscriber)      │                        │
│              └─────────────────────┘                        │
│                         │                                   │
│                         ▼                                   │
│              ┌─────────────────────┐                        │
│              │   Database          │  ← MySQL / PostgreSQL  │
│              │   (Port 3306)       │                        │
│              └─────────────────────┘                        │
│                         │                                   │
│                         ▼                                   │
│              ┌─────────────────────┐                        │
│              │   API Server        │  ← Node.js / FastAPI   │
│              │   (Port 3000)       │                        │
│              └─────────────────────┘                        │
│                         │                                   │
│                         ▼                                   │
│              ┌─────────────────────┐                        │
│              │   Frontend          │  ← React / Vue         │
│              │   (Port 80/443)     │                        │
│              └─────────────────────┘                        │
└─────────────────────────────────────────────────────────────┘

Each component will run in its own container, orchestrated by Docker Compose.

---

3. Step‑by‑Step: Containerising Your Stack

3.1 Project Structure

Start with a clean project structure:

iot-backend/
├── docker-compose.yml
├── .env
├── backend/
│   ├── Dockerfile
│   ├── package.json
│   └── src/
├── mqtt-worker/
│   ├── Dockerfile
│   ├── package.json
│   └── worker.js
├── frontend/
│   ├── Dockerfile
│   └── (React/Vue app)
├── mosquitto/
│   └── config/
│       └── mosquitto.conf
└── mysql/
    └── init/
        └── schema.sql

3.2 The Docker Compose File

The docker-compose.yml file defines all services, their images, ports, volumes, and dependencies.

version: '3.8'

services:
  # ── MQTT Broker ──
  mosquitto:
    image: eclipse-mosquitto:2
    container_name: mosquitto
    restart: unless-stopped
    ports:
      - "1883:1883"      # MQTT TCP
      - "9001:9001"      # MQTT over WebSocket
    volumes:
      - ./mosquitto/config:/mosquitto/config
      - mosquitto_data:/mosquitto/data
      - mosquitto_log:/mosquitto/log
    networks:
      - iot_network

  # ── Database ──
  mysql:
    image: mysql:8
    container_name: mysql
    restart: unless-stopped
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    ports:
      - "3306:3306"
    volumes:
      - mysql_data:/var/lib/mysql
      - ./mysql/init:/docker-entrypoint-initdb.d
    networks:
      - iot_network

  # ── Backend API Server ──
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    container_name: backend
    restart: unless-stopped
    environment:
      NODE_ENV: production
      MYSQL_HOST: mysql
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MQTT_HOST: mosquitto
      MQTT_PORT: 1883
    ports:
      - "3000:3000"
    depends_on:
      - mysql
      - mosquitto
    networks:
      - iot_network

  # ── MQTT Worker (Subscriber) ──
  mqtt_worker:
    build:
      context: ./mqtt-worker
      dockerfile: Dockerfile
    container_name: mqtt_worker
    restart: unless-stopped
    environment:
      MYSQL_HOST: mysql
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MQTT_HOST: mosquitto
      MQTT_PORT: 1883
    depends_on:
      - mysql
      - mosquitto
    networks:
      - iot_network

  # ── Frontend ──
  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    container_name: frontend
    restart: unless-stopped
    ports:
      - "80:80"
    depends_on:
      - backend
    networks:
      - iot_network

  # ── Reverse Proxy (Optional) ──
  nginx:
    image: nginx:alpine
    container_name: nginx
    restart: unless-stopped
    ports:
      - "443:443"
    volumes:
      - ./nginx/conf:/etc/nginx/conf.d
      - ./nginx/ssl:/etc/nginx/ssl
    depends_on:
      - backend
      - frontend
    networks:
      - iot_network

networks:
  iot_network:
    driver: bridge

volumes:
  mysql_data:
  mosquitto_data:
  mosquitto_log:

3.3 Environment Variables (.env)

Keep sensitive configuration outside the compose file:

# Database
MYSQL_ROOT_PASSWORD=your_root_password
MYSQL_DATABASE=iot_db
MYSQL_USER=iot_user
MYSQL_PASSWORD=your_db_password

# MQTT (optional auth)
MQTT_USER=iot_device
MQTT_PASS=device_password

3.4 Dockerfile for Backend API (Node.js Example)

A multi‑stage Dockerfile optimises the image for production:

# ── Build Stage ──
FROM node:20-alpine AS builder

WORKDIR /app

# Copy package files and install dependencies
COPY package*.json ./
RUN npm ci --only=production

# Copy source code
COPY . .

# ── Production Stage ──
FROM node:20-alpine

WORKDIR /app

# Copy only production artifacts from builder
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/src ./src
COPY --from=builder /app/package.json ./

# Run as non-root user for security
USER node

EXPOSE 3000

CMD ["node", "src/server.js"]

3.5 Dockerfile for MQTT Worker

FROM node:20-alpine

WORKDIR /app

COPY package*.json ./
RUN npm ci --only=production

COPY worker.js ./

USER node

CMD ["node", "worker.js"]

3.6 Dockerfile for Frontend (React with Nginx)

# ── Build Stage ──
FROM node:20-alpine AS builder

WORKDIR /app

COPY package*.json ./
RUN npm ci

COPY . .
RUN npm run build

# ── Production Stage ──
FROM nginx:alpine

# Copy built assets
COPY --from=builder /app/dist /usr/share/nginx/html

# Custom Nginx config (optional)
COPY nginx.conf /etc/nginx/conf.d/default.conf

EXPOSE 80

CMD ["nginx", "-g", "daemon off;"]

3.7 Mosquitto Configuration

mosquitto/config/mosquitto.conf:

persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log
log_type all

listener 1883
protocol mqtt

listener 9001
protocol websockets

allow_anonymous true
# For production, enable authentication:
# password_file /mosquitto/config/passwd
# allow_anonymous false

---

4. Building and Running

First-Time Setup

# Clone or create your project
cd iot-backend

# Create .env file with your credentials
cp .env.example .env

# Build and start all services
docker-compose up -d

# View logs
docker-compose logs -f

# Check running containers
docker-compose ps

Common Operations

Command	Description
docker-compose up -d	Start all services in detached mode
docker-compose down	Stop and remove all containers
docker-compose down -v	Stop and remove containers + volumes (⚠️ deletes data)
docker-compose logs -f backend	Follow logs for a specific service
docker-compose restart backend	Restart a single service
docker-compose exec backend bash	Open a shell inside a running container
docker-compose build --no-cache	Rebuild all images without cache

Verifying the Stack

1. MQTT Broker: Connect with any MQTT client to localhost:1883
2. Database: Connect with MySQL client to localhost:3306
3. API: Access http://localhost:3000/api/health
4. Frontend: Open http://localhost in your browser

---

5. Best Practices for Production

5.1 Security

Practice	Why
Run as non‑root user	Prevents a compromised container from modifying host files
Use environment variables	Keep secrets out of images and version control
Read‑only root filesystem	Prevents runtime modifications to container binaries
Use a reverse proxy	Terminate TLS at Nginx, not in each service
Regularly update base images	Patch vulnerabilities in Alpine/Node images

5.2 Performance Optimisation

Practice	Benefit
Alpine‑based images	Smaller images (~5 MB vs ~50 MB), faster pulls and less memory
Multi‑stage builds	Exclude build tools and dev dependencies from production images
Layer caching	Order Dockerfile commands from least to most frequently changing
Resource limits	Set mem_limit and cpus in compose to prevent resource contention

Example resource limits:

services:
  backend:
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
        reservations:
          cpus: '0.2'
          memory: 256M

5.3 Data Persistence

Volume Type	Use Case
Named volumes (mysql_data:)	Persistent data managed by Docker, ideal for databases
Bind mounts (./mosquitto/config:)	Development—live updates from host to container
tmpfs	Temporary data that should not persist (e.g., cache)

Backup strategy:

# Backup MySQL data
docker exec mysql mysqldump -u root -p$MYSQL_ROOT_PASSWORD $MYSQL_DATABASE > backup.sql

# Backup volume data
docker run --rm -v mysql_data:/data -v $(pwd):/backup alpine tar czf /backup/mysql_data.tar.gz -C /data .

5.4 Logging

Centralise logs for easier debugging:

services:
  backend:
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

For production, consider forwarding logs to a central system (ELK stack, Loki, or a cloud logging service).

5.5 Health Checks

Add health checks to ensure services are ready before dependent services start:

services:
  mysql:
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5

  backend:
    depends_on:
      mysql:
        condition: service_healthy
      mosquitto:
        condition: service_started

---

6. Scaling Beyond a Single Host

6.1 Docker Swarm (Simple)

For modest scaling, Docker Swarm provides built‑in orchestration:

# Initialize swarm
docker swarm init

# Deploy stack
docker stack deploy -c docker-compose.yml iot-stack

# Scale a service
docker service scale iot-stack_backend=3

6.2 Kubernetes (Enterprise)

For large‑scale deployments, Kubernetes offers advanced features like auto‑scaling, rolling updates, and self‑healing.

Key concepts:

• Deployments: Manage replica sets and rolling updates
• Services: Expose pods with stable network identities
• PersistentVolumeClaims: Abstract storage from underlying infrastructure
• ConfigMaps/Secrets: Manage configuration and secrets

A minimal Kubernetes deployment for the backend:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
spec:
  replicas: 2
  selector:
    matchLabels:
      app: backend
  template:
    metadata:
      labels:
        app: backend
    spec:
      containers:
      - name: backend
        image: your-registry/iot-backend:latest
        ports:
        - containerPort: 3000
        env:
        - name: MYSQL_HOST
          value: mysql-service
---
apiVersion: v1
kind: Service
metadata:
  name: backend-service
spec:
  selector:
    app: backend
  ports:
  - port: 3000
    targetPort: 3000
  type: ClusterIP

---

7. Real‑World Example: The N3xar Stack

The N3xar platform follows the containerisation principles described above:

Services:

• Mosquitto – MQTT broker for device communication
• MySQL – Relational database for device metadata and historical data
• Backend (Node.js) – API server and business logic
• MQTT Worker (Node.js) – Subscribes to MQTT topics and writes to MySQL
• Frontend (React) – Served via Nginx
• Nginx – Reverse proxy with TLS termination

Deployment Options:

• Development: docker-compose up on a local machine
• Self‑Hosted: Deploy the same compose file on a client’s server
• SaaS: Deploy the stack on a cloud VM with monitoring and auto‑scaling

“The platform supports both Docker/cloud deployment anywhere and offers custom development for enterprise clients.”

---

8. Troubleshooting Common Issues

Issue	Likely Cause	Solution
Container exits immediately	Missing environment variable or startup script error	Check logs: docker-compose logs service_name
Port already in use	Another process using the same port	Change host port mapping (e.g., "3001:3000")
Database connection refused	MySQL not ready when backend starts	Add depends_on with condition: service_healthy
MQTT connection refused	Mosquitto not running or wrong hostname	Verify MQTT_HOST=mosquitto (service name, not localhost)
Volume permission errors	Container user can’t write to bind‑mounted directory	Use named volumes or set correct ownership
Image build slow	Large dependencies or no layer caching	Reorder Dockerfile, use Alpine base, leverage build cache
Out of memory	Container exceeds resource limits	Set mem_limit or increase available host memory

---

9. Conclusion

Containerising your IoT backend with Docker transforms a complex, brittle deployment into a reproducible, scalable, and maintainable system. The key takeaways:

1. Define your stack in a docker-compose.yml file—every service, its dependencies, and its configuration in one place.

2. Use multi‑stage Dockerfiles to keep production images small and secure.

3. Keep configuration in environment variables—the same image works everywhere.

4. Persist data with named volumes and back up regularly.

5. Add health checks to ensure services start in the correct order.

6. Plan for scaling—start with Docker Compose, evolve to Swarm or Kubernetes as needed.

7. Adopt best practices—non‑root users, Alpine images, read‑only filesystems, and regular updates.

“The goal of containerisation is not just to run software in a box, but to make your entire infrastructure reproducible, auditable, and boringly reliable.”

With Docker, your IoT backend becomes a first‑class citizen of the cloud‑native ecosystem—ready for development, testing, and production, anywhere you choose to deploy.

---

Further Reading

• Designing Scalable IoT Systems: A Practical Guide for Integrators – the main article providing a broad overview of IoT architecture.

---

This guide is based on practical experience building and deploying the N3xar IoT platform using Docker containers across development, staging, and production environments.