1. What is an OCI ?
2. Does Docker Create OCI Images?
3. What is a Buildpack ?
4. Overview of Buildpack Process
5. Builder: The Image That Executes the Build
   1. Components of a Builder Image
   2. Stack: The Combination of Build and Run Images
6. Installation and Initial Setups
7. Basic Build of an Image (Python Project)
   1. Building an image using buildpack
   2. Building an Image using Dockerfile
8. Unique Benefits of Buildpacks
   1. No Need for a Dockerfile (Auto-Detection)
   2. Automatic Security Updates
   3. Standardized & Reproducible Builds
   4. Extensibility: Custom Buildpacks
9. Generating SBOM in Buildpacks
   1. a) Using pack CLI to Generate SBOM
   2. b) Generate SBOM in Docker

Last few days, i was exploring on Buildpacks. I am amused at this tool features on reducing the developer’s pain. In this blog i jot down my experience on Buildpacks.

Before going to try Buildpacks, we need to understand what is an OCI ?

What is an OCI ?

An OCI Image (Open Container Initiative Image) is a standard format for container images, defined by the Open Container Initiative (OCI) to ensure interoperability across different container runtimes (Docker, Podman, containerd, etc.).

It consists of,

1. Manifest – Metadata describing the image (layers, config, etc.).
2. Config JSON – Information about how the container should run (CMD, ENV, etc.).
3. Filesystem Layers – The actual file system of the container.

OCI Image Specification ensures that container images built once can run on any OCI-compliant runtime.

Does Docker Create OCI Images?

Yes, Docker creates OCI-compliant images. Since Docker v1.10+, Docker has been aligned with the OCI Image Specification, and all Docker images are OCI-compliant by default.

• When you build an image with docker build, it follows the OCI Image format.
• When you push/pull images to registries like Docker Hub, they follow the OCI Image Specification.

However, Docker also supports its legacy Docker Image format, which existed before OCI was introduced. Most modern registries and runtimes (Kubernetes, Podman, containerd) support OCI images natively.

What is a Buildpack ?

A buildpack is a framework for transforming application source code into a runnable image by handling dependencies, compilation, and configuration. Buildpacks are widely used in cloud environments like Heroku, Cloud Foundry, and Kubernetes (via Cloud Native Buildpacks).

Overview of Buildpack Process

The buildpack process consists of two primary phases

• Detection Phase: Determines if the buildpack should be applied based on the app’s dependencies.
• Build Phase: Executes the necessary steps to prepare the application for running in a container.

Buildpacks work with a lifecycle manager (e.g., Cloud Native Buildpacks’ lifecycle) that orchestrates the execution of multiple buildpacks in an ordered sequence.

Builder: The Image That Executes the Build

A builder is an image that contains all necessary components to run a buildpack.

Components of a Builder Image

1. Build Image – Used during the build phase (includes compilers, dependencies, etc.).
2. Run Image – A minimal environment for running the final built application.
3. Lifecycle – The core mechanism that executes buildpacks, orchestrates the process, and ensures reproducibility.
   

Stack: The Combination of Build and Run Images

• Build Image + Run Image = Stack
• Build Image: Base OS with tools required for building (e.g., Ubuntu, Alpine).
• Run Image: Lightweight OS with only the runtime dependencies for execution.

Installation and Initial Setups

• For Installation: https://buildpacks.io/docs/app-journey/
• For App Developers: https://buildpacks.io/docs/for-app-developers/

Basic Build of an Image (Python Project)

Project Source: https://github.com/syedjaferk/gh_action_docker_build_push_fastapi_app

Building an image using buildpack

Before running these commands, ensure you have Pack CLI (pack) installed.

a) Detect builder suggest

pack builder suggest

b) Build the image

pack build my-app --builder paketobuildpacks/builder:base

c) Run the image locally

docker run -p 8080:8080 my-python-app

Building an Image using Dockerfile

a) Dockerfile

FROM python:3.9-slim
WORKDIR /app
COPY requirements.txt .

RUN pip install -r requirements.txt

COPY ./random_id_generator ./random_id_generator
COPY app.py app.py

EXPOSE 8080

CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]

b) Build and Run

docker build -t my-python-app .
docker run -p 8080:8080 my-python-app

Unique Benefits of Buildpacks

No Need for a Dockerfile (Auto-Detection)

Buildpacks automatically detect the language and dependencies, removing the need for Dockerfile.

pack build my-python-app --builder paketobuildpacks/builder:base

It detects Python, installs dependencies, and builds the app into a container. Docker requires a Dockerfile, which developers must manually configure and maintain.

Automatic Security Updates

Buildpacks automatically patch base images for security vulnerabilities.

If there’s a CVE in the OS layer, Buildpacks update the base image without rebuilding the app.

pack rebase my-python-app

No need to rebuild! It replaces only the OS layers while keeping the app the same.

Standardized & Reproducible Builds

Ensures consistent images across environments (dev, CI/CD, production). Example: Running the same build locally and on Heroku/Cloud Run,

pack build my-app

Extensibility: Custom Buildpacks

Developers can create custom Buildpacks to add special dependencies.

Example: Adding ffmpeg to a Python buildpack,

pack buildpack package my-custom-python-buildpack --path .

Generating SBOM in Buildpacks

a) Using pack CLI to Generate SBOM

After building an image with pack, run,

pack sbom download my-python-app --output-dir ./sbom

• This fetches the SBOM for your built image.
• The SBOM is saved in the ./sbom/ directory.

Supported formats:

• SPDX (sbom.spdx.json)
• CycloneDX (sbom.cdx.json)

b) Generate SBOM in Docker

trivy image --format cyclonedx -o sbom.json my-python-app

Both are helpful in creating images. Its all about the tradeoffs.

GitHub Actions is a powerful tool for automating workflows directly in your repository.In this blog, we’ll explore how to efficiently set up GitHub Actions to handle Docker workflows with environments, secrets, and protection rules.

Why Use GitHub Actions for Docker?

My Code base is in Github and i want to tryout gh-actions to build and push images to docker hub seamlessly.

Setting Up GitHub Environments

GitHub Environments let you define settings specific to deployment stages. Here’s how to configure them:

1. Create an Environment

Go to your GitHub repository and navigate to Settings > Environments. Click New environment, name it (e.g., production), and save.

2. Add Secrets and Variables

Inside the environment settings, click Add secret to store sensitive information like DOCKER_USERNAME and DOCKER_TOKEN.

Use Variables for non-sensitive configuration, such as the Docker image name.

3. Optional: Set Protection Rules

Enforce rules like requiring manual approval before deployments. Restrict deployments to specific branches (e.g., main).

Sample Workflow for Building and Pushing Docker Images

Below is a GitHub Actions workflow for automating the build and push of a Docker image based on a minimal Flask app.

Workflow: .github/workflows/docker-build-push.yml

name: Build and Push Docker Image

on:
  push:
    branches:
      - main  # Trigger workflow on pushes to the `main` branch

jobs:
  build-and-push:
    runs-on: ubuntu-latest
    environment: production  # Specify the environment to use

    steps:
      # Checkout the repository
      - name: Checkout code
        uses: actions/checkout@v3

      # Log in to Docker Hub using environment secrets
      - name: Log in to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_TOKEN }}

      # Build the Docker image using an environment variable
      - name: Build Docker image
        env:
          DOCKER_IMAGE_NAME: ${{ vars.DOCKER_IMAGE_NAME }}
        run: |
          docker build -t ${{ secrets.DOCKER_USERNAME }}/$DOCKER_IMAGE_NAME:${{ github.run_id }} .

      # Push the Docker image to Docker Hub
      - name: Push Docker image
        env:
          DOCKER_IMAGE_NAME: ${{ vars.DOCKER_IMAGE_NAME }}
        run: |
          docker push ${{ secrets.DOCKER_USERNAME }}/$DOCKER_IMAGE_NAME:${{ github.run_id }}

To Actions on live: https://github.com/syedjaferk/gh_action_docker_build_push_fastapi_app/actions

Kubernetes is a powerful platform that simplifies the management of containerized applications. If you’re familiar with the fundamentals, it’s time to take a step further and explore intermediate concepts that enhance your ability to manage and optimize Kubernetes clusters.

1. Understanding Deployments
   A Deployment ensures your application runs reliably by managing scaling, updates, and rollbacks.

2. Using ConfigMaps and Secrets
   Kubernetes separates application configuration and sensitive data from the application code using ConfigMaps and Secrets.

   ConfigMaps

   Store non-sensitive configurations, such as environment variables or application settings.
   

kubectl create configmap app-config --from-literal=ENV=production 

3. Liveness and Readiness Probes

Probes ensure your application is healthy and ready to handle traffic.

Liveness Probe
Checks if your application is running. If it fails, Kubernetes restarts the pod.

Readiness Probe
Checks if your application is ready to accept traffic. If it fails, Kubernetes stops routing requests to the pod.

4.Resource Requests and Limits
To ensure efficient resource utilization, define requests (minimum resources a pod needs) and limits (maximum resources a pod can use).

5.Horizontal Pod Autoscaling (HPA)
Scale your application dynamically based on CPU or memory usage.
Example:

kubectl autoscale deployment my-app --cpu-percent=70 --min=2 --max=10 

This ensures your application scales automatically when resource usage increases or decreases.

6.Network Policies
Control how pods communicate with each other and external resources using Network Policies.

Conclusion
Kubernetes has revolutionized the way we manage containerized applications. By automating tasks like deployment, scaling, and maintenance, it allows developers and organizations to focus on innovation. Whether you're a beginner or a seasoned developer, mastering Kubernetes is a skill that will enhance your ability to build and manage modern applications.

By mastering these slightly advanced Kubernetes concepts, you’ll improve your cluster management, application reliability, and resource utilization. With this knowledge, you’re well-prepared to dive into more advanced topics like Helm, monitoring with Prometheus, and service meshes like Istio.

Follow for more and Happy learning :)

In today’s tech-driven world, Kubernetes has emerged as one of the most powerful tools for container orchestration. Whether you’re managing a few containers or thousands of them, Kubernetes simplifies the process, ensuring high availability, scalability, and efficient resource utilization. This blog will guide you through the basics of Kubernetes, helping you understand its core components and functionality.

What is Kubernetes?
Kubernetes, often abbreviated as K8s, is an open-source platform developed by Google that automates the deployment, scaling, and management of containerized applications. It was later donated to the Cloud Native Computing Foundation (CNCF).
With Kubernetes, developers can focus on building applications, while Kubernetes takes care of managing their deployment and runtime.

Key Features of Kubernetes

1. Automated Deployment and Scaling Kubernetes automates the deployment of containers and can scale them up or down based on demand.
2. Self-Healing If a container fails, Kubernetes replaces it automatically, ensuring minimal downtime.
3. Load Balancing Distributes traffic evenly across containers, optimizing performance and preventing overload.
4. Rollbacks and Updates Kubernetes manages seamless updates and rollbacks for your applications without disrupting service.
5. Resource Management Optimizes hardware utilization by efficiently scheduling containers across the cluster.

Core Components of Kubernetes
To understand Kubernetes, let’s break it down into its core components:

1. Cluster A Kubernetes cluster consists of:
2. Master Node: The control plane managing the entire cluster.
3. Worker Nodes: Machines where containers run.
4. Pods :The smallest deployable unit in Kubernetes. A pod can contain one or more containers that share resources like storage and networking.
5. Nodes : Physical or virtual machines that run the pods. Managed by the Kubelet, a process ensuring pods are running as expected.
6. Services : Allow communication between pods and other resources, both inside and outside the cluster. Examples include ClusterIP, NodePort, and LoadBalancer services.
7. ConfigMaps and Secrets : ConfigMaps: Store configuration data for your applications. Secrets: Store sensitive data like passwords and tokens securely.
8. Namespaces Virtual clusters within a Kubernetes cluster, used for organizing and isolating resources.

Conclusion
Kubernetes has revolutionized the way we manage containerized applications. By automating tasks like deployment, scaling, and maintenance, it allows developers and organizations to focus on innovation. Whether you're a beginner or a seasoned developer, mastering Kubernetes is a skill that will enhance your ability to build and manage modern applications.

Follow for more and Happy learning :)

Here’s a Docker cheat sheet that covers the most commonly used Docker commands, organized by categories.

Docker Basics

• docker --version: Show the Docker version installed on your system.
• docker info: Display system-wide information, including Docker version, number of containers, and images.
• docker help: Get help on Docker commands.

Docker Images

• docker images: List all Docker images on your system.
• docker pull <image>: Download an image from a Docker registry (e.g., Docker Hub).
• docker build -t <image_name> .: Build an image from a Dockerfile in the current directory and tag it with a name.
• docker tag <image_id> <new_image_name>: Tag an image with a new name.
• docker rmi <image>: Remove one or more images.
• docker history <image>: Show the history of an image (layers).

Docker Containers

• docker ps: List all running containers.
• docker ps -a: List all containers (running and stopped).
• docker run <image>: Run a container from an image.
• docker run -d <image>: Run a container in detached mode (in the background).
• docker run -it <image>: Run a container in interactive mode with a terminal.
• docker run -p <host_port>:<container_port> <image>: Map a port from the host to the container.
• docker stop <container>: Stop a running container.
• docker start <container>: Start a stopped container.
• docker restart <container>: Restart a running container.
• docker rm <container>: Remove a stopped container.
• docker logs <container>: View the logs of a container.
• docker exec -it <container> <command>: Execute a command inside a running container (e.g., bash to open a shell).

Docker Networks

• docker network ls: List all Docker networks.
• docker network create <network_name>: Create a new Docker network.
• docker network inspect <network_name>: View detailed information about a network.
• docker network connect <network_name> <container>: Connect a container to a network.
• docker network disconnect <network_name> <container>: Disconnect a container from a network.
• docker network rm <network_name>: Remove a Docker network.

Docker Volumes

• docker volume ls: List all Docker volumes.
• docker volume create <volume_name>: Create a new Docker volume.
• docker volume inspect <volume_name>: View detailed information about a volume.
• docker volume rm <volume_name>: Remove a Docker volume.
• docker run -v <volume_name>:<container_path> <image>: Mount a volume inside a container.

Docker Compose

• docker-compose up: Start the services defined in a docker-compose.yml file.
• docker-compose down: Stop and remove containers, networks, volumes, and images created by docker-compose up.
• docker-compose build: Build or rebuild services defined in a docker-compose.yml file.
• docker-compose ps: List containers managed by Docker Compose.
• docker-compose logs: View logs for services managed by Docker Compose.
• docker-compose exec <service> <command>: Execute a command in a running service.

Dockerfile Directives

• FROM: Specifies the base image.
• WORKDIR: Sets the working directory inside the container.
• COPY: Copies files from the host to the container.
• RUN: Executes a command in the container.
• CMD: Specifies the command to run when the container starts.
• EXPOSE: Specifies the port on which the container will listen.
• ENV: Sets environment variables.
• ENTRYPOINT: Configures the container to run as an executable.

Docker Cleanup Commands

• docker system prune: Remove unused data (stopped containers, unused networks, dangling images, etc.).
• docker container prune: Remove all stopped containers.
• docker image prune: Remove unused images.
• docker volume prune: Remove all unused volumes.
• docker network prune: Remove all unused networks.

Miscellaneous

• docker inspect <container_or_image>: Return low-level information on Docker objects (containers, images, volumes, etc.).
• docker stats: Display a live stream of container(s) resource usage statistics.
• docker top <container>: Display the running processes of a container.
• docker cp <container>:<path> <local_path>: Copy files from a container to the host or vice versa.

The Client-Server Architecture

Once upon a time in the electronic city of Banglore, there was a popular digital tea kadai. This cafe was unique because it didn’t serve traditional coffee or pastries. Instead, it served data and services to its customers developers, businesses, and tech enthusiasts who were hungry for information and resources.

The Client:

One day, a young developer named Dinesh walked into tea kadai. He was working on a new app and needed to fetch some data from the cafe’s servers. In this story, Dinesh represents the client. As a client, his role was to request specific services and data from the cafe. He approached the counter and handed over his order slip, detailing what he needed.

The Server:

Behind the counter was Syed, the tea master, representing the server. Syed’s job was to take Dinesh’s request, process it, and deliver the requested data back to him.

Syed had access to a vast array of resources stored in the cafe’s back room, where all the data was kept. When Dinesh made his request, Syed quickly went to the back, gathered the data, and handed it back to Dinesh.

The client-server architecture at Tea Kadai worked seamlessly.

Dinesh, as the client, could make requests whenever he needed, and

Syed, as the server, would respond by providing the requested data.

This interaction was efficient, allowing many clients to be served by a single server at the cafe.

Docker’s Client-Server Technology

As Tea Kadai grew in popularity, it decided to expand its services to deliver data more efficiently and flexibly. To do this, they adopted a new technology called Docker, which helped them manage their operations more effectively.

Docker Client:

In the world of Docker at Tea Kadai, Dinesh still played the role of the client. But now, instead of just making simple data requests, she could request entire environments where he could test and run his applications.

These environments, called containers, were like personalized booths in the cafe where Alice could have her own setup with everything she needed to work on her app.

Dinesh used a special tool called the Docker Client to place his order. With this tool, he could specify exactly what he wanted in his container like the operating system, libraries, and applications needed for his app. The Docker Client was her interface for communicating with the cafe’s new backend system.

Docker Server (Daemon):

Behind the scenes, Tea Kadai had installed a powerful system known as the Docker Daemon, which acted as the server in this setup. The Docker Daemon was responsible for creating, running, and managing the containers requested by clients like Dinesh.

When Dinesh sent his container request using the Docker Client, the Docker Daemon received it, built the container environment, and handed it back to Dinesh for use.

Docker Images:

The Tea Kadai had a collection of premade recipes called Docker Images. These images were like blueprints for creating containers, containing all the necessary ingredients and instructions.

When Dinesh requested a new container, the Docker Daemon used these images to quickly prepare the environment.

Flexibility and Isolation:

The beauty of Docker at Tea Kadai was that it allowed multiple clients like Dinesh to have their containers running simultaneously, each isolated from the others. This isolation ensured that one client’s work wouldn’t interfere with another’s, just like having separate booths in the cafe for each customer. Dinesh could run, test, and even destroy his environment without affecting anyone else.

At the end,

In the vibrant city of Banglore, Tea Kadai thrived by adopting client-server architecture and Docker’s client-server technology. This approach allowed them to efficiently serve data and services while providing flexible, isolated environments for their clients. Dinesh and many others continued to come to tea kadai, knowing they could always get what they needed in a reliable and innovative way.