Write-Ahead Logging (WAL) is a fundamental feature of PostgreSQL, ensuring data integrity and facilitating critical functionalities like crash recovery, replication, and backup.

This series of experimentation explores WAL in detail, its importance, how it works, and provides examples to demonstrate its usage.

What is Write-Ahead Logging (WAL)?

WAL is a logging mechanism where changes to the database are first written to a log file before being applied to the actual data files. This ensures that in case of a crash or unexpected failure, the database can recover and replay these logs to restore its state.

Your question is right !

Why do we need a WAL, when we do a periodic backup ?

Write-Ahead Logging (WAL) is critical even when periodic backups are in place because it complements backups to provide data consistency, durability, and flexibility in the following scenarios.

1. Crash Recovery

• Why It’s Important: Periodic backups only capture the database state at specific intervals. If a crash occurs after the latest backup, all changes made since that backup would be lost.
• Role of WAL: WAL ensures that any committed transactions not yet written to data files (due to PostgreSQL’s lazy-writing behavior) are recoverable. During recovery, PostgreSQL replays the WAL logs to restore the database to its last consistent state, bridging the gap between the last checkpoint and the crash.

Example:

• Backup Taken: At 12:00 PM.
• Crash Occurs: At 1:30 PM.
• Without WAL: All changes after 12:00 PM are lost.
• With WAL: All changes up to 1:30 PM are recovered.

2. Point-in-Time Recovery (PITR)

• Why It’s Important: Periodic backups restore the database to the exact time of the backup. However, this may not be sufficient if you need to recover to a specific point, such as just before a mistake (e.g., accidental data deletion).
• Role of WAL: WAL records every change, enabling you to replay transactions up to a specific time. This allows fine-grained recovery beyond what periodic backups can provide.

Example:

• Backup Taken: At 12:00 AM.
• Mistake Made: At 9:45 AM, an important table is accidentally dropped.
• Without WAL: Restore only to 12:00 AM, losing 9 hours and 45 minutes of data.
• With WAL: Restore to 9:44 AM, recovering all valid changes except the accidental drop.

3. Replication and High Availability

• Why It’s Important: In a high-availability setup, replicas must stay synchronized with the primary database to handle failovers. Periodic backups cannot provide real-time synchronization.
• Role of WAL: WAL enables streaming replication by transmitting logs to replicas, ensuring near real-time synchronization.

Example:

• A primary database sends WAL logs to replicas as changes occur. If the primary fails, a replica can quickly take over without data loss.

4. Handling Incremental Changes

• Why It’s Important: Periodic backups store complete snapshots of the database, which can be time-consuming and resource-intensive. They also do not capture intermediate changes.
• Role of WAL: WAL allows incremental updates by recording only the changes made since the last backup or checkpoint. This is crucial for efficient data recovery and backup optimization.

5. Ensuring Data Durability

• Why It’s Important: Even during normal operations, a database crash (e.g., power failure) can occur. Without WAL, transactions committed by users but not yet flushed to disk are lost.
• Role of WAL: WAL ensures durability by logging all changes before acknowledging transaction commits. This guarantees that committed transactions are recoverable even if the system crashes before flushing the changes to data files.

6. Supporting Hot Backups

• Why It’s Important: For large, active databases, taking a backup while the database is running can result in inconsistent snapshots.
• Role of WAL: WAL ensures consistency by recording changes that occur during the backup process. When replayed, these logs synchronize the backup, ensuring it is valid and consistent.

7. Debugging and Auditing

• Why It’s Important: Periodic backups are static snapshots and don’t provide a record of what happened in the database between backups.
• Role of WAL: WAL contains a sequential record of all database modifications, which can help in debugging issues or auditing transactions.

In upcoming sessions, we will all experiment each one of the failure scenarios for understanding.

Load balancing helps distribute client requests across multiple servers to ensure high availability, performance, and reliability. Weighted Round Robin Load Balancing is an extension of the round-robin algorithm, where each server is assigned a weight based on its capacity or performance capabilities. This approach ensures that more powerful servers handle more traffic, resulting in a more efficient distribution of the load.

What is Weighted Round Robin Load Balancing?

Weighted Round Robin Load Balancing assigns a weight to each server. The weight determines how many requests each server should handle relative to the others. Servers with higher weights receive more requests compared to those with lower weights. This method is useful when backend servers have different processing capabilities or resources.

Step-by-Step Implementation with Docker

Step 1: Create Dockerfiles for Each Flask Application

We’ll use the same three Flask applications (app1.py, app2.py, and app3.py) as in previous examples.

• Flask App 1 (app1.py):

from flask import Flask

app = Flask(__name__)

@app.route("/")
def home():
    return "Hello from Flask App 1!"

@app.route("/data")
def data():
    return "Data from Flask App 1!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5001)

• Flask App 2 (app2.py):

from flask import Flask

app = Flask(__name__)

@app.route("/")
def home():
    return "Hello from Flask App 2!"

@app.route("/data")
def data():
    return "Data from Flask App 2!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5002)

• Flask App 3 (app3.py):

from flask import Flask

app = Flask(__name__)

@app.route("/")
def home():
    return "Hello from Flask App 3!"

@app.route("/data")
def data():
    return "Data from Flask App 3!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5003)

Step 2: Create Dockerfiles for Each Flask Application

Create Dockerfiles for each of the Flask applications:

• Dockerfile for Flask App 1 (Dockerfile.app1):

# Use the official Python image from Docker Hub
FROM python:3.9-slim

# Set the working directory inside the container
WORKDIR /app

# Copy the application file into the container
COPY app1.py .

# Install Flask inside the container
RUN pip install Flask

# Expose the port the app runs on
EXPOSE 5001

# Run the application
CMD ["python", "app1.py"]

• Dockerfile for Flask App 2 (Dockerfile.app2):

FROM python:3.9-slim
WORKDIR /app
COPY app2.py .
RUN pip install Flask
EXPOSE 5002
CMD ["python", "app2.py"]

• Dockerfile for Flask App 3 (Dockerfile.app3):

FROM python:3.9-slim
WORKDIR /app
COPY app3.py .
RUN pip install Flask
EXPOSE 5003
CMD ["python", "app3.py"]

Step 3: Create the HAProxy Configuration File

Create an HAProxy configuration file (haproxy.cfg) to implement Weighted Round Robin Load Balancing

global
    log stdout format raw local0
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms

frontend http_front
    bind *:80
    default_backend servers

backend servers
    balance roundrobin
    server server1 app1:5001 weight 2 check
    server server2 app2:5002 weight 1 check
    server server3 app3:5003 weight 3 check

Explanation:

• The balance roundrobin directive tells HAProxy to use the Round Robin load balancing algorithm.
• The weight option for each server specifies the weight associated with each server:
  • server1 (App 1) has a weight of 2.
  • server2 (App 2) has a weight of 1.
  • server3 (App 3) has a weight of 3.
• Requests will be distributed based on these weights: App 3 will receive the most requests, App 2 the least, and App 1 will be in between.

Step 4: Create a Dockerfile for HAProxy

Create a Dockerfile for HAProxy (Dockerfile.haproxy):

# Use the official HAProxy image from Docker Hub
FROM haproxy:latest

# Copy the custom HAProxy configuration file into the container
COPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg

# Expose the port for HAProxy
EXPOSE 80

Step 5: Create a docker-compose.yml File

To manage all the containers together, create a docker-compose.yml file

version: '3'

services:
  app1:
    build:
      context: .
      dockerfile: Dockerfile.app1
    container_name: flask_app1
    ports:
      - "5001:5001"

  app2:
    build:
      context: .
      dockerfile: Dockerfile.app2
    container_name: flask_app2
    ports:
      - "5002:5002"

  app3:
    build:
      context: .
      dockerfile: Dockerfile.app3
    container_name: flask_app3
    ports:
      - "5003:5003"

  haproxy:
    build:
      context: .
      dockerfile: Dockerfile.haproxy
    container_name: haproxy
    ports:
      - "80:80"
    depends_on:
      - app1
      - app2
      - app3

Explanation:

• The docker-compose.yml file defines the services (app1, app2, app3, and haproxy) and their respective configurations.
• HAProxy depends on the three Flask applications to be up and running before it starts.

Step 6: Build and Run the Docker Containers

Run the following command to build and start all the containers

docker-compose up --build

This command builds Docker images for all three Flask apps and HAProxy, then starts them.

Step 7: Test the Load Balancer

Open your browser or use curl to make requests to the HAProxy server

curl http://localhost/
curl http://localhost/data

Observation:

• With Weighted Round Robin Load Balancing, you should see that requests are distributed according to the weights specified in the HAProxy configuration.
• For example, App 3 should receive three times more requests than App 2, and App 1 should receive twice as many as App 2.

Conclusion

By implementing Weighted Round Robin Load Balancing with HAProxy, you can distribute traffic more effectively according to the capacity or performance of each backend server. This approach helps optimize resource utilization and ensures a balanced load across servers.

Load balancing helps distribute traffic across multiple servers, enhancing performance and reliability. One common strategy is Source IP Hash load balancing, which ensures that requests from the same client IP are consistently directed to the same server.

This method is particularly useful for applications requiring session persistence, such as shopping carts or user sessions. In this blog, we’ll implement Source IP Hash load balancing using Flask and HAProxy, all within Docker containers.

What is Source IP Hash Load Balancing?

Source IP Hash Load Balancing is a technique that uses a hash function on the client’s IP address to determine which server should handle the request. This guarantees that a particular client will always be directed to the same backend server, ensuring session persistence and stateful behavior.

Consistent Hashing: https://parottasalna.com/2024/06/17/why-do-we-need-to-maintain-same-hash-in-load-balancer/

Step-by-Step Implementation with Docker

Step 1: Create Flask Application

We’ll create three separate Dockerfiles, one for each Flask app.

Flask App 1 (app1.py)

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello from Flask App 1!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5001)

Flask App 2 (app2.py)

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello from Flask App 2!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5002)

Flask App 3 (app3.py)

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello from Flask App 3!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5003)

Each Flask app listens on a different port (5001, 5002, 5003).

Step 2: Dockerfiles for each flask application

Dockerfile for Flask App 1 (Dockerfile.app1)

# Use the official Python image from the Docker Hub
FROM python:3.9-slim

# Set the working directory inside the container
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY app1.py .

# Install Flask inside the container
RUN pip install Flask

# Expose the port the app runs on
EXPOSE 5001

# Run the application
CMD ["python", "app1.py"]

Dockerfile for Flask App 2 (Dockerfile.app2)

FROM python:3.9-slim
WORKDIR /app
COPY app2.py .
RUN pip install Flask
EXPOSE 5002
CMD ["python", "app2.py"]

Dockerfile for Flask App 3 (Dockerfile.app3)

FROM python:3.9-slim
WORKDIR /app
COPY app3.py .
RUN pip install Flask
EXPOSE 5003
CMD ["python", "app3.py"]

Step 3: Create a configuration for HAProxy

global
    log stdout format raw local0
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms

frontend http_front
    bind *:80
    default_backend servers

backend servers
    balance source
    hash-type consistent
    server server1 app1:5001 check
    server server2 app2:5002 check
    server server3 app3:5003 check

Explanation:

• The balance source directive tells HAProxy to use Source IP Hashing as the load balancing algorithm.
• The hash-type consistent directive ensures consistent hashing, which is essential for minimizing disruption when backend servers are added or removed.
• The server directives define the backend servers and their ports.

Step 4: Create a Dockerfile for HAProxy

Create a Dockerfile for HAProxy (Dockerfile.haproxy)

# Use the official HAProxy image from Docker Hub
FROM haproxy:latest

# Copy the custom HAProxy configuration file into the container
COPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg

# Expose the port for HAProxy
EXPOSE 80

Step 5: Create a Dockercompose file

To manage all the containers together, create a docker-compose.yml file

version: '3'

services:
  app1:
    build:
      context: .
      dockerfile: Dockerfile.app1
    container_name: flask_app1
    ports:
      - "5001:5001"

  app2:
    build:
      context: .
      dockerfile: Dockerfile.app2
    container_name: flask_app2
    ports:
      - "5002:5002"

  app3:
    build:
      context: .
      dockerfile: Dockerfile.app3
    container_name: flask_app3
    ports:
      - "5003:5003"

  haproxy:
    build:
      context: .
      dockerfile: Dockerfile.haproxy
    container_name: haproxy
    ports:
      - "80:80"
    depends_on:
      - app1
      - app2
      - app3

Explanation:

• The docker-compose.yml file defines four services: app1, app2, app3, and haproxy.
• Each Flask app is built from its respective Dockerfile and runs on its port.
• HAProxy is configured to wait (depends_on) for all three Flask apps to be up and running.

Step 6: Build and Run the Docker Containers

Run the following commands to build and start all the containers:

# Build and run the containers
docker-compose up --build

This command will build Docker images for all three Flask apps and HAProxy and start them up in the background.

Step 7: Test the Load Balancer

Open your browser or use a tool like curl to make requests to the HAProxy server:

curl http://localhost

Observation:

• With Source IP Hash load balancing, each unique IP address (e.g., your local IP) should always be directed to the same backend server.
• If you access the HAProxy from different IPs (e.g., using different devices or by simulating different client IPs), you will see that requests are consistently sent to the same server for each IP.

For the URI based hashing we just need to add,

global
    log stdout format raw local0
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms

frontend http_front
    bind *:80
    default_backend servers

backend servers
    balance uri
    hash-type consistent
    server server1 app1:5001 check
    server server2 app2:5002 check
    server server3 app3:5003 check

Explanation:

• The balance uri directive tells HAProxy to use URI Hashing as the load balancing algorithm.
• The hash-type consistent directive ensures consistent hashing to minimize disruption when backend servers are added or removed.
• The server directives define the backend servers and their ports.

Load balancing is crucial for distributing incoming network traffic across multiple servers, ensuring optimal resource utilization and improving application performance. One of the simplest and most popular load balancing algorithms is Round Robin. In this blog, we’ll explore how to implement Least Connection load balancing using Flask as our backend application and HAProxy as our load balancer.

What is Least Connection Load Balancing?

Least Connection Load Balancing is a dynamic algorithm that distributes requests to the server with the fewest active connections at any given time. This method ensures that servers with lighter loads receive more requests, preventing any single server from becoming a bottleneck.

Step-by-Step Implementation with Docker

Step 1: Create Dockerfiles for Each Flask Application

We’ll create three separate Dockerfiles, one for each Flask app.

Flask App 1 (app1.py) – Introduced Slowness by adding sleep

from flask import Flask
import time

app = Flask(__name__)

@app.route("/")
def hello():
    time.sleep(5)
    return "Hello from Flask App 1!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5001)

Flask App 2 (app2.py)

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello from Flask App 2!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5002)

Flask App 3 (app3.py) – Introduced Slowness by adding sleep.

from flask import Flask
import time

app = Flask(__name__)

@app.route("/")
def hello():
    time.sleep(5)
    return "Hello from Flask App 3!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5003)

Each Flask app listens on a different port (5001, 5002, 5003).

Step 2: Dockerfiles for each flask application

Dockerfile for Flask App 1 (Dockerfile.app1)

# Use the official Python image from the Docker Hub
FROM python:3.9-slim

# Set the working directory inside the container
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY app1.py .

# Install Flask inside the container
RUN pip install Flask

# Expose the port the app runs on
EXPOSE 5001

# Run the application
CMD ["python", "app1.py"]

Dockerfile for Flask App 2 (Dockerfile.app2)

FROM python:3.9-slim
WORKDIR /app
COPY app2.py .
RUN pip install Flask
EXPOSE 5002
CMD ["python", "app2.py"]

Dockerfile for Flask App 3 (Dockerfile.app3)

FROM python:3.9-slim
WORKDIR /app
COPY app3.py .
RUN pip install Flask
EXPOSE 5003
CMD ["python", "app3.py"]

Step 3: Create a configuration for HAProxy

global
    log stdout format raw local0
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms

frontend http_front
    bind *:80
    default_backend servers

backend servers
    balance leastconn
    server server1 app1:5001 check
    server server2 app2:5002 check
    server server3 app3:5003 check

Explanation:

• frontend http_front: Defines the entry point for incoming traffic. It listens on port 80.
• backend servers: Specifies the servers HAProxy will distribute traffic evenly the three Flask apps (app1, app2, app3). The balance leastconn directive sets the Least Connection for load balancing.
• server directives: Lists the backend servers with their IP addresses and ports. The check option allows HAProxy to monitor the health of each server.

Step 4: Create a Dockerfile for HAProxy

Create a Dockerfile for HAProxy (Dockerfile.haproxy)

# Use the official HAProxy image from Docker Hub
FROM haproxy:latest

# Copy the custom HAProxy configuration file into the container
COPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg

# Expose the port for HAProxy
EXPOSE 80

Step 5: Create a Dockercompose file

To manage all the containers together, create a docker-compose.yml file

version: '3'

services:
  app1:
    build:
      context: .
      dockerfile: Dockerfile.app1
    container_name: flask_app1
    ports:
      - "5001:5001"

  app2:
    build:
      context: .
      dockerfile: Dockerfile.app2
    container_name: flask_app2
    ports:
      - "5002:5002"

  app3:
    build:
      context: .
      dockerfile: Dockerfile.app3
    container_name: flask_app3
    ports:
      - "5003:5003"

  haproxy:
    build:
      context: .
      dockerfile: Dockerfile.haproxy
    container_name: haproxy
    ports:
      - "80:80"
    depends_on:
      - app1
      - app2
      - app3

Explanation:

• The docker-compose.yml file defines four services: app1, app2, app3, and haproxy.
• Each Flask app is built from its respective Dockerfile and runs on its port.
• HAProxy is configured to wait (depends_on) for all three Flask apps to be up and running.

Step 6: Build and Run the Docker Containers

Run the following commands to build and start all the containers:

# Build and run the containers
docker-compose up --build

This command will build Docker images for all three Flask apps and HAProxy and start them up in the background.

You should see the responses alternating between “Hello from Flask App 1!”, “Hello from Flask App 2!”, and “Hello from Flask App 3!” as HAProxy uses the Round Robin algorithm to distribute requests.

Step 7: Test the Load Balancer

Open your browser or use a tool like curl to make requests to the HAProxy server:

curl http://localhost

You should see responses cycling between “Hello from Flask App 1!”, “Hello from Flask App 2!”, and “Hello from Flask App 3!” according to the Least Connection strategy.

Load balancing is crucial for distributing incoming network traffic across multiple servers, ensuring optimal resource utilization and improving application performance. One of the simplest and most popular load balancing algorithms is Round Robin. In this blog, we’ll explore how to implement Round Robin load balancing using Flask as our backend application and HAProxy as our load balancer.

What is Round Robin Load Balancing?

Round Robin load balancing works by distributing incoming requests sequentially across a group of servers.

For example, the first request goes to Server A, the second to Server B, the third to Server C, and so on. Once all servers have received a request, the cycle repeats. This algorithm is simple and works well when all servers have similar capabilities.

Step-by-Step Implementation with Docker

Step 1: Create Dockerfiles for Each Flask Application

We’ll create three separate Dockerfiles, one for each Flask app.

Flask App 1 (app1.py)

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello from Flask App 1!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5001)

Flask App 2 (app2.py)

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello from Flask App 2!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5002)

Flask App 3 (app3.py)

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello from Flask App 3!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5003)

Each Flask app listens on a different port (5001, 5002, 5003).

Step 2: Dockerfiles for each flask application

Dockerfile for Flask App 1 (Dockerfile.app1)

# Use the official Python image from the Docker Hub
FROM python:3.9-slim

# Set the working directory inside the container
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY app1.py .

# Install Flask inside the container
RUN pip install Flask

# Expose the port the app runs on
EXPOSE 5001

# Run the application
CMD ["python", "app1.py"]

Dockerfile for Flask App 2 (Dockerfile.app2)

FROM python:3.9-slim
WORKDIR /app
COPY app2.py .
RUN pip install Flask
EXPOSE 5002
CMD ["python", "app2.py"]

Dockerfile for Flask App 3 (Dockerfile.app3)

FROM python:3.9-slim
WORKDIR /app
COPY app3.py .
RUN pip install Flask
EXPOSE 5003
CMD ["python", "app3.py"]

Step 3: Create a configuration for HAProxy

global
    log stdout format raw local0
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms

frontend http_front
    bind *:80
    default_backend servers

backend servers
    balance roundrobin
    server server1 app1:5001 check
    server server2 app2:5002 check
    server server3 app3:5003 check

Explanation:

• frontend http_front: Defines the entry point for incoming traffic. It listens on port 80.
• backend servers: Specifies the servers HAProxy will distribute traffic evenly the three Flask apps (app1, app2, app3). The balance roundrobin directive sets the Round Robin algorithm for load balancing.
• server directives: Lists the backend servers with their IP addresses and ports. The check option allows HAProxy to monitor the health of each server.

Step 4: Create a Dockerfile for HAProxy

Create a Dockerfile for HAProxy (Dockerfile.haproxy)

# Use the official HAProxy image from Docker Hub
FROM haproxy:latest

# Copy the custom HAProxy configuration file into the container
COPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg

# Expose the port for HAProxy
EXPOSE 80

Step 5: Create a Dockercompose file

To manage all the containers together, create a docker-compose.yml file

version: '3'

services:
  app1:
    build:
      context: .
      dockerfile: Dockerfile.app1
    container_name: flask_app1
    ports:
      - "5001:5001"

  app2:
    build:
      context: .
      dockerfile: Dockerfile.app2
    container_name: flask_app2
    ports:
      - "5002:5002"

  app3:
    build:
      context: .
      dockerfile: Dockerfile.app3
    container_name: flask_app3
    ports:
      - "5003:5003"

  haproxy:
    build:
      context: .
      dockerfile: Dockerfile.haproxy
    container_name: haproxy
    ports:
      - "80:80"
    depends_on:
      - app1
      - app2
      - app3

Explanation:

• The docker-compose.yml file defines four services: app1, app2, app3, and haproxy.
• Each Flask app is built from its respective Dockerfile and runs on its port.
• HAProxy is configured to wait (depends_on) for all three Flask apps to be up and running.

Step 6: Build and Run the Docker Containers

Run the following commands to build and start all the containers:

# Build and run the containers
docker-compose up --build

This command will build Docker images for all three Flask apps and HAProxy and start them up in the background.

You should see the responses alternating between “Hello from Flask App 1!”, “Hello from Flask App 2!”, and “Hello from Flask App 3!” as HAProxy uses the Round Robin algorithm to distribute requests.

Step 7: Test the Load Balancer

Open your browser or use a tool like curl to make requests to the HAProxy server:

curl http://localhost

In the city of Data, the citizens relied heavily on organizing their information. The city was home to many different types of data numbers, names, addresses, and even some exotic types like images and documents. But as the city grew, so did the complexity of managing all this information.

One day, the city’s leaders called a meeting to discuss how best to handle the growing data. They were split between two different systems

1. the old and trusted Relational Database Management System (RDBMS)
2. the new, flashy NoSQL databases.

Enter Relational Databases:

Relational databases were like the city’s libraries. They had rows of neatly organized shelves (tables) where every book (data entry) was placed according to a specific category (columns).

Each book had a unique ID (primary key) so that anyone could find it quickly. These libraries had been around for decades, and everyone knew how to use them.

The RDBMS was more than just a library. It enforced rules (constraints) to ensure that no book went missing, was duplicated, or misplaced. It even allowed librarians (queries) to connect different books using relationships (joins).

If you wanted to find all the books by a particular author that were published in the last five years, the RDBMS could do it in a heartbeat.

The Benefits of RDBMS:

The citizens loved the RDBMS because it was:

1. Organized: Everything was in its place, and data was easy to find.
2. Reliable: The rules ensured data integrity, so they didn’t have to worry about inconsistencies.
3. Powerful: It could handle complex queries, making it easy to get insights from their data.
4. Secure: Access to the data could be controlled, keeping it safe from unauthorized users.

The Rise of NoSQL:

But then came the NoSQL databases, which were more like vast, sprawling warehouses. These warehouses didn’t care much about organization; they just stored everything in a big open space. You could toss in anything, and it would accept it—no need for strict categories or relationships. This flexibility appealed to the tech-savvy citizens who wanted to store newer, more diverse types of data like social media posts, images, and videos.

NoSQL warehouses were fast. They could handle enormous amounts of data without breaking a sweat and were perfect for real-time applications like chat systems and analytics.

The PostgreSQL Advantage:

PostgreSQL was a superstar in the world of RDBMS. It combined the organization and reliability of traditional relational databases with some of the flexibility of NoSQL. It allowed citizens to store structured data in tables while also offering support for unstructured data types like JSON. This made PostgreSQL a versatile choice, bridging the gap between the old and new worlds.

For installing postgres : https://www.postgresql.org/download/

The Dilemma: PostgreSQL vs. NoSQL:

The city faced a dilemma. Should they stick with PostgreSQL, which offered the best of both worlds, or fully embrace NoSQL for its speed and flexibility? The answer wasn’t simple. It depended on what the city valued more: the structured, reliable nature of PostgreSQL or the unstructured, flexible approach of NoSQL.

For applications that required strict data integrity and complex queries, PostgreSQL was the way to go. But for projects that needed to handle massive amounts of unstructured data quickly, NoSQL was the better choice.

Conclusion:

In the end, the city of Data realized that there was no one-size-fits-all solution. They decided to use PostgreSQL for applications where data relationships and integrity were crucial, and NoSQL for those that required speed and flexibility with diverse data types.

And so, the citizens of Data lived happily, managing their information with the right tools for the right tasks, knowing that both systems had their place in the ever-growing city.

Let’s develop a simple flask application,

1. Set up the project directory: Create a new directory for your Flask project.

mkdir flask-docker-app
cd flask-docker-app

2. Create a virtual environment (optional but recommended):

python3 -m venv venv
source venv/bin/activate

3. Install Flask

pip install Flask

4. Create a simple Flask app:

In the flask-docker-app directory, create a file named app.py with the following content,

from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, Dockerized Flask!'

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

5. Test the Flask app: Run the Flask application to ensure it’s working.

python app.py

Visit http://127.0.0.1:5000/ in your browser. You should see “Hello, Dockerized Flask!”.

Dockerize the Flask Application

1. Create a Dockerfile: In the flask-docker-app directory, create a file named Dockerfile with the following content:

# Use the official Python image from the Docker Hub
FROM python:3.9-slim

# Set the working directory in the container
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY . /app

# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir Flask

# Make port 5000 available to the world outside this container
EXPOSE 5000

# Define environment variable
ENV FLASK_APP=app.py

# Run app.py when the container launches
CMD ["python", "app.py"]

2. Create a .dockerignore file:

In the flask-docker-app directory, create a file named .dockerignore to ignore unnecessary files during the Docker build process:

venv
__pycache__
*.pyc
*.pyo

3. Build the Docker image:

In the flask-docker-app directory, run the following command to build your Docker image:

docker build -t flask-docker-app .

4. Run the Docker container:

Run the Docker container using the image you just built,

docker run -p 5000:5000 flask-docker-app

5. Access the Flask app in Docker: Visit http://localhost:5000/ in your browser. You should see “Hello, Dockerized Flask!” running in a Docker container.

You have successfully created a simple Flask application and Dockerized it. The Dockerfile allows you to package your app with its dependencies and run it in a consistent environment.

The selection sort algorithm sorts an array by iteratively finding the minimum (for ascending order) / maximum (for descending order) element from the unsorted part and putting it at the beginning of the array.

The algorithm will be considering two subarrays in a given array.

1. The subarray which is already sorted.
2. Remaining subarray which is unsorted.

In every iteration of selection sort, the minimum element (considering ascending order)/ the maximum element (considering descending order) from the unsorted subarray is picked and moved to the sorted subarray. The movement is the process of swapping the minimum element with the current element.

How selection sort works ?

We will see the implementation of sorting in ascending order.

Consider the array : 29,10,14,37,14

First Iteration

Initially the val of min_index is 0. The element at the min_index is 29.

Then we can iterate over the rest of the array to find if there are any element which are minimum than the element at min_index.

By iterating, we found 10 is the minimum element present in the array. Thus, replace 29 with 10. After one iteration 10, which happens to be the least value in the array, tends to appear in the first position of the sorted list.

Second Iteration

Now the value of min_index will be incremented by 1. So now it will be min_index = 1 . Then we can iterate over the rest of the array to find if the there are any element which are minimum than the element at min_index.

By iterating, we found 14 is the minimum element present in the array. Thus, swap 29 with 14.

Third Iteration

We can repeat the same process till the entire array is sorted.

Now the value of min_index will be incremented by 1. So now it will be min_index = 2 . Then we can iterate over the rest of the array to find if the there are any element which are minimum than the element at min_index.

By iterating, we found 14 is the minimum element present in the array. Thus, swap 29 with 14.

Fourth Iteration

Now the value of min_index will be incremented by 3. So now it will be min_index = 3 . Then we can iterate over the rest of the array to find if the there are any element which are minimum than the element at min_index.

By iterating, we found 29 is the minimum element present in the array. Thus, swap 29 with 37.

Its Done.

After 4th iteration, the entire array is sorted.

So we can define the approach like,

• Initialize minimum value(min_index) to location 0
• Traverse the array to find the minimum element in the array
• While traversing if any element smaller than min_index is found then swap both the values.
• Then, increment min_index to point to next element
• Repeat until array is sorted

Code

def selection_sort(array, size):
    
    for itr in range(size):
        min_index = itr
        for ctr in range(itr + 1, size):
            if array[ctr] < array[min_index]:
                min_index = ctr
        array[itr], array[min_index] = array[min_index], array[itr]
 
arr = [29,10,14,37,14]
size = len(arr)
selection_sort(arr, size)

print(arr)

Complexity analysis

Time Complexity

The sort complexity is used to express the number of execution times it takes to sort the list. The implementation has two loops. The outer loop which picks the values one by one from the list is executed n times where n is the total number of values in the list. The inner loop, which compares the value from the outer loop with the rest of the values, is also executed n times where n is the total number of elements in the list.

Therefore, the number of executions is (n * n), which can also be expressed as O(n2).

1. Worst case – this is where the list provided is in descending order. The algorithm performs the maximum number of executions which is expressed as [Big-O] O(n2)
2. Best case – this occurs when the provided list is already sorted. The algorithm performs the minimum number of executions which is expressed as [Big-Omega] Ω(n2)
3. Average case – this occurs when the list is in random order. The average complexity is expressed as [Big-theta] Θ(n2)

Space Complexity

Since selection sort is an inplace sorting algorithm, it has a space complexity of O(1) as it requires one temporal variable used for swapping values.

Is Selection sort algorithm stable ?

The default implementation is not stable. However it can be made stable. But it can be achived using stable selection sort.

Is Selection Sort Algorithm in-place?

Yes, it does not require extra space.

When to use selection sort ?

1. Selection sort can be good at checking if everything is already sorted .
2. It is also good to use when memory space is limited. This is because unlike other sorting algorithms, selection sort doesn’t go around swapping things until the very end, resulting in less temporary storage space used.
3. When a simple sorting implementation is desired
4. When the array to be sorted is relatively small

When to avoid selection sort ?

1. The array to be sorted has a large number of elements
2. The array is nearly sorted
3. You want a faster run time and memory is not a concern.

Summary

1. Selection sort is an in-place comparison algorithm that is used to sort a random list into an ordered list. It has a time complexity of O(n2)
2. The list is divided into two sections, sorted and unsorted. The minimum value is picked from the unsorted section and placed into the sorted section.
3. This thing is repeated until all items have been sorted.
4. The time complexity measures the number of steps required to sort the list.
5. The worst-case time complexity happens when the list is in descending order. It has a time complexity of [Big-O] O(n2)
6. The best-case time complexity happens when the list is already in ascending order. It has a time complexity of [Big-Omega] O(n2)
7. The average-case time complexity happens when the list is in random order. It has a time complexity of [Big-theta] O(n2)
8. The selection sort is best used when you have a small list of items to sort, the cost of swapping values does not matter, and checking of all the values is mandatory.
9. The selection sort does not perform well on huge lists

Bonus Section

For the bigger array, how selection sort will work with insertion sort ?

The size of the array involved is rarely of much consequence. The real question is the speed of comparison vs. copying. The time a selection sort will win is when a comparison is a lot faster than copying. Just for example, let’s assume two fields: a single int as a key, and another megabyte of data attached to it. In such a case, comparisons involve only that single int, so it’s really fast, but copying involves the entire megabyte, so it’s almost certainly quite a bit slower.

Since the selection sort does a lot of comparisons, but relatively few copies, this sort of situation will favor it.

The Secure Boot Case Study

Attackers can break through the Secure Boot process on millions of computers using Intel and ARM processors due to a leaked cryptographic key that many manufacturers used during the startup process. This key, called the Platform Key (PK), is meant to verify the authenticity of a device’s firmware and boot software.

Unfortunately, this key was leaked back in 2018. It seems that some manufacturers used this key in their devices instead of replacing it with a secure one, as was intended. As a result, millions of devices from brands like Lenovo, HP, Asus, and SuperMicro are vulnerable to attacks.

If an attacker has access to this leaked key, they can easily bypass Secure Boot, allowing them to install malicious software that can take control of the device. To fix this problem, manufacturers need to replace the compromised key and update the firmware on affected devices. Some have already started doing this, but it might take time for all devices to be updated, especially those in critical systems.

The problem is serious because the leaked key is like a master key that can unlock many devices. This issue highlights poor cryptographic key management practices, which have been a problem for many years.

What Are “Not Replaced Constants”?

In software, constants are values that are not meant to change during the execution of a program. They are often used to define configuration settings, cryptographic keys, and other critical values.

When these constants are hard-coded into a system and not updated or replaced when necessary, they become a code smell known as “Not Replaced Constants.”

Why Are They a Problem?

When constants are not replaced or updated:

1. Security Risks: Outdated or exposed constants, such as cryptographic keys, can become security vulnerabilities. If these constants are publicly leaked or discovered by attackers, they can be exploited to gain unauthorized access or control over a system.
2. Maintainability Issues: Hard-coded constants can make a codebase less maintainable. Changes to these values require code modifications, which can be error-prone and time-consuming.
3. Flexibility Limitations: Systems with hard-coded constants lack flexibility, making it difficult to adapt to new requirements or configurations without altering the source code.

The Secure Boot Case Study

The recent Secure Boot vulnerability is a perfect example of the dangers posed by “Not Replaced Constants.” Here’s a breakdown of what happened:

The Vulnerability

Researchers discovered that a cryptographic key used in the Secure Boot process of millions of devices was leaked publicly. This key, known as the Platform Key (PK), serves as the root of trust during the Secure Boot process, verifying the authenticity of a device’s firmware and boot software.

What Went Wrong

The leaked PK was originally intended as a test key by American Megatrends International (AMI). However, it was not replaced by some manufacturers when producing devices for the market. As a result, the same compromised key was used across millions of devices, leaving them vulnerable to attacks.

The Consequences

Attackers with access to the leaked key can bypass Secure Boot protections, allowing them to install persistent malware and gain control over affected devices. This vulnerability highlights the critical importance of replacing test keys and securely managing cryptographic constants.

Sample Code:

Wrong

def generate_pk() -> str:
    return "DO NOT TRUST"

# Vendor forgets to replace PK
def use_default_pk() -> str:
    pk = generate_pk()
    return pk  # "DO NOT TRUST" PK used in production

Right

def generate_pk() -> str:
    # The documentation tells vendors to replace this value
    return "DO NOT TRUST"

def use_default_pk() -> str:
    pk = generate_pk()

    if pk == "DO NOT TRUST":
        raise ValueError("Error: PK must be replaced before use.")

    return pk  # Valid PK used in production

Ignoring important security steps, like changing default keys, can create big security holes. This ongoing problem shows how important it is to follow security procedures carefully. Instead of just relying on written instructions, make sure to test everything thoroughly to ensure it works as expected.

Creating a simple alarm clock application can be a fun project to develop programming skills. Here are the steps, input ideas, and additional features you might consider when building your alarm clock

Game Steps

1. Define the Requirements:
   • Determine the basic functionality your alarm clock should have (e.g., set alarm, snooze, dismiss).
2. Choose a Programming Language:
   • Select a language you are comfortable with, such as Python, JavaScript, or Java.
3. Design the User Interface:
   • Decide if you want a graphical user interface (GUI) or a command-line interface (CLI).
4. Implement Core Features:
   • Set Alarm: Allow users to set an alarm for a specific time.
   • Trigger Alarm: Play a sound or display a message when the alarm time is reached.
   • Snooze Functionality: Enable users to snooze the alarm for a set period.
   • Dismiss Alarm: Allow users to turn off the alarm once it’s triggered.
5. Test the Alarm Clock:
   • Ensure that all functions work as expected and fix any bugs.
6. Refine and Enhance:
   • Improve the interface and add additional features based on user feedback.

Input Ideas

• Set Alarm Time:
  • Input format: “HHAM/PM” or 24-hour format “HH”.
• Snooze Duration:
  • Allow users to input a snooze time in minutes.
• Alarm Sound:
  • Let users choose from a list of available alarm sounds.
• Repeat Alarm:
  • Options for repeating alarms (e.g., daily, weekdays, weekends).
• Custom Alarm Message:
  • Input a custom message to display when the alarm goes off.

Additional Features

• Multiple Alarms:
  • Allow users to set multiple alarms for different times and days.
• Customizable Alarm Sounds:
  • Let users upload their own alarm sounds.
• Volume Control:
  • Add an option to control the alarm sound volume.
• Alarm Labels:
  • Enable users to label their alarms (e.g., “Wake Up,” “Meeting Reminder”).
• Weather and Time Display:
  • Show current weather information and time on the main screen.
• Recurring Alarms:
  • Allow users to set recurring alarms on specific days.
• Dark Mode:
  • Implement a dark mode for the UI.
• Integration with Calendars:
  • Sync alarms with calendar events or reminders.
• Voice Control:
  • Add support for voice commands to set, snooze, or dismiss alarms.
• Smart Alarm:
  • Implement a smart alarm feature that wakes the user at an optimal time based on their sleep cycle (e.g., using a sleep tracking app).

Implementing a simple grocery list management tool can be a fun and practical project. Here’s a detailed approach including game steps, input ideas, and additional features:

Game Steps

1. Introduction: Provide a brief introduction to the grocery list tool, explaining its purpose and how it can help manage shopping lists.
2. Menu Options: Present a menu with options to add, view, update, delete items, and clear the entire list.
3. User Interaction: Allow the user to select an option from the menu and perform the corresponding operation.
4. Perform Operations: Implement functionality to add items, view the list, update quantities, delete items, or clear the list.
5. Display Results: Show the updated grocery list and confirmation of any operations performed.
6. Repeat or Exit: Allow the user to perform additional operations or exit the program.

Input Ideas

1. Item Name: Allow the user to enter the name of the grocery item.
2. Quantity: Prompt the user to specify the quantity of each item (optional).
3. Operation Choice: Provide options to add, view, update, delete, or clear items from the list.
4. Item Update: For updating, allow the user to specify the item and new quantity.
5. Clear List Confirmation: Ask for confirmation before clearing the entire list.

Additional Features

1. Persistent Storage: Save the grocery list to a file (e.g., JSON or CSV) and load it on program startup.
2. GUI Interface: Create a graphical user interface using Tkinter or another library for a more user-friendly experience.
3. Search Functionality: Implement a search feature to find items in the list quickly.
4. Sort and Filter: Allow sorting the list by item name or quantity, and filtering by categories or availability.
5. Notification System: Add notifications or reminders for items that are running low or need to be purchased.
6. Multi-user Support: Implement features to manage multiple lists for different users or households.
7. Export/Import: Allow users to export the grocery list to a file or import from a file.
8. Item Categories: Organize items into categories (e.g., dairy, produce) for better management.
9. Undo Feature: Implement an undo feature to revert the last operation.
10. Statistics: Provide statistics on the number of items, total quantity, or other relevant data.

Implementing a simple key-value storage system is a great way to practice data handling and basic file operations in Python. Here’s a detailed approach including game steps, input ideas, and additional features:

Game Steps

1. Introduction: Provide an introduction explaining what a key-value storage system is and its uses.
2. Menu Options: Present a menu with options to add, retrieve, update, and delete key-value pairs.
3. User Interaction: Allow the user to interact with the system based on their choice from the menu.
4. Perform Operations: Implement functionality to perform the chosen operations (add, retrieve, update, delete).
5. Display Results: Show the results of the operations (e.g., value retrieved or confirmation of deletion).
6. Repeat or Exit: Allow the user to perform additional operations or exit the program.

Input Ideas

1. Key Input: Allow the user to enter a key for operations. Ensure that keys are unique for storage operations.
2. Value Input: Prompt the user to enter a value associated with a key. Values can be strings or numbers.
3. Operation Choice: Present options to add, retrieve, update, or delete key-value pairs.
4. File Handling: Optionally, allow users to specify a file to save and load the key-value pairs.
5. Validation: Ensure that keys and values are entered correctly and handle any errors (e.g., missing keys).

Additional Features

1. Persistent Storage: Save key-value pairs to a file (e.g., JSON or CSV) and load them on program startup.
2. Data Validation: Implement checks to validate the format of keys and values.
3. GUI Interface: Create a graphical user interface using Tkinter or another library for a more user-friendly experience.
4. Search Functionality: Add a feature to search for keys or values based on user input.
5. Data Backup: Implement a backup system to periodically save the key-value pairs.
6. Data Encryption: Encrypt the stored data for security purposes.
7. Command-Line Arguments: Allow users to perform operations via command-line arguments.
8. Multi-key Operations: Support operations on multiple keys at once (e.g., batch updates).
9. Undo Feature: Implement an undo feature to revert the last operation.
10. User Authentication: Add user authentication to secure access to the key-value storage system.

Implementing a Pomodoro technique timer is a practical way to manage time effectively using a simple and proven productivity method. Here’s a detailed approach for creating a Pomodoro timer, including game steps, input ideas, and additional features.

Game Steps

1. Introduction: Provide an introduction to the Pomodoro Technique, explaining that it involves working in 25-minute intervals (Pomodoros) followed by a short break, with longer breaks after several intervals.
2. Start Timer: Allow the user to start the timer for a Pomodoro session.
3. Timer Countdown: Display a countdown for the Pomodoro session and break periods.
4. Notify Completion: Alert the user when the Pomodoro session or break is complete.
5. Record Sessions: Track the number of Pomodoros completed and breaks taken.
6. End Session: Allow the user to end the session or reset the timer if needed.
7. Play Again Option: Offer the user the option to start a new session or stop the timer.

Input Ideas

1. Session Duration: Allow users to set the duration for Pomodoro sessions and breaks. The default is 25 minutes for work and 5 minutes for short breaks, with a longer break (e.g., 15 minutes) after a set number of Pomodoros (e.g., 4).
2. Custom Durations: Enable users to customize the duration of work sessions and breaks.
3. Notification Preferences: Allow users to choose how they want to be notified (e.g., sound alert, visual alert, or popup message).
4. Number of Pomodoros: Ask how many Pomodoro cycles the user wants to complete before taking a longer break.
5. Reset and Stop Options: Provide options to reset the timer or stop it if needed.

Additional Features

1. GUI Interface: Create a graphical user interface using Tkinter or another library for a more user-friendly experience.
2. Notifications: Implement system notifications or sound alerts to notify the user when a Pomodoro or break is over.
3. Progress Tracking: Track and display the number of completed Pomodoros and breaks, providing visual feedback on progress.
4. Task Management: Allow users to input and track tasks they want to accomplish during each Pomodoro session.
5. Statistics: Provide statistics on time spent working and taking breaks, possibly with visual charts or graphs.
6. Customizable Alerts: Enable users to set custom alert sounds or messages for different stages (start, end of Pomodoro, end of break).
7. Integration with Calendars: Integrate with calendar applications to schedule Pomodoro sessions and breaks automatically.
8. Desktop Widgets: Create desktop widgets or applets that display the remaining time for the current session and next break.
9. Focus Mode: Implement a focus mode that minimizes distractions by blocking certain apps or websites during Pomodoro sessions.
10. Daily/Weekly Goals: Allow users to set and track daily or weekly productivity goals based on completed Pomodoros.

Caesar Cipher: https://en.wikipedia.org/wiki/Caesar_cipher

Game Steps

1. Introduction: Provide a brief introduction to the Caesar Cipher, explaining that it’s a substitution cipher where each letter in the plaintext is shifted a fixed number of places down or up the alphabet.
2. Choose Operation: Ask the user whether they want to encrypt or decrypt a message.
3. Input Text: Prompt the user to enter the text they want to encrypt or decrypt.
4. Input Shift Value: Request the shift value (key) for the cipher. Ensure the value is within a valid range (typically 1 to 25).
5. Perform Operation: Apply the Caesar Cipher algorithm to the input text based on the user’s choice of encryption or decryption.
6. Display Result: Show the resulting encrypted or decrypted text to the user.
7. Play Again Option: Ask the user if they want to perform another encryption or decryption with new inputs.

Input Ideas

1. Text Input: Allow the user to input any string of text. Handle both uppercase and lowercase letters. Decide how to treat non-alphabetic characters (e.g., spaces, punctuation).
2. Shift Value: Ask the user for an integer shift value. Ensure it is within a reasonable range (1 to 25). Handle cases where the shift value is negative or greater than 25 by normalizing it.
3. Mode Selection: Provide options to select between encryption and decryption. For encryption, the shift will be added; for decryption, the shift will be subtracted.
4. Case Sensitivity: Handle uppercase and lowercase letters differently or consistently based on user preference.
5. Special Characters: Decide whether to include special characters and spaces in the encrypted/decrypted text. Define how these characters should be treated.

Additional Features

1. Input Validation: Implement checks to ensure the shift value is an integer and falls within the expected range. Validate that text input does not contain unsupported characters (if needed).
2. Help/Instructions: Provide an option for users to view help or instructions on how to use the tool, explaining the Caesar Cipher and how to enter inputs.
3. GUI Interface: Create a graphical user interface using Tkinter or another library to make the tool more accessible and user-friendly.
4. File Operations: Allow users to read from and write to text files for encryption and decryption. This is useful for larger amounts of text.
5. Brute Force Attack: Implement a brute force mode that tries all possible shifts for decryption and displays all possible plaintexts, useful for educational purposes or cracking simple ciphers.
6. Custom Alphabet: Allow users to define a custom alphabet or set of characters for the cipher, making it more flexible and adaptable.
7. Save and Load Settings: Implement functionality to save and load encryption/decryption settings, such as shift values or custom alphabets, for future use.

Creating a simple version of Hangman is a fun way to practice programming and game logic.

Here’s a structured approach to building this game, including game steps, input ideas, and additional features to enhance it.

Game Steps (Workflow)

1. Introduction:
   • Start with a welcome message explaining the rules of Hangman.
   • Provide brief instructions on how to play (guessing letters, how many guesses are allowed, etc.).
2. Word Selection:
   • Choose a word for the player to guess. This can be randomly selected from a predefined list or from a file.
3. Display State:
   • Show the current state of the word with guessed letters and placeholders for remaining letters.
   • Display the number of incorrect guesses left (hangman stages).
4. User Input:
   • Prompt the player to guess a letter.
   • Check if the letter is in the word.
5. Update Game State:
   • Update the display with the correct guesses.
   • Keep track of incorrect guesses and update the hangman drawing if applicable.
6. Check for Win/Loss:
   • Determine if the player has guessed the word or used all allowed guesses.
   • Display a win or loss message based on the result.
7. Replay Option:
   • Offer the player the option to play again or exit the game.

Input Ideas

1. Guess Input:
   • Prompt the player to enter a single letter.
   • Validate that the input is a single alphabetic character.
2. Replay Input:
   • After a game ends, ask the player if they want to play again (e.g., y for yes, n for no).
3. Word List:
   • Provide a list of words to choose from, which can be hardcoded or read from a file.

Additional Features

1. Difficulty Levels:
   • Implement difficulty levels by varying word length or allowing more or fewer incorrect guesses.
2. Hangman Drawing:
   • Add a visual representation of the hangman that updates with each incorrect guess.
3. Hints:
   • Offer hints if the player is struggling (e.g., reveal a letter or provide a clue).
4. Word Categories:
   • Categorize words into themes (e.g., animals, movies) and allow players to choose a category.
5. Score Tracking:
   • Keep track of the player’s score across multiple games and display statistics.
6. Save and Load:
   • Allow players to save their progress and load a game later.
7. Custom Words:
   • Allow players to input their own words for the game.
8. Leaderboard:
   • Create a leaderboard to track high scores and player achievements.

Creating a command-line to-do list application is a fantastic way to practice Python programming and work with basic data management. Here’s a structured approach to building this application, including game steps, input ideas, and additional features:

Game Steps (Workflow)

1. Introduction:
   • Start with a welcome message and brief instructions on how to use the application.
   • Explain the available commands and how to perform actions like adding, removing, and viewing tasks.
2. Main Menu:
   • Present a main menu with options for different actions:
     • Add a task
     • View all tasks
     • Mark a task as complete
     • Remove a task
     • Exit the application
3. Task Management:
   • Implement functionality to add, view, update, and remove tasks.
   • Store tasks with details such as title, description, and completion status.
4. Data Persistence:
   • Save tasks to a file or database so that they persist between sessions.
   • Load tasks from the file/database when the application starts.
5. User Interaction:
   • Use input prompts to interact with the user and execute their commands.
   • Provide feedback and confirmation messages for actions taken.
6. Exit and Save:
   • Save the current state of tasks when the user exits the application.
   • Confirm that tasks are saved and provide an exit message.

Input Ideas

1. Command Input:
   • Use text commands to navigate the menu and perform actions (e.g., add, view, complete, remove, exit).
2. Task Details:
   • For adding tasks, prompt the user for details like title and description.
   • Use input fields for the task details:
     • Title: Enter task title:
     • Description: Enter task description:
3. Task Identification:
   • Use a unique identifier (like a number) or task title to reference tasks for actions such as marking complete or removing.
4. Confirmation:
   • Prompt the user to confirm actions such as removing a task or marking it as complete.

Additional Features

1. Task Prioritization:
   • Allow users to set priorities (e.g., low, medium, high) for tasks.
   • Implement sorting or filtering by priority.
2. Due Dates:
   • Add due dates to tasks and provide options to view tasks by date or sort by due date.
3. Search and Filter:
   • Implement search functionality to find tasks by title or description.
   • Add filters to view tasks by status (e.g., completed, pending) or priority.
4. Task Categories:
   • Allow users to categorize tasks into different groups or projects.
5. Export and Import:
   • Provide options to export tasks to a file (e.g., CSV or JSON) and import tasks from a file.
6. User Authentication:
   • Add user authentication if multiple users need to manage their own tasks.
7. Reminders and Notifications:
   • Implement reminders or notifications for tasks with upcoming due dates.
8. Statistics:
   • Show statistics such as the number of completed tasks, pending tasks, or tasks by priority.

1. Create a dictionary named student with the following keys and values. and print the same
   • "name": "Alice"
   • "age": 21
   • "major": "Computer Science"
2. Using the student dictionary, print the values associated with the keys "name" and "major".
3. Add a new key-value pair to the student dictionary: "gpa": 3.8. Then update the "age" to 22.
4. Remove the key "major" from the student dictionary using the del statement. Print the dictionary to confirm the removal.
5. Check if the key "age" exists in the student dictionary. Print True or False based on the result.
6. Create a dictionary prices with three items, e.g., "apple": 0.5, "banana": 0.3, "orange": 0.7. Iterate over the dictionary and print each key-value pair.
7. Use the len() function to find the number of key-value pairs in the prices dictionary. Print the result.
8. Use the get() method to access the "gpa" in the student dictionary. Try to access a non-existing key, e.g., "graduation_year", with a default value of 2025.
9. Create another dictionary extra_info with the following keys and values. Also merge extra_info into the student dictionary using the update() method.
   • "graduation_year": 2025
   • "hometown": "Springfield"
10. Create a dictionary squares where the keys are numbers from 1 to 5 and the values are the squares of the keys. Use dictionary comprehension.
11. Using the prices dictionary, print the keys and values as separate lists using the keys() and values() methods.
12. Create a dictionary school with two nested dictionaries. Access and print the age of "student2".
    • "student1": {"name": "Alice", "age": 21}
    • "student2": {"name": "Bob", "age": 22}
13. Use the setdefault() method to add a new key "advisor" with the value "Dr. Smith" to the student dictionary if it does not exist.
14. Use the pop() method to remove the "hometown" key from the student dictionary and store its value in a variable. Print the variable.
15. Use the clear() method to remove all items from the prices dictionary. Print the dictionary to confirm it’s empty.
16. Make a copy of the student dictionary using the copy() method. Modify the copy by changing "name" to "Charlie". Print both dictionaries to see the differences.
17. Create two lists: keys = ["name", "age", "major"] and values = ["Eve", 20, "Mathematics"]. Use the zip() function to create a dictionary from these lists.
18. Use the items() method to iterate over the student dictionary and print each key-value pair.
19. Given a list of fruits: ["apple", "banana", "apple", "orange", "banana", "banana"], create a dictionary fruit_count that counts the occurrences of each fruit.
20. Use collections.defaultdict to create a dictionary word_count that counts the number of occurrences of each word in a list: ["hello", "world", "hello", "python"].

Introduction to the Botanical Garden

We are planning to opening a botanical garden with flowers which will attract people to visit.

Morning: Planting Unique Flowers

One morning, we decides to plant flowers in the garden. They ensure that each flower they plant is unique.

botanical_garden = {"Rose", "Lily", "Sunflower"}

Noon: Adding More Flowers

At noon, they find some more flowers and add them to the garden, making sure they only add flowers that aren’t already there.

Adding Elements to a Set:

# Adding more unique flowers to the enchanted garden
botanical_garden.add("Jasmine")
botanical_garden.add("Hibiscus")
print(botanical_garden)
# output: {'Hibiscus', 'Rose', 'Tulip', 'Sunflower', 'Jasmine'}

Afternoon: Trying to Plant Duplicate Flowers

In the afternoon, they accidentally try to plant another Rose, but the garden’s rule prevents any duplicates from being added.

Adding Duplicate Elements:

# Attempting to add a duplicate flower
botanical_garden.add("Rose")
print(botanical_garden)
# output: {'Lily', 'Sunflower', 'Rose'}

Evening: Removing Unwanted Plants

As evening approaches, they decide to remove some flowers they no longer want in their garden.

Removing Elements from a Set:

# Removing a flower from the enchanted garden
botanical_garden.remove("Lily")
print(botanical_garden)
# output: {'Sunflower', 'Rose'}

Night: Checking Flower Types

Before going to bed, they check if certain flowers are present in their botanical garden.

Checking Membership:

# Checking if certain flowers are in the garden
is_rose_in_garden = "Rose" in botanical_garden
is_tulip_in_garden = "Tulip" in botanical_garden

print(f"Is Rose in the garden? {is_rose_in_garden}")
print(f"Is Tulip in the garden? {is_tulip_in_garden}")

# Output
# Is Rose in the garden? True
# Is Tulip in the garden? False

Midnight: Comparing with Rose Garden

Late at night, they compare their botanical garden with their rose garden to see which flowers they have in common and which are unique to each garden.

Set Operations:

Intersections:

# Neighbor's enchanted garden
rose_garden = {"Rose", "Lavender"}

# Flowers in both gardens (Intersection)
common_flowers = botanical_garden.intersection(rose_garden)
print(f"Common flowers: {common_flowers}")

# Output
# Common flowers: {'Rose'}
# Unique flowers: {'Sunflower'}
# All unique flowers: {'Sunflower', 'Lavender', 'Rose'}

Difference:

# Flowers unique to their garden (Difference)
unique_flowers = botanical_garden.difference(rose_garden)
print(f"Unique flowers: {unique_flowers}")

#output
# Unique flowers: {'Sunflower'}

Union:

# All unique flowers from both gardens (Union)
all_unique_flowers = botanical_garden.union(rose_garden)
print(f"All unique flowers: {all_unique_flowers}")
# Output: All unique flowers: {'Sunflower', 'Lavender', 'Rose'}

In a vibrant town in Tamil Nadu, there is a popular grocery store called Annachi Kadai. This store is always bustling with fresh deliveries of items.

The store owner, Pandian, uses a special inventory system to track the products. This system functions like a dictionary in Python, where each item is labeled with its name, and the quantity available is recorded.

Morning: Delivering Items to the Store

One bright morning, a new delivery truck arrives at the grocery store, packed with fresh items. Pandian records these new items in his inventory list.

Creating and Updating the Inventory:

# Initial delivery of items to the store
inventory = {
    "apples": 20,
    "bananas": 30,
    "carrots": 15,
    "milk": 10
}

print("Initial Inventory:", inventory)
# Output: Initial Inventory: {'apples': 20, 'bananas': 30, 'carrots': 15, 'milk': 10}

Noon: Additional Deliveries

As the day progresses, more deliveries arrive with additional items that need to be added to the inventory. Pandian updates the system with these new arrivals.

Adding New Items to the Inventory:

# Adding more items from the delivery
inventory["bread"] = 25
inventory["eggs"] = 50

print("Updated Inventory:", inventory)
# Output: Updated Inventory: {'apples': 20, 'bananas': 30, 'carrots': 15, 'milk': 10, 'bread': 25, 'eggs': 50}

Afternoon: Stocking the Shelves

In the afternoon, Pandian notices that some items are running low and restocks them by updating the quantities in the inventory system.

Updating Quantities:

# Updating item quantities after restocking shelves
inventory["apples"] += 10  # 10 more apples added
inventory["milk"] += 5     # 5 more bottles of milk added

print("Inventory after Restocking:", inventory)
# Output: Inventory after Restocking: {'apples': 30, 'bananas': 30, 'carrots': 15, 'milk': 15, 'bread': 25, 'eggs': 50}

Evening: Removing Sold-Out Items

As evening falls, some items are sold out, and Pandian needs to remove them from the inventory to reflect their unavailability.

Removing Items from the Inventory:

# Removing sold-out items
del inventory["carrots"]

print("Inventory after Removal:", inventory)
# Output: Inventory after Removal: {'apples': 30, 'bananas': 30, 'milk': 15, 'bread': 25, 'eggs': 50}

Night: Checking Inventory

Before closing the store, Pandian checks the inventory to ensure that all items are accurately recorded and none are missing.

Checking for Items:

# Checking if specific items are in the inventory
is_bananas_in_stock = "bananas" in inventory
is_oranges_in_stock = "oranges" in inventory

print(f"Are bananas in stock? {is_bananas_in_stock}")
print(f"Are oranges in stock? {is_oranges_in_stock}")
# Output: Are bananas in stock? True
# Output: Are oranges in stock? False

Midnight: Reviewing Inventory

After a busy day, Pandian reviews the entire inventory to ensure all deliveries and sales are accurately recorded.

Iterating Over the Inventory:

# Reviewing the final inventory
for item, quantity in inventory.items():
    print(f"Item: {item}, Quantity: {quantity}")

# Output:
# Item: apples, Quantity: 30
# Item: bananas, Quantity: 30
# Item: milk, Quantity: 15
# Item: bread, Quantity: 25
# Item: eggs, Quantity: 50