Caching is an essential technique for improving application performance and reducing the load on databases. However, improper caching strategies can lead to serious issues.

I got inspired from ByteByteGo https://www.linkedin.com/posts/bytebytego_systemdesign-coding-interviewtips-activity-7296767687978827776-Dizz

In this blog, we will discuss four common cache problems: Thundering Herd Problem, Cache Penetration, Cache Breakdown, and Cache Crash, along with their causes, consequences, and solutions.

1. Thundering Herd Problem
   1. What is it?
   2. Example Scenario
   3. Solutions
2. Cache Penetration
   1. What is it?
   2. Example Scenario
   3. Solutions
3. Cache Breakdown
   1. What is it?
   2. Example Scenario
   3. Solutions
4. Cache Crash
   1. What is it?
   2. Example Scenario
   3. Solutions

Thundering Herd Problem

What is it?

The Thundering Herd Problem occurs when a large number of keys in the cache expire at the same time. When this happens, all requests bypass the cache and hit the database simultaneously, overwhelming it and causing performance degradation or even a system crash.

Example Scenario

Imagine an e-commerce website where product details are cached for 10 minutes. If all the products’ cache expires at the same time, thousands of users sending requests will cause an overwhelming load on the database.

Solutions

1. Staggered Expiration: Instead of setting a fixed expiration time for all keys, introduce a random expiry variation.
2. Allow Only Core Business Queries: Limit direct database access only to core business data, while returning stale data or temporary placeholders for less critical data.
3. Lazy Rebuild Strategy: Instead of all requests querying the database, the first request fetches data and updates the cache while others wait.
4. Batch Processing: Queue multiple requests and process them in batches to reduce database load.

Cache Penetration

What is it?

Cache Penetration occurs when requests are made for keys that neither exist in the cache nor in the database. Since these requests always hit the database, they put excessive pressure on the system.

Example Scenario

A malicious user could attempt to query random user IDs that do not exist, forcing the system to repeatedly query the database and skip the cache.

Solutions

1. Cache Null Values: If a key does not exist in the database, store a null value in the cache to prevent unnecessary database queries.
2. Use a Bloom Filter: A Bloom filter helps check whether a key exists before querying the database. If the Bloom filter does not contain the key, the request is discarded immediately.
3. Rate Limiting: Implement request throttling to prevent excessive access to non-existent keys.
4. Data Prefetching: Predict and load commonly accessed data into the cache before it is needed.

Cache Breakdown

What is it?

Cache Breakdown is similar to the Thundering Herd Problem, but it occurs specifically when a single hot key (a frequently accessed key) expires. This results in a surge of database queries as all users try to retrieve the same data.

Example Scenario

A social media platform caches trending hashtags. If the cache expires, millions of users will query the same hashtag at once, hitting the database hard.

Solutions

1. Never Expire Hot Keys: Keep hot keys permanently in the cache unless an update is required.
2. Preload the Cache: Refresh the cache asynchronously before expiration by setting a background task to update the cache regularly.
3. Mutex Locking: Ensure only one request updates the cache, while others wait for the update to complete.
4. Double Buffering: Maintain a secondary cache layer to serve requests while the primary cache is being refreshed.

Cache Crash

What is it?

A Cache Crash occurs when the cache service itself goes down. When this happens, all requests fall back to the database, overloading it and causing severe performance issues.

Example Scenario

If a Redis instance storing session data for a web application crashes, all authentication requests will be forced to hit the database, leading to a potential outage.

Solutions

1. Cache Clustering: Use a cluster of cache nodes instead of a single instance to ensure high availability.
2. Persistent Storage for Cache: Enable persistence modes like Redis RDB or AOF to recover data quickly after a crash.
3. Automatic Failover: Configure automated failover with tools like Redis Sentinel to ensure availability even if a node fails.
4. Circuit Breaker Mechanism: Prevent the application from directly accessing the database if the cache is unavailable, reducing the impact of a crash.

class CircuitBreaker:
    def __init__(self, failure_threshold=5):
        self.failure_count = 0
        self.failure_threshold = failure_threshold
    
    def call(self, func, *args, **kwargs):
        if self.failure_count >= self.failure_threshold:
            return "Service unavailable"
        try:
            return func(*args, **kwargs)
        except Exception:
            self.failure_count += 1
            return "Error"

Caching is a powerful mechanism to improve application performance, but improper strategies can lead to severe bottlenecks. Problems like Thundering Herd, Cache Penetration, Cache Breakdown, and Cache Crash can significantly degrade system reliability if not handled properly.

Today, I learnt about Valet Key Pattern, which helps clients to directly access the resources without the server using a token. In this blog, i jot down notes on valet key pattern for better understanding.

The Valet Key Pattern is a security design pattern used to provide limited access to a resource or service without exposing full access credentials or permissions. It is akin to a physical valet key for a car, which allows the valet to drive the car without accessing the trunk or glove box. This pattern is widely employed in distributed systems, cloud services, and API design to ensure secure and controlled resource sharing.

Why Use the Valet Key Pattern?

Modern systems often require sharing access to specific resources while minimizing security risks. For instance:

• A mobile app needs to upload files to a storage bucket but shouldn’t manage the entire bucket.
• A third-party service requires temporary access to a user’s resource, such as a document or media file.
• A system needs to allow time-bound or operation-restricted access to sensitive data.

In these scenarios, the Valet Key Pattern provides a practical solution by issuing a scoped, temporary, and revocable token (valet key) that grants specific permissions.

Core Principles of the Valet Key Pattern

1. Scoped Access: The valet key grants access only to specific resources or operations.
2. Time-Limited: The access token is typically valid for a limited duration to minimize exposure.
3. Revocable: The issuing entity can revoke the token if necessary.
4. Minimal Permissions: Permissions are restricted to the least privilege required to perform the intended task.

How the Valet Key Pattern Works

1. Resource Owner Issues a Valet Key

The resource owner (or controlling entity) generates a token with limited permissions. This token is often a signed JSON Web Token (JWT) or a pre-signed URL in the case of cloud storage.

2. Token Delivery to the Client

The token is securely delivered to the client or third-party application requiring access. For instance, the token might be sent via HTTPS or embedded in an API response.

3. Client Uses the Valet Key

The client includes the token in subsequent requests to access the resource. The resource server validates the token, checks its permissions, and allows or denies the requested operation accordingly.

4. Expiry or Revocation

Once the token expires or is revoked, it becomes invalid, ensuring the client can no longer access the resource.

Examples of the Valet Key Pattern in Action

1. Cloud Storage (Pre-signed URLs)

Amazon S3, Google Cloud Storage, and Azure Blob Storage allow generating pre-signed URLs that enable temporary, scoped access to specific files. For example, a user can upload a file using a URL valid for 15 minutes without needing direct access credentials.

2. API Design

APIs often issue temporary access tokens for limited operations. OAuth 2.0 tokens, for instance, can be scoped to allow access to specific endpoints or resources.

3. Media Sharing Platforms

Platforms like YouTube or Dropbox use the Valet Key Pattern to provide limited access to files. A shareable link often embeds permissions and expiration details.

Implementation Steps

1. Define Permissions Scope

Identify the specific operations or resources the token should allow. Use the principle of least privilege to limit permissions.

2. Generate Secure Tokens

Create tokens with cryptographic signing to ensure authenticity. Include metadata such as:

• Resource identifiers
• Permissions
• Expiry time
• Issuer information

3. Validate Tokens

The resource server must validate incoming tokens by checking the signature, expiration, and permissions.

4. Monitor and Revoke

Maintain a mechanism to monitor token usage and revoke them if misuse is detected.

Best Practices

1. Use HTTPS: Always transmit tokens over secure channels to prevent interception.
2. Minimize Token Lifetime: Short-lived tokens reduce the risk of misuse.
3. Implement Auditing: Log token usage for monitoring and troubleshooting.
4. Employ Secure Signing: Use robust cryptographic algorithms to sign tokens and prevent tampering.

Challenges

• Token Management: Requires robust infrastructure for token generation, validation, and revocation.
• Revocation Delays: Invalidation mechanisms may not instantly propagate in distributed systems.

package taskPkg;

public class PatternsPrintNos {

	public static void main(String[] args) {
		// TODO Auto-generated method stub
		PatternsPrintNos ptnNos = new PatternsPrintNos();
		int n=9;
		for (int i=0; i<=9; i++) {
			ptnNos.printNo(i,n);
			System.out.println();
		}
	}
	private void printNo(int num, int n) {
		int m=n/2+1;
		switch(num) {
		case 0:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if (r==1 || r==n || c==1 || c==n) System.out.print(" *");
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;			
		case 1:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if (c==m) System.out.print(" *");
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;
		case 2:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if (r==1 || r==n || r==m || (r<=m && c==n) || (c==1 && r>=m)) System.out.print(" *");
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;
		case 3:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if (r==1 || r==n || r==m || c==n) System.out.print(" *"); 
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;
		case 4:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if ((c==1 && r<=m) || c==m ||  r==m ) System.out.print(" *");
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;
		case 5:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if (r==1 || r==n || r==m || (r<=m && c==1) || (c==n && r>=m)) System.out.print(" *");
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;
		case 6:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if (c==1  ||  r==n || r==m || (r>m && c==n)) System.out.print(" *");
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;
		case 7:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if (r==1 || (c==n-r+1)) System.out.print(" *");
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;
		case 8:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if (r==1 || r==m || r==n || c==1 || c==n) System.out.print(" *");
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;		
		case 9:
			for (int r=1; r<=n; r++) {
				for (int c=1; c<=n; c++) {
					if (r==1 || r==m || (c==1 && r<=m) || c==n) System.out.print(" *");
					else  System.out.print("  ");
				}
				System.out.println();
			}
			break;				
		}

	}
}

Output: