In the rapidly evolving landscape of Large Language Models (LLMs), effective prompt engineering has become a crucial skill. While much attention is given to the art of crafting effective prompts, less focus has been placed on how to efficiently manage these prompts programmatically. Python, with its rich set of built-in features, offers powerful tools to dynamically construct, optimize, and manage LLM prompts.
This article explores how Python's built-in features can transform your approach to LLM prompt engineering, making your code more efficient, maintainable, and powerful.

1. Using locals() for Dynamic Context Injection

The Problem
When working with LLMs, we often need to inject contextual information into our prompts. The traditional approach involves manual string formatting:

def generate_response(user_name, user_query, previous_context):
    prompt = f"""
    User name: {user_name}
    User query: {user_query}
    Previous context: {previous_context}

    Please respond to the user's query considering the context above.
    """

    return call_llm_api(prompt)

This works well for simple cases, but becomes unwieldy as the number of variables increases. It's also error-prone – you might forget to include a variable or update a variable name.

The Solution with locals()
Python's locals() function returns a dictionary containing all local variables in the current scope. We can leverage this to automatically include all relevant context:

def generate_response(user_name, user_query, previous_context, user_preferences=None, user_history=None):
    # All local variables are now accessible
    context_dict = locals()

    # Build a dynamic prompt section with all available context
    context_sections = []
    for key, value in context_dict.items():
        if value is not None:  # Only include non-None values
            context_sections.append(f"{key}: {value}")

    context_text = "\n".join(context_sections)

    prompt = f"""
    Context information:
    {context_text}

    Please respond to the user's query considering the context above.
    """

    return call_llm_api(prompt)

Benefits:

Automatic variable inclusion: If you add a new parameter to your function, it's automatically included in the context.
Reduced errors: No need to manually update string formatting when variables change.
Cleaner code: Separates the mechanism of context injection from the specific variables.

2. Using inspect for Function Documentation

The Problem
When creating LLM prompts that involve function execution or code generation, providing accurate function documentation is crucial:

def create_function_prompt(func_name, params):
    prompt = f"""
    Create a Python function named '{func_name}' with the following parameters:
    {params}
    """
    return prompt

This approach requires manually specifying function details, which can be tedious and error-prone.

The Solution with inspect
Python's inspect module allows us to extract rich metadata from functions:

import inspect

def create_function_prompt(func_reference):
    # Get the function signature
    signature = inspect.signature(func_reference)

    # Get the function docstring
    doc = inspect.getdoc(func_reference) or "No documentation available"

    # Get source code if available
    try:
        source = inspect.getsource(func_reference)
    except:
        source = "Source code not available"

    prompt = f"""
    Function name: {func_reference.__name__}

    Signature: {signature}

    Documentation:
    {doc}

    Original source code:
    {source}

    Please create an improved version of this function.
    """

    return prompt

# Example usage
def example_func(a, b=10):
    """This function adds two numbers together."""
    return a + b

improved_function_prompt = create_function_prompt(example_func)
# Send to LLM for improvement

This dynamically extracts all relevant information about the function, making the prompt much more informative.

3. Context Management with Class Attributes

The Problem
Managing conversation history and context with LLMs often leads to repetitive code:

conversation_history = []

def chat_with_llm(user_input):
    # Manually build the prompt with history
    prompt = "Previous conversation:\n"
    for entry in conversation_history:
        prompt += f"{entry['role']}: {entry['content']}\n"

    prompt += f"User: {user_input}\n"
    prompt += "Assistant: "

    response = call_llm_api(prompt)

    # Update history
    conversation_history.append({"role": "User", "content": user_input})
    conversation_history.append({"role": "Assistant", "content": response})

    return response

The Solution with Class Attributes and dict
We can create a conversation manager class that uses Python's object attributes:

class ConversationManager:
    def __init__(self, system_prompt=None, max_history=10):
        self.history = []
        self.system_prompt = system_prompt
        self.max_history = max_history
        self.user_info = {}
        self.conversation_attributes = {
            "tone": "helpful",
            "style": "concise",
            "knowledge_level": "expert"
        }

    def add_user_info(self, **kwargs):
        """Add user-specific information to the conversation context."""
        self.user_info.update(kwargs)

    def set_attribute(self, key, value):
        """Set a conversation attribute."""
        self.conversation_attributes[key] = value

    def build_prompt(self, user_input):
        """Build a complete prompt using object attributes."""
        prompt_parts = []

        # Add system prompt if available
        if self.system_prompt:
            prompt_parts.append(f"System: {self.system_prompt}")

        # Add conversation attributes
        prompt_parts.append("Conversation attributes:")
        for key, value in self.conversation_attributes.items():
            prompt_parts.append(f"- {key}: {value}")

        # Add user info if available
        if self.user_info:
            prompt_parts.append("\nUser information:")
            for key, value in self.user_info.items():
                prompt_parts.append(f"- {key}: {value}")

        # Add conversation history
        if self.history:
            prompt_parts.append("\nConversation history:")
            for entry in self.history[-self.max_history:]:
                prompt_parts.append(f"{entry['role']}: {entry['content']}")

        # Add current user input
        prompt_parts.append(f"\nUser: {user_input}")
        prompt_parts.append("Assistant:")

        return "\n".join(prompt_parts)

    def chat(self, user_input):
        """Process a user message and get response from LLM."""
        prompt = self.build_prompt(user_input)

        response = call_llm_api(prompt)

        # Update history
        self.history.append({"role": "User", "content": user_input})
        self.history.append({"role": "Assistant", "content": response})

        return response

    def get_state_as_dict(self):
        """Return a dictionary of the conversation state using __dict__."""
        return self.__dict__

    def save_state(self, filename):
        """Save the conversation state to a file."""
        import json
        with open(filename, 'w') as f:
            json.dump(self.get_state_as_dict(), f)

    def load_state(self, filename):
        """Load the conversation state from a file."""
        import json
        with open(filename, 'r') as f:
            state = json.load(f)
            self.__dict__.update(state)```



Using this approach:

# Create a conversation manager
convo = ConversationManager(system_prompt="You are a helpful assistant.")

# Add user information
convo.add_user_info(name="John", expertise="beginner", interests=["Python", "AI"])

# Set conversation attributes
convo.set_attribute("tone", "friendly")

# Chat with the LLM
response = convo.chat("Can you help me understand how Python dictionaries work?")
print(response)

# Later, save the conversation state
convo.save_state("conversation_backup.json")

# And load it back
new_convo = ConversationManager()
new_convo.load_state("conversation_backup.json")

4. Using dir() for Object Exploration

The Problem
When working with complex objects or APIs, it can be challenging to know what data is available to include in prompts:

def generate_data_analysis_prompt(dataset):
    # Manually specifying what we think is available
    prompt = f"""
    Dataset name: {dataset.name}
    Number of rows: {len(dataset)}

    Please analyze this dataset.
    """
    return prompt

The Solution with dir()
Python's dir() function lets us dynamically discover object attributes and methods:

def generate_data_analysis_prompt(dataset):
    # Discover available attributes
    attributes = dir(dataset)

    # Filter out private attributes (those starting with _)
    public_attrs = [attr for attr in attributes if not attr.startswith('_')]

    # Build metadata section
    metadata = []
    for attr in public_attrs:
        try:
            value = getattr(dataset, attr)
            # Only include non-method attributes with simple values
            if not callable(value) and not hasattr(value, '__dict__'):
                metadata.append(f"{attr}: {value}")
        except:
            pass  # Skip attributes that can't be accessed

    metadata_text = "\n".join(metadata)

    prompt = f"""
    Dataset metadata:
    {metadata_text}

    Please analyze this dataset based on the metadata above.
    """

    return prompt

This approach automatically discovers and includes relevant metadata without requiring us to know the exact structure of the dataset object in advance.

5. String Manipulation for Prompt Cleaning

The Problem
User inputs and other text data often contain formatting issues that can affect LLM performance:

def process_document(document_text):
    prompt = f"""
    Document:
    {document_text}

    Please summarize the key points from this document.
    """
    return call_llm_api(prompt)

The Solution with String Methods
Python's rich set of string manipulation methods can clean and normalize text:

def process_document(document_text):
    # Remove excessive whitespace
    cleaned_text = ' '.join(document_text.split())

    # Normalize line breaks
    cleaned_text = cleaned_text.replace('\r\n', '\n').replace('\r', '\n')

    # Limit length (many LLMs have token limits)
    max_chars = 5000
    if len(cleaned_text) > max_chars:
        cleaned_text = cleaned_text[:max_chars] + "... [truncated]"

    # Replace problematic characters
    for char, replacement in [('\u2018', "'"), ('\u2019', "'"), ('\u201c', '"'), ('\u201d', '"')]:
        cleaned_text = cleaned_text.replace(char, replacement)

    prompt = f"""
    Document:
    {cleaned_text}

    Please summarize the key points from this document.
    """

    return call_llm_api(prompt)

Conclusion

Python's built-in features offer powerful capabilities for enhancing LLM prompts:

Dynamic Context: Using locals() and dict to automatically include relevant variables
Introspection: Using inspect and dir() to extract rich metadata from objects and functions
String Manipulation: Using Python's string methods to clean and normalize text

By leveraging these built-in features, you can create more robust, maintainable, and dynamic LLM interactions. The techniques in this article can help you move beyond static prompt templates to create truly adaptive and context-aware LLM applications.
Most importantly, these approaches scale well as your LLM applications become more complex, allowing you to maintain clean, readable code while supporting sophisticated prompt engineering techniques.
Whether you're building a simple chatbot or a complex AI assistant, Python's built-in features can help you create more effective LLM interactions with less code and fewer errors.

Introduction

As artificial intelligence advances, there is growing interest in evaluating how different AI models perform in specialized domains like clinical trial statistics. This article compares two state-of-the-art large language models — Llama 4 Scout Reasoning and Claude 3.7 — on their ability to solve common statistical problems in clinical trials. It’s important to emphasize that this study examines only a limited set of three clinical trial problems and should not be interpreted as a comprehensive assessment of these models’ overall capabilities.

Llama 4 Scout Instruct Model

Claude 3.7

Problem Selection

Three foundational clinical trial statistical problems were selected to evaluate the models:

Treatment Effect Analysis: Calculating response rates, absolute risk reduction (ARR), and number needed to treat (NNT) in a cancer treatment study comparing experimental and control arms

Non-inferiority Trial Design: Determining the minimum cure rate required for a new antibiotic to be considered non-inferior to the standard of care

Interim Analysis Decision-Making: Applying O’Brien-Fleming boundaries to decide whether to stop a trial early based on interim results

Evaluation Criteria

The outputs from both models were compared across several dimensions:

• Mathematical accuracy
• Statistical reasoning approach
• Clarity of explanation
• Contextual understanding
• Presentation format
• Result interpretation

Detailed Findings

Mathematical Precision

Both models demonstrated excellent mathematical precision, arriving at identical numerical answers for all three problems:

• In Problem 1, both correctly calculated the response rates (55.6% vs 44.4%), ARR (11.2%), and NNT (9)
• In Problem 2, both determined the minimum acceptable cure rate to be 70%
• In Problem 3, both correctly concluded that the trial should not be stopped based on the interim analysis

Approach to Statistical Reasoning

Llama 4 Scout Reasoning

Llama 4 Scout approached problems with a highly structured methodology:

• Began by clearly organizing given information in bulleted lists
• Used explicit section headings to demarcate reasoning steps
• Provided direct formula applications with clear variable substitutions
• Included practical interpretations of the final statistical outcomes

Claude 3.7

Claude 3.7 demonstrated a more narrative reasoning style:

• Used numbered steps with detailed explanations before formula application
• Provided more extensive context about the statistical principles being applied
• Explained the reasoning behind formula selection
• Included additional interpretation of why certain approaches were appropriate

Explanation Quality

The models differed somewhat in their explanatory approaches:

Llama 4 Scout Reasoning

• Provided concise but complete explanations
• Excellent at clarifying practical implications (e.g., “This means we would need to treat 9 patients with the experimental treatment instead of the control treatment to achieve one additional response”)
• Included additional context about threshold interpretations

• Explicit about Type I error control in the interim analysis problem

  Claude 3.7

• Offered more detailed contextual explanations of statistical concepts

• Provided more extensive rationale for calculation approaches

• Explained why certain statistical formulas were appropriate in each context

• Included more discussion of underlying statistical principles (e.g., family-wise error rate)

Visual Presentation

The models showed distinct presentation styles:

Llama 4 Scout Reasoning

• More visually structured with effective use of bulleted lists
• Clear section labeling and organization
• Highlighted key information more prominently
• More consistent formatting throughout responses

Claude 3.7

• More narrative in style with detailed steps
• Used numbered steps with explanatory headings
• Integrated formulas within explanatory text
• Maintained professional presentation but with less visual structuring

Discussion

Complementary Strengths
This comparative analysis revealed that both models possess strong capabilities in clinical trial statistical reasoning but with slightly different emphases:

• Llama 4 Scout Reasoning excelled in structural organization, practical interpretation, and concise presentation of statistical concepts
• Claude 3.7 demonstrated particular strength in contextual explanation, statistical rationale, and discussion of underlying principles.

Practical Implications

For professionals working in clinical trial statistics, the choice between these models might depend on specific needs:

• For quick statistical calculations with a clear structure: Llama 4 Scout Reasoning’s concise and well-organized approach might be preferred.
• Claude 3.7’s more detailed explanations might offer more excellent educational value for a deeper understanding of statistical principles.

However, the mathematical precision demonstrated by both models suggests that either would be reliable for calculating vital clinical trial statistics.

Comparing Llama 4 Scout Reasoning vs Claude 3.7 in Clinical Trial Problem Solving

Comparing Llama 4 Scout Reasoning vs Claude 3.5 in Clinical Trial Problem Solving

Conclusion

This limited comparison suggests that Llama 4 Scout Reasoning and Claude 3.7 demonstrate strong capabilities in solving clinical trial statistical problems. However, Llama 4 Scout is open-source, and you can fine-tune it with your data, which will be more powerful.

It’s worth emphasizing that this analysis is based on only three specific problems and should not be extrapolated to represent overall model capabilities across the broad and complex domain of clinical trial statistics. A more comprehensive evaluation would require testing across a broader range of problem types, complexity levels, and specialized statistical methods used in clinical trials.

In today's data-driven world, the ability to quickly extract insights from documents is becoming increasingly valuable. Whether you're a researcher sifting through academic papers, a legal professional analyzing contracts, or a business analyst reviewing reports, the challenge of efficiently navigating through lengthy documents remains universal.

Enter Llama-4-Scout, Meta's latest language model that's showing remarkable capabilities in document question-answering tasks. Released on April 5, 2025, Scout is a groundbreaking addition to the new Llama 4 family, specifically designed to be more reliable, precise, and effective at finding and extracting information from documents.

What Makes Llama-4-Scout Special?

Llama-4-Scout utilizes a Mixture-of-Experts (MoE) architecture with 17 billion active parameters (out of 109 billion total parameters across 16 experts) that strikes an optimal balance between performance and efficiency. What truly sets Scout apart is its unprecedented 10-million token context window, making it best in class for long-context tasks like summarizing huge codebases and analyzing extensive documents.

Scout was pretrained on approximately 40 trillion tokens of multimodal data and excels at:

Document comprehension: Understanding context across extremely long documents

Information retrieval: Precisely locating specific details in complex content

Source attribution: Providing clear references to where information was found

Multimodal reasoning: Visual recognition, image reasoning, captioning, and answering questions about document visuals

This focused design makes Scout particularly well-suited for professional document analysis tasks that require both accuracy and reasonable computational resources. Remarkably, the model fits on a single H100 GPU when quantized to Int4, making it accessible for organizations without massive computing infrastructure.

Real-World Example: Document Q&A on the AGENTS Framework

I recently had the opportunity to test Llama-4-Scout's capabilities by building a simple PDF Q&A application with a research paper on the AGENTS framework (a system for autonomous language agents). The results demonstrated Scout's remarkable ability to extract, understand, and contextualize information from technical documents. Here's what I observed:

Source Paper: https://arxiv.org/pdf/2309.07870

Example 1: Identifying Core Framework Components

When asked about the main features of the framework described in the document, Scout successfully extracted and summarized the key components:

Query: "What are the main features of the AGENTS framework?"

Scout's Response: "The main features of the AGENTS framework include planning, memory (specifically long-short term memory implementation), tool usage capabilities, multi-agent communication protocols, and fine-grained symbolic control through SOPs (Standard Operating Procedures). Source: Page 4, Framework Architecture section"

Example 2: Understanding Technical Implementation Details

Scout excelled at extracting specific technical implementation details related to memory systems:

Query: "How does long-short term memory work in AGENTS?"

Scout's Response: "The AGENTS framework implements a dual-memory system where long-term memories are constructed as action histories embedded by sentence-transformers and stored in VectorDB for semantic search retrieval. In contrast, short-term memories (working memories) are maintained in natural language form and dynamically updated by an LLM through a carefully tuned prompt system. Source: Pages 7-8, Memory Architecture section"

Example 3: Explaining Human-Agent Interaction Mechanisms

The model demonstrated its ability to identify specific configuration details that enable key functionalities:

Query: "How does the AGENTS framework support human-agent interaction?"

Scout's Response: "The AGENTS framework supports human-agent interaction through a simple but effective configuration mechanism. Users can change an agent's 'is_human' field to 'True' in the config file, which enables humans to take the role of an agent and interact with other language agents in both single-agent and multi-agent scenarios. Source: Page 12, Human-Agent Interaction section"

Example 4: Clarifying Domain-Specific Terminology

When asked about specialized terminology in the document, Scout provided clear and accurate explanations:

Query: "What is SOP in the AGENTS framework?"

Scout's Response: "In the AGENTS framework, SOP (Standard Operating Procedures) is a symbolic plan that provides fine-grained control of agents' behaviors. Technically, it's implemented as a graph of multiple states that define different situations an agent may encounter, with explicit transition rules between these states. This architecture makes agents' behavior more stable and predictable across various scenarios. Source: Page 9, Agent Control Mechanisms section"

Example 5: Identifying Ecosystem Components

Scout successfully extracted information about the broader ecosystem supporting the framework:

Query: "What is the Agent Hub in the AGENTS framework?"

Scout's Response: "The Agent Hub is a platform component of the AGENTS framework that enables users to share their fine-tuned language agents and search/download useful language agents created by others. This significantly reduces the effort required for designing, testing, and tuning language agents from scratch, creating a collaborative ecosystem for agent development. Source: Page 15, Ecosystem and Tools section"

Building Your Own Document Intelligence Tool

The ability to build custom applications around Llama-4-Scout is straightforward through providers like Together.ai. I created a simple Streamlit application that allows users to:

• Upload any PDF document
• Process both the text and visual content
• Ask questions in natural language
• Receive answers with source references
• Evaluate answer quality with multiple criteria

This type of application can transform how teams work with documents, significantly reducing the time spent searching for information and improving the accuracy of insights extracted. Scout's ability to process images in documents means it can work with charts, tables, diagrams, and other visual elements that are often critical for full document understanding and is "optimized for visual recognition, image reasoning, captioning, and answering general questions about an image."

Technical Capabilities and Performance

Llama-4-Scout demonstrates impressive performance relative to competing models. In comparative evaluations, Scout has shown "superior performance relative to contemporary models such as Gemma 3, Gemini 2.0 Flash-Lite, and Mistral 3.1 across recognized benchmark datasets."

What makes Scout particularly practical is its efficiency. Scout "fits on a single H100 GPU when quantized to Int4" while still delivering high-quality results. This efficiency means organizations can implement advanced document intelligence without requiring massive computational resources.

Looking Ahead: The Future of Document Intelligence

As models like Llama-4-Scout continue to evolve, we can expect even more sophisticated document intelligence capabilities. Future developments will likely include:

• Deeper reasoning across multiple documents
• More nuanced understanding of domain-specific content
• Better handling of ambiguity and uncertain information
• Enhanced multimodal capabilities for complex visual content

Conclusion

Llama-4-Scout represents a significant step forward in making advanced document intelligence accessible. Its balanced approach to performance and efficiency makes it particularly valuable for professional applications where accuracy and attribution matter.

For organizations dealing with large volumes of documents, investing in tools built around models like Scout could yield substantial returns through improved information accessibility and insight generation. The model's ability to "process and work with extremely lengthy documents" makes it ideal for enterprises with extensive documentation needs.

Have you experimented with Llama-4-Scout or similar models for document analysis? I'd love to hear about your experiences and applications in the comments below.

Note: The examples provided are based on actual testing of Llama-4-Scout through Together.ai's API integration. Results may vary depending on document complexity and specific implementation details.

VibeCoding

As a CTO, I don’t typically get a lot of time to sit and code—there’s a lot of grunt work involved in my role. Quite a bit of my time goes into the operational aspects of running an engineering team: feature prioritization, production issue reviews, cloud cost reviews, one-on-ones, status updates, budgeting, etc.

Although I’m deeply involved in architecture, design, and scaling decisions, I’m not contributing as a senior developer writing code for features in the product as much as I’d like. It’s not just about writing code—it’s about maintaining it. And with everything else I do, I felt I didn’t have the time or energy to maintain a complex feature.

Over the past couple of years—apart from AI research work—my coding has been pretty much limited to pair programming or contributing to some minor, non-critical features. Sometimes I end up writing small utilities here and there.

Pair Programming as a Connection Tool

I love to pair program with at least two engineers for a couple of hours every week. This gives me an opportunity to connect with them 1:1, understand the problems on the ground, and help them see the bigger picture of how the features they’re working on fit into our product ecosystem.

1:1s were never effective for me. In that 30-minute window, engineers rarely open up. But if you sit and code for 2–3 hours at a stretch, you get to learn a lot about them—their problem-solving approach, what motivates them, and more.

I also send out a weekly video update to the engineering team, where I talk about the engineers I pair programmed with, their background, and what we worked on. It helps the broader engineering team learn more about their peers as well.

The Engineer in Me

The engineer in me always wants to get back to coding—because there’s no joy quite like building something and making it work. I’m happiest when I code.

I’ve worked in several languages over the years—Java, VB6, C#, Perl, good old shell scripting, Python, JavaScript, and more. Once I found Python, I never looked back. I absolutely love the Python community.

I’ve been a full-stack developer throughout my career. My path to becoming a CTO was non-traditional (that’s a story for another blog). I started in a consulting firm and worked across different projects and tech stacks early on, which helped me become a well-rounded full-stack engineer.

I still remember building a simple timesheet entry application in 2006 using HTML and JavaScript (with AJAX) for a client’s invoicing needs. It was a small utility, but it made timesheet entry so much easier for engineers. That experience stuck with me.

I’ll get to why being a full-stack engineer helped me build the app using VibeCoding shortly.

The Spark: Coffee with Shuveb Hussain

I was catching up over coffee with Shuveb Hussain, founder and CEO of ZipStack. Their product, Unstract, is really good for extracting entities from different types of documents. They’ve even open-sourced a version—go check it out.

Shuveb, being a seasoned engineer’s engineer, mentioned how GenAI code editors helped him quickly build a few apps for his own use over a weekend. That sparked something in me: why wasn’t I automating my grunt work with one of these GenAI code editors?

I’ve used GitHub Copilot for a while, but these newer GenAI editors—like Cursor and Windsurf—are in a different league. Based on Shuveb’s recommendation, I chose Windsurf.

Let’s be honest though—I don’t remember any weekend project of mine that ended in just one weekend 😅

The Grunt Work I Wanted to Automate

I was looking for ways to automate the boring but necessary stuff, so I could focus more on external-facing activities.

Every Thursday, I spent about 6 hours analyzing production issues before the weekly Friday review with engineering leaders, the SRE team, product leads, and support. I’d get a CSV dump of all the tickets and manually go through each one to identify patterns or repeated issues. Then I’d start asking questions on Slack or during the review meeting.

The process was painful and time-consuming. I did this for over 6 months and knew I needed to change something.

In addition to that:

• I regularly reviewed cloud compute costs across environments and products to identify areas for optimization.
• I monitored feature usage metrics to see what customers actually used.
• I examined job runtime stats (it’s a low-code platform, so this matters).
• I looked at engineering team metrics from the operations side.

Each of these lived in different tools, dashboards, or portals. I was tired of logging into 10 places and context-switching constantly while fielding distractions.

The Build Begins: CTO Dashboard

I decided to build an internal tool I nicknamed CTODashboard—to consolidate everything I needed.

My Tech Stack (via Windsurf):

• Frontend: ReactJS
• Backend: Python (FastAPI)
• Database: Postgres
• Deployment: EC2 (with some help from the SRE team)

I used Windsurf’s Cascade interface to prompt out code, even while attending meetings. It was surprisingly effective… except for one time when a prompt completely messed up my day’s work. Lesson learned: commit code at every working logical step.

In a couple of days, I had:

• A feature to upload the CSV dump
• Filters to slice and dice data
• A paginated data table
• Trend analytics with visualizations

Even when I hit errors, I just screenshot them or pasted logs into Windsurf and asked it to fix them. It did a decent job. When it hallucinated or got stuck, I just restarted with a fresh Cascade.

I had to rewrite the CSV upload logic manually when the semantic mapping to the backend tables went wrong. But overall, 80% of the code was generated—20% I wrote myself. And I reviewed everything to ensure it worked as intended.

Early Feedback & Iteration

I gave access to a couple of colleagues for early feedback. It was overwhelmingly positive. They even suggested new features like:

• Summarizing long tickets
• Copy-pasting ticket details directly into Slack
• Copying visualizations without taking screenshots

I implemented all of those using Windsurf in just a few more days.

In under a week, I had an MVP that cut my Thursday analysis time from 6+ hours to under 2.

Then Abhay Dandekar, another senior developer, offered to help. He built a Lambda function to call our Helpdesk API every hour to fetch the latest tickets and updates. He got it working in 4 hours—like a boss.

Growing Usage

Word about the dashboard started leaking (okay, I may have leaked it myself 😉). As more people requested access, I had to:

• Add Google Sign-In
• Implement authorization controls
• Build a user admin module
• Secure the backend APIs with proper access control
• Add audit logs to track who was using what

I got all this done over a weekend. It consumed my entire weekend, but it was worth it.

AWS Cost Analytics Module

Next, I added a module to analyze AWS cost trends across production and non-prod environments by product.

Initially, it was another CSV upload feature. Later, Abhay added a Lambda to fetch the data daily. I wanted engineering directors to see the cost implications of design decisions—especially when non-prod environments were always-on.

Before this, I spent 30 minutes daily reviewing AWS cost trends. Once the dashboard launched, engineers started checking it themselves. That awareness led to much smarter decisions and significant cost savings.

I added visualizations for:

• Daily cost trends
• Monthly breakdowns
• Environment-specific views
• Product-level costs

More Modules Coming Soon

I’ve since added:

• Usage metrics
• Capitalization tracking
• (In progress): Performance and engineering metrics

The dashboard now has 200+ users, and I’m releasing access in batches to manage performance.

Lessons Learned from VibeCoding

This was a fun experiment to see how far I could go with GenAI-based development using just prompts.

What I Learned:

1. Strong system design fundamentals are essential.
2. Windsurf can get stuck in loops—step in and take control.
3. Commit frequently. Mandatory.
4. You’re also the tester—don’t skip this.
5. If the app breaks badly, roll back. Don’t fix bad code.
6. GenAI editors are great for senior engineers; less convinced about junior devs.
7. Model training cutoffs matter—it affects library choices.
8. Write smart prompts with guardrails (e.g., “no files >300 lines”).
9. GenAI tools struggle to edit large files (my app.py hit 5,000 lines; I had to refactor manually).
10. Use virtual environments—GenAI often forgets.
11. Deployments are tedious—I took help from the SRE team for Jenkins/Terraform setup.
12. If you love coding, VibeCoding is addictive.

Gratitude

Special thanks to:

• Bhavani Shankar, Navin Kumaran, Geetha Eswaran, and Sabyasachi Rout for helping with deployment scripts and automation (yes, I bugged them a lot).
• Pravin Kumar, Vikas Kishore, Nandini PS, and Prithviraj Subburamanian for feedback and acting as de facto product managers for dashboard features.

Introduction

Ever wished you could create stunning portraits with just a few clicks? Meet Gibili Portrait Assistance, an AI-powered tool that makes generating high-quality portraits effortless. Whether you’re an artist, designer, or simply someone who loves experimenting with AI, Gibili can help bring your ideas to life.

In this post, we’ll walk you through how to use Gibili Portrait Assistance and explore the OpenAI architecture behind it.

How to Use Gibili Portrait Assistance

Using Gibili is straightforward and requires no prior technical knowledge. Here’s a simple step-by-step guide:

1. Enter Your Description or Upload an Image
You can either type a text description of the portrait you want or upload an existing image to be enhanced or transformed by AI.

Text Prompt Example:

• “A realistic portrait of a woman with curly brown hair, wearing a red scarf, in a cinematic lighting style.”

Image Upload:

• If you have an image you want to modify or enhance, simply upload it, and Gibili will apply AI-powered enhancements or transformations.

2. Customize Your Preferences
You can fine-tune details such as:

• Art Style: Realistic, digital painting, anime, etc.
• Background: Solid color, blurred, natural scenery.
• Facial Expressions: Smiling, neutral, surprised.
• Additional Features: Glasses, hats, jewelry, etc.

3. Generate the Image
Press Enter, and within seconds, Gibili will produce a high-resolution portrait based on your input or uploaded image.

4. Refine and Download
If you want adjustments, you can tweak your input and regenerate until you’re satisfied. Once ready, download your portrait in high-quality format.

The OpenAI Architecture Behind Gibili

Gibili Portrait Assistance is powered by OpenAI’s advanced image generation models, leveraging diffusion models to create highly detailed and realistic portraits. Here’s a simplified breakdown:

1. Text-to-Image & Image-to-Image Generation
When you provide a text prompt, the AI model translates it into a visual representation using deep learning techniques. If you upload an image, the model can enhance, transform, or stylize it while maintaining its core structure.

2. Fine-Tuned on Portrait Data
The model has been trained on a vast dataset of portraits across different styles, ensuring high accuracy and creativity in generated images.

3. Iterative Refinement
Instead of creating the final image instantly, the AI gradually refines it through multiple steps, ensuring greater precision and quality.

4. User-Guided Adjustments
Users can modify parameters like style and background, and the model will intelligently adjust the portrait while maintaining coherence.

Why Use Gibili Portrait Assistance?

✅ Easy to Use

No need for advanced design skills — just describe what you want or upload an image, and AI does the rest.

🎨 Customizable Output

From photorealistic portraits to artistic illustrations, you can tailor the results to your liking.

🚀 Fast & High-Quality

Generate high-resolution images within seconds.

🖌️ Creative Freedom

Perfect for artists, marketers, and content creators looking for unique visuals.

Get Started with Gibili Today!

Ready to create amazing AI-generated portraits? Try Gibili Portrait Assistance now and explore the limitless possibilities of AI-powered creativity!

Introduction

Kubernetes (K8s) is a powerful container orchestration platform that simplifies application deployment and scaling. In this guide, we’ll set up Kubernetes on an AWS EC2 instance, install the Nginx Ingress Controller, and configure Ingress rules to expose multiple services (app1 and app2).

Step 1: Setting Up Kubernetes on an EC2 Instance
1.1 Launch an EC2 Instance
Choose an instance with enough resources (e.g., t3.medium or larger) and install Ubuntu 20.04 or Amazon Linux 2.
1.2 Update Packages

sudo apt update && sudo apt upgrade -y  # For Ubuntu 

1.3 Install Docker

sudo apt install -y docker.io  
sudo systemctl enable --now docker

1.4 Install Kubernetes (kubectl, kubeadm, kubelet)

sudo apt install -y apt-transport-https ca-certificates curl
sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update
sudo apt install -y kubelet kubeadm kubectl

1.5 Initialize Kubernetes

sudo kubeadm init --pod-network-cidr=192.168.0.0/16

Follow the output instructions to set up kubectl for your user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

1.6 Install a Network Plugin (Calico)

For Calico:
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

Now, Kubernetes is ready!

Step 2: Install Nginx Ingress Controller
Nginx Ingress Controller helps manage external traffic to services inside the cluster.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

Wait until the controller is running:

kubectl get pods -n ingress-nginx

You should see ingress-nginx-controller running.

Step 3: Deploy Two Applications (app1 and app2)
3.1 Deploy app1
Create app1-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app1
spec:
  replicas: 2
  selector:
    matchLabels:
      app: app1
  template:
    metadata:
      labels:
        app: app1
    spec:
      containers:
      - name: app1
        image: nginx
        ports:
        - containerPort: 80

Create app1-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: app1-service
spec:
  selector:
    app: app1
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: ClusterIP

Apply the resources:

kubectl apply -f app1-deployment.yaml 
kubectl apply -f app1-service.yaml

3.2 Deploy app2
Create app2-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app2
spec:
  replicas: 2
  selector:
    matchLabels:
      app: app2
  template:
    metadata:
      labels:
        app: app2
    spec:
      containers:
      - name: app2
        image: nginx
        ports:
        - containerPort: 80

Create app2-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: app2-service
spec:
  selector:
    app: app2
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: ClusterIP

Apply the resources:

kubectl apply -f app2-deployment.yaml 
kubectl apply -f app2-service.yaml

Step 4: Configure Ingress for app1 and app2
Create nginx-ingress.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: app1.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app1-service
            port:
              number: 80
  - host: app2.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app2-service
            port:
              number: 80

Apply the Ingress rule:

kubectl apply -f nginx-ingress.yaml

Step 5: Verify Everything
5.1 Get Ingress External IP

kubectl get ingress

5.2 Update /etc/hosts (Local Testing Only)
If you're testing on a local machine, add this to /etc/hosts:

<EXTERNAL-IP> app1.example.com
<EXTERNAL-IP> app2.example.com

Replace with the actual external IP of your Ingress Controller.
5.3 Test in Browser or Curl

curl http://app1.example.com
curl http://app2.example.com

If everything is set up correctly, you should see the default Nginx welcome page for both applications.

Conclusion
In this guide, we:

• Installed Kubernetes on an EC2 instance
• Set up Nginx Ingress Controller
• Deployed two services (app1 and app2)
• Configured Ingress to expose them via domain names

Now, you can easily manage multiple applications in your cluster using a single Ingress resource.

Follow for more . Happy learning :)

In this blog, I will guide you through step-by-step instructions to set up a two-tier architecture on AWS using VPC, Subnets, Internet Gateway, Route Tables, RDS, EC2, Apache, MySQL, PHP, and HTML. This project will allow you to host a registration web application where users can submit their details, which will be stored in an RDS MySQL database.

Step 1: Create a VPC
1.1 Login to AWS Management Console

• Navigate to the VPC service
• Click Create VPC
• Enter the following details:
• VPC Name: my-vpc
• IPv4 CIDR Block: 10.0.0.0/16
• Tenancy: Default
• Click Create VPC

Step 2: Create Subnets
2.1 Create a Public Subnet

• Go to VPC > Subnets
• Click Create Subnet
• Choose my-vpc
• Set Subnet Name: public-subnet
• IPv4 CIDR Block: 10.0.1.0/24
• Click Create

2.2 Create a Private Subnet
Repeat the steps above but set:

• Subnet Name: private-subnet
• IPv4 CIDR Block: 10.0.2.0/24

Step 3: Create an Internet Gateway (IGW) and Attach to VPC
3.1 Create IGW

• Go to VPC > Internet Gateways
• Click Create Internet Gateway
• Set Name: your-igw
• Click Create IGW 3.2 Attach IGW to VPC
• Select your-igw
• Click Actions > Attach to VPC
• Choose my-vpc and click Attach

Step 4: Configure Route Tables
4.1 Create a Public Route Table

• Go to VPC > Route Tables
• Click Create Route Table
• Set Name: public-route-table
• Choose my-vpc and click Create
• Edit Routes → Add a new route:
• Destination: 0.0.0.0/0
• Target: my-igw
• Edit Subnet Associations → Attach public-subnet

Step 5: Create an RDS Database (MySQL)

• Go to RDS > Create Database
• Choose Standard Create
• Select MySQL
• Set DB instance identifier: my-rds
• Master Username: admin
• Master Password: yourpassword
• Subnet Group: Select private-subnet
• VPC Security Group: Allow 3306 (MySQL) from my-vpc
• Click Create Database

Step 6: Launch an EC2 Instance

• Go to EC2 > Launch Instance
• Choose Ubuntu 22.04
• Set Instance Name: my-ec2
• Select my-vpc and attach public-subnet
• Security Group: Allow
• SSH (22) from your IP
• HTTP (80) from anywhere
• MySQL (3306) from my-vpc
• Click Launch Instance

Step 7: Install Apache, PHP, and MySQL Client
7.1 Connect to EC2

ssh -i your-key.pem ubuntu@your-ec2-public-ip

7.2 Install LAMP Stack

sudo apt update && sudo apt install -y apache2 php libapache2-mod-php php-mysql mysql-client

7.3 Start Apache

sudo systemctl start apache2
sudo systemctl enable apache2

Step 8: Configure Web Application
8.1 Create the Registration Form

cd /var/www/html

sudo nano index.html

<!DOCTYPE html>
<html>
<head>
    <title>Registration Form</title>
</head>
<body>
    <h2>User Registration</h2>
    <form action="submit.php" method="POST">
        Name: <input type="text" name="name" required><br>
        DOB: <input type="date" name="dob" required><br>
        Email: <input type="email" name="email" required><br>
        <input type="submit" value="Register">
    </form>
</body>
</html>

8.2 Create PHP Script (submit.php)

sudo nano /var/www/html/submit.php

<?php
$servername = "your-rds-endpoint";
$username = "admin";
$password = "yourpassword";
$dbname = "registration";
$conn = new mysqli($servername, $username, $password, $dbname);
if ($conn->connect_error) {
    die("Connection failed: " . $conn->connect_error);
}
$name = $_POST['name'];
$dob = $_POST['dob'];
$email = $_POST['email'];
$stmt = $conn->prepare("INSERT INTO users (name, dob, email) VALUES (?, ?, ?)");
$stmt->bind_param("sss", $name, $dob, $email);
if ($stmt->execute()) {
    echo "Registration successful";
} else {
    echo "Error: " . $stmt->error;
}
$stmt->close();
$conn->close();
?>

Step 9: Create Target Group

1. Go to the AWS EC2 Console → Navigate to Target Groups
2. Click Create target group
3. Choose Target type: Instance
4. Enter Target group name: my-target-group
5. Select Protocol: HTTP
6. Select Port: 80
7. Choose the VPC you created earlier
8. Click Next
9. Under Register Targets, select your EC2 instances
10. Click Include as pending below, then Create target group

Step 10: Create an Application Load Balancer (ALB)

1. Go to AWS EC2 Console → Navigate to Load Balancers
2. Click Create Load Balancer
3. Choose Application Load Balancer
4. Enter ALB Name: my-alb
5. Scheme: Internet-facing
6. IP address type: IPv4
7. Select the VPC
8. Select at least two public subnets (for high availability)
9. Click Next

Step 11: Test the Application

1. Restart Apache sudo systemctl restart apache2
2. Open your browser and visit: http://your-ec2-public-ip/
3. Fill in the form and Submit
4. Check MySQL Database:

mysql -u admin -p -h your-rds-endpoint
USE your_database;
SELECT * FROM table_name;

This setup ensures a scalable, secure, and high-availability application on AWS! 🚀

Follow for more and happy learning :)

Introduction

Large Language Model (LLM) based AI agents represent a new paradigm in artificial intelligence. Unlike traditional software agents, these systems leverage the powerful capabilities of LLMs to understand, reason, and interact with their environment in more sophisticated ways. This guide will introduce you to the basics of LLM agents and their think-act-observe cycle.

What is an LLM Agent?

An LLM agent is a system that uses a large language model as its core reasoning engine to:

1. Process natural language instructions
2. Make decisions based on context and goals
3. Generate human-like responses and actions
4. Interact with external tools and APIs
5. Learn from interactions and feedback

Think of an LLM agent as an AI assistant who can understand, respond, and take actions in the digital world, like searching the web, writing code, or analyzing data.

The Think-Act-Observe Cycle in LLM Agents

Observe (Input Processing)

LLM agents observe their environment through:

1. Direct user instructions and queries
2. Context from previous conversations
3. Data from connected tools and APIs
4. System prompts and constraints
5. Environmental feedback

Think (LLM Processing)

The thinking phase for LLM agents involves:

1. Parsing and understanding input context
2. Reasoning about the task and requirements
3. Planning necessary steps to achieve goals
4. Selecting appropriate tools or actions
5. Generating natural language responses

The LLM is the "brain," using its trained knowledge to process information and make decisions.

Act (Execution)

LLM agents can take various actions:

1. Generate text responses
2. Call external APIs
3. Execute code
4. Use specialized tools
5. Store and retrieve information
6. Request clarification from users

Key Components of LLM Agents

Core LLM

1. Serves as the primary reasoning engine
2. Processes natural language input
3. Generates responses and decisions
4. Maintains conversation context

Working Memory

1. Stores conversation history
2. Maintains current context
3. Tracks task progress
4. Manages temporary information

Tool Use

1. API integrations
2. Code execution capabilities
3. Data processing tools
4. External knowledge bases
5. File manipulation utilities

Planning System

1. Task decomposition
2. Step-by-step reasoning
3. Goal tracking
4. Error handling and recovery

Types of LLM Agent Architectures

Simple Agents

1. Single LLM with basic tool access
2. Direct input-output processing
3. Limited memory and context
4. Example: Basic chatbots with API access

ReAct Agents

1. Reasoning and Acting framework
2. Step-by-step thought process
3. Explicit action planning
4. Self-reflection capabilities

Chain-of-Thought Agents

1. Detailed reasoning steps
2. Complex problem decomposition
3. Transparent decision-making
4. Better error handling

Multi-Agent Systems

1. Multiple LLM agents working together
2. Specialized roles and capabilities
3. Inter-agent communication
4. Collaborative problem-solving

Common Applications

LLM agents are increasingly used for:

1. Personal assistance and task automation
2. Code generation and debugging
3. Data analysis and research
4. Content creation and editing
5. Customer service and support
6. Process automation and workflow management

Best Practices for LLM Agent Design

Clear Instructions

1. Provide explicit system prompts
2. Define constraints and limitations
3. Specify available tools and capabilities
4. Set clear success criteria

Effective Memory Management

1. Implement efficient context tracking
2. Prioritize relevant information
3. Clean up unnecessary data
4. Maintain conversation coherence

Robust Tool Integration

1. Define clear tool interfaces
2. Handle API errors gracefully
3. Validate tool outputs
4. Monitor resource usage

Safety and Control

1. Implement ethical guidelines
2. Add safety checks and filters
3. Monitor agent behavior
4. Maintain user control

Understanding Attention in Large Language Models: A Beginner's Guide

Have you ever wondered how ChatGPT or other AI models can understand and respond to your messages so well? The secret lies in a mechanism called ATTENTION - a crucial component that helps these models understand relationships between words and generate meaningful responses. Let's break it down in simple terms!

What is Attention?

Imagine you're reading a long sentence: "The cat sat on the mat because it was comfortable." When you read "it," your brain naturally connects back to either "the cat" or "the mat" to understand what "it" refers to. This is exactly what attention does in AI models - it helps the model figure out which words are related to each other.

How Does Attention Work?

The attention mechanism works like a spotlight that can focus on different words when processing each word in a sentence. Here's a simple breakdown:

1. For each word, the model calculates how important every other word is in relation to it.
2. It then uses these importance scores to create a weighted combination of all words.
3. This helps the model understand context and relationships between words.

Let's visualize this with an example:

In this diagram, the word "it" is paying attention to all other words in the sentence. The thickness of the arrows could represent the attention weights. The model would likely assign higher attention weights to "cat" and "mat" to determine which one "it" refers to.

Multi-Head Attention: Looking at Things from Different Angles

In modern language models, we don't just use one attention mechanism - we use several in parallel! This is called Multi-Head Attention. Each "head" can focus on different types of relationships between words.

Let's consider the sentence: The chef who won the competition prepared a delicious meal.

• Head 1 could focus on subject-verb relationships (chef - prepared)
• Head 2 might attend to adjective-noun pairs (delicious - meal)
• Head 3 could look at broader context (competition - meal)

Here's a diagram:

This multi-headed approach helps the model understand text from different perspectives, just like how we humans might read a sentence multiple times to understand different aspects of its meaning.

Why Attention Matters

Attention mechanisms have revolutionized natural language processing because they:

1. Handle long-range dependencies better than previous methods.
2. Can process input sequences in parallel.
3. Create interpretable connections between words.
4. Allow models to focus on relevant information while ignoring irrelevant parts.

Recent Developments and Research

The field of LLMs is rapidly evolving, with new techniques and insights emerging regularly. Here are a few areas of active research:

Contextual Hallucinations

Large language models (LLMs) can sometimes hallucinate details and respond with unsubstantiated answers that are inaccurate with respect to the input context.

The Lookback Lens technique analyzes attention patterns to detect when a model might be generating information not present in the input context.

Extending Context Window

Researchers are working on extending the context window sizes of LLMs, allowing them to process longer text sequences.

Conclusion

While the math behind attention mechanisms can be complex, the core idea is simple: help the model focus on the most relevant parts of the input when processing each word. This allows language models to understand the context and relationships between words better, leading to more accurate and coherent responses.

Remember, this is just a high-level overview - there's much more to learn about attention mechanisms! Hopefully, this will give you a good foundation for understanding how modern AI models process and understand text.