Dev Log: July 31 Wrap-up

Overview

Today was one of those days where I felt like I was playing both the architect and the bouncer. Most of my energy went into hardening our AI flow against weird prompts and cleaning up some data-fetching logic that was starting to feel a bit sluggish. It’s the kind of work that isn’t always flashy but makes the whole system feel way more robust.

What I Worked On

Setting up the Guardrails

I spent a good chunk of time building out a new GuardRail node. As we scale out these AI agents, you realize pretty quickly that you can't just trust every user input. People will try to break things—whether it's through prompt injection or just asking the bot things that have nothing to do with our specific domain.

I implemented a multi-layered check. It starts with some regex patterns to catch obvious "jailbreak" attempts and then uses an LLM-based semantic check to see if the query actually belongs in the conversation. It now returns specific status codes like 403 for security violations or 400 for off-topic stuff, which makes it much easier for the frontend to handle errors gracefully.

# Generalizing the check logic for prompt injections
def check_safety(user_input):
    patterns = [
        r"ignore previous instructions",
        r"bypass safety filters",
        r"system prompt:"
    ]
    
    for pattern in patterns:
        if re.search(pattern, user_input, re.IGNORECASE):
            return 403 # Security violation
    return 200

Trimming the Fat in Chat Sessions

I noticed something annoying in our chat embed component. Every time a session loaded, we were fetching every single session associated with a flow just to find the active one. It was a classic "why are we doing this?" moment. I refactored it to just target the specific session ID we already have in storage. It's a small change, but the UI feels much snappier now since we aren't waiting on a massive JSON array of old conversations.

Smarter Agent Responses

I also updated the agent execution logic to be a bit more communicative. I wanted a way for the LLM itself to signal its status back to the system. I settled on having the agent emit a specific tag—something like <responseCode>XXX</responseCode>—inside its text output. I added a bit of logic to the backend to sniff these out, extract the code, and then clean the tag out of the final message so the user never sees the "mechanical" parts of the response.

# Extracting custom status codes from LLM output
rc_match = re.search(r'<responseCode>(\d{3})</responseCode>', raw_output)
if rc_match:
    extracted_code = int(rc_match.group(1))
    # Clean up the output for the end user
    clean_text = re.sub(r'\s*<responseCode>\d{3}</responseCode>\s*', '', raw_output).strip()

Making "Get Started" More Flexible

Lastly, I had to rework the "Get Started" module. We needed it to work as a standalone, login-less component that can be easily embedded into other sites (like a WordPress landing page). I moved some routes around and added the chat assistant directly into that flow. It's much cleaner now and serves as a better entry point for new users who aren't fully onboarded yet.

Wrapping Up

I'm pretty happy with the GuardRail progress. Security for LLMs is such a moving target, so having a dedicated node for it feels like the right move. Tomorrow, I’ll probably dive deeper into testing how the LLM handles the semantic classification part of those guardrails. Hopefully, it’s not too over-sensitive.

Catch you later.