Dev Log: September 05 Wrap-up

Overview

Today was mostly about closing a blind spot in how we track our AI agent's behavior. It’s one of those things where everything looks fine on the surface, but the moment something breaks, you realize you're missing the most important piece of evidence.

Tracking what actually happens

I spent a good chunk of the day digging into our tool-calling logic. Right now, when our agent decides to use a tool, we log what the LLM asks it to do (the arguments). That’s great, but it’s not the full story. Often, our tools take those arguments and merge them with internal configs or transform them before actually hitting an external API.

If the final API call fails, just knowing what the LLM suggested isn't enough to debug it. I needed to see the actual, final payload that left our system.

I implemented a way to capture the 'resolved' request. The trickiest part was making sure I wasn't logging stale data. If a tool call fails halfway through, I didn't want the logs to accidentally show the payload from the previous successful call just because it was still sitting in the buffer. I added a quick step to clear out the state before each execution starts.

It's a small change, but it makes the difference between "I think I know why this failed" and "I can see exactly why this failed."

Here’s a simplified version of how I handled the capture logic inside the agent node:

# Make sure we're starting with a clean slate
final_request_sink = tool_metadata.get("resolved_payload_buffer")
if isinstance(final_request_sink, dict):
    final_request_sink.clear()

try:
    # Execute the actual tool logic
    tool_output = await execute_tool(t_name, t_args)
except Exception as e:
    logger.error(f"Tool {t_name} failed: {e}")
    raise e

# Map the final state back to our logs
log_entry = {
    "tool_name": t_name,
    "input_args": t_args,
    "actual_request_sent": dict(final_request_sink) if final_request_sink else None,
    "response": tool_output
}

Debugging these agent workflows is starting to feel a lot more manageable now that the logs actually reflect reality. No more guessing if a middleware layer accidentally stripped a header or mangled a JSON body.

Wrapping Up

It feels good to get this into the repo. It’s not a flashy feature, but future-me will definitely be thankful the next time a tool starts acting up in production. Tomorrow I might look into some more granular timing logs for these calls, as some of the external services are feeling a bit sluggish lately. Catch you then.