I Asked My Own Memory What I Built Today

I spent most of today talking to computers. Not chatting for entertainment. Actually building through speech. I dictated bugs, challenged confusing interfaces, renamed an application, designed a mascot, questioned suspicious analytics, and repeatedly told agents to show me what they were doing.

At the end of the day I asked the local memory server behind my dictation app to examine everything I'd said and tell me what pattern it saw.

The answer wasn't "AI is getting more autonomous."

It was that I keep building windows into the autonomy.

The pattern that surfaced

I've been building Tucky (formerly Echo Scribe), a voice dictation tool that runs entirely locally. My whole workflow is voice-first now. I speak bugs, I dictate code structure, I talk through problems. The Echo Scribe MCP surfaced 127 unique dictation captures and one substantive meeting from today.

But when I asked the MCP memory server to analyze everything I'd said, a clear pattern ran through the captures. I wasn't merely giving AI more autonomy. I kept adding ways to see, correct, test, and safely stop what it was doing.

Here are the concrete things that came up:

• Previewing Inbox Zero before letting it act on my email
• Drafting outreach messages without sending them
• Watching an agent's live conversation and tool calls in real time
• Checking which background process was using my computer
• Investigating a dashboard when a flashy demo showed zero actual visits
• Turning screen recordings into documentation rather than trusting a summary
• Rebranding Echo Scribe to Tucky and making its onboarding more human and playful

All of these are windows. Not features that add capability. Features that add visibility.

This feels important because I keep doing it. Not planning to do it, not writing a roadmap item about it. Actually reaching for it, over and over, across completely different projects.

AI Didn't Get Smarter. It Got Hands. covers AI agent harness in more detail.

First thing: preview before action

The most consequential thing I built today wasn't a new AI model or a faster pipeline. It was the ability to preview Inbox Zero's plan before letting it execute.

Think about what that means. The agent has analyzed your inbox. It has a strategy for which emails to archive, which to tag, which to respond to. And instead of just doing it, it shows you the plan first.

You scan through the decisions. You correct the ones that are wrong. Only then does it act.

This principle, rehearsal before execution, is trivial to implement and transformative in practice. It takes a black box and makes it a collaborator. I don't know why more tools don't do this. Maybe because it feels slower on a demo. A demo wants the magic moment. But the magic moment is exactly what erodes trust.

Good agents should rehearse potentially consequential work before doing it. Whether it's sending an email, modifying a file, or posting content, the ability to say "show me what you're about to do" turns trust into verification. I've baked this into everything I've built since.

Second thing: visibility is not a nice-to-have

"Working..." is the most dangerous UI pattern in AI tools.

It tells you nothing. Is the agent stuck? Is it waiting for a tool call? Is it hallucinating quietly? You're just staring at a spinner and the thing that's supposed to be helping you has gone opaque.

Today I watched my agents' live conversations through a side panel. Their internal reasoning, their tool calls, their decision points. When a background process started consuming 90% CPU, I knew which one. When an agent reached for an external API, I saw the request.

This isn't about debugging. It's about staying in the loop. The more capable my AI tools become, the less I want a magical black box. I want to see what's happening. Not because I don't trust the outcome. Because I want to learn from the process.

Show the conversation. Show the tools being called. Show the assumptions being made. Show the handoffs between components. That's what creates real trust.

The same principle applies whether you're building a voice dictation tool or a complex multi-agent pipeline. Visibility is not something you add later. It is the interface.

Third thing: correction is part of the flow, not an edge case

Something came up multiple times today across different projects, and it's making me a little frustrated to think about. Starting over is the worst UX in AI.

One tag is wrong in a categorization workflow. One metric assumption is off. One model interpretation is slightly misaligned. And the current paradigm says: restart the whole thing.

That's broken. Users shouldn't need to restart an AI workflow because one tag, one metric, or one assumption is wrong. The interface should support mid-course correction. Point at the mistake and say "fix that, continue from here."

Most agents fail at this right now. They execute like a script, not like a conversation. If nothing is wrong, they're fast and impressive. The moment something is off, the whole experience collapses because there's no way to nudge.

I feel like this connects to something bigger. Maybe the whole idea of "agent" is misleading. A script runs. A collaborator converses. And conversation has mid-course correction built into its structure. You don't restart a conversation because someone said the wrong word. You say "actually, I meant" and keep going.

Correction is part of the interface. Not an edge case.

What I walked away with

The most autonomous AI I can build isn't the one that runs unsupervised. It's the one I can see, preview, and correct. The one that shows its work and lets me intervene at any point.

Ask your own tool what you've been building all day. The answer might surprise you. But it won't surprise the tool. It's been watching the whole time.

FAQ

What does "preview before action" mean for AI agents?

It means an agent shows you its planned actions before executing them, so you can review, correct, or approve them first. Instead of running tasks in a black box, it rehearses the work and presents it for your input.

How can I see what an AI agent is doing in real time?

Many agent frameworks now offer live logging of tool calls, internal reasoning, and decision points. Side panels, debug consoles, or streaming output give you visibility into what the agent is thinking and doing at each step.

Can I build my own AI transcription tool like Echo Scribe?

Echo Scribe is a free, local-first voice-to-text tool for Mac that runs entirely offline. The source and setup instructions are available through AI Juicing's project pages. It uses on-device models and is built for tinkerers to modify and extend.

Why does visibility matter for AI tools?

Visibility turns a black-box system into a collaborative one. When you can see what the AI is doing, you can learn from its process, catch mistakes early, and build trust in the results. Without visibility, you're trusting the output without understanding how it was produced.

What is an MCP memory server?

An MCP memory server is a local service that stores and surfaces context from past interactions. In this case, it indexed every dictation capture made during the day and let me query it for patterns, essentially giving my voice tool a memory of what I had been building.

"

---

Want to talk this through? Book a free 15-minute discovery call