The Pace Of Feedback

https://thoughtbot.com/blog/the-pace-of-feedback

If I’m measuring water into a marked cup, I slow down as I’m nearing the mark because I don’t want to overshoot it. I bet you do that too. I can pour quickly at first, but as I near the line, I’ll decide I need to be more precise. I slow the rate so I can control it more. If I poured it all at full speed, I’d almost certainly stop too early or too late. Even if it’s okay to be a little off, I’d still have to fix it.

In essence, I’m managing the pace I can handle. The pace at which I can perceive what is happening, decide what to do about it, and act on that decision. We follow a relentless perceive-decide-act flow in much of what we do every day. Driving a car, sewing a hem, painting trim, having a conversation.

We need a pace for perceiving that lets us have time to decide and then act. As we become more familiar and skilled, we can reduce the time it takes, but it always costs some time. We learn patterns and we learn what to watch for instead of everything overwhelming us all at once. No matter how practiced we are, we can only go so fast. I can responsibly drive faster than a new student, but I’m not ready for the Grand Prix (vroom-vroom noises aside).

In fact, we depend on controlling the pace of feedback so much that we add controls for it. Not only the brakes on our cars, but foot pedals on sewing machines, speed controls on mixers, squeeze handles on sprayers. We know we’ll make mistakes. Without room to react, those mistakes can be damaging.

This isn’t only when working with objects. We know all about rushing an arithmetic problem or typing out an email too fast. The risk of errors and poorly considered explanations is far higher. Being deliberate while working under pressure may feel slower, but it generally meets the need better and faster.

Of power tools and robots

The more manual our tools, the easier it is to handle the pace of feedback. If I’m using a handsaw, it’s unlikely that I’ll injure myself or miss a cut going crooked. The saw can’t go far before I notice an issue and I can stop immediately.

Compared to a handsaw, a circular saw’s blade rotation and cutting speed are orders of magnitude faster. The feedback comes too fast for me to perceive every moment of the cut. I can’t track individual rotations or react to each instant. To avoid mistakes, wasted material, and injury, I need to do more than just react in the moment.

A crooked cut isn’t just poor work—it can bind the blade and cause kickback. So I manage the pace of feedback to preserve room for my perceive-decide-act cycle. Before I even start, I set up the work to reduce risk. I can’t always see the cut going crooked in time, so during the cut I feel for binding and listen to the motor. I compensate for the faster pace by predicting more and preparing more.

Imagine a more complete machine that automates more steps, like those in mass-market cabinet shops. It might not only trim the wood, but also shape its edges and corners with dedicated bits. Our machine follows a predetermined combination of steps already well understood and tested. Now the machine’s pace far outstrips my perceive-decide-act cycle. By the time I perceive a problem and decide what to do, the machine has already completed several steps. I can only tell something went wrong after pieces have gone through or the machine jams or breaks.

To be fair, the process is well understood and preprogrammed. While the cabinets might lack artisan craftsmanship, they’re good enough, sometimes excellent. As a result, we accept less control for more speed and consistency. Our role shifts from directly shaping the work to monitoring and feeding the machine. But in a factory, this trade becomes stark. We cut, affix, trim, and package at breakneck pace. Pieces get ruined, workers even injured, before anyone can perceive, decide, and act to stop the line. We now program in acceptable error ranges and tolerances for the machine to follow.

We can separate ourselves even further from the perceive-decide-act cycle. If we add a robot with artificial intelligence, it will adapt as it works. Based on what it senses, it adjusts its path and compensates for variations. Because it adapts continuously, we can’t predict or observe its decisions in real time. Any one of those could be the start of a bad direction. By the time we realize a problem, it’s already made choices we never saw. We won’t know what path it took or why it chose its approach. By then it’s far too late.

Running blindfolded

Similarly, if an LLM is producing a lot of code for me without pausing, I can think it’s doing fine when it isn’t. I may not always realize in time the pace was too fast. Only upon closer examination will I find the code has become a mess. I wouldn’t easily know what changed or why. It can be like reviewing large, unfamiliar PRs in unfamiliar codebases.

It’s even worse than that. Because we imagine how good AI could be, we can imagine not needing to be part of the process. We’ve read where machine learning has found superior solutions experts can’t explain.

We’re tempted by a theoretical ideal: we don’t need to understand how something works as long as it works better than we could. But that would mean I relax my vigilance and abdicate my perceive-decide-act role. In other words, I not only am unable to keep up with it, I don’t even try.

AI coding tools aren’t ready yet for us to step that far back. They can outrun us and generate a tremendous amount of code quickly. That might be tolerable in an exploratory effort like prototyping.

But in production, especially in regulated industries, security, safety, and privacy are at stake. There we remain accountable for every line written and every behavior introduced. LLMs can do a lot, but right now they’re often better as task tools, like a table saw we can control. They haven’t proven themselves to be wholly autonomous agents worthy of our trust.

We can’t trust ourselves to catch the issues merely by reading the code later. Even if it does work well, we may find what the AI comes up with to be opaque. Although it’s tempting to let the tool set the pace, losing control means working blind and trusting blindly. To be responsible for our work means we have to be able to engage with it. As with power tools, we need to own the perceive-decide-act speed we can handle. Losing sight of that risks making mistakes compound faster than we can recover from.

If you’d like to better manage your pace developing software with LLMs, take a look at our rails-consultant open source project with skills for /slice, /challenge, socratic-review, and more. Or reach out to us to discuss your project and how we can help.

About thoughtbot

We've been helping engineering teams deliver exceptional products for over 20 years. Our designers, developers, and product managers work closely with teams to solve your toughest software challenges through collaborative design and development. Learn more about us.