Yesterday on November 4th at 3:40 pm I was clicking around our dev environment and typed a small compound sentence explaining that the user account menu belonged in the top right of my web app with the ability to create organization accounts. My factory agents built this feature overnight. It was in production 12 hours later as release 0.59.1.
This is what my factory does and is: a queue, agents that follow a defined process to build, integrate, verify, ship and me.
Two months ago I wrote building a software factory ain’t hard. My view hasn’t changed. explaining what running one is like is the purpose of this meditation.
The one I describe below builds RunStory. All the numbers come from my own logs which began in late August. My attention and tokens is what my factory runs on.
Learn to ask of all actions, “Why are they doing that?” Starting with your own. — X. 37
What I actually do all day
Mostly I type short prompts.
Over the past month I sent 863 prompts to my main Claude Code orchestrator session that runs the factory. About 24 a day at just ~60 characters in length.
The main session turns my intent into work I don’t see happening for the most part. It writes a goal with rules and decisions, launches a workflow (a script that runs a fixed sequence of agents) and reports back. Since I began I’ve typed 1,980 prompts.
Each open decision for a particular phase of work is read back to me with a default predetermined, so I answer only the ones I’d change. I made a median of 7 decisions a day, and 26 on the most active day.
I don’t watch the factory but I do read what it reports.
The SDLC then and now
The SDLC we are used to had a person at almost step and they needed to be actively involved in nearly all the work. Regarding code review of yore: SmartBear advised reviewing 200 to 400 lines at a time, for no more than 60 minutes. Google’s guide gives a reviewer 1 business day to answer. We know that review did more than catch bugs. It spread knowledge of the code around the team.
What the SDLC used to look like
Here are the same four stages in my factory. Notice how many fewer folks are required. And where, specifically they’re incorporated.
What the SDLC looks like w/ my factory
I ask, I say go, and I try what comes out. I don’t write code anymore or read it, either.
What comes out of this is substantial. The median change the factory landed from over the last 30 days modified 4,578 lines of code across 47 files, counting tests and leaving out docs. Only 9 changes were 400 lines or fewer. My factory deployed to production 72 times in 31 days.
What this buys you
The upside isn’t just speed. Its that trying ideas costs very little.
RunStory is testing infrastructure that runs your software at a particular commit to find bugs. For a while it had a third-party classifier in place picked “lenses” for each check run. These lenses would change the nature of the type of prompt that would be used based on if you wanted to guide a particular scenario. It was built on belief but not originally measured. In late October we started to doubt it’s efficacy.
The next day 12 checks ran six open source commits two ways, with the lenses feature and without. My blind judges found that the product performed better without the feature and so we removed it
Research works the same way. At night on November 2nd I asked for a design study of every screen across the web app and cli. About four hours later I had a HTML mockup of 57 proposed changes, and its verifier had already caught and fixed 12 problems in it. I picked 19 to change the next morning, and they shipped within the day.
Where time goes
Each phase of work that flows through my factory follows a standard design shape. On average verifying and fixing the work takes as long as building it. The other night when I was adding the organization accounts to the login menu I also was building functionality to link commits to the coding session that originated the change. Below is a time breakdown of what that phase looked like.
In June I wrote about calibrated distrust. I’ve started calling the factory’s side of it manufactured trust. I mean manufactured the way a part is, on an actual factory line because once you start building this way and understanding how it works, it will start to make sense.
What it costs you
Beyond attention and tokens there are definitely kinks.
In mid September a cleanup loop force removed 83 git worktrees including one that held design work that had never been committed. I acquiesced that “it was ok for now”.
In early October my orchestrator that drives the queue filled its context, its next scheduled wake-up failed and nothing alerted me. The queue sat idle for 12 hours overnight. Wasted time but not a huge deal.
The more significant issues include:
Mental models of an evolving product can be difficult to share.
Mastery in a particular domain, language or implementation is delegated not learned.
It’s a factory of one. I think its much harder to try and do this with multiple people actually deciding how to queue and phase work.
And so what does this all actually feel like? No, not like building the way you were used to.
Its deciding what to believe, all day, at a speed you need to keep pace with.