The Day Job · The Durability Curve

You Built a Number You Will Not Trust

Fri, 28 Aug 2026 00:00:00 GMT

There is probably something you are part of the way through building. A spreadsheet that will score your sales leads so you know which to ring first. A scorecard that will rank the people you interviewed. A formula that will tell you which product line to drop. You have the columns and the weights, and it more or less works. And some part of you already knows you will not trust the number it gives you. When it puts the wrong lead at the top, you will override it, the way you did last time.

That thing was built well and it is not going to help you, and the reason is not that you got the weights wrong. You could tune them all weekend. The reason is that you never said what would make its answer right. You built a machine to rank the leads before you had settled what a good ranking even was, so when its number surprises you, you have no honest reason to trust the surprise over your own read, and you override it. Working that out before you start is worth more than any amount of tuning, because it changes what you build.

What “it works” does not tell you

When a thing you made runs, you feel finished. The columns add up, the report generates, the score comes out. But working only tells you the thing does what you built it to do. It says nothing about the two questions that can kill the thing before you waste a weekend on it: whether you could ever tell that its answer was wrong, and whether it beats what you would have done without it. You can ask both before you write a line, and most of the cost of a wasted build is the cost of not asking.

The one people skip: what would tell you it was wrong

Start with whether you could ever tell the answer was wrong, because that is the question that gets skipped, and it is easy to hear it as a question about whether the tool runs. Those are not the same: a tool can run perfectly and still hand you an answer you could never catch out. A total, a count, a due date, an amount owed: each has an obvious check, because there is an answer outside the tool to compare it with. Those you can build and know you built them well.

Past those, it comes down to one thing that is easy to miss: whether the outcome that grades the answer arrives on its own, or whether acting on the answer is what decides which outcome you ever see. A forecast can be the first kind, as long as the thing you are predicting arrives regardless of what you do with the prediction. Predict how many people will show up or how many inbound calls will arrive, and the real number turns up whatever you guessed. Footfall, call volume, next week’s demand: the world returns a verdict on these whether you like it or not, which makes them possible to test and improve even when the answers are hard. You always get to find out.

The lead score looks like one of those and is not. The outcome you get to see is the one the score chose for you. You ring the leads at the top, some of them buy, and the ranking looks confirmed, but the leads it sent to the bottom you never ring, so they never get the chance to prove it wrong. It can bury half your real buyers and still show you a good week, because the ones it buried are the ones you will never check. The number decides which evidence you ever see, and the mistakes that matter are the ones it never shows you. That is the dangerous kind of number: the one whose very use hides its own mistakes, so the more you lean on it the less you can see them.

A tool like that leaves you two choices, and neither is simply to trust the score. You can keep the decision for yourself and build the thing that lays the evidence out, the renewal date, the value, the last time you spoke, so the call takes seconds and the decision stays where it can actually be made. Or, if you want the score, you keep a corner of the world honest: at random, ring some of the leads it tells you to skip, so it cannot set its own exam and then pass it. What you cannot do is act on it everywhere and call the result proof.

So the question that matters is not whether the answer is a fact or a judgement. Plenty of recommendations can be tested against what actually happens, over enough cases. The question is whether you would ever find out this one was wrong, for the options it turns down as much as the ones it takes. When nothing would, you have automated the decision without earning any reason to trust it.

I have watched myself skip this question, so I know why it gets skipped. Asking is quick and uncomfortable, because the honest answer is sometimes that nothing would ever tell you, and that ends the project in the first ten minutes. Building is slow and feels like progress, and every hour in makes the thing harder to walk away from. So the quick question loses to the comfortable work, and you find out at the end.

What it cost me to learn this

I learned this by paying for it twice in one evening. I had built a small tool that read a page of written instructions, the kind you might write to hand a job over to someone, and sorted each line into two piles: the lines that told you to do something, and the lines that were only describing the setup. It ran. Then I tested it against a page it had never seen, marking the lines myself first, and on the ones it was willing to call, it agreed with me a little over half the time. The tempting lesson is that the tool was bad. The real lesson was worse. I had never pinned down what the right pile was, and I could have: written down the rule for what counts as an instruction, had someone else label the same page, checked whether we agreed. That would have given me something to hold the tool to. I never did it, so the tool’s score could tell me how often it matched me, but not whether it had learned a rule worth trusting.

The target was buildable. I had just built everything except the target.

So I rebuilt it as the other kind of tool, one that only counts, showing each section’s share of the page and letting me tick what to cut. That has an obvious check, and I built it properly, with a list of thirty-four ways it could go wrong, twenty-seven test cases worked by hand, seven real pages it passed cleanly. It was correct. It was also useless, because the thing it replaced was opening the file and reading it, and it beat that by nothing. The first question had sent me to a tool that could at least be right. The second question killed it anyway. That second failure is the easier one to miss, because a tool that is correct feels like a tool that is worth having. It can be measurable, tested, and right, and still lose to the thing you would have done without it. What you would do instead belongs on the page next to the answer you are after.

Later I wrote out what could have sunk each build. The thing that actually sank the first one was not among the bugs I had spent the night fixing. It had been sitting above them from the start: I never said what a right answer would be.

Run it on the thing you are building

So take the thing you are building now. Before you add another column, write down the answer it is meant to give, and put it to the two questions.

First, if that answer came out wrong, would anything ever tell you? If the outcome arrives independently of what you do with the answer, build it and keep score. If acting on the answer is what picks the evidence you see, randomly test some of the choices it would otherwise turn down, or leave the decision to yourself and build the tool that lays the evidence out.

Second, say in one plain sentence what you would do instead if this did not exist, and why what you are building beats it. If that sentence will not come, you have your answer, and it is cheaper now than after the weekend.

The tell that you skipped both: you are deep in the build and neither answer is written down anywhere.

What the questions do not do

Two honest limits. The first is that even a target you can check can be the wrong target. Rank the leads by who converts and the tool may learn to love the small easy accounts and starve the big awkward ones, and it will be hitting the number you set the entire time. You can define success exactly and still have defined the wrong success. These two questions do not protect you from that. What they buy you is the right to find out you were wrong, not a promise you aimed at the right thing.

The second is that they do not make you wise in advance. Both times, I killed the tool after I had built it, not before. Asking first lowers how often you pay, but it will not take the count to zero, because now and then the only way to learn what a thing is worth is to build it and look. The questions are there to catch the builds you start because building feels easier than asking, not to promise that you will never lose another evening.

There is one last thing the questions never reach, and it was always going to be yours. When you have a target you can genuinely test, software can hand you a better answer than you would reach alone, and you should let it. What it cannot do is settle what counts as a good answer, settle how much evidence is enough, or answer for the choice to act on it. A machine can be right. Being the one who is answerable for trusting it is the part that stays with you.

The Work That Comes Due After You Leave

Tue, 25 Aug 2026 00:00:00 GMT

You finish something. A project wraps, a client signs off, a piece of work goes out for the last time. Then there is the tail, the small handful of things you do afterwards, none of which take any real time. Mark it done. Tell them it is finished. Cancel the paid seat you bought for it. Switch off the weekly update that goes out to them every Monday.

Four steps, four different places: the tracker, your email, wherever the card gets charged, whatever tool sends that update. Later you check one of them, probably the tracker, because that is where you look to see whether things are finished. It tells you the job is done, and it is telling the truth about the only step it can see.

Switching off the update is the one that did not happen. Months later it is still arriving, every Monday at nine, to someone who stopped being your client a long time ago. Nothing is wrong with the system that sends it. It is doing exactly what it was told, on time. From where you are standing, the failure looks exactly like everything working, and that is the whole of the problem.

The tracker is not lying. A job that touches four systems has four different ways of still being open, and the tracker sees only the one it holds. Done was never one state; a single word just made it look like one.

Why the small steps are the ones that go missing

The tempting explanation is that you were busy, or careless, or need a better checklist. I want to offer a more specific one, because it tells you which steps will go wrong instead of telling you to try harder.

A checklist is a list of the steps you thought of. It is good at holding you to those. What it cannot do is mention a step that never went on it, and the steps that never go on it are not random. They are the ones that cross into a system you do not quite think of as part of the job. You wrote the list around the place you do the work, and the step that lives somewhere else did not occur to you, for the same reason it will not later occur to you to check whether it happened.

Making a second list does not save you, and that is the part worth sitting with. If you build the second list from the same picture of the job, the same step is missing from it too, and now you have two records that agree with each other and are both wrong. That is not a hypothetical: your tracker is that second list. You filled it from the same picture of the job, so it agreed the work was done and was wrong in the same place you were.

And a missed closing step does not stay missed quietly. An ordinary task you skip just sits there until you come back to it; a closing step you skip stays open until something closes it, and until then it keeps acting, every day or every month, on its own.

The check has to come from somewhere you did not write

So the thing that catches the missing step cannot be your own account of the work. It has to be a record that something else kept, for its own reasons, whether or not you remembered the step.

You already have several of these. You just do not read them against the job. The card statement is one: the bank records the charge whether or not you remember the seat you meant to cancel, so the seat that is still billing turns up as a line you cannot attach to any live piece of work. The access list is another: the system logs who can get in whether or not anyone told it that a person left, so the account that outlived the project is a login with no current owner. What actually shipped is recorded by the thing that shipped it, so a promise you made and never delivered stands as a commitment on one side with no send on the other.

Even with a checklist I take seriously, I did this. I keep a written routine for finishing an essay, detailed, with a warning next to the item that slips most, and my archive quietly slipped twenty-three pieces behind what I had published since late spring. Copying each finished piece across to that archive had never been a line on the routine at all: it lived on a different system, so it never occurred to me to write it down. What caught it was the published record of what had actually gone out, kept by the platform and owing nothing to my memory. Held against the archive, it showed the twenty-three at once.

The move itself is old. Accountants have reconciled two sets of books this way for centuries, and there is nothing here to invent. What is easy to get wrong is what makes the second record worth anything: not that it is a second record, but that something other than your own memory produced it. Two dashboards drawn from the same database, or two lists built from the same picture of the job, only look like a check, because the same forgetting shaped both. A record can catch you only when your forgetting could not have reached it too.

One question to carry

That gives you a single question, and it is worth more than any checklist. Of anything you lean on to tell you a job is finished, ask: would this still be here, and still say the same thing, if I had forgotten the step entirely?

The tracker fails that question: you fill it in yourself, so a step you forget is a step you also forget to log, and it stays green over a gap it never knew about. The card statement passes, because the charge is there whether or not you remembered the seat. A check built from your own memory cannot expose the step that memory left out.

The question keeps its shape as the instrument gets bigger. A tracker, a dashboard, a report you write on your own project: each is an instrument you fill from your own picture of the work, and each is blind in the same place you are. The statement is worth more than any account you write of what you meant to do, for the same reason an audit leans hardest on evidence the audited side did not get to shape.

The part you can use

Here is the version you can run on your own job this week.

Write out the routine you run after you finish something, every step, including the ones that feel too small to be worth writing down. Mark each with the system it touches: the tracker, the calendar, the billing account, the shared drive, the tool somebody set up before you arrived. This is not the check yet. It is how you find which records are worth reading against each other, and it usually turns what felt like one job into the three or four systems it was always made of.

Then there are two ways to keep a step from being lost, and the first is much stronger. Where you can, do not rely on catching the step at all; arrange things so that forgetting it does no harm. Anything that runs on its own, a payment, a subscription, a recurring invite, an access granted for a single project, gets its end date on the day you set it up, while you still know what it was for. Something that expires unless it is renewed cannot outlast your forgetting, because forgetting it and ending it become the same act. Reach for this first; it removes the obligation instead of watching it. Its limit is the one this piece began with: you can only set an end date on a step you thought of, and the step that never made the list cannot be made self-closing.

For everything you could not foresee, or cannot make expire, there is the slower move: read your own record against one you did not produce. Your active-projects list against the vendor or card statement, looking for a charge attached to work that has already finished. Your list of who is on the team against the access export from whatever holds the accounts, looking for a login with no owner. The commitments in a signed contract against what your team actually sent, looking for a promise with no matching send. Choose the second record by the causal test, not by where it happens to be stored: pick the one your own memory did not shape.

How often you look depends on how much damage you will let build up first. A ten-pound seat can wait a month; a former colleague who can still open every file cannot, and something confidential still reaching the wrong person is not a scheduled job at all. None of it needs a tool you have to build: a read-only export or a screenshot is enough, and where you cannot pull the record yourself, the person who can is an email away, not a project. And reading across only points to a mismatch; you still have to look and decide whether it is a real miss, a timing lag or a duplicate, and keep that verdict for yourself.

What none of this fixes

Two things survive all of it, and I would rather say so than leave the tidy version standing.

The first is the record that was never kept. If something gets finished and lands in no system at all, no charge, no log, no row anywhere, then there is no second record to read it against. You cannot check against a record that does not exist. That case surfaces only when a person happens to notice, or is told.

The second is quieter, and more common. If the same blind spot sits in both records, they agree, and the agreement looks like an all-clear. This is the failure I walked into the first time I tried to build a check like this for myself. I searched my files for links to the publication. That sounds like reading an independent record, until you notice it read the same surface I would have: it counted the times I had linked to old pieces inside new ones as though that proved the old ones had shipped. Independence is the whole of the mechanism, and when it is missing it fails without a sound.

So the honest tally is smaller than the tidy one. The obligations that leave a trace in a record I did not write, I can now catch, once in a while, in half an hour. The ones that touch nothing outside my own attention, I am still carrying in my head, and I have learned how little the word covers when I say nothing is wrong. Nothing is wrong and nothing I can see is wrong are different sentences, and most of the time only one of them is available to any of us.