mattwood.blog

The Half-Life of an Assumption

A nautical chart is as authoritative as an official document gets: surveyed, compiled, checked, and issued under a national hydrographic office. It is also out of date almost as soon as it is printed. A buoy drags out of position. A wreck settles onto a shoal. A dredged channel silts in. The sea starts disagreeing with the chart almost immediately, and navigation, as a discipline, is built around that fact.

The British Admiralty began issuing Notices to Mariners in 1834, at first sent individually to the ships and squadrons that needed them; by 1890 they had become a regular weekly publication, and hydrographic offices around the world now issue their own. Navigators marked each change onto the paper chart by hand and logged the notice number in the bottom-left corner of the sheet. A chart's reliability was understood to depend on two things: the quality of the original survey, and whether its corrections were up to date. Small changes were penned in. When enough of them accumulated, or when a change was too large for a pen, the office issued a new edition.

Nothing in the original chart had to be wrong. The surveyors did their work; the cartographers drew what was true. The world changed after publication. Being authoritative and being current are different properties, and the notice system exists because the difference can put a ship on the rocks.

Organizations produce decisions with the care of charts and preserve them like monuments. The analysis is researched, reviewed, approved, and embedded in roadmaps, budgets, and controls. What changes afterward is rarely attached to it. Somewhere in your organization there is a document that says AI cannot do something it can now do, and no notice has ever been issued against it.

That is not carelessness. Organizations could not function if every decision remained permanently open; converting judgment into structure is how anything gets built. The practice fails only when the world changes faster than the structure that recorded it.

Every consequential decision has a half-life. Not a number anyone can calculate, but a decay rate all the same: the conditions that made it reliable erode, and some erode far faster than others. Some of an organization's charts stay true for years. Others now go stale between planning cycles. AI is increasing the rate at which the territory changes, and the problem is not that organizations make bad maps. It is that they have no Notice to Mariners for their assumptions.

For the past few years, AI improved quickly but at a relatively regular cadence, with each model generation raising familiar benchmarks and different providers trading the lead. Over the past few months, gains in coding, reasoning, computer use, tool use, and long-running work have arrived together and reinforced one another, moving models into work that was previously beyond their reach. The rate itself is rising, and this looks like just the start.

The organizational impact does not rise smoothly with capability. A modest improvement can move an entire workload from impractical to routine. A process that once needed constant attention may now run with a few checkpoints; automation that cost more than it saved may suddenly pay for itself; a product experience that could not be built may become the natural place to begin.

Any one of these shifts is manageable. The difficulty comes when several land at once across engineering, operations, customer service, and product development. Organizations that discover each change only when it interrupts an existing plan will experience the period as a succession of surprises. Those that expect their assumptions to expire will still need to adapt, but they will have made room for adaptation before knowing exactly what would change.

These threshold crossings invalidate decisions that were entirely reasonable when they were made. The human review step remains after the failure mode it was designed to catch has faded. The model selected for its reasoning stays in place after a cheaper one catches up. The workflow rejected on economic grounds is not reconsidered after its cost falls by an order of magnitude.

None of these decisions became foolish. Their useful life just ended.

Why sound decisions outlive their evidence

An assumption that customers will continue to care about reliability, price, and ease of use can hold for years, while an assumption about what the technology can do is now among the fastest-decaying beliefs an organization holds: a view about which model is best for a workload can go stale in weeks, and the right agent configuration in days.

Most organizations, though, record the answer without recording its shelf life. The more work that went into a decision, the more authority it acquires. Then the artifact built from it begins to reinforce it: a system with years of investment looks appropriate because it exists, a large team appears to prove that the problem requires a large team, a detailed roadmap makes the destination look understood. The artifact becomes evidence for the decision that produced it.

People feel this personally. A leader sponsored the roadmap. An architect defended the design. A team spent months making it work. Reopening the premise can feel like diminishing all of that, and committing fully to a decision without making loyalty to it part of your identity is easy to endorse in principle and hard to practice after budgets and reputations have accumulated.

The instinct is to forecast better. It helps a little, but capabilities emerge unevenly, and several modest improvements can combine to cross a threshold that no single benchmark captures. Prediction cannot carry the weight that planning wants it to.

A notice system for assumptions

The practical response is the mariners': make correction a normal part of operating the system, not an exception to it.

That starts with recording why a decision was made, not only what was decided. A workflow excluded from automation was rarely judged inherently unsuitable; it was excluded because computer use failed too often, supervision erased the economic benefit, or the remaining errors carried too much risk. Those are testable conditions, and the decision can carry its own triggers for reconsideration. When computer use materially improves, or the cost falls below the threshold in the original business case, the evaluation runs again.

Keeping the original test cases makes that retest cheap, and AI can stand the watch itself: agents can track new models against the old evaluation, attempt the same requests and the same exceptions, and return the question to its owner with evidence attached rather than a general stream of AI news. The organization never has to debate computer use in the abstract. It learns whether the result on its own work has crossed the threshold that made the original decision fail.

The point is not to keep every decision open; permanent uncertainty would make coordinated action impossible. The point is to know which decisions deserve to be reopened, and when. The cadence should follow the decay rate: model selection becomes continuous rather than ceremonial, workflow design moves more slowly on operational data, customer needs are read through sustained contact with customers, and purpose barely moves at all.

Corrections and new editions

Not every expired assumption calls for the same response. Some are handwritten corrections: a review point moves, a model assignment changes, a threshold gets updated. Others call for a new edition, whether because small corrections have accumulated until the structure no longer fits the territory, or because a single change is large enough to invalidate the design on its own.

Starting over was historically the expensive part. Architectures represented years of engineering, roadmaps carried commitments across teams, governance encoded hard-won agreements, and even when the evidence called for a rebuild, patching could be the least bad economic choice. AI is changing that calculation. Prototyping a replacement, testing an alternative architecture, migrating a system: each is becoming cheap enough to change the choice between continued patching and replacement, and the same capability jump that invalidates an implementation also reduces the cost of replacing it. A team revisiting an expired exclusion can sketch several versions of a workflow, from fully automated to human-approved at the riskiest step, and build enough of each to learn which deserves the commitment.

None of this makes starting over free, and it should never mean starting ignorant. Production systems still carry customer commitments, data, security requirements, and accumulated complexity. But if what the organization learned lives in reusable test cases, decision records, and observed customer outcomes rather than only inside the artifact, a replacement begins with everything the previous attempt discovered. Knowledge that exists only inside the system dies with it, which is one more reason the artifact is hard to release.

This is where stability belongs. Purpose, customer promises, accountability, security boundaries, and the evidence required to trust a result should be durable. Models, configurations, workflow designs, and implementation plans can be held more lightly, and a change of route should not be read as a retreat from the objective or a repudiation of the people who pursued it. The quality of their work may be the reason the organization knows enough to choose differently now.

AI is reducing the useful life of many assumptions while lowering the cost of testing and replacing them. Organizations now have better tools for noticing change, determining whether it matters, and rebuilding around what they learn. The outcome will depend on whether a decision that was once correct is allowed to become a question again.