Update 18: Same job, different odds

This week I have been thinking about what AI is doing to our product team. The short version: it is quietly becoming an AI product team.

Fifteen years ago I started a design and development agency. Software back then was obedient. It did what we told it to (mostly). Checking our work was simple: does the feature match the spec? Yes or no.

The core job has not changed. We still dig into user needs, define roadmaps, build, and ship. Product management still sits where business, technology, and user experience meet.

What changes is the ground beneath it. Traditional software is deterministic. AI is probabilistic. The product will sometimes be wrong - not through bugs, but by design: it makes educated guesses. We must build the whole user experience around that.

That one shift reshuffles everything else. QA turns into statistical evaluation rather than pass-fail. Data becomes a first-class responsibility: code is increasingly a commodity; our messy, real-world data is the asset. Fewer user stories about button placement; more specifications for labelling pipelines. Success gets two scoreboards: product metrics like user retention, and model metrics like precision and recall. Roadmaps become rolling experiments. And shipping is no longer the finish line: models drift, so we monitor them forever.

There is a new duty, too: asking whether a problem needs AI at all, and whether we have the right data to feed it. Sometimes the honest answer is a boring old rules engine.

Notice the theme? Statistics. Everywhere. A team raised on deterministic software suddenly needs to understand distributions, error analysis, and thresholds. That is a lot of learning.

And because this wouldn't be a post about AI without a twist: we may not have to learn it the hard way. I have started looking at the teach skill, which turns Claude into a patient personal tutor. Early signs are promising. If it survives contact with the whole team, you will read about it here.

Author: Petr Klus