Two out of three should fail.

A venture fund where every investment succeeds has a problem. It's taking no risk, so it's leaving the big outcomes on the table.

Investors know this in their bones. They expect most of a portfolio to disappoint. What they won't accept is losing money without knowing why, or letting a bad bet limp along for three years because nobody wanted to be the one to call it.

I don't have a study behind two out of three. Call it my rule of thumb. If most of your AI pilots work, you picked safe ones. The failures don't worry me. What worries me is how few companies can say, before a pilot starts, what failure would even look like.

Most pilots aren't experiments

Picture a pilot that starts the way a lot of them do. Someone senior sees a demo. A team gets a budget and a loose brief: "see what AI can do for claims processing." Six months later there's a slide with a few promising screenshots, a cost line, and the word "learnings" doing a lot of heavy lifting.

Did it succeed? Nobody can say, because nobody said what success meant. Should it continue? Nobody wants to say no, because no feels like admitting the money was wasted. So it rolls into next year with a smaller team and a new sponsor.

Calling that a failed experiment is generous. Nobody designed it.

A real experiment has a question, a test and an ending, set up so that either result tells you something. Most corporate pilots have the question, sort of, and skip the rest.

Declare both before you start

The rule I give teams working on anything new is simple. When you conceive the idea, declare how you'll know it worked and how you'll know it failed. Both. Early, before anyone has fallen in love with what they built.

The failure half is the one people leave out, and it's the one that does the work. Success criteria are easy to write and easy to stretch. Somebody will always find a number that went up. A failure criterion written in advance is much harder to argue with.

When I ran a concept of mine through G.I.V.E., every canvas carried both. One failure criterion read: "claim denial rate unchanged 90 days post-deployment." Another: "provider still needs to ask the patient what brought them in." A stranger could check either one. If it's true at the end, the thing didn't work, however good the demo was. ("G.I.V.E. on one real decision" walks through that concept.)

You won't get the criteria perfect. On anything truly new you're measuring something nobody has measured before, so you're working with a proxy whether you admit it or not. That's fine. The way you measure success and failure needs to be developed just as much as the concept itself. Start fuzzy, write it down, and sharpen it as you learn. ("Use fuzzy metrics until you get good ones" is about exactly that.)

How I'd run an AI portfolio

• Every pilot gets success and failure criteria before it starts. "If we don't see this by this date, we stop." Written down, agreed, boring. By the checkpoint the team will be a little in love with what they built. That's human. The failure criterion is the note you wrote yourself before it happened.
• Every pilot gets a learning goal, separate from its success goal. Even if it fails, what will we know that we don't know now? Can the model handle our messiest documents? Will the people doing the work trust its output enough to change how they work? Is our data clean enough to try anything at all? A failed pilot that answers a real question is a good investment. One that answers nothing is just a cost.
• Size the bets differently. Most should be small and fast. One or two can be big. A fund doesn't write the same check for a seed round and a growth round. Small bets buy information cheaply. Big bets are for ideas that already survived a small one.
• Review the set as a portfolio. The question moves from "did this pilot work?" to "is the portfolio producing more than it costs, and are we getting smarter?" Reviewing pilots one at a time turns every review into a trial, and teams learn to defend instead of report. Looking at the whole set lets you celebrate a clean kill as loudly as a win.

What a good failure looks like

A good failure is fast. It hits its failure criterion on the date you set, and the team moves on to the next bet within a week.

It's written down. Someone puts on one page what was tried, what happened, and what it means for the next bet. That page is the asset. It's what stops another team from running the same pilot in a different department a year and a half later, which in big companies happens more than anyone likes to admit.

And it changes something. If a pilot showed your data isn't ready, the next bet goes on the data, not on a fancier model sitting on top of the same mess.

The bad failure is the one that never officially ends. It can't hit a failure criterion because it never had one. It just gets quieter. People drift to other work, and a year later it's still on someone's budget.

A clean record is a warning

If none of your AI pilots have failed, that's a warning. Calling it discipline doesn't change what it is.

A clean record usually means the bets were too safe. Summarize the meeting notes, tag the support tickets, draft the weekly report: projects that were always going to work and were never going to matter much. They have their place. They won't show you where the big outcomes are, and a portfolio made only of them looks great on a slide right up until a competitor finds what you didn't look for.