Making sense of a startup or innovation can be difficult. There are risks inherent wether you’re a VC or other investor, a ninja/rockstar or founder being asked to take equity as payment, or just someone that’s excited to work on something new. Maybe failure is part of it, but that doesn’t mean you have to go down with the ship. I was recenty able to protect 50% of a six-figure investment by approaching it not as a one big bet to be made, but rather as series of measurable risks to be managed.
I’ve developed a flexible framework to model startups that can be matched to varied contexts and risk appetites. The model can’t predict success, rather it provides a way of thinking about the risks in a more holistic way geared toward actually running, and therefore governing, the company.
I’ve outlined the thinking behind it below by first looking at a common thread of reasoning with startups, appling it to three concrete cases, and then examining the logic (where it can go wrong, especially), to develop our model.
And because models aren’t always super-useful in the real world, I’ve also brought it back to some concrete things you can look for when deciding if you’re willing to take that leap.
Three start-up case studies
First, let’s start with a common refrain - the startup conjecture:
“It’s expensive to [do some business processes]. What if I told you we can [technology] and make lots of money. Here’s my demo…”
let’s break that statement down a bit further. what we’re saying is:
- current business process is done like this today
- technology now exist that can be used do the same thing, like so (flashy demo)
- example cost savings for this scenario
- multiply it across the industry
- profit!
We’re all familiar with this pattern, it makes sense, but how can we evaluate the claim, given that the new company often can’t be measured? Models exist that take into acount varying degrees of market analysis, or other factors like “founder strength”. But, when it’s something completely unbuilt, investors are largely going “with their gut”.
Assume you really trust your gut. What then?
Surely we can do better. Let’s look at three cases to start to identify factors that matter in a startup.
Case 1 - Automated, grid-distributed, real-time energy analysis for building architecture models
The industry problem at the time was that energy analysis is manual, labour intensive, and expensive. Analysis from an engineering company can take weeks and cost tens or even hundreds of thousands of dollars, making iteration cost-prohibitive. The startup is founded on the theory that using cloud technology, one can automate the process. Rather than having a bunch of engineers doing excel, we can do it in near real-time with tech. Sounds good, right?
Assumptions & solutions
First, let’s look at the key technology claims being made. We can:
- decompose 3-d model into geometry planes programmatically (using code)
- then automate the hookup of geometry to maths models (imagine cutting and pasting the dimensions of a wall from a 3-d model into excel)
- create a user interface for fast changes to the model
- use cloud computing to run the calculations
- a weeks-long process is now seconds
- free money!
What went wrong
Well, a few things:
- The company initially tried to farm out the 3-d model decomposition, but getting it right for a variety of models and edge cases turned out to be cost-prohibitive. This was solved by finding someone who was doing their PhD in this area. They found the problem interesting & challenging- it was a borderline academic. This was a key innovation the company had missed along with required resourcing.
- The user interface required many custom UI widgets. At the time, people who could build javascript UI libraries were uncommon. For instance, interfaces for manipulating architecture models and handling multiple asynchronous data streams and joining them back up on the client-side had to be built. Another missed key innovation that required specialised resourcing.
- While cloud-computing now made servers cheap, three features we take for granted today were not available at the time:
- Lambdas - the geometry of any building is of unknown size, but the compute requirements for any single piece of geometry are known. Technology was required to split the calculation into [n] sub-functions across [n] servers. Done easily with lambdas today, but not so in 2010.
- Auto-scaling – ensuring that the servers could be spun up and down for the above mentioned calculations in a neat manner was required
- Measuring & monitoring - the ability to provide to the customer the predicted, live, and past costs for all of the above, based on server utilisation did not exist.
Outcome
This company burned through its seed, then A round of investment and was sold in a firesale to an established competitor.
Case 2 - Real-time, graph-based, automated news production
Running tapes around for news production, searching and archiving is difficult. Finding relevant and related news is also time consuming. Social media is eating the lunch of traditional media. This startup was founded on the theory that one can automate these processes and do real-time social media search to provide a modern news product along with faster, more cost-effective production processes. Slurp in the video in real-time, turn it into text and do graph-search on existing, traditional media and social media. News production of the future!
Assumptions & solutions
Here’s what’s in play:
- online video players exist
- speech-to-text exists
- graph data storage is a thing that exists
- we can combine the above
- bing-bang-boom!
What went wrong
- Oversimplification of each step. For instance, an online video editor and player is theoretically a company on its own. The video player was outsourced as a UI issue to a third-party that wasn’t a specialist in the area.
- Speech-to-text at the time without machine learning of today. Take the simple case, and then try the wide-and-varied regional accents of any nation. Another business on its own.
- Speech-to-text compute requirements, even without AI are ginormous.
- Graph data storage working in conjunction with social media knowledge graphs and search in real-time was outsourced. The start-up was effectively paying a second company to build their own product.
- None of the founders were experts in any of the above technology areas, driving many of the decisions to outsource.
Outcome
The burn-rate for each of these innovations was not correctly predicted. Key engineering staff, understanding the depths of technical debt to address, left well-ahead of the inevitable outcome. The project, which was spun off from a larger organisation, was shut down eventually. The start-up never got into production, the parent company eventually went into bankruptcy.
Case 3 - Medical AI Bot
AI has had some recent press as a panacea for many things, but medical AI really took off when IBM started touting Watson. This startup, like many AI-chatbots, was founded on the theory that handling ‘low value contact’ – common, repeatable tasks– will provide magnitudes of efficiency, but for medical organisations.
Assumptions & solutions
Here’s what we’re working with:
- medical systems are clogged / there aren’t enough doctors
- AI Language models are now a mature thing
- there’s LOADS of medical information that can be used to train a bot
- provide AI Chatbot to deal with the low-value stuff
- doctors are free to deal with the hard stuff
- Bob’s your uncle
What went wrong
- Many medical things that might seem ‘small’ at first are potentially ‘big’ requiring lab work. This required a key innovation: a secure lab-to-consumer distribution network which was never part of the plan.
- Efficacy is difficult to measure (training is a key machine learning step). Many AI/ML systems rely on what’s called ‘human in the loop’ to fix errors. It’s often outsourced to developing nations. Imagine doing this, but needing doctors.
- Modelling chat data for data science is not the same thing as medical analysis. Building and maintaining a data pipeline that can be quickly improved and used for multiple purposes is … expensive.
- Multiply all of the above effort with life and death risk
- Huge ‘unknown unknowns’ that don’t even come into the technical problem equation
- political things – never to be underestimated
- an established industry already structured around gaming the ratio of low-value to high-value interactions
Outcome
The startup burned through $550 million investment, went bankrupt and its assets have been liquidated.
Extracting the model from evidence
So, you can start to see some patterns, but let’s be a little bit more specific. If we look at the above cases, the claim being made is about functional equivalency and optimisation. Let’s break down what that actually means:
When we’re talking about some existing business model in comparative terms, we’re saying that:
- we put some effort (E) into a series of steps, let’s assume time or money
- from this you get something of value - a product (P)
- the product has some measurable value, for instance you can sell it (more money) or take the day off (time)
- our business process is a value-generating black box
So, you put in some effort, you get some product, and you get some value. In our first case, the product is the energy analysis, in the second it’s relevant news, in the last it’s medical diagnosis.
By saying it has Value, the claim that we’re making with our innovation is that, all things being the same, our new black box is:
- more efficient (less effort)
- potentially more valuable
So, even if it’s just more efficient, the amount of effort we put into it will be reduced for the same product. That alone is great. But often, we’re actually making a second claim. For instance, the lower cost or faster production makes our product available to a wider audience. Growing the market, as they say.
This is often enough for investors or potential employees, and sometimes it goes right. But, we can do better. We can look at why it goes wrong to reduce our risk.
First, let’s ask where does that value come from? When we look inside the black box of our business process, we see that the business process is made up of a bunch of little steps working in concert:
- we can now see that steps can ‘cost’ effort
- some steps may ‘cost’ different amounts of effort
- assume a step is a combination of process and tech
- the total effort is expended at the end (we’re very efficient)
With this in mind, we can now look at our innovation again, and go back to comparison.
We’re saying that technologies have either reduced the cost or eliminated the step altogether. That’s an efficiency. Maybe the product is faster, making it more attractive, the value goes up. You can now sell it to more people. Etc.
However, as we’ve seen in the above examples, things don’t always turn out this way for various reasons. If we consider factors external to the process, for instance:
- making a technical thing work within our business context can require ‘secret sauce’ in itself that must be developed
- politics & market conditions
- maturity or experience of the team
Additionally, it’s assumed the ‘problem’ that our business problem is working against is one thing. In fact, businesses are often directed at a range of related problems. Taking just a couple of these external factors, our diagram might actually look like this:
As with the medical/AI example, these external factors can be just as powerful as the internal ones.
Data that matters
When we make claims of equivalency like this what’s known as a transitive fallacy – the two things being compared are said to be equal because their parts are the same. However, as we’ve seen this isn’t always the case, and we could be comparing apples to oranges.
But how to know when we’re making valid comparisons? Well, context matters. From the examples above, let’s extract three of the risk factors to test our model:
- hidden process/informal knowledge (validate the things we are sure we know)
- key innovations that are unidentified or minimised (discover unknown unknowns)
- management inexperience in the target business domain or domains
There are surely many more, but let’s go with these for now.
Applying the model
The next step is pretty straightforward: score each bet we’re making and decide how to allocate risk. A lot of the relevant information for deciding the risk is hidden inside people at the beginning and requires in-person research. We can also extrapolate from our own experience, and of course, one should bring in an expert when unsure.
In this example, we’re going to look at three ‘bets’ we’ve made about how our new tech is going to help us. Then, we’re going to think about the risks that we’re right. We’ll divide the risk amongst three things in this simplified model:
- risk of doing the things that we think we’re sure of (never to be ignored), which must be validated
- risk that we’ve ignored unknowns & key innovations
- whether we think the founding team understands both both business processes BP1 & BP2 sufficiently
We’ll keep scoring simple: thumbs-up or thumbs down (score with a 1 or a 0), moving toward 1 if we think we can work the risk down, and 0 if it’s insurmountable. And we can line them up and do some calculation.
In this model, we then weigh the bets by relative priority. In the example below, we’ve divided a priority of 1 by the three items (it’s represented as decimal instead of percent for easier reading). We can then add up our total.
| Bet | Validated knowns (33%) | Discovered Unknowns (33%) | Domain Expert (33%) | Score | Priority (weight) | Weighted Score |
|---|---|---|---|---|---|---|
| BP2, Step 4’s ‘cost’ is less than BP1, Step 4’ | 1 | 0 | 0 | 33 | .1 | 3.3 |
| BP2, Step 5’s ‘cost’ is eliminated by BP2, Step 4’ | 1 | 1 | 1 | 100 | .4 | 40 |
| Combining BP2S5 and BP2S4 doesn’t require a significant innovation | 1 | 0 | 1 | 75 | .5 | 37.5 |
| Total | 80% |
In this case, we have a 80% likelihood that we’re going to succeed, IF we weigh unknowns equally with knowns.
But as we’ve seen in our examples, unknown unknowns can actually have an outsized effect. If we try it again, adding a bit more weight to the unknowns:
| Bet | Validated knowns (33%) | Discovered Unknowns (33%) | Domain Expert (33%) | Score | Priority (weight) | Weighted Score |
|---|---|---|---|---|---|---|
| BP2, Step 4’s ‘cost’ is less than BP1, Step 4’ | 1 | 0 | 0 | 25 | .1 | 2.5 |
| BP2, Step 5’s ‘cost’ is eliminated by BP2, Step 4’ | 1 | 1 | 1 | 100 | .4 | 40 |
| Combining BP2S5 and BP2S4 doesn’t require a significant innovation | 1 | 0 | 1 | 50 | .5 | 25 |
| Total | 60% |
… we go from thinking it’s very likely to starting to think that we’re just beyond even odds. You can imagine how this will now drop if we further weigh the whole endeavour of combining our things together.
This is a very simple model. Imagine that as we discover new unknowns, each one is a new “Bet”. Domain expertise has been simplified. In our examples, domain expertise can be overridden by a founder that isn’t an expert. As an investor, this would mean more funding being required to fulfill the requirement. From a governance perspective, founders that don’t recognise their shortcomings would drive this risk up. We could weigh that at the combine step differently than the individual Bets. And so on.
In this simple example, we can see the benefits of many first-year alpha exercises with tech and the importance not just of validating the core assumptions, but specifically looking for the things that weren’t envisioned at the outset. I would be suspicious of any endeavour that doesn’t admit to hitting at least some of these problems.
What this means for you (taking action)
You don’t have to do all the work above or do the math. Let’s face it, coming up with the exact values for many of these factors is probably going to be time and cost prohibitive relative to the scale and velocity of most startups. That’s not the point anyway, unless you’re thinking about valuations, which is a different beast altogether.
For investors, the elevator pitch and financial projections aren’t enough, and yet it’s what innovators are taught to focus on from day one. Attempts to address the human factor such as “founder strength” are pretty weak. Instead what we should be asking is things like:
- what is the distance to contextualise the technologies
- what are the key innovations required to combine these technologies
- who, at this time, can actually do the work
For anyone looking to work for a start-up, the questions should be:
- what are the core problems to crack
- who here can crack them – is it supposed to be me
- can the people in charge crack it, but I’m here to scale
- do they have the money to hire the people to crack/scale these problems
The above model can provide a framework for objective discussion and decision-making for either.
Again, this isn’t about determining success, but rather identifying risks and finding a way for everybody on board to agree what they are. Understanding the problems ahead and agreeing that the hard things are hard makes for really exciting and positive innovation projects for some. Too exciting for others, but first we have to know how exciting it is.
The bottom line is that you don’t have to use this model, but you need something. Be suspicious of anybody who’s working against you to rationalise the risks – they’re either wilfully ignorant or incompetent. Neither is good.