The Lean Startup - by Eric Ries
ISBN: 0307887898Date read: 2011-10-23
How strongly I recommend it: 6/10
(See my list of 360+ books, for more.)
Go to the Amazon page for details and reviews.
The methodology here is the one I recommend the most. The stuff I preach is like a cute casual intro to the real deal: the Lean Startup methodology. (As an aside: this book is the one that pushed my book out of the #1 slot on Amazon's Entrepreneur charts. Quite an honor.)
my notes
My definition of a startup: a human institution designed to create new products and services under conditions of extreme uncertainty.
The stories in the magazines are lies: hard work and perseverance don’t lead to success.
It’s the boring stuff that matters the most.
Startup success is not a consequence of good genes or being in the right place at the right time. Startup success can be engineered by following the right process, which means it can be learned, which means it can be taught.
Early adopters: we often talked to them and asked for their feedback. But we emphatically did not do what they said. We viewed their input as only one source of information about our product and overall vision. In fact, we were much more likely to run experiments on our customers than we were to cater to their whims.
An extremely fast cycle time, a focus on what customers want (without asking them), and a scientific approach to making decisions.
Startups exist to learn how to build a sustainable business. This learning can be validated scientifically by running frequent experiments.
The fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere. All successful startup processes should be geared to accelerate that feedback loop.
Focus on the boring stuff: how to measure progress, how to set up milestones, and how to prioritize work. This requires a new kind of accounting.
Planning and forecasting are only accurate when based on a long, stable operating history and a relatively static environment. Startups have neither.
Many entrepreneurs take a “just do it” attitude, avoiding all forms of management, process, and discipline. Unfortunately, this approach leads to chaos more often than it does to success.
Entrepreneurship requires a managerial discipline to harness the entrepreneurial opportunity.
Progress in manufacturing is measured by the production of high-quality physical goods. As we’ll see in Chapter 3, the Lean Startup uses a different unit of progress, called validated learning. With scientific learning as our yardstick, we can discover and eliminate the sources of waste that are plaguing entrepreneurship.
A method for measuring progress in the context of extreme uncertainty.
Clear guidance on how to make the many trade-off decisions they face: whether and when to invest in process; formulating, planning, and creating infrastructure; when to go it alone and when to partner; when to respond to feedback and when to stick with vision; and how and when to invest in scaling the business. Most of all, it must allow entrepreneurs to make testable predictions.
The goal of a startup is to figure out the right thing to build - the thing customers want and will pay for - as quickly as possible.
The Lean Startup is a new way of looking at the development of innovative new products that emphasizes fast iteration and customer insight.
Too many startup business plans look more like they are planning to launch a rocket ship than drive a car. They prescribe the steps to take and the results to expect in excruciating detail, and as in planning to launch a rocket, they are set up in such a way that even tiny errors in assumptions can lead to catastrophic outcomes.
The customers failed to materialize, the company had committed itself so completely that they could not adapt in time. They had “achieved failure” - successfully, faithfully, and rigorously executing a plan that turned out to have been utterly flawed.
Instead of making complex plans that are based on a lot of assumptions, you can make constant adjustments with a steering wheel called the Build-Measure-Learn feedback loop. Through this process of steering, we can learn when and if it’s time to make a sharp turn called a pivot or whether we should persevere along our current path.
The strategy may have to change (called a pivot). However, the overarching vision rarely changes. Entrepreneurs are committed to seeing the startup through to that destination. Every setback is an opportunity for learning how to get where they want to go.
Startups use many kinds of innovation: novel scientific discoveries, repurposing an existing technology for a new use, devising a new business model that unlocks value that was hidden, or simply bringing a product or service to a new location or a previously underserved set of customers.
Innovation is at the heart of the company’s success.
Most tools from general management are not designed to flourish in the harsh soil of extreme uncertainty in which startups thrive. The future is unpredictable, customers face a growing array of alternatives, and the pace of change is ever increasing. Yet most startups - in garages and enterprises alike - still are managed by using standard forecasts, product milestones, and detailed business plans.
Innovation is a bottoms-up, decentralized, and unpredictable thing, but that doesn’t mean it cannot be managed. It can, but to do so requires a new management discipline, one that needs to be mastered not just by practicing entrepreneurs seeking to build the next big thing but also by the people who support them, nurture them, and hold them accountable. In other words, cultivating entrepreneurship is the responsibility of senior management.
Validated learning is the process of demonstrating empirically that a team has discovered valuable truths about a startup’s present and future business prospects. It is more concrete, more accurate, and faster than market forecasting or classical business planning.
Invite one of your friends to chat.” And she says, “No way!” We say, “Why not?” And she says, “Well, I don’t know if this thing is cool yet. You want me to risk inviting one of my friends? What are they going to think of me? If it sucks, they’re going to think I suck, right?”
We had a mental model for how people used software that was years out of date.
Bit by bit, customers tore apart our seemingly brilliant initial strategy.
I had committed the biggest waste of all: building a product that our customers refused to use.
One last refuge for people aching to justify their own failure. I consoled myself that if we hadn’t built this first product - mistakes and all - we never would have learned these important insights about customers. We never would have learned that our strategy was flawed. There is truth in this excuse: what we learned during those critical early months set IMVU on a path that would lead to our eventual breakout success. For a time, this “learning” consolation made me feel better, but my relief was short-lived. Here’s the question that bothered me most of all: if the goal of those months was to learn these important insights about customers, why did it take so long? How much of our effort contributed to the essential lessons we needed to learn? Could we have learned those lessons earlier if I hadn’t been so focused on making the product “better” by adding features and fixing bugs? VALUE VS. WASTE In other words, which of our efforts are value-creating and which are wasteful?
Lean thinking defines value as providing benefit to the customer; anything else is waste.
But in a startup, who the customer is and what the customer might find valuable are unknown.
What we had learned over those first months about what creates value for customers. Anything we had done during those months that did not contribute to our learning was a form of waste.
Would it have been possible to learn the same things with less effort?
Think of all the debate and prioritization of effort that went into features that customers would never discover. If we had shipped sooner, we could have avoided that waste. Also consider all the waste caused by our incorrect strategic assumptions. I had built interoperability for more than a dozen different IM clients and networks. Was this really necessary to test our assumptions? Could we have gotten the same feedback from our customers with half as many networks? With only three? With only one? Since the customers of all IM networks found our product equally unattractive, the level of learning would have been the same, but our effort would have been dramatically less.
Did we have to support any networks at all? Is it possible that we could have discovered how flawed our assumptions were without building anything? For example, what if we simply had offered customers the opportunity to download the product from us solely on the basis of its proposed features before building anything?
(Note that this is different from asking customers what they want. Most of the time customers don’t know what they want in advance.)
Learning is the essential unit of progress for startups. The effort that is not absolutely necessary for learning what customers want can be eliminated. I call this validated learning because it is always demonstrated by positive improvements in the startup’s core metrics. As we’ve seen, it’s easy to kid yourself about what you think customers want. It’s also easy to learn things that are completely irrelevant. Thus, validated learning is backed up by empirical data collected from real customers.
Certainly our stories of failure were entertaining, and we had fascinating theories about what we had done wrong and what we needed to do to create a more successful product. However, the proof did not come until we put those theories into practice and built subsequent versions of the product that showed superior results with actual customers.
The hard work of discovering what customers really wanted and adjusting our product and strategy to meet those desires.
It is often easier to raise money or acquire other resources when you have zero revenue, zero customers, and zero traction than when you have a small amount. Zero invites imagination, but small numbers invite questions about whether large numbers will ever materialize. Everyone knows (or thinks he or she knows) stories of products that achieved breakthrough success overnight. As long as nothing has been released and no data have been collected, it is still possible to imagine overnight success in the future. Small numbers pour cold water on that hope.
Learn to see every startup in any industry as a grand experiment. The question is not “Can this product be built?” In the modern economy, almost any product that can be imagined can be built. The more pertinent questions are “Should this product be built?” and “Can we build a sustainable business around this set of products and services?” To answer those questions, we need a method for systematically breaking down a business plan into its component parts and testing each part empirically.
One of the most important lessons of the scientific method: if you cannot fail, you cannot learn.
A true experiment follows the scientific method. It begins with a clear hypothesis that makes predictions about what is supposed to happen. It then tests those predictions empirically. Just as scientific experimentation is informed by theory, startup experimentation is guided by the startup’s vision. The goal of every startup experiment is to discover how to build a sustainable business around that vision.
To sell the shoes, Zappos had to interact with customers: taking payment, handling returns, and dealing with customer support. This is decidedly different from market research. If Zappos had relied on existing market research or conducted a survey, it could have asked what customers thought they wanted. By building a product instead, albeit a simple one, the company learned much more:
1. It had more accurate data about customer demand because it was observing real customer behavior, not asking hypothetical questions.
2. It put itself in a position to interact with real customers and learn about their needs.
3. It allowed itself to be surprised when customers behaved in unexpected ways, revealing information Zappos might not have known to ask about. For example, what if customers returned the shoes?
Break It Down
The first step would be to break down the grand vision into its component parts. The two most important assumptions entrepreneurs make are what I call the value hypothesis and the growth hypothesis. The value hypothesis tests whether a product or service really delivers value to customers once they are using it.
The growth hypothesis tests how new customers will discover a product or service.
The point is not to find the average customer but to find early adopters: the customers who feel the need for the product most acutely. Those customers tend to be more forgiving of mistakes and are especially eager to give feedback.
This specification will be rooted in feedback on what is working today rather than in anticipation of what might work tomorrow.
Traditionally, the product manager says, ‘I just want this.’ In response, the engineer says, ‘I’m going to build it.’
Instead, I try to push my team to first answer four questions:
1. Do consumers recognize that they have the problem you are trying to solve?
2. If there was a solution, would they buy it?
3. Would they buy it from us?
4. Can we build a solution for that problem?”
The common tendency of product development is to skip straight to the fourth question and build a solution before confirming that customers have the problem.
“Until we could figure out how to sell and make the product, it wasn’t worth spending any engineering time on.”
Returning to India, Akshay joined the Village Laundry Services (VLS), created by Innosight Ventures. VLS began a series of experiments to test its business assumptions. For their first experiment, VLS mounted a consumer-grade laundry machine on the back of a pickup truck parked on a street corner in Bangalore. The experiment cost less than $8,000 and had the simple goal of proving that people would hand over their laundry and pay to have it cleaned. The entrepreneurs did not clean the laundry on the truck, which was more for marketing and show, but took it off-site to be cleaned and brought it back to their customers by the end of the day. The VLS team continued the experiment for a week, parking the truck on different street corners, digging deeper to discover all they could about their potential customers. They wanted to know how they could encourage people to come to the truck. Did cleaning speed matter? Was cleanliness a concern? What were people asking for when they left their laundry with them? They discovered that customers were happy to give them their laundry to clean. However, those customers were suspicious of the washing machine mounted on the back of the truck, concerned that VLS would take their laundry and run. To address that concern, VLS created a slightly more substantial mobile cart that looked more like a kiosk. VLS also experimented with parking the carts in front of a local minimarket chain. Further iterations helped VLS figure out which services people were most interested in and what price they were willing to pay. They discovered that customers often wanted their clothes ironed and were willing to pay double the price to get their laundry back in four hours rather than twenty-four hours. As a result of those early experiments, VLS created an end product that was a three-foot by four-foot mobile kiosk that included an energy-efficient, consumer-grade washing machine, a dryer, and an extra-long extension cord. The kiosk used Western detergents and was supplied daily with fresh clean water delivered by VLS. Since then, the Village Laundry Service has grown substantially, with fourteen locations operational in Bangalore, Mysore, and Mumbai. As CEO Akshay Mehra shared with me, “We have serviced 116,000 kgs. in 2010 (vs. 30,600 kg. in 2009). And almost 60 percent of the business is coming from repeat customers. We have serviced more than 10,000 customers in the past year alone across all the outlets.”5
VLS story was recounted by Elnor Rozenrot, formerly of Innosight Ventures. Additional detail was provided by Akshay Mehra. For more on the VLS, see the article in Harvard Business Review
Their challenge is to overcome the prevailing management thinking that puts its faith in well-researched plans. Remember, planning is a tool that only works in the presence of a long and stable operating history.
To apply the scientific method to a startup, we need to identify which hypotheses to test. I call the riskiest elements of a startup’s plan, the parts on which everything depends, leap-of-faith assumptions. The two most important assumptions are the value hypothesis and the growth hypothesis. These give rise to tuning variables that control a startup’s engine of growth. Each iteration of a startup is an attempt to rev this engine to see if it will turn. Once it is running, the process repeats, shifting into higher and higher gears. Once clear on these leap-of-faith assumptions, the first step is to enter the Build phase as quickly as possible with a minimum viable product (MVP). The MVP is that version of the product that enables a full turn of the Build-Measure-Learn loop with a minimum amount of effort and the least amount of development time.
Creating a MVP requires extra work: we must be able to measure its impact. For example, it is inadequate to build a prototype that is evaluated solely for internal quality by engineers and designers. We also need to get it in front of potential customers to gauge their reactions.
When we enter the Measure phase, the biggest challenge will be determining whether the product development efforts are leading to real progress.
The method I recommend is called innovation accounting, a quantitative approach that allows us to see whether our engine-tuning efforts are bearing fruit. It also allows us to create learning milestones,
Finally, and most important, there’s the pivot.
If we’ve discovered that one of our hypotheses is false, it is time to make a major change to a new strategic hypothesis.
How Facebook was able to raise so much money when its actual usage was so small: By all accounts, what impressed investors the most were two facts about Facebook’s early growth. The first fact was the raw amount of time Facebook’s active users spent on the site. More than half of the users came back to the site every single day. This is an example of how a company can validate its value hypothesis - that customers find the product valuable. The second impressive thing about Facebook’s early traction was the rate at which it had taken over its first few college campuses.
Facebook also had validated its growth hypothesis. These two hypotheses represent two of the most important leap-of-faith questions any new startup faces.
Is the lesson of Facebook that startups should not charge customers money in the early days? Or is it that startups should never spend money on marketing? These questions cannot be answered in the abstract;
Instead, as we saw in Part One, startups need to conduct experiments that help determine what techniques will work in their unique circumstances. For startups, the role of strategy is to help figure out the right questions to ask.
Every business plan begins with a set of assumptions. It lays out a strategy that takes those assumptions as a given and proceeds to show how to achieve the company’s vision. Because the assumptions haven’t been proved to be true (they are assumptions, after all) and in fact are often erroneous, the goal of a startup’s early efforts should be to test them as quickly as possible.
Assumptions that require more courage to state - in the present tense - with a straight face: we assume that customers have a significant desire to use a product like ours, or we assume that supermarkets will carry our product. Acting as if these assumptions are true is a classic entrepreneur superpower. They are called leaps of faith precisely because the success of the entire venture rests on them. If they are true, tremendous opportunity awaits. If they are false, the startup risks total failure.
Venture capitalist Randy Komisar, whose book "Getting to Plan B" discussed the concept of leaps of faith in great detail, uses a framework of “analogs” and “antilogs” to plot strategy.
What differentiates the success stories from the failures is that the successful entrepreneurs had the foresight, the ability, and the tools to discover which parts of their plans were working brilliantly and which were misguided, and adapt their strategies accordingly.
Genchi gembutsu, which is one of the most important phrases in the lean manufacturing vocabulary. In English, it is usually translated as a directive to “go and see for yourself” so that business decisions can be based on deep firsthand knowledge.
You cannot be sure you really understand any part of any business problem unless you go and see for yourself firsthand. It is unacceptable to take anything for granted or to rely on the reports of others.
Craft a customer archetype, a brief document that seeks to humanize the proposed target customer. This archetype is an essential guide for product development and ensures that the daily prioritization decisions that every product team must make are aligned with the customer to whom the company aims to appeal.
A minimum viable product (MVP) is simply the fastest way to get through the Build-Measure-Learn feedback loop with the minimum amount of effort.
The goal of the MVP is to begin the process of learning,
Its goal is to test fundamental business hypotheses.
Early adopters are suspicious of something that is too polished: if it’s ready for everyone to adopt, how much advantage can one get by being early? As a result, additional features or polish beyond what early adopters demand is a form of wasted resources and time.
Early adopters use their imagination to fill in what a product is missing. They prefer that state of affairs, because what they care about above all is being the first
Most entrepreneurs and product development people dramatically overestimate how many features are needed in an MVP. When in doubt, simplify.
Most entrepreneurs approach a question like this by building the product and then checking to see how customers react to it. I consider this to be exactly backward because it can lead to a lot of waste. First, if it turns out that we’re building something nobody wants, the whole exercise will be an avoidable expense of time and money. If customers won’t sign up for the free trial, they’ll never get to experience the amazing features that await them. Even if they do sign up, there are many other opportunities for waste. For example, how many features do we really need to include to appeal to early adopters? Every extra feature is a form of waste, and if we delay the test for these extra features, it comes with a tremendous potential cost in terms of learning and cycle time. The lesson of the MVP is that any additional work beyond what was required to start learning is waste, no matter how important it might have seemed at the time.
(Dropbox:) To avoid the risk of waking up after years of development with a product nobody wanted, Drew did something unexpectedly easy: he made a video. The video is banal, a simple three-minute demonstration of the technology as it is meant to work.
Only at the point where the founders were too busy to bring on additional customers did Manuel and his team start to invest in automation
Always focused on scaling something that was working rather than trying to invent something that might work in the future.
Contrast this with the case of a small business, in which it is routine to see the CEO, founder, president, and owner serving customers directly, one at a time. In a concierge MVP, this personalized service is not the product but a learning activity designed to test the leap-of-faith assumptions in the company’s growth model.
“We continued to run with humans replicating pieces of the backend for nine months. We hired eight people to manage queries, classify conversations, etc. We actually raised our seed and series A rounds before the system was automated. As we refined the product, we would bring in six to twelve people weekly to react to mockups, prototypes, or simulations that we were working on. It was a mix of existing users and people who never saw the product before. We had our engineers join for many of these sessions, both so that they could make modifications in real time, but also so we could all experience the pain of a user not knowing what to do.”
If we do not know who the customer is, we do not know what quality is.
As you consider building your own minimum viable product, let this simple rule suffice: remove any feature, process, or effort that does not contribute directly to the learning you seek.
The most common objection I have heard over the years to building an MVP is fear of competitors - especially large established companies - stealing a startup’s ideas.
I have often given entrepreneurs fearful of this issue the following assignment: take one of your ideas (one of your lesser insights, perhaps), find the name of the relevant product manager at an established company who has responsibility for that area, and try to get that company to steal your idea. Call them up, write them a memo, send them a press release - go ahead, try it. The truth is that most managers in most companies are already overwhelmed with good ideas. Their challenge lies in prioritization and execution.
If a competitor can outexecute a startup once the idea is known, the startup is doomed anyway. You believe you can accelerate through the Build-Measure-Learn feedback loop faster than anyone else can. If that’s true, it makes no difference what the competition knows. If it’s not true, a startup has much bigger problems, and secrecy won’t fix them. Sooner or later, a successful startup will face competition from fast followers.
The only way to win is to learn faster than anyone else.
MVP can seem like a dangerous branding risk. Easy solution: launch the MVP under a different brand name. Experiment under the radar and then do a public marketing launch once the product has proved itself with real customers.
Prepare for the fact that MVPs often result in bad news.
The solution to this dilemma is a commitment to iteration. You have to commit to a locked-in agreement - ahead of time - that no matter what comes of testing the MVP, you will not give up hope.
A startup’s job is to
(1) rigorously measure where it is right now, confronting the hard truths that assessment reveals, and then
(2) devise experiments to learn how to move the real numbers closer to the ideal reflected in the business plan.
A simple question that I make a habit of asking startups whenever we meet:
Are you making your product better?
They always say yes.
Then I ask: how do you know?
I invariably get this answer: well, we are in engineering and we made a number of changes last month, and our customers seem to like them, and our overall numbers are higher this month. We must be on the right track.
This is the kind of storytelling that takes place at most startup board meetings. Most milestones are built the same way: hit a certain product milestone, maybe talk to a few customers, and see if the numbers go up. Unfortunately, this is not a good indicator of whether a startup is making progress. How do we know that the changes we’ve made are related to the results we’re seeing? More important, how do we know that we are drawing the right lessons from those changes? To answer these kinds of questions, startups have a strong need for a new kind of accounting geared specifically to disruptive innovation.
Innovation accounting works in three steps:
(1) Use a minimum viable product to establish real data on where the company is right now.
(2) Startups must attempt to tune the engine from the baseline toward the ideal. This may take many attempts. After the startup has made all the micro changes and product optimizations it can to move its baseline toward the ideal, the company reaches a decision point. That is the third step:
(3) Pivot or persevere. If the company is making good progress toward the ideal, that means it’s learning appropriately and using that learning effectively, in which case it makes sense to continue. If not, the management team eventually must conclude that its current product strategy is flawed and needs a serious change. When a company pivots, it starts the process all over again, reestablishing a new baseline and then tuning the engine from there. The sign of a successful pivot is that these engine-tuning activities are more productive after the pivot than before.
Smoke test with its marketing materials. This is an old direct marketing technique in which customers are given the opportunity to preorder a product that has not yet been built. A smoke test measures only one thing: whether customers are interested in trying a product. By itself, this is insufficient to validate an entire growth model. Nonetheless, it can be very useful to get feedback on this assumption.
These MVPs provide the first example of a learning milestone.
Test the riskiest assumptions first.
For example, a company might spend time improving the design of its product to make it easier for new customers to use. This presupposes that the activation rate of new customers is a driver of growth and that its baseline is lower than the company would like. To demonstrate validated learning, the design changes must improve the activation rate of new customers. If they do not, the new design should be judged a failure. This is an important rule: a good design is one that changes customer behavior for the better.
Five dollars bought us a hundred clicks - every day. From a marketing point of view this was not very significant, but for learning it was priceless. Every single day we were able to measure our product’s performance with a brand new set of customers. Also, each time we revised the product, we got a brand new report card on how we were doing the very next day.
Cohort analysis. This is one of the most important tools of startup analytics. Although it sounds complex, it is based on a simple premise. Instead of looking at cumulative totals or gross numbers such as total revenue and total number of customers, one looks at the performance of each group of customers that comes into contact with the product independently. Each group is called a cohort.
Every company depends for its survival on sequences of customer behavior called flows. Customer flows govern the interaction of customers with a company’s products. They allow us to understand a business quantitatively and have much more predictive power than do traditional gross metrics.
The sign of a successful pivot: the new experiments you run are overall more productive than the experiments you were running before.
This is the pattern: poor quantitative results force us to declare failure and create the motivation, context, and space for more qualitative research. These investigations produce new ideas - new hypotheses - to be tested, leading to a possible pivot. Each pivot unlocks new opportunities for further experimentation, and the cycle repeats. Each time we repeat this simple rhythm: establish the baseline, tune the engine, and make a decision to pivot or persevere.
Tools for product improvement do not work the same way for startups. If you are building the wrong thing, optimizing the product or its marketing will not yield significant results. A startup has to measure progress against a high bar: evidence that a sustainable business can be built around its products or services. That’s a standard that can be assessed only if a startup has made clear, tangible predictions ahead of time. In the absence of those predictions, product and strategy decisions are far more difficult and time-consuming.
Because Grockit was using the wrong kinds of metrics, the startup was not genuinely improving. Farb was frustrated in his efforts to learn from customer feedback. In every cycle, the type of metrics his team was focused on would change: one month they would look at gross usage numbers, another month registration numbers, and so on. Those metrics would go up and down seemingly on their own. He couldn’t draw clear cause-and-effect inferences. Prioritizing work correctly in such an environment is extremely challenging. Farb could have asked his data analyst to investigate a particular question. For example, when we shipped feature X, did it affect customer behavior? But that would have required tremendous time and effort. When, exactly, did feature X ship? Which customers were exposed to it? Was anything else launched around that same time? Were there seasonal factors that might be skewing the data? Finding these answers would have required parsing reams and reams of data. The answer often would come weeks after the question had been asked. In the meantime, the team would have moved on to new priorities and new questions that needed urgent attention.
Grockit changed the metrics they used to evaluate success in two ways. Instead of looking at gross metrics, Grockit switched to cohort-based metrics, and instead of looking for cause-and-effect relationships after the fact, Grockit would launch each new feature as a true split-test experiment. A split-test experiment is one in which different versions of a product are offered to customers at the same time. By observing the changes in behavior between the two groups, one can make inferences about the impact of the different variations.
Although working with split tests seems to be more difficult because it requires extra accounting and metrics to keep track of each variation, it almost always saves tremendous amounts of time.
User stories were not considered complete until they led to validated learning. Thus, stories could be cataloged as being in one of four states of development: in the product backlog, actively being built, done (feature complete from a technical point of view), or in the process of being validated. Validated was defined as “knowing whether the story was a good idea to have been done in the first place.” This validation usually would come in the form of a split test showing a change in customer behavior but also might include customer interviews or surveys. The kanban rule permitted only so many stories in each of the four states. As stories flow from one state to the other, the buckets fill up. Once a bucket becomes full, it cannot accept more stories. Only when a story has been validated can it be removed from the kanban board. If the validation fails and it turns out the story is a bad idea, the relevant feature is removed from the product.
I have implemented this system with several teams, and the initial result is always frustrating: each bucket fills up, starting with the “validated” bucket and moving on to the “done” bucket, until it’s not possible to start any more work.
The only way to start work on new features is to investigate some of the stories that are done but haven’t been validated.
If they include the validation exercise from the beginning, the whole team can be more productive. For example, why build a new feature that is not part of a split-test experiment? It may save you time in the short run, but it will take more time later to test, during the validation phase.
The three A’s of metrics: actionable, accessible, and auditable.
Actionable: For a report to be considered actionable, it must demonstrate clear cause and effect. Make the reports as simple as possible so that everyone understands them.
Cohort-based reports are the gold standard of learning metrics: they turn complex actions into people-based reports. Each cohort analysis says: among the people who used our product in this period, here’s how many of them exhibited each of the behaviors we care about.
Each employee could log in to the system at any time, choose from a list of all current and past experiments, and see a simple one-page summary of the results.
Be able to check if the reports contain true facts. Managers need the ability to spot check the data with real customers.
Gain insights into why customers are behaving the way the data indicate.
Reports should be drawn directly from the master data, rather than from an intermediate system, which reduces opportunities for error.
There is no bigger destroyer of creative potential than the misguided decision to persevere.
Companies that cannot bring themselves to pivot to a new direction on the basis of feedback from the marketplace can get stuck in the land of the living dead, neither growing enough nor dying, consuming resources and commitment from employees.
David had identified his leap-of-faith questions explicitly at the outset and, more important, had made quantitative predictions about each of them.
It is only because David focused on actionable metrics for each of his leap-of-faith questions that he was able to accept that his company was failing.
Because David had not wasted energy on premature PR, he was able to make this determination without public embarrassment or distraction.
A pivot requires that we keep one foot rooted in what we’ve learned so far, while making a fundamental change in strategy in order to seek even greater validated learning.
A STARTUP’S RUNWAY IS THE NUMBER OF PIVOTS IT CAN STILL MAKE
Get to each pivot faster. In other words, the startup has to find ways to achieve the same amount of validated learning at lower cost or in a shorter time. All the techniques in the Lean Startup model that have been discussed so far have this as their overarching goal.
The failure of the “launch it and see what happens” approach should now be evident: you will always succeed - in seeing what happens. Except in rare cases, the early results will be ambiguous, and you won’t know whether to pivot or persevere, whether to change direction or stay the course.
Entrepreneurs need to face their fears and be willing to fail, often in a public way. In fact, entrepreneurs who have a high profile, either because of personal fame or because they are operating as part of a famous brand, face an extreme version of this problem.
I recommend that every startup have a regular “pivot or persevere” meeting.
Remember that the rationale for building low-quality MVPs is that developing any features beyond what early adopters require is a form of waste. However, the logic of this takes you only so far. Once you have found success with early adopters, you want to sell to mainstream customers. Mainstream customers have different requirements and are much more demanding. The kind of pivot we needed is called a customer segment pivot. In this pivot, the company realizes that the product it’s building solves a real problem for real customers but that they are not the customers it originally planned to serve. In other words, the product hypothesis is confirmed only partially.
The very actions that made us successful with early adopters were diametrically opposed to the actions we’d have to master to be successful with mainstream customers.
"Pivot" sometimes is used incorrectly as a synonym for change. A pivot is a special kind of change designed to test a new fundamental hypothesis about the product, business model, and engine of growth.
A CATALOG OF PIVOTS
A platform pivot. Instead of selling an application to one customer at a time, David envisioned a new growth model inspired by Google’s AdWords platform. He built a self-serve sales platform where anyone could become a customer with just a credit card.
Zoom-in Pivot In this case, what previously was considered a single feature in a product becomes the whole product. A zoom-in pivot, refocusing the product on what previously had been considered just one feature of a larger whole.
Zoom-out Pivot In the reverse situation, sometimes a single feature is insufficient to support a whole product. In this type of pivot, what was considered the whole product becomes a single feature of a much larger product.
Customer Segment Pivot In this pivot, the company realizes that the product it is building solves a real problem for real customers but that they are not the type of customers it originally planned to serve. In other words, the product hypothesis is partially confirmed, solving the right problem, but for a different customer than originally anticipated. A customer segment pivot, keeping the functionality of the product the same but changing the audience focus. In other words, David went from being a business-to-consumer (B2C) company to being a business-to-business (B2B) company.
Customer Need Pivot: The problem we’re trying to solve for them is not very important. However, because of this customer intimacy, we often discover other related problems that are important and can be solved by our team. The target customer has a problem worth solving, just not the one that was originally anticipated.
Platform Pivot A platform pivot refers to a change from an application to a platform or vice versa. Most commonly, startups that aspire to create a new platform begin life by selling a single application, the so-called killer app, for their platform. Only later does the platform emerge as a vehicle for third parties to leverage as a way to create their own related products.
Business Architecture Pivot: a startup switches architectures. Some companies change from high margin, low volume by going mass market (e.g., Google’s search “appliance”); others, originally designed for the mass market, turned out to require long and expensive sales cycles.
Value Capture Pivot: capturing value is an intrinsic part of the product - and changes to the way a company captures value can have far-reaching consequences for the rest of the business, product, and marketing strategies.
Engine of Growth Pivot: three primary engines of growth that power startups: the viral, sticky, and paid growth models. In this type of pivot, a company changes its growth strategy to seek faster or more profitable growth. Commonly but not always, the engine of growth also requires a change in the way value is captured.
Channel Pivot: the sales channel or distribution channel. For example, consumer packaged goods are sold in a grocery store, cars are sold in dealerships, and much enterprise software is sold (with extensive customization) by consulting and professional services firms. Often, the requirements of the channel determine the price, features, and competitive landscape of a product. A channel pivot is a recognition that the same basic solution could be delivered through a different channel with greater effectiveness.
It is precisely because of its destructive effect on sales channels that the Internet has had such a disruptive influence in industries that previously required complex sales and distribution channels, such as newspaper, magazine, and book publishing.
Technology Pivot: Occasionally, a company discovers a way to achieve the same solution by using a completely different technology. Technology pivots are much more common in established businesses.
Technology life cycle ideas of theorists such as Geoffrey Moore know certain later-stage pivots by the names he has given them: the Chasm, the Tornado, the Bowling Alley.
Modern managers cannot have escaped the deluge of recent books calling on them to adapt, change, reinvent, or upend their existing businesses. Many of the works in this category are long on exhortations and short on specifics.
On the page, these processes may seem clinical, slow, and simple. In the real world, something different is needed. We have learned to steer when moving slowly. Now we must learn to race. Laying a solid foundation is only the first step toward our real destination: acceleration.
Which activities create value and which are a form of waste? Once you understand this distinction, you can begin using lean techniques to drive out waste and increase the efficiency of the value-creating activities.
What products do customers really want? How will our business grow? Who is our customer? Which customers should we listen to and which should we ignore? These are the questions that need answering as quickly as possible
The one envelope at a time approach is called “single-piece flow” in lean manufacturing. It works because of the surprising power of small batches. When we do work that proceeds in stages, the “batch size” refers to how much work moves from one stage to the next at a time.
In process-oriented work like this, individual performance is not nearly as important as the overall performance of the system.
Instead of working in separate departments, engineers and designers would work together side by side on one feature at a time. Whenever that feature was ready to be tested with customers, they immediately would release a new version of the product, which would go live on our website for a relatively small number of people. The team would be able immediately to assess the impact of their work, evaluate its effect on customers, and decide what to do next. For tiny changes, the whole process might be repeated several times per day.
In School of One, students have daily “playlists” of their learning tasks that are attuned to each student’s learning needs, based on that student’s readiness and learning style.
There are assessments built into each activity so that data can be fed back to the teacher to choose appropriate tasks for the next playlist.
The longer we worked, the more afraid we became of how customers would react when they finally saw the new version. As our plans became more ambitious, so too did the number of bugs, conflicts, and problems we had to deal with. Pretty soon we got into a situation in which we could not ship anything. Our launch date seemed to recede into the distance. The more work we got done, the more work we had to do. The lack of ability to ship eventually precipitated a crisis and a change of management, all because of the trap of large batches.
Lean production solves the problem of stockouts with a technique called pull. When you bring a car into the dealership for repair, one blue 2011 Camry bumper gets used. This creates a “hole” in the dealer’s inventory, which automatically causes a signal to be sent to a local restocking facility called the Toyota Parts Distribution Center (PDC). The PDC sends the dealer a new bumper, which creates another hole in inventory. This sends a similar signal to a regional warehouse called the Toyota Parts Redistribution Center (PRC), where all parts suppliers ship their products. That warehouse signals the factory where the bumpers are made to produce one more bumper, which is manufactured and shipped to the PRC. The ideal goal is to achieve small batches all the way down to single-piece flow along the entire supply chain. Each step in the line pulls the parts it needs from the previous step. This is the famous Toyota just-in-time production method.8
The amount of just-in-case inventory [called work-in-progress (WIP) inventory] is reduced dramatically.
All the work that goes into designing the minimum viable product is - until the moment that product is shipped - just WIP inventory. Incomplete designs, not-yet-validated assumptions, and most business plans are WIP.
Almost every Lean Startup technique we’ve discussed so far works its magic in two ways: by converting push methods to pull and reducing batch size. Both have the net effect of reducing WIP.
Customers often don’t know what they want. Our goal in building products is to be able to run experiments that will help us learn how to build a sustainable business. Thus, the right way to think about the product development process in a Lean Startup is that it is responding to pull requests in the form of experiments that need to be run.
As soon as we formulate a hypothesis that we want to test, the product development team should be engineered to design and run this experiment as quickly as possible, using the smallest batch size that will get the job done. Remember that although we write the feedback loop as Build-Measure-Learn because the activities happen in that order, our planning really works in the reverse order: we figure out what we need to learn and then work backwards to see what product will work as an experiment to get that learning. Thus, it is not the customer, but rather our hypothesis about the customer, that pulls work from product development and other functions. Any other work is waste.
Toyota has built the most advanced learning organization in history. It has demonstrated an ability to unleash the creativity of its employees, achieve consistent growth, and produce innovative new products relentlessly over the course of nearly a century.11 This is the kind of long-term success to which entrepreneurs should aspire. Although lean production techniques are powerful, they are only a manifestation of a high-functioning organization that is committed to achieving maximum performance by employing the right measures of progress over the long term. Process is only the foundation upon which a great company culture can develop. But without this foundation, efforts to encourage learning, creativity, and innovation will fall flat - as
Sustainable growth is characterized by one simple rule: New customers come from the actions of past customers. There are four primary ways past customers drive sustainable growth:
1. Word of mouth. Embedded in most products is a natural level of growth that is caused by satisfied customers’ enthusiasm for the product.
2. As a side effect of product usage. Fashion or status, such as luxury goods products, drive awareness of themselves whenever they are used.
3. Through funded advertising. For this to be a source of sustainable growth, the advertising must be paid for out of revenue, not one-time sources such as investment capital. As long as the cost of acquiring a new customer (the so-called marginal cost) is less than the revenue that customer generates (the marginal revenue), the excess (the marginal profit) can be used to acquire more customers. The more marginal profit, the faster the growth.
4. Through repeat purchase or use. Some products are designed to be purchased repeatedly either through a subscription plan (a cable company) or through voluntary repurchases (groceries or lightbulbs). By contrast, many products and services are intentionally designed as one-time events, such as wedding planning.
These sources of sustainable growth power feedback loops that I have termed engines of growth. Each is like a combustion engine, turning over and over. The faster the loop turns, the faster the company will grow. Each engine has an intrinsic set of metrics that determine how fast a company can grow when using it.
Startups don’t starve; they drown.
Startups have to focus on the big experiments that lead to validated learning. The engines of growth framework helps them stay focused on the metrics that matter.
Companies using the sticky engine of growth track their attrition rate or churn rate very carefully. The churn rate is defined as the fraction of customers in any period who fail to remain engaged with the company’s product. The rules that govern the sticky engine of growth are pretty simple: if the rate of new customer acquisition exceeds the churn rate, the product will grow. The speed of growth is determined by what I call the rate of compounding, which is simply the natural growth rate minus the churn rate.
Focus needs to be on improving customer retention. This goes against the standard intuition in that if a company lacks growth, it should invest more in sales and marketing. This counterintuitive result is hard to infer from standard vanity metrics.
THE THREE ENGINES OF GROWTH:
(1) = The Sticky Engine of Growth
(2) = The Viral Engine of Growth:
Depend on person-to-person transmission as a necessary consequence of normal product use. Customers are not intentionally acting as evangelists; they are not necessarily trying to spread the word about the product. Growth happens automatically as a side effect of customers using the product. Viruses are not optional.
The viral engine is powered by a feedback loop that can be quantified. It is called the viral loop, and its speed is determined by a single mathematical term called the viral coefficient. The higher this coefficient is, the faster the product will spread. The viral coefficient measures how many new customers will use a product as a consequence of each new customer who signs up.
For a product with a viral coefficient of 0.1, one in every ten customers will recruit one of his or her friends. This is not a sustainable loop. Imagine that one hundred customers sign up. They will cause ten friends to sign up. Those ten friends will cause one additional person to sign up, but there the loop will fizzle out. By contrast, a viral loop with a coefficient that is greater than 1.0 will grow exponentially, because each person who signs up will bring, on average, more than one other person
Companies that rely on the viral engine of growth must focus on increasing the viral coefficient more than anything else, because even tiny changes in this number will cause dramatic changes in their future prospects.
Many viral products do not charge customers directly but rely on indirect sources of revenue such as advertising. This is the case because viral products cannot afford to have any friction impede the process of signing customers up and recruiting their friends. This can make testing the value hypothesis for viral products especially challenging. The true test of the value hypothesis is always a voluntary exchange of value between customers and the startup that serves them. A lot of confusion stems from the fact that this exchange can be monetary, as in the case of Tupperware, or nonmonetary, as in the case of Facebook. In the viral engine of growth, monetary exchange does not drive new growth; it is useful only as an indicator that customers value the product enough to pay for it. If Facebook or Hotmail had started charging customers in their early days, it would have been foolish, as it would have impeded their ability to grow. However, it is not true that customers do not give these companies something of value: by investing their time and attention in the product, they make the product valuable to advertisers.
Each customer pays a certain amount of money for the product over his or her “lifetime” as a customer. Once variable costs are deducted, this usually is called the customer lifetime value (LTV). This revenue can be invested in growth by buying advertising. Suppose an advertisement costs $100 and causes fifty new customers to sign up for the service. This ad has a cost per acquisition (CPA) of $2.00. In this example, if the product has an LTV that is greater than $2, the product will grow. The margin between the LTV and the CPA determines how fast the paid engine of growth will turn (this is called the marginal profit). Conversely, if the CPA remains at $2.00 but the LTV falls below $2.00, the company’s growth will slow.
(3) = The Paid Engine of Growth
Startups that employ an outbound sales force are also using this engine, as are retail companies that rely on foot traffic. All these costs should be factored into the cost per acquisition.
The startup had an “unlimited” pricing plan, its most expensive, that cost only a few hundred dollars per month. The NGO literally could not make the purchase because it had no process in place for buying something so inexpensive. Additionally, the NGO needed substantial help in managing the rollout, educating its staff on the new tool, and tracking the impact of the change; those were all services the company was ill equipped to offer. Changing customer segments required them to switch to hiring a sizable outbound sales staff that spent time attending conferences, educating executives, and authoring white papers. Those much higher costs came with a corresponding reward: the company switched from making only a few dollars per customer to making tens and then hundreds of thousands of dollars per much larger customer. Their new engine of growth led to sustained success.
Advertising that is targeted to more affluent customers generally costs more than advertising that reaches the general public. What determines these prices is the average value earned in aggregate by the companies that are in competition for any given customer’s attention. Wealthy consumers cost more to reach because they tend to become more profitable customers. Over time, any source of customer acquisition will tend to have its CPA bid up by this competition.
Successful startups usually focus on just one engine of growth, specializing in everything that is required to make it work.
I strongly recommend that startups focus on one engine at a time.
ENGINES OF GROWTH DETERMINE PRODUCT/MARKET FIT
Product/market fit to describe the moment when a startup finally finds a widespread set of customers that resonate with its product:
Conversely, in a terrible market, you can have the best product in the world and an absolutely killer team, and it doesn’t matter - you’re going to fail.3
To many entrepreneurs it implies that a pivot is a failure event - “our startup has failed to achieve product/market fit.” It also implies the inverse - that once our product has achieved product/market fit, we won’t have to pivot anymore. Both assumptions are wrong.
If a startup is attempting to use the viral engine of growth, it can focus its development efforts on things that might affect customer behavior - on the viral loop - and safely ignore those that do not. Such a startup does not need to specialize in marketing, advertising, or sales functions. Conversely, a company using the paid engine needs to develop those marketing and sales functions urgently.
What really matters is not the raw numbers or vanity metrics but the direction and degree of progress.
Overarchitecture failure, in which attempting to prevent all the various kinds of problems that could occur wound up delaying the company from putting out any product.
Friendster effect, suffering a high-profile technical failure just when customer adoption is going wild.
One of the most important discoveries of the lean manufacturing movement: you cannot trade quality for time. If you are causing (or missing) quality problems now, the resulting defects will slow you down later. Defects cause a lot of rework, low morale, and customer complaints, all of which slow progress and eat away at valuable resources.
Use a system called the Five Whys to make incremental investments and evolve a startup’s processes gradually. The core idea of Five Whys is to tie investments directly to the prevention of the most problematic symptoms. The system takes its name from the investigative method of asking the question “Why?” five times to understand what has happened (the root cause).
At the root of every seemingly technical problem is a human problem. Five Whys provides an opportunity to discover what that human problem might be.
Chronic problems are caused by bad process, not bad people, and remedy them accordingly.
Managers and employees can fall into the trap of using the Five Blames as a means for venting their frustrations and calling out colleagues for systemic failures.
We routinely asked new engineers to make a change to the production environment on their first day.
“If our production process is so fragile that you can break it on your very first day of work, shame on us for making it so easy to do so.”
Everyone came through it with a visceral understanding of our values.
Ask teams to adopt these simple rules:
1. Be tolerant of all mistakes the first time.
2. Never allow the same mistake to be made twice.