Lars Vonk recently posted this comment on Joshua’s talk about Lean Startup: Why It Rocks Far More Than Agile Development
So I listened 30 minutes and stopped because I am thinking if this leanstartup is really an ugly baby? I heard a couple of things which feels like an agile rebranding thing:
- The example on the 9 months building and then launching. What book tells you that? How about Principle #1 from the Manifesto: Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Are you doing customer tests in the definition of done? Then it is the leanstartup definition of done. What is that about?
Should I continue and read the book or is it a waste of time? What does it really add?
The first thing I need to get off my chest however is that it is not Lean Startup vs Agile. It is Lean Startup & Agile. It is like Extreme Programming and Agile. On the surface they are similar, but with a small twist. And once you dig deeper you find elements in both that feed of each other. And today almost all decent Agile teams are using both Extreme Programming techniques with Agile.
The short answer to question 1 is: You should certainly continue and read the book.
The longer answer to question 2:
For me the best summary is Agile is the right way to build software. Lean Startup is about finding out what the right software is to build in environments of extreme uncertainty.
First of all, the uncertainty thing is pretty important. If you are replacing system A with system B which is supposed to do pretty much the same thing, or if you are automating a business process that you have been doing for 25 years, you are not going to be needing Lean Startup techniques. You know what you want and you just need to build it as quickly as possible. There is no extreme uncertainty.
User Stories and Hypothesis
Agile has pretty much universally adopted the user stories from XP as their way of managing backlogs. These are usually written by a Product Owner/Manager in the form of: As a I want so that . But there are a lot of assumptions underlying them. “Does really want ?” and “Does the really lead to the ?” and more important ones such as: “How much is willing to pay to achieve ?”
Lean Startup works with these assumptions. It challenges you to define hypothesis around them. That is just a fancy word for a prediction about an assumption. Once we have such a hypothesis we can design experiments around them to validate or disprove them.
Eric Ries calls them Minimum Viable Products in his book, but I think that’s a terrible name because it implies you should be building something. I call them Minimum Viable Experiments. The great thing is that a hypothesis allows you to think outside the software box.
Take for example the video that Drew Houston from Dropbox fame made before even going live:
But it can also be 404 testing. Create a link to a new feature where you apologize and explain it will come soon. Record how many people click on it and see if that’s enough for you to warrant building it.
Vanity Metrics
Lean Startup also challenges you to think only of the real metrics for your business. Often we see companies publish extremely impressive numbers that have absolutely nothing to do with building a sustainable business. One of the ones I can remember was Whatsapp publishing some insane number about messages delivered per day. They make exactly $0 on every message, and probably lose some, so this is not a healthy metric to use to measure your business results.
Analysis
Agile is about often delivering software to your customers, but Lean Startup goes a bit further. It challenges you to track the effects your change has on your core business metrics with things like A/B testing. Someone recently told me a story that I hope will illustrate things.
His client wanted a big (and expensive) change to the flow of the checkout process. He thought the change was not worth the money and they would not make back the initial investment, but his client insisted and he developed the change and put it live. Now everything he puts live is A/B tested. 20% of the users get to see the change and 80% of the users still see the old version (control group). The surprising thing was that not only did the 20% not increase revenues, but it decreased! If the change would have been put live for everyone, they would lose $10,000 a day in revenues. No need to add the change was reverted after only a couple of days.
In summary
Lean Startup has a lot to add to the Agile Development. Especially in cases with uncertainty is forces you to look critically at your requirements. Look through your user stories to see the assumptions underlying them and tackle those. Do not get side tracked by vanity metrics. How does a particular user story affect our real metrics. Does it increase our Life Time Value of a customer? Will it increase sign-up rate? Conversion rates? Retention? If only you added an item to the user story:
As a I want so that which should improve by
Now we also have a way to Business Value. The improvement of the metric should be fairly easy to calculate into a number.
But it is also about the effect of that change. What really happened? Did the metric really improve? And by how much?
So.. Would you do Scrum or Kanban without XP practices like TDD and User Stories? Of course you would not.
So why not consider expanding them with a focus on assumptions and hypothesis and a focus on the effects on your core business metrics?
Hi Erwin,
Thanks for replying through this blogpost. Allmost all the things you write in this blogpost make perfectly sense. However to me Agile has always been about these things, so saying “So why not consider expanding them with a focus on assumptions and hypothesis and a focus on the effects on your core business metrics?” sounds a bit strange. Agile methods have it’s core in PDCA which is all about assumptions and hypothesis. So saying that it is good to add this to Agile doesn’t fit my understanding of it. In that light to me there is not really much new in the mindset of it. I do value the tools mentioned, like 404, A/B, we can use to achieve that.
Secondly I totally disagree with you when you say: “If you are replacing system A with system B which is supposed to do pretty much the same thing, or if you are automating a business process that you have been doing for 25 years, you are not going to be needing Lean Startup techniques. You know what you want and you just need to build it as quickly as possible.”. Rebuilding projects are, in my experience, the projects that fail the most because of the lack of focussing on core business metrics, lack of real user testing etc. etc. So I definitely use these techniques when doing projects like these.
So to sum up: I like the (re)emphasize on finding the (right) business case and validating this quickly with whatever tools possible. (But first I will have to read the book
).
Lars
Lars,
It is certainly possible, also not necessary, that existing teams are already using some or all of the ideas in Lean Startup.
What with regard to the PDCA I find that most of the focus in the Agile Community is with regards to the process, not so much the product and hardly ever the desired outcomes.
And I don’t see many companies trace the performance of their changes. They made the business case for the change at some point and rarely close the loop to track what actually happened when life punches them in the face.
Yes that is indeed true and a real pity. But I wonder if those companies are helped by introducing a new name for the (in my opinion) same thing. In such cases it is better to inspect and adept and see why they are doing a half ass implementation of X (Enter favorite agile process here). That is basically my main concern.
Pingback: Splitting up my split (blogging) personalities | Erronis