Lean Software Development - build v1s and v2s

Once upon a time (and isn’t that how all good stories start)….<!--mep-nl-->I was managing a software team and we were working several initiatives.  The way projects were assigned was based on who was available, their skillset, and their developmental goals.  This ended up resulting on two developers, let’s call them Mary and Melissa, being assigned to the same project.Mary and Melissa had been working together a few weeks when I started hearing complaints in my one on ones with them about the other.  Mary was complaining that Melissa was taking too long to do her part, and spending time on unit tests that didn’t make sense because things were in flux with the project.  And meanwhile, Melissa was complaining that Mary wrote sloppy code and didn’t write enough tests (and even showed me comments in the code like “hack:….” and “to do: someone should add more error handling here”.  Both of them had valid points and feedback for the other.I spent the next few weeks coaching both of them on how improve and address the other’s concerns.  With Mary I was focused on pushing her to write more tests and not just throw things together, and with Melissa I was focused on having her prototype something quickly and then add the polish after it was working.  Both of them tried really hard, and both were miserable.  It just wasn’t what they were meant to do.These two ladies were just not meant to collaborate on projects together.And this got me thinking: how could we change the process so both could write code the way they like to create (and that caters to their strengths) and would still allow us to meet our team objectives?And from this was born the v1/v2 development process.You see, there are certain types of developers that love building the first versions or prototyping – they are the ones that love hacking things together to get something working quickly.  They love (and are best at) building the version 1 of a product.  The other type is the one that loves building the second version.   They see their code as a craft and write unit tests for everything.  “Test coverage” and “beautiful code” are phrases often used in their vocabulary.And of course this definition isn’t black and white, as there are certainly people who fall on both sides of the line, so it is more a spectrum.Typically, the v1 type of person doesn’t like working with the v2 person, and vice versa – not because of personal reasons, but because of the way they think and create software.  At the heart of it, the fundamental thing they enjoy about what they do is different.By changing the way we thought about software, we were able to address this concern and actually make the team more agile and solve some business problems along the way. 

The v1/v2 Process

When it comes to software development there are many ways to build and create products.In my experience it is difficult to build the right product the first time.  And by "right" I don't just mean something customers will want, use, and hopefully generate revenue; but also the "right" technical solution.  Often times the way customers use products can be hard to predict.  For example, before you ship the first version of a product it can be difficult to answer questions like:

• How quickly will the data grow?
• How fast will writes need to be?  Will it be feasible to write directly to the data store or do you need a queue to manage writes?
• What will the throughput be, and is it enough?
• Will the system scale with usage?

But even if you could reasonably answer these questions, or you build a system that takes all these things into account such that you can just “add hardware”, then it is likely that this effort was larger than if you had simply pushed to ship a first version quickly.  And it is possible (if not probable in some cases) that such a system may be over engineered (it was designed to solve problems or scale in a way that may not be useful), or even have other unforeseen issues.  While there are certainly pros to doing it right the first time, in practice it is very hard to get it right; and building the end all system will take longer, delaying the answers to all of the questions indicating you are moving in the right direction.There is a popular movement called Lean Startup that advocates fast iteration and data collection to hone in on the right product and requirements as quickly as possible.  It's all about figuring out what customers will use and then building those things.  And these same ideas can be applied to the technical ways we build products, too. 

Build v1 fast, then build v2 right.

Similar to creating a minimum viable product (MVP), build the minimum viable technical implementation that meets the business goals (the product can meet current usage requirements and performance).Most engineering teams have been taught to think about the software design and architecture from the beginning and build something that can scale.  However, if you don’t have any customers yet, scaling really isn’t your challenge.  So in some ways, designing for scale when you don’t have to scale yet, is solving the wrong problems.Of course I am not advocating doing something stupid, or writing bad code, but don’t over engineer or solve for problems until you have them.  For example:

• Forgoing the deployment of a multi-node Cassandra cluster and storing the initial data in a single db (with backup) that is fast and easy to make arbitrary queries against.
• Faster UI development using standard forms and charts vs. interesting shapes or fancy custom graphs.
• Cutting corners on unit tests (because the units are in flux) or comments and documentation.

All if these short cuts will help you get something out faster, and while not necessarily best practices they do solve the issue to ensure you are building the right thing.In many ways, this process is about looking at the problems differently, and instead of solving the “big” problem, focusing on speed and efficiency – getting the highest ROI (return on investment) for your development resources.So you modify your development process to ship the first version quickly, and then once the v1 is out in the wild, plan to start on the v2 right after.Then your v2 can address all the problems with the first version of the product.   You can improve the usability, cut or add features based on customer usage and feedback, pick the right technology and libraries to power the product, and spend time writing acceptance tests and unit tests that are unlikely to change.  And from a business perspective you get the product out there faster, so it is easy to understand if this product is going to drive revenue and success. 

Why this process makes sense…

Reduce the risk of over engineering.As noted above, getting something out quicker and understanding how the product is used (or how successful it is in the market) will help confirm that your are building the right product and solving the right technical problems for your business.  This allows you to mitigate the risk of building or scaling out systems that may or may not address the production issues.If you build the wrong thing you can correct it.Since the product is truly the minimum viable product, you can mitigate the risk of wasting resources creating the wrong product.  Furthermore, you can get the feedback earlier in the release cycle and pivot or make changes faster (since it is always harder to change a system with a plethora of moving parts).Chances are you will get to market faster.Building something quickly will help you launch sooner, and this will help you test and determine the viability or your product faster than waiting for the bigger launch.  This means you won't spend time building software your customers won't use or you can't sell.Staffing and personnel happiness.This is perhaps one of the biggest upsides to this method.  Software engineers get to do what they love, and what caters to their strengths.  Moreover, they aren’t forced to work with people that don’t share their same values, and they don’t have to work in a code base that they can’t sculpt into their desired piece of art.

So what are the downsides? 

Total time to build is more than if you just got it right the first time.In this model it is likely that overall the total development time will be longer and cumulatively greater than building the product once.  If you build the v1 on certain technologies (like say a MySQL DB that won’t scale the way you need it to), and then the v2 needs a different one (say a proper key value store for faster queries), then the team will spend time shipping and building expertise on one database, only to have to learn and manage another.Operations can be a headache.Most seasoned engineers will tell you that building something really fast can result in operations nightmares if the usage or data grows in a way that wasn’t considered.   And of course no team likes being operational and fighting fires.  Of course, if this is a problem it also probably means that your product has achieved some adoption and success, so at least you are solving something that derives real value. However, even for the v1 it is still worthwhile to identify risks and potential bottlenecks, so in the event there are problems the team will have some ideas for solutions.

So you want to try it at home? Errr….work?

Like any software process or methodology it is important to assess if this makes sense for your business, product, and company – it certainly isn’t a one size fits all.  However, it definitely has its merits, so feel free to steal the pieces you like  :)If you do embark on this journey though, here are some pointers from my experience:

• Get management buy-in.  This is really critical, because a key part of this being successful is always building a v1 and a v2.  If you don’t have buy off and your boss is content to keep and operate the v1, your team is going to hate you.  Seriously.  No one wants to get stuck supporting crufty v1 software indefinitely.  So make sure everyone knows and understands the plan ahead of time.
• Measure everything.  In order to build the right v2 you need to know what to build.  Where are your bottlenecks?  How is the data growing?  Without this data then it is very hard to realize very many of the pro reasons to adopt this model.  Make sure that you thing about what you need to measure, and how you will measure it before the product ships.
• The v2 will take much longer than the v1.  I have personally seen this process evolve across 4 projects (so not a large sample set), but in each case the v2 always took more than 2x the amount of time the v1 took to create and launch.  Of course every v2 I have launched a lot of extra features and didn’t always leverage a lot of the v1 code.  However, some people assume the v2 will be faster, since “you have already built is once before”, but I haven’t experienced this to be the case.
• Have a tight feedback loop with your customers.  Make sure it is easy to get insight into what people use, what they like and what they don't like - that way you can build a great v2.
• Make sure v1 & v2s are celebrated equally.  Sometimes teams or companies celebrate the first launch of a feature in a much grander way than the next version. But in this case the next version is what you need to grow your business and is just as important, if not more so.  Make sure you value each of these equally or else people will gravitate to the project with the most fanfare, not necessarily the one suited to their strengths and talents.
• Be open to people being both v1/v2s.  Sometimes I think I have someone pegged and then they prove me wrong.  Be open to the fact that there aren’t two types of people, and some people are just good at everything... (I just wish I was one of those people!)

 Hopefully some of these ideas prove useful for you and your team.   If anything, let this anecdote inspire you to question the way you are doing things, and look critically at innovative ways to improve the way you build software.And if you liked this or have other better ideas, definitely post them in the comments – I am always looking for ways and ideas to make us more efficient and effective as an engineering organization.

(Check out the best java software development services for a better software development experience.)