The Valley of Success

The idea behind this term comes from a concept created by Rico Mariani when discussing the development of low level computer software code in the early 2000s. He gave a talk centered around the idea that there were huge potential performance gains that weren’t being seen in the real world because most of the design choices for implementation led users down paths that were ultimately less performant for external reasons. The recommendation for developers was to think about building systems and APIs that were obvious and encouraged implementations that made it easy for users to just fall into a “pit of success.”

We’re constantly making decisions that might make us more or less successful. Often, the decisions we made previously that we thought were setting ourselves up for success, turn out to actually encourage failure down the road. As such, it is useful to think about it as traveling through a valley where you constantly have the option of taking various paths. Some of these paths might take you up the mountain, some might include a circuitous route, some will keep you in the valley. Often there are choices that will move you back into the valley after you realize you’ve started climbing the mountain. The key is to continually think about the choices being made to consider what is most likely to keep you in the Valley where the most progress can be made with the least cost.

When creating an automated testing strategy, consider these ideas:

Don’t focus on being successful

There are many ways to accomplish something, but most of them are actually brittle and prone to failure. The key is to find an implementation that is flexible and resilient. Instead of just focusing on making your project successful in the short term, think about what choices you can make so that it is harder for the project to fail down the road.

There are many perfectly fine approaches and designs that should be avoided in preference to something just as effective, but more straightforward in the long run. Avoid clever code.

You can make any process “work”

I’ve seen very smart people put together very complicated approaches that, with sufficient resources, they make entirely successful, and they are often extremely satisfied with the results. Often the thrill of taking the harder path is more alluring than what can be accomplished with a more simple and straightforward approach.

Just because you can make it work doesn’t mean it is a good thing to do. In fact, it is often actively a bad thing to do; even if your test framework is working fine, that only means it is working fine “right now.”

Don’t improve the end user experience

When you are creating a framework for other people to use, it is absolutely a good practice to adopt a more complex implementation In order to facilitate a simpler and more obvious solution for the end user.

The danger with this approach is that the author is unlikely to be the only person who will have to maintain this code. Framework design choices are made based on conventions that make sense to the author’s past history and frame of reference.

If the framework implementation is too complicated or non-obvious, users will find creative and less-maintainable workarounds rather than figuring out the correct fix. If the framework author leaves the company, users will frequently give up and rewrite everything in a way that fits their own frame of reference. Usually in JavaScript. (It’s always in JavaScript these days).

Don’t follow prescribed object-oriented design principles

Overcomplicating your code is worse than bad development practices. This is the opposite of what you’ll hear from many people who bemoan the lack of software engineering rigor by testers.

Consultants make a lot of money convincing people that they aren’t following the correct rules as decided by various luminaries in the industry. In my experience, however, too many developers become focused on strictly following proscribed rules instead of keeping in mind the actual purpose behind them.

The principles should serve to make the code you are writing easier to work with and more maintainable. What that looks like is much more context dependent than you’ll find written about in any book. I’m not saying it isn’t useful to understand these ideas, but blindly following them, regardless of the context causes more harm than good.

When it comes down to it, I’ve found that it is much easier to fix under-developed code than over-developed code.

Every day, as you are making decisions about your testing, you should be thinking about if there’s a more maintainable way to validate something or a less clever way to implement something. We need to resist the urge to start climbing mountains and stick to the valley of success.