Jun 20, 2021
Projects
10
 min read

Evaluating Startup Ideas and When to Pivot: Learn From Our Mistakes

Lots of writing out there suggests that the best way to start almost every startup is by first working on the problem as a side project. But how do you know if your product idea is best to be kept as a side project or should be pursued as something greater? If your idea doesn’t seem to be gaining any traction, when should you call it quits and move on to something else?

Warning: don’t simply hope early on that your idea for a side project will turn into a fully-fledged startup and don’t simply move on to your next idea when progress seems weak.

This is a story of how we learnt those lessons, plus the tools we now use to avoid the same mistakes moving forward.

Some context:

In spring 2020, Thomas, Modest and I (Alex) applied to Y Combinator’s summer batch. We received an invitation to interview. We had applied with an idea that was aiming to solve a problem that my 15-year old brother had: trying to assess the career options ahead of him and not knowing where to start. Our initial solution was dubbed ‘Wayfinder’, which we stated was going to 'gamify career discovery for high schoolers'; basically a fun-to-use learning resource to allow high school students to conveniently explore different career paths.

Three weeks after submitting the application, and five product iterations later, we started to feel our hearts weren’t in the problem we were trying to solve. We (and more importantly, my brother) weren’t excited by any of the initial products we had developed.

We had read a lot of online literature stating that when working on projects you wish may one day turn into fully-fledged companies, you must work on solving a problem you are familiar with and one that fascinates you. We began to convince ourselves that we knew far too little about what 15-year olds truly wanted and couldn’t build anything that would prove both informational and ‘cool to use’ for high school students. During this time, we made the decision to pivot to a different problem and product entirely.

Like many wannabe entrepreneurs, we had a notes app filled with ideas. Most of these ideas were terrible. Luckily, we could identify these ones and kept them in our notes simply as a humorous reminder. However, some ideas, at face value, appear to veer on the line of good, yet may in fact be very bad in reality. Paul Graham refers to these ideas as “sitcom” ideas, because they sound plausible enough that they may even be included in TV writing as a ‘truthful sounding’ fictional startup idea. These ideas that appear plausible on the surface are dangerous. They can convince you to give them your full attention and effort, when deep down there will be some underlying flaw with the prospect. You need to stay vigilant and utilise mental models, such as the framework discussed below, to avoid being consumed by these plausible-sounding ideas.    

We pivoted from the Wayfinder career discovery tool to a music exploration tool. Why? Because Thomas and I are fascinated with underground dance music. We love discovering and listening to DJ sets (long-form recordings of a collection of different songs blended together) where they introduce you to new songs (most of them soon-to-be released) in fun and playful ways. We felt there was no platform currently out there that was optimised for this content - nowhere to save the moments within a mix, seamlessly jump between different mixes and explore new parts of different mixes recommended based on your tastes. This was a ‘problem’, however niche, that we believed we faced. Motivated to see if we could solve this problem, and excited to be building something in the space we were so passionate about, we built a duct-taped-together Minimal Viable Product (MVP) of the platform. We called it Audiex.

Screenshots and promo material of the Audiex app

We made two big mistakes pivoting from Wayfinder to Audiex, and then going on to pursue Audiex and pitch the idea to Y Combinator.

Mistake 1: Not Being SMART

The first issue was that we hadn’t objectively timeboxed our efforts when starting with Wayfinder. Objectively timeboxing means that you set a SMART goal (Specific, Measurable, Actionable, Result-based, Time-orientated) with regard to the current idea you want to pursue. An example would be “We will work on this idea for three months and aim to get 10 users that are happy and willing to pay at least £50 for the product.”

Objectivley Timebox by using SMART goals

When setting SMART goals, make sure they are ambitious but still attainable, so that you don’t get dismayed when you miss a completely unrealistic goal set by yourself. Another key part of the SMART framework that directly relates to achievability is to make sure the time you give yourself is enough for you to put a lot of focused effort into building the product and talking to users.

As always, you should priotise building product as efficiently as possible in the initial validation phase. This relates to the principle of MVP: testing the hypothesis of your idea by making something of value in the most efficient way possible. You should also accommodate for the crucial step of talking to users; an essential process of obtaining open and explicit feedback on your product.

Of course, the time taken to develop product and talk to users depends upon resources such as your current free time and availability. However, always aim to invoke tight deadlines - this will give some motivational pressure to encourage you to work towards the goal efficiently, as well as showcase what is truly possible with the time you have currently available.

A commonly-used heuristic is that a software-based MVP should take no more than a couple of months to put together. The renowned investor Garry Tan has a great video on timeboxing and how it relates to validating your ideas and pivoting.

One last thing on timeboxing and setting goals. You will of course have preconceived ideas and expectations for how your product idea will progress. Just make sure you truly accept how hard it will be to achieve your goal. This isn’t to deter you from starting. If it did, you wouldn’t (and shouldn’t) be pursuing your own product anyway. These thoughts are to try and help you realise that, when you are in those deep difficult periods, that they were already accommodated for. Don’t jump out of an idea at the first sign of trouble. If you have successfully timeboxed your idea validation with a SMART goal, then have faith in the process and don’t give up early.

If you get to the end of the time allotted during your SMART goal and you don’t achieve the result laid out, then you have a decision to make on whether you continue to try and solve the problem you set out to solve, or move on to validating a different product in a different problem space. This decision is difficult. But at least you have some objective metrics to support your decision now. You must take the learnings from speaking to your prospect customers and understand if there is anything you can do to amend or change your product to meet their true perceived needs. This, of course, is a pivot in itself, albeit a less drastic one than moving onto an entirely different problem space. You must make the call whether you perform a minor or extreme pivot at this point, but make sure you utilise the results and insights of your timeboxed objective.

There are different types of business pivots


I believe not implementing a SMART goal and objectively timeboxing our idea validation with Wayfinder resulted in us becoming deterred too early in the process. If we had done what we preach here, we would have at least systematically and objectively tested the hypothesis of our idea.

Mistake 2: Product Idea ≠ Business Idea

As stated at the beginning of this article, lots of writing suggests that the best way to start most startups is by first working on the problem as a side project. Working on something as a side project, because you find it interesting, whilst having no intention of turning it into a business is perfectly fine. What’s dangerous is when you try to turn a side project into a business, before validating whether it would work as a suitable prospect business idea. We found Audiex interesting and liked to work on it as a side project, but we didn’t rigorously and pragmatically assess whether it could turn into a suitable business idea. I like to blame the rushed anticipation of building the MVP in time for the YC interview, as it was the only product we were feeling proud enough to present at the time, as the reason why. But in reality, we were foolish to not stop and assess our idea through a framework such as the one below. Don’t be foolish like us.

The building blocks of the framework I like to use were inspired by the mental models of Courtland Allen, founder of Indiehackers.

The framework is as follows - any legitimate business idea has three components:

The problem component

This aspect relates to what problem you are trying to solve. How big is the problem? How many people face this problem? How often do they face the problem? What are current solutions?

The product component

This aspect relates to the product you intend to develop. How does it solve the problem? How will the product be used? How will users perceive the product? Does the product solve the root cause of the problem, or simply help relieve symptoms of the problem?

The business component

The aspect we had missed the most when assessing Audiex.

This aspect relates to the commercial side of the product. How will you charge for the product? How much will you charge for the product? What are the costs associated with providing the product to each user? How will you get the product to users at scale?

The three components of a business idea

The former YC partner Kevin Hale has a fantastic video outlining how to evaluate venture-suitable startup ideas.

The truth with Audiex was that the product was interesting and exciting to us personally as a side project to work on, but terrible for trying to turn into a business. There are, of course, a litany of legal issues within the music streaming space. We knew how to reach initial users for the app - as well as curate content from our friends for the app itself - but the long-term business prospects were dire.

You need all three components to weigh in your favour in order to pass the principle that the idea you want to pursue may work out as a legitimate business. Though, of course, it goes without saying that there is no guarantee!  

Our Result:

So we didn’t objectively timebox the validation of our first product idea and we didn’t assess our second idea using the three-component framework. You can guess what happened next.

Our YC Summer 2020 rejection email for Audiex

Audiex was a fun and interesting side project, and had a set of dedicated users. (Some used the app for more than two hours per day!) But it was a terrible idea to pursue as a startup business. We had jumped from one idea to another, with no real process or decision-making system involved. This was our huge mistake.

Now:

Now we just treat Audiex as a fun side project that we tinker with on occasion. We have renamed and rebranded it as Sunset Radio.

We are mainly working on another side project now: Jelly; a gamified maths learning app for primary school kids. We have assessed this idea against the above framework and it seems pretty promising. We will update on how it goes!

Like what you read? Want more?

Subscribe to the mailing list to receive content we publish!

Thanks for joining our newsletter.
Oops! Something went wrong.