Feedback vs. insights: A practical example for building a successful MVP

Feedback is a loaded word. Asking for it in the wrong way will not only result in poor information, but may also actually hurt your business. More often than not, it’s predicated on opinions and not actual facts. Bad feedback sends you in wrong directions and provides false positives and false negatives.

So, how does one get valuable, actionable input from users, potential customers, and stakeholders? Focus on insights, not feedback.

Understanding the problem

Filtered.ai is a Techstars startup led by CEO Paul Bilodeau and Derek Bugley, whom I had the good fortune of mentoring as part of the 2018 Techstars Anywhere cohort.

Filtered started as a data science consulting firm. Seeking to hire software developers, they received hundreds of unqualified resumes from recruiters. In response, they developed a simple coding test to be used as a filter.

Their MVP immediately paid dividends. Recruiters could only send resumes of engineers who achieved a specific score on the test, dramatically cutting the time and cost of recruiting new developers.

Unbeknownst to the consulting firm, some savvy recruiters started using the tool with their large enterprise clients. These corporations are often stuck in decades-old HR recruiting systems and processes that cannot take advantage of modern technology.

One particular Fortune 500 chemicals manufacturer recruiting software engineers had hiring results that looked like this:

• Out of 10,000 phone screens, they whittled down to 1,627 final coding interviews, experienced 23 percent rejected offers, and completed the hiring for only 4 percent (the industry standard is 30 percent).
• The average time to fill vacancies was 126 days.
• The cost of each hire was greater than US$40,000.

This particular company asked Paul and his team to come in to discuss how their new technology might stop the bleeding.

Differentiating opinions from insights

While Filtered’s MVP solved their own small-scale problem, what did the MVP look like for a behemoth enterprise locked into a decades-old applicant tracking system and anachronistic HR practices? How did anyone figure that out, at the startup or enterprise level?

Here are some possible tactics, both at the surface level and in greater depth:

• Request feature input using online tools such as UserVoice
• Send a demo or provide a trial to an MVP and ask for feedback with a survey
• Pitch the solution by phone or in person and see if potential customer likes the idea
• Ask potential customers by phone or in person what they’d like to see in a modern recruiting software system
• Ask potential customers what their existing process looks like
• Ask potential users what they dislike about existing systems
• Interview other stakeholders, such as hiring managers, about existing systems, processes, and outcomes
• Go hang out on site and observe the systems and processes at work
• Run experiments that measure behavior indicating whether or not user receives value

Filtered used a combination of these tactics, but with the key objective of discovering customer insights. Insights are the competitive advantage, not feedback. Feedback can only be given in response to something you do, so it limits the context and your universe of potential insight.

Paul and his team shadowed the HR people who brought them in, and observed problems in the process:

• It took five days for the HR to even look at a resume after it was uploaded into the applicant tracking system. It took another five days to schedule a phone call.
• The HR asked technical interview questions they didn’t understand, like Javascript programming questions.
• Qualification questions boiled down to “why do you want to work here?”

Building for the end user

The results