Why You Shouldn’t Survey Your Own Invention

Inventors and companies of every size need an outside view.
(Understanding the problems of DIY can help you overcome them)

In 1988, R.J. Reynolds test-marketed Premier, a new kind of cigarette that delivered nicotine without conventional tobacco combustion. The idea was compelling: smokers could get a cigarette-like experience with less smoke and odor. R.J. Reynolds spent hundreds of millions of dollars developing it. The company's own consumer research had already identified serious problems before launch. They launched anyway. Consumers disliked the taste and found the experience unsatisfying. Even the familiar cloud of exhaled smoke was largely missing. Premier was withdrawn within months.

Yet the concept wasn't wrong. Years later, vaping showed that millions of smokers wanted an alternative to conventional cigarettes. Vaping even brought back the visible exhale. Premier failed because the product didn't deliver an experience consumers wanted. A good consumer evaluation can help distinguish between a bad concept and bad execution. That difference matters, because one may mean walking away while the other means fixing the product.

Premier also shows a basic danger in product development. When people have invested heavily in an idea, it becomes very hard for them to judge that idea objectively.

The Problem With Surveying Your Own Invention

Inventors are optimists. They have to be. But that optimism becomes a liability when it's time to evaluate commercial potential. An inventor who has spent years on a product wants consumers to like it. A company that has invested millions wants research that justifies moving forward. Neither one has to deliberately manipulate a survey for bias to creep in. It enters through how the product is presented, which questions get asked, who gets selected to answer, and how the results are interpreted.

Consider two questions. "Would you be interested in a revolutionary cigarette that substantially reduces smoke and odor?" Or: "Would you buy a cigarette that costs more than your usual brand, tastes different, and produces little visible vapor when you exhale?" Both describe the same product, but they could produce dramatically different answers. The first sells the concept. The second explores the actual experience. Someone with a stake in success will naturally lean toward the first, often without realizing it.

A Good Survey Begins Before the First Question

A survey starts with an accurate presentation of the product. The goal isn't to make the invention look as appealing as possible. It's to set an accurate expectation of what a customer will experience after buying it. Images, video, and descriptions should communicate meaningful limitations as well as benefits.

Think about the Amazon reviews a product might get six months after launch: "Bigger than I expected." "Too complicated." "Doesn't work as well as advertised." "Not worth the price." A good survey anticipates those reactions before the product reaches the market. If images and video can't adequately communicate the real experience, hands-on testing may be necessary.

Asking the Right Questions

People are generous with compliments when they aren't spending money. "Great idea!" doesn't mean "I'll buy it." A useful survey explores whether consumers actually have the problem, what they've already spent on solutions, how the invention compares with alternatives, and whether they would buy it at a realistic price.

The results also require skepticism. Did respondents spend enough time to understand the product? Do their stated purchase intentions match the value they assign to it? Are the enthusiastic respondents actually representative of potential customers? An independent researcher has to ask these questions even when the answers are disappointing. Interpreting survey results is a topic for another post.

AI Can Make the Problem Worse

AI makes it easier than ever for inventors to create professional-looking surveys. But it doesn't remove bias, and it can make the problem worse. AI assistants are designed to be helpful and responsive to their users. When an inventor enthusiastically describes an invention and asks AI to build a survey, the resulting questions can reflect that enthusiasm rather than challenge it. A biased survey doesn't become unbiased because AI wrote it. If you use AI, try asking it to find every reason the product might fail, then make sure the survey addresses those concerns.

The Value of Independence

We should be upfront: Invention City sells surveys, so we have a stake in this argument. Here's what that stake looks like in practice. We recently ran a survey for an inventor that showed respondents valued his product at about half his target retail price. There was no way to reduce the cost to what the market wanted and still make a profit. So he decided not to proceed, avoiding more than $100,000 in planned tooling, inventory, and marketing expenses on a product that appeared unlikely to succeed.

We regularly encounter inventors who conducted their own surveys, received encouraging results, and then struggled to sell their products. Their surveys told them what they wanted to hear, rather than what they needed to know.

Independent surveys can also be valuable when seeking a licensing deal. Companies hear enthusiastic claims from inventors all the time. What they rarely see is credible evidence that consumers actually want the product. A well-designed survey can demonstrate demand, identify the most promising customers, establish acceptable pricing, and show how the invention compares with existing alternatives. Positive results from an independent survey carry more weight than an inventor's own claims and can help persuade a prospective licensee that an invention deserves serious consideration.

Hiring an outside firm doesn't guarantee good research. Independence has to be combined with competence and experience, appropriate respondent selection, unbiased questions, and objective analysis. The researcher's job isn't to prove that an invention will succeed. It's to find out whether there's credible evidence that it might.

For an inventor about to invest in patents, tooling, manufacturing, and marketing, an independent survey costs a fraction of what's at risk. It may reveal an opportunity, suggest improvements, or prevent an expensive mistake.

The most valuable survey is one that gets closest to the truth. That's hard to do on your own.

Learn more about surveys here: https://www.inventioncity.com/survey