5 Flaws with Consumer Panels

Consumer panels are a common data collection method. Businesses gather a group of 10 to 20 people into a room and have them discuss a topic or several topics. They may show a video, ask questions about product features, and brainstorm thoughts on products or needs and more.

But the basic idea is that there is a set, small panel that is prodded to provide their thoughts and opinions by a trained panel leader. It's a useful method of collecting data, but it has its flaws, especially as a standalone data collection method. Some of the disadvantages of using consumer panels include: Small Sample – Without follow up quantitative research, a panel sample is essentially meaningless when it comes to decision making.

It is far too small to have an impact on what is a "good idea" versus a "bad idea" and even if the group overwhelmingly supports a single idea, it may be because someone is persuasively discussing it – not because it's the best idea available or will be represented by the rest of the population. Group Psychology – Group psychology is something that's been mentioned often as a problem with consumer panels. When you put people in a group, creativity tends to stagger and most look to a leader to tell them what to think and do.

Challenging thoughts and opinions is rare, and some people won't even share their ideas because of fear or ridicule. No Creative Thinkers – When you have a small panel, there is no guarantee that someone with intelligence or creative thought processes is even going to be included. While it would ideally be a room full of people that can have great ideas, even those that have been in panels before may be limited in their ability to think creatively.

Data Collection – It's very hard to collect and evaluate any type of data or even the quality of ideas when you have a panel. Two people may both have interesting ideas, but if one person drowns out the other person or the other person is not a persuasive or brave enough speaker, and idea may seem less important. Biases – Both the leader and the panel can introduce biases into the research.

This may be in how a question is asked, how a follow up is asked, body language and more. There is no way to control for these when something is completed in person and with free flowing dialogue. Consumer panels will always be an interesting method of collecting data, and can help you come up with ideas for your quantitative surveys, but it is prone to some disadvantages that make it hard to stand by your data.