In my early days as a researcher I studied statistics quite deep. Although I am trained as a qualitative researcher I wanted to understand the quantitative view as well. Since I had the exact variant’ in highschool, I had enough maths background to appreciate this stuff and even to like it. I recall the distinction between ‘alpha’ and ‘beta’ mistakes. As I recall it, alpha mistake meant that you had rejected the hypothesis where shouldn’t have rejected it. The beta mistake was the other way round: you rejected it but you shouldn’t have. It depends on the type of hypothesis which mistake is worse. You could use the alpha and beta faults in qual as well. A beta mistake would occur for in stance if you reject an idea on basis of your qualitative findings, that would actually have worked well in reality. This is the sort of crime that research accused of by idea makers: research kills ideas.
I have been given a hilarious video about a qualitative example of a ‘beta mistake’: a focus group to find out the attitude towards a new invention (the wheel). The research method seemed to have been designed in order to ‘falsify’ the new invention. The video is a must see. It is also a very strong stereotypical story of How Research Kills New Ideas. It sort of demonetises the focus group method, showing how a researcher COULD go completely astray by encouraging the wrong kind of rationalisations. Click on the link to see it if you want to have laugh.
It could be used as an instruction video, asking students to pinpoint the Four Major Mistakes.The biggest mistake is setting up a focus group about this as an idea. Innovation can only be discussed in a focus group if it is incremental innovation as opposed to rule breaking innovation. The second mistake is the test material. If you would have rule breaking innnovation, you would need to show context and future use situations. Third is the recruitment. If you have a rule breaking innovation and you would like to discuss it in you shouldn’t invite the average consumer. You recruit either consumers who are into innovation or even ’semi-experts’. The fourth mistake is the moderation. The moderator gives plenty of room for post rationalisations. The human brain works counter intuitive. We think that our thoughts and opinions precede our choices. In reality it is reverse: we choose and than create opinions according to the choice. The reversed timing can be monitored: they occor as recurrent patterns in the brain after the choice has been made. It is therefor a big mistake to dive to deep into opinions and beliefs, especially in innovation, because there is a tendency in the human brain to dislike the new. A logo is liked more as it has been seen more often. This is referred to as the ‘mere-exposure’ effect. Since innovative stuff can never have been ’seen often’ it is disadvantaged in a test situation (and in the real situation as a matter of fact).
Tags: innovation
Jochum, do you mean alpha is accepting the hypothesis when you shouldn’t have, and beta is rejecting the hypothesis when you should have?
regards, John
[...] Doesn’t the consumer stick with ‘the old rules’? Here we see the old fear as described in my ‘Alfa or beta’ blog. The consumer as destroyer of ideas. And the researcher as the facilitator of this mass idea [...]