TV3: The Algorithm Game Show
2015.07.07The idea of a game-show built around survey data isn’t an altogether unfamiliar concept; most of us have seen Family Fortunes or 8 out of 10 cats and wondered who did these surveys! As researchers of course we probably wished it was us. So when Soshefeigh Productions and TV3 approached us with a game-show concept that included a research component we were naturally excited. However this was not to be a simple survey akin to the aforementioned show formats, it needed to be something a lot more sophisticated than that.
“The Algorithm” is a game-show format that pits a group of contestant’s loved-ones against a statistical algorithm, to see whether they can better-predict the answers the contestant gave to a variety of interesting questions. The contestant, who is the main player, provides their answers to a question in private, unseen by their group of four friends. The friends then try to rationalise and form an opinion regarding the answer they expect that their loved one would give. Meanwhile the “Algorithm” is crunching the numbers on what it knows about the contestant; demographic details, height, eye colour, hair colour and a range of screening data that was collected when casting the contestant to form its prediction on how the contestant would answer this question. If the Algorithm is correct and the family members are wrong, the contestant loses out. If the family members are right and the Algorithm is wrong in its prediction, the contestant wins and moves on. If they are both either right or wrong another question is posed. This is a money game, so the stakes are high.
The foundation for this show is powerful statistical algorithms that can predict answers to a host of questions using only the screening information provided during casting. The excellence in this piece of work lies in its scale and in the development of these statistical models; and the success of the outcome is evidenced by the show being the second most sought-after format at this year’s Global Cannes MIPFormat event and in the excellent response from both local and global production teams. The first series has been shot and will be aired in Ireland in January.
A large-scale online survey was undertaken with n=2,680 adults across Ireland, each of whom completed a 20 minute survey that featured two key elements:
- An “inventory” of demographic, attitudinal and behavioural questions, and
- A randomised selection of fun “gameplay” questions
In total there were 306 “gameplay” questions; questions that would be asked as part of the show itself. Each of these questions had three answer options: A, B or C. This made the questions snappy to complete and each were sufficiently interesting to retain respondent’s attention. A rotation system was used so each respondent answered no more than 70 such questions delivering a total sample for each gameplay question of approximately n=600.
As there were 306 potential questions within the game-show for which predictions were required. RED C produced 306 individual predictive models; one for each question. Each of these needed to be built, validated, tested and deployed. To do this we used two classification modelling techniques, CHAID (Chi Squared Automatic Interaction Detection) and C&RT (Classification and Regression Trees). These models allow us to understand:
- Which of the “inventory” items are most predictive,
- How accurately we can predict the answer to gameplay questions based on knowing the “inventory” items
The resulting models included a set of classification rules. These rules can then be used to produce predicted answers to any of the 306 gameplay questions based on an individual’s responses to the inventory of classification and behavioural questions. The resulting predictions are considerably better in accuracy than guesswork would produce.
The average accuracy of prediction across these questions was 64%, with individual questions ranging from 48% to over 90% accuracy. When one compares this to the 33% accuracy that a pure guess would produce it is an impressive result. Also, from a game-show perspective this level of accuracy is ideal as there is some room for error, which is required for the contestant to have a chance of winning some money.
We delivered the specific predictions for each of the contestants selected from the casting process; 24 in total. This allowed the production team to select the questions to be asked of each individual and develop the show to maximise the entertainment value and deliver a reasonable chance of winning for the contestant.
