Sinks at Scale: Adventures in Behaviour Recognition

We’ve talked for a long time about Video As Data, and the opportunities for brands in accessing high quality, passively observed behaviour at scale. We’ve observed, counted and analysed wine spills, breakfasts, smart device activations, fixture visits, mopping, hair straightening and toilet visits (yes, really). 

Our industry-leading tagging system has helped our analysts process 1000s of hours of video quickly and accurately. But although we’ve dabbled, we’ve struggled with automated behaviour recognition: too many camera angles, too much variation, too much context and nuance. 

And – as you’ve heard us say a lot – we’ve always been concerned that while most automation of visual data extraction may look super shiny, it hasn't delivered enough value against client problems. It only worked in a narrow range of non-real world use cases and/or generated pretty poor quality data (despite what you may hear or read about supposedly great AIs!), which then defeats the point of working with the best quality data source available in consumer research.

The New World

But, things are changing. In the last few months, we’ve integrated with IBM Watson and other NLP tools as we see them start to match human analyst classifications; we've built our own people detection API that's working with 95% accuracy and saving us hundreds of hours on observational projects; and we've now successfully deployed behaviour recognition in a live study.

(more on that exciting development another time!)

This is a big step. 

Our automated plug’n’play fixed camera solution streamed video to us for consumers’ homes in real time across a month. Existing models combined with client, project and household specific overlays to produce accurate 2,600 detections of dishwashing behaviour out of 27,000 files.

We still used our expert content analysts to nail the fine details of product usage, context and outcomes. And we ended up with a rich, qual-quant data set: searchable, sliceable, scaleable. 

We’re lining up the next set of projects and behaviours with the client at the moment. Needless to say, as much as we loved scrubbing 1000s of hours of video; we’re excited.

For the full case study, drop us a line at or contact your account manager. (Just don’t ask us to count any more toilet trips - please.)