Emil Pawłowski on the measurement of OOH in an interview with Screenlovers

Full, Polish version of the interview you can find here.

From this November, all advertising agencies will square up according to the new currency, extended by Gemius research. The change surprised the agencies and advertisers. The multi-threaded discussion about it is driven mainly by unintuitive Out-of-home viewership results, but also by the future shape of single-source media measurement or the future of Nielsen company. Today we are talking about the controversy of the OOH audience measurement – but not only about this topic - with Emil Pawłowski, Chief Strategy Officer at Gemius.

- Screenlovers Blog

Emil Pawłowski, Chief Strategy Officer at Gemius

ScreenLovers: Some believe that recruitment to the Gemius panel is the source of the non always intuitive OOH Viewing results. It is said that initially you were looking for people willing for free smartphones, and the specificity of the respondents obtained this way affects the provided data. Is it so?

Emil Pawłowski, Gemius: A few years ago, we created a device, originally designed to measure the Internet, mainly Facebook and Youtube mobile applications. These were specially prepared smartphones with our proprietary measuring software. At that time, we conducted tests to check whether such a measure would be accepted by the respondents. The tests lasted two years. At that time, we recruited people using the CAWI method.

In parallel with the tests, we developed the measurement of radio and television, as well as the location of research participants. The aim was to provide data for these media, including the OOH television viewership, and in the future - the panelists' contact with outdoor advertising during their presence in stores. The tests finished, and about 600 people took part in them for two years. Currently, there is not a single panelist in the study who would participate in those previous tests.

ScreenLovers: So what does recruitment look like today?

E.P.: Most of the current sample is recruited using the CATI method, in collaboration with certified research agencies. The current wave of recruitment is carried out by using the face to face method. By the end of the year, approximately 1/3 of our panel will be recruited by using this method. We are signing a two-year contract with each panelist. After two years we are extending our cooperation with 15% of them, of course if they meet our quality criteria.

ScreenLovers: How many panellists research include?

E.P.: About 2100. We plan to have 2500 panellists by the end of the year.

ScreenLovers: What are the further plans on this field, e.g. by the end of next year?

E.P.: It depends on the results of the current wave of recruitment, on costs and on the needs of the market, which has just received a new impulse and is wondering what to do about it. I can declare that number of our panellists will not be less than 2500. And what - and how many - more? We are waiting for feedback from market organizations such as MOC and our customers.

ScreenLovers: From what you say, it would appear that it is not the method of recruiting to the panel that affects the anomalies which the market is discussing today, i.e. the phenomenon of music channels that record greater reach outside the home than at home. The famous Eska Rock station turns out to be a secret market giant.

E.P.: We test various types of hypotheses to see if there is any hidden error in the research or methodology that affects the phenomenon of music channels. So far, however, we did not find any irregularities. So I am now assuming that the panelists' behavioral schemes are simply outside-home viewing of music stations.

I think it is worth reflecting on your own beliefs about how viewership should look like, based on the schemes identified in the Nielsen study. Gemius's data sometimes follow them, and sometimes show different behaviors. I don't think that delivering counterintuitive results is a downside to our project - it's more an advantage. Otherwise, we could just add 10% to the Nielsen stats for each station and finish the topic.

ScreenLovers: With regard to music stations, there have been theories that it may be a matter of the audiomatching method using by Gemius to recognize the TV signal. Nielsen TV research is using the same method, but here we have two filters along the way: the Respondent has to click on the remote control, that he or she is watching and the TV must be turned on.

E.P.: Let us make it clear: only the turned on TV is this kind of filter. Clicking the remote control is an additional burden on the respondent. We do not require this from our panelists - we are using a personal device.

Coming back to the question - of course, we tested this intuitive hypothesis that music stations can adapt to other music sources than TV. And since such a phenomenon can actually occur, the data we provide to Nielsen only includes live away from home viewership. We do not provide data with a time shift. So when it comes to catching fragments of songs from the radio, Youtube and all other sources, the possibility that someone is listening to the same song, playing at the same time at for example - Eska Rock, is close to zero.

ScreenLovers: What about spikes in viewership at night? Is the fact that a large channel can generate several hundred percent GRP less in given time at home than out from home, does not point that you should look for a mistake in a research or fusion?

E.P.: We also looked at this issue. And we did not find any reasons to question the accuracy of the data yet. I think that one thing worth considered is the specificity of watching TV outside the home, which many people stereotypically perceive as watching a match in the fan zone. This is a much broader topic.

OOH is not only about watching outside, at work, in the pub. It is also watching in a second home, in a dormitory or a hostel. Collective living quarters are not included in surveys based on household measurements, so standard telemetry measurements will not capture any TV viewership there, although it does exist. Paradoxically, the examined person can watch more television at night, when he is alone outside the home.

Gemius analyze behaviour of people equipped with personal measures and assigns them viewership regardless of where they are. When analyzing OOH data, we should put this in mind and be open to new behavior schemes.

ScreenLovers: A few years ago, Gemius considered various definitions of Out-of-home audience viewership. Which of them and why this one did you choose?

E.P.: When creating Mediapanel in Poland, we wanted to supplement the scope of the research with the part that had not been available here so far. In the case of OOH television, our goal was to estimate its popularity at the bottom. Whenever we suspected that there might be a misclassification error, implying that someone was away from home despite being at home, we further sharpened the definition.

Let me give you an example: if our measurer did not provide us with information about the location in a given period, we assume that the panellist is at home. With each location measurement, our device also provides us with information about the possible margin of error, which we include "in favor" of viewership at home. This allows to minimalize the phenomenon of false OOH audience counting.

As for the definition itself - we consider a person to be away from home if they are more than 100 meters away from it, taking under consider the mentioned before measurement error. If panellist is 99 meters from home and watches TV there, Gemius will not report it as OOH viewership. It seems to us that this is a very conservative classification model.

ScreenLovers: Is it taken from other countries?

E.P.: No, this is a Gemius proprietary solution.

ScreenLovers: The more the OOH viewership seems to be high. Didn't your own data surprise you?

E.P.: They surprised us. At first, I did not believe in them myself. Following the intuition derived from the guest viewing research, i.e. part of the TV audience survey, I expected a share of 1 percent, maybe two. And here suddenly we have 7, 9, and on some days even 13 percent. However, the longer we conducted this study, the more we realized that it was not a measurement error, but our cognitive bias. Daniel Kahneman bows here.

ScreenLovers: Why the assumption that this is a cognitive bias?

E.P.: Let us return to the assumptions of the OOH measurement. It includes work - we spend 8 hours there - going out to family, friends, etc. If we consider in how many moments of our day we have a chance to watch TV outside the home, then suddenly this 10% share may seem modest. Anyway, this may be the result of the conservative definition of OOH viewership, which I talked about earlier, and on which we will be working next year.

As our conversation proves, we rightly created a bottom-estimate model for Out-of-home TV viewing. So we started with a conservative definition of OOH viewership, but I believe that it is time to make this model real together with experts, i.e. our clients, at the same time avoiding the mistake of false classification. If we prove that the appropriate limit will be, e.g. 50 meters, not 100 meters, we will think about the change.

ScreenLovers: You are talking about consulting topic with clients. Were there such consultations and talks before imposing new billing data on the market?

E.P.: Nielsen has implemented a data fusion project commissioned by Polsat Media. Gemius was a subcontractor providing some of the data from the Mediapanel study in this case. When it comes to the communication of the fusion project, we were not the subject here, as we are with Mediapanel, about which we very willingly and openly discuss with the market, both at industry conferences and meetings in organizations, i.e. MOC Tv, SAR or KMdLB.

ScreenLovers: So the issue of a possible audit was also not in your hands?

E.P.: Gemius's CEO, Marcin Pery, has been declaring his willingness to submit the study to an audit for at least two years. However, we need an organization that will commission such an audit. We are more than open to such an initiative from MOC Tv or any other industry organization. However, the decision must be made by the recipients of our study.

ScreenLovers: You probably do not want to comment on why Polsat did not take care of the audit issue?

E.P.: This is a question to Polsat. However, I can share my personal thoughts. Many of our clients, who buy data from, for example, the AdReal - well-received by the market - which also come from a single-source study, decided that they do not need an audit at the moment. I think Polsat also got to know our product well. Personally, I have repeatedly provided details of Gemius's research at meetings attended by researchers from Polsat Media. It should also be remembered that our method has been thoroughly checked by PBI and KBR experts, the commissioners of the Gemius study.

ScreenLovers: What are your business plans for the next 5 years? Replacing Nielsen as the TV research and data provider is the basic one?

E.P.: Let me answer the simpler part of the question first - no, we do not want to replace Nielsen in a standardized, worldwide model of telemetry research. Dot. Our goal is to offer the market a new quality and supplement our clients' knowledge of those areas that are not covered by other data sources. OOH viewership is a good example. This is not a competitive project, but complementary to Nielsen's data. Similarly to the issue of radio audience among TV viewers, viewership of ads on mobile Facebook or contact with external advertising or e-commerce data.

ScreenLovers: And for the next 5 years you just want to suplement knowledge this way? Modest.

E.P.: Today we measure radio, Internet, television, including its OOH audience and the popularity of advertisements on Walled Gardens mobile platforms. So the question arises: what complements what? Assuming that an average person sleeps 8 hours per day, the time spent watching TV accounts for 25% its active time, assuming the ATV is just over 4 hours. That's a lot, but for the advertiser the remaining 75% time is equally important, and indeed more difficult to study. During this time, Poles listen to the radio - on average 2.2 hours a day, use the Internet - 2.3 hours, go out, go to the cinema and, above all, do shopping. This means that television is still an important but no longer dominant medium. Gemius by providing Mediapanel and AdReal allows you to see a combined TV, Internet and radio audience.

Summing up, we focus on the research covering the widest possible scope of consumer behavior, in the digital sphere and beyond. In my opinion, the goal we have set ourselves is not modest.

ScreenLovers: Mediapanel study, despite the fact that perhaps some of the agencies and clients were aware of it, has not been very popular in recent years. If Polsat would not make its brutal performance, the situation would probably still looks this way. Market inertia or other considerations?

E.P.: This is a wrong observation. The Mediapanel study is selling itself. After all, our measurement is not only about OOH viewership - it covers the entire Internet and radio, mobile platforms, etc. Data about them is already widely used to evaluate 'post' campaigns.

Indeed, cross-media data has not been widely used for campaign planning, especially in traditional media. But their implementation is not an overnight process, we have to be patient in this case. Now the first step has been made. But the market has to learn, i.e. it doesn't have to ...

ScreenLovers: Let's be honest - it must.

E.P.: …learn how to use this information in the planning. This may take many months.

Joanna Nowakowska interviewed