Ep.133 Finding ground truth: measuring & testing incrementality

For this episode our guest is Brian Krebs, the CEO of MetricWorks, an expert on incrementality measurement and testing. Brian discusses how the use of incrementality has evolved to its present popularity within the mobile marketing industry and the confluence of issues that make it complex to test incremental lift. Brain says Mixed Media Modeling is the way to fill in the gaps with attribution, but it must be validated and calibrated with geo level ground truth experimentation. Find out why and how this works.

MetricWorks is a third-party solution for measuring incremental lift. Their solution only uses aggregate data points impervious to privacy changes, making it “future proof.” MetricWorks also provides LTV predictions and full UA automation, driven by machine learning.

Listen on Spotify

Listen on Apple Podcasts

Listen on Google Podcasts

• What drives your entrepreneurial spirit?
• How has MetricWorks changed over the years and what is your focus today?
• How have you seen the evolution of incrementality measurement?
• What’s the process of working with MetricWorks to conduct an incremental lift study?
• Do you have a solution for apps that are only based in one country?
• Who stands to gain the most from incrementality measurement? What channels have been undervalued?

• 2:45 Brian’s background
• 7:20 About MetricWorks
• 13:40 The evolution of incrementality
• 15:55 Confluence of issues with testing incrementality
• 19:05 Mixed Media Modeling: strengths and weaknesses
• 20:05 MetricWorks’s incrementality measurement solution
• 24:54 Ground truth incrementality testing
• 28:26 Solutions for apps running in one country only
• 33:33 What channels are being undervalued

(18:47-19:02) “Ghost bidding is just a magical solution. And not only is it really hurt by losing the device id, but as a marketer you’re finding yourself trusting the vendor. The vendor is necessary to do ghost bidding or ghost ads.”

(19:10-19:36) “[MMM] is complex but it’s simple in fundamental understanding of it, and, completely privacy compatible – doesn’t require any device-level data. And, there’s no need to get rid of incrementality testing. You have to go to a geo level, which is what we do. But, you can still use those signals to calibrate the MMM over time, which is probably MMM’s biggest weakness: not having some truth to ground in.”