Miniclip Verifies Remerge's Randomization Method

Miniclip develops, publishes, and distributes highly engaging games to an audience of 200 million monthly active users, across mobile, social and online platforms. Founded in 2001, the company has successfully grown an organic global audience with over 2 billion downloads.

Miniclip ran retargeting campaigns for their mobile game 8 Ball Pool to re-engage lapsed users and increase the conversion rate of their in-app offerings. Miniclip was also interested in running an incrementality test to accurately measure the ROI of retargeting, on top of other marketing activities.

When running an incrementality test, it is incredibly important to ensure that the test and control groups are balanced, as this is the only way to guarantee that the results of the test will be meaningful.

After running their first incrementality test with us, Miniclip was keen to do a deep dive into our methodology and analyze the results of the test as well as the group balance. They compared the data provided by Remerge (which outlined the randomization along with which users were in which group) with their own internal behavioral user data (which Remerge had no access to nor awareness of).

To ensure that the groups were truly randomized, Miniclip applied Distribution Matching, an advanced technique used to verify the balancing of both test and control groups based on secondary attributes. In this case, parameters included activity, matches played, player level, life-time-value, and retention.

We were delighted when Miniclip informed us that, after a rigorous analysis, our groups were perfectly balanced on all KPIs.

What you can see below is not one graph - it’s two graphs overlapping perfectly, indicating a perfectly balanced control and test group.