Get Help From Experts in Data Science and Mathematics
ePortfolios |STEM Gender Equality | ZOOM | Slack | Spreading Mathematical Happiness

You can support MathsGee with your DONATION

1 like 0 dislike

What two major issues do users of the Uber Experimentation platform watch for when doing A/B experiments?

in Data Science by anonymous
edited | 67 views

1 Answer

1 like 0 dislike
Best answer


After gathering data, their XP’s analytic platform validates the data and detects two major issues for experimenters to watch for and to keep a healthy skepticism in their A/B experiments:

  • Sample size imbalance, meaning that the sample size ratio in the control and treatment groups is significantly different from what was expected. In these scenarios, experimenters must double check their randomization mechanisms.
  • Flickers, which refers to users that have switched between control and treatment groups. For example, a rider purchases a new Android cell phone to replace an old iPhone, while the treatment of the experiment was only configured for iOS. The rider would switch from the treatment group to the control group. Existence of such users might contaminate the experiment results, so we would exclude these users (flickers) in our analyses.
by Wooden (1,460 points)

Related questions

1 like 0 dislike
1 answer
0 like 0 dislike
1 answer
1 like 0 dislike
1 answer

Welcome to MathsGee Q&A Bank, Africa’s largest personalized FREE Study Help network that helps people find answers to problems and connect with experts for improved outcomes.

Get help from experts - simply ask your question

You can Support MathsGee with your DONATION

Enter your email address:

13,102 questions
10,327 answers
11,179 users