MathsGee Answers is Zero-Rated (You do not need data to access) on: Telkom | Dimension Data | Rain | MWEB
First time here? Checkout the FAQs!
x

ASK - ANSWER - COMMENT - VOTE - DONATE


Network: Global | Joburg Libraries | MOOCs | StartUpTribe | Zimbabwe | Donate

MathsGee is Zero-Rated (You do not need data to access) on: Telkom |Dimension Data | Rain | MWEB

0 like 0 dislike
46 views
Why are A/B test experiments impossible at times?
in Data Science by Diamond (67,406 points) | 46 views

1 Answer

0 like 0 dislike

Sometimes it’s just not possible to set up an experiment. Here are a few reasons why A/B tests won’t work in every situation:

  • Lack of tooling. For example, if your code can’t be modified in certain parts of the product.
  • Lack of time to implement the experiment.
  • Ethical concerns  for example, at an e-commerce company like Shopify, randomly leaving some merchants out of a new feature that could help them with their business is sometimes not an option).
  • Just plain oversight (for example, a request to study the data from a launch that happened in the past).

Fortunately, if you find yourself in one of the above situations, there are methods that exist  which enable you to obtain causal estimates.

A quasi-experiment  is an experiment where your treatment and control group are divided by a natural process that isn’t truly random, but are considered close enough to compute estimates. Quasi-experiments frequently occur in product companies, for example, when a feature rollout happens at different dates in different countries, or if eligibility for a new feature is dependent on the behaviour of other features (like in the case of a deprecation). In order to compute causal estimates when the control group is divided using a non-random criterion, you’ll use different methods that correspond to different assumptions on how “close” you are to the random situation.

by Diamond (67,406 points)

MathsGee Answers is a global, STEM-focused Q&A platform where you can ask people from all over the world educational questions for improved outcomes.



On MathsGee Answers, you can:


1. Ask questions
2. Answer questions
3. Comment on Answers
4. Vote on Questions and Answers
5. Donate to your favourite users

Registered Members Online
MathsGee Tools

Math Worksheet Generator

Math Algebra Solver

Trigonometry Simulations

Vectors Simulations

Matrix Arithmetic Simulations

Matrix Transformations Simulations

Quadratic Equations Simulations

Probability & Statistics Simulations

PHET Simulations

Visual Statistics

Interactive Courseware

ZeroEd Search Engine

Article Rewriter Tool

Word Counter Tool

Other Tools

MathsGee TV |ZOOM | Slack | eBook