Technical on Sandeep Gangarapu

Essays on Causal Inference

Sat, 29 Oct 2022 15:30:00 -0600

The fundamental problem of causal inference

You probably know the COVID-19 pandemic and the race between all the pharma companies to create a vaccine. There was a lot of talk about different phases of clinical trials and the effectiveness of various vaccines. In clinical trials, companies are trying to estimate the causal effect of giving a drug on patient’s health.

Causal effect is the difference between what happened if the treatment was given and what would have happened if the treatment was not given

Under null hypothesis, p-values are uniformly distributed

Thu, 06 Oct 2022 16:44:00 -0600

Central limit theorem states that sample means follow normal distribution. People often confuse this with and question the validity of uniform distribution of p-values under nul hypothesis. Should we not be observing extreme p-values much less ofter than usual pvalues like 0.2 or 0.3?

Imagine if null hypothesis is true. This means there is no difference betweem treatment and control.

import numpy as np
import pandas as pd
import statsmodels.api as sm
from scipy import stats
import matplotlib.pyplot as plt

pval_lis = []
t_stat_lis = []
samp_mean_lis = []

for i in range(100000):
    sample = np.random.normal(0, 1, 10)
    samp_mean = sample.mean()
    samp_mean_lis.append(samp_mean)
    t_stat = sample.mean()/(sample.std()/np.sqrt(sample.size))
    t_stat_lis.append(t_stat)
    pval = stats.t.sf(abs(t_stat), df=sample.size)*2
    pval_lis.append(pval)

plt.hist(np.array(samp_mean_lis))

Distribution of sample means

Practical significance is the North Star

Wed, 03 Mar 2021 06:53:00 -0600

Practical significance is the North Star. 🌟

Why experiment?

At the end of the day we want to know if treatment is better than control and whether changing the status quo is worth it. That is why we do experimentation or derive inference from observational data. Being better is not enough to take a practical decision. Is it worth changing the status quo? Both go in conjunction. That’s where practical significance comes in.

A layman's explanation of Law of Large numbers

Thu, 17 Dec 2020 17:50:00 -0600

The first step is to nail their intuition for randomness. Here’s how I will do it.

I will ask them if I flip a coin once, will it be heads or tails. The answer would be It will be heads! or It can be either! (smart kid). If it is former, I can prove them wrong in 1-3 tries and lay the path for explaining randomness. If it is later, they already have some intuition for randomness.

Who and when should one use Multi-Armed Bandits? A scenario.

Sun, 11 Oct 2020 23:13:00 -0600

💡 This is a humor piece. I am using the scenario to communicate the use of MAB’s. Please take it with a pinch of salt.

A manager at a tech company that sells Alphonso mangoes online is stuck with one of the most difficult decisions. What should the color of the “Buy Now” button be? She recently came to know that data-driven decision making is the new fad and wants to follow suit. She has a meeting with her team of data scientists and marketing folks.

Thompson Sampling Algorithm for Normal outcome distribution

Thu, 10 Sep 2020 21:46:00 -0600

Thompson Sampling is one of the most popular Multi-Armed bandit (MAB) algorithms - the main reason being its explainability (imagine explaining upper confidence bound to your manager) and decent performance in practice [1].

Many blog posts on the Internet show how to implement Thompson sampling (here, here, here and here). Almost all of them consider Bernoulli outcome distribution (e.g. click or no click, purchase, or no purchase) and use the Beta-Bernoulli Bayesian update procedure for simulations and usually compare the performance (Regret) to other MAB algorithms like UCB or sometimes to A/B testing. However, none of them consider Gaussian outcome distribution and especially the case when both mean and variance of the distributions are unknown (which is usually the case when you are conducting an experiment where the outcome is continuous, ex: dollars spent, time spent, etc.).

Learning Resources

Tue, 18 Aug 2020 00:00:00 -0600

Over the years, I have come across multiple resources I learned things from with ease. This is a page to track those resources. Please reach out to me if you think something belongs to this list. I will add it.

General

Julia Evans has online zines for topics ranging from SQL, Shell scripts to Linux. These are hand-drawn explanations of stuff related to a particular topic. I stumbled upon here twitter post about how SQL actually executes a query and have been a fan ever since.