Research

Abstract: We propose a prescriptive analytics framework that addresses the needs of a constrained decision-maker facing, ex-ante, unknown costs and benefits of multiple policy levers. The framework is general in nature and can be deployed in any utility maximizing context, public or private. It relies on randomized field experiments for causal inference, machine learning for estimating heterogeneous treatment effects, and the optimization of an integer linear program for converting predictions into decisions. The net result is the discovery of individual-level targeting of policy interventions to maximize overall utility under a budget constraint. The framework is set in the context of the four pillars of analytics and is especially valuable for companies that already have an existing practice of running A/B tests. The key contribution in this work is to develop and operationalize a framework to exploit both within- and between-treatment arm heterogeneity in the utility response function, in order to derive extreme benefits from future (optimized) prescriptions. We demonstrate the value of this framework as compared to benchmark practices–i.e., the use of the average treatment effect as well as uplift modeling–in two different settings. Unlike these standard approaches, our framework is able to recognize, adapt to, and exploit the (potential) presence of different subpopulations that experience varying costs and benefits within a treatment arm, while also exhibiting differential costs and benefits across treatment arms. As a result, we find a targeting strategy that produces an order of magnitude improvement in expected total utility, for the case where significant within- and between-treatment arm heterogeneity exists.

Multi-armed bandits (MAB) are sequential experimentation procedures that use a combination of exploration and exploitation techniques to reduce allocations to interventions with sub-optimal outcomes. MAB’s are very effective in reducing the regret of the experimentation process compared to A/B testing, especially in the presence of multiple policy levers. However, unlike A/B testing, MAB’s may fail to accurately estimate the parameters of treatment effect distributions of interventions. In many Marketing, Clinical Trials, and Public Policy settings, estimating the parameters of treatment effect distributions is as crucial as that of identifying the best intervention, e.g. feedback for intervention designers. In this paper, we propose a new MAB algorithm called UCB-INF that solves the above problem. We show that UCB-INF has regret comparable to the best MAB algorithms while having the parameter estimation properties of A/B testing.

Electronic word of mouth plays an important role in driving purchase decisions. There is enough evidence is literature that claims that referral marketing works [Bapna and Umyarov (2015), Jung et al. (2020)]. In this paper, we investigate if referrals are are better than other customer acquisition approaches like machine learning targeting? If referral effectiveness due to influence, or better identification of potential members? If referral process is cost effective?

Internet changed the way people consume media every day. People read their news online, catch their favorite TV show online, and stay in touch with their friends online. There are many studies that investigate how online media affects online traditional incumbents. However, we do not know how new forms of entertainment that are created and exist only on the internet like Vlogging, Podcasts, eSports affect traditional media consumption. In this paper, we investigate this question in the context of eSports viewing, a new leisure activity afforded by the internet, and exists only on the internet. We study how eSports effect one of the most favorite leisure-time activities of Americans, watching sports on TV. We use panel data on the TV viewership of popular traditional sports and exogenous scheduling of popular eSports watched in the United States in order to estimate the effect that eSports have on traditional sports viewership. Though we find the negative relationship indicating that traditional sports viewership drops on the days' eSports are scheduled, we do not find a statistically significant effect with the data at hand.