September 23 2021, 1pm Paris time / 7am NY time
Duality and Estimation of Undiscounted Markov Decision Processes
Abstract: This paper studies estimation of undiscounted Markov decision processes (MDPs). Exploiting convex analytic methods, it argues that undiscounted MDPs can be treated as static discrete choice models over state-action frequencies, leveraging this idea to derive a conjugate duality framework for studying this type of models. It then exploits this framework to draw implications in several dimensions. First, it characterizes the empirical content of undiscounted MDPs, analyzing how exclusion or parametric restrictions can produce identification of agents’ payoffs, and providing an axiomatic characterization of the undiscounted dynamic logit model; second, it proves convergence of simple inversion algorithms based on progressive Tâtonnements, and investigates novel estimation strategies based on these. Finally, it shows that the dual framework extends to models with persistent fixed effects and to models where certain actions or states are unobserved.
December 2 2021, 5pm Paris time / 11am NY time
Sorting with Team Formation
(with Aleh Tsyvinski and Alexander P. Zimin)
Abstract: We fully solve an assignment problem with heterogeneous firms and multiple heterogeneous workers whose skills are imperfect substitutes, that is, when production is submodular. We show that sorting is neither positive nor negative and is characterized sufficiently by two regions. In the first region, mediocre firms sort with mediocre workers and coworkers such that output losses are equal across all these pairings (complete mixing). In the second region, high skill workers sort with a low skill coworker and a high productivity firm, while high productivity firms employ a low skill worker and a high skill coworker (pairwise countermonotonicity). The equilibrium assignment is also necessarily characterized by product countermonotonicity, meaning that sorting is negative for each dimension of heterogeneity with the product of heterogeneity in the other dimensions. The equilibrium assignment as well as wages and firm values are completely characterized in closed form. We illustrate our theory with an application to show that our model is consistent with the observed dispersion of earnings within and across U.S. firms. Our counterfactual analysis gives evidence that the change in the firm project distribution between 1981 and 2013 has a larger effect on the observed change in earnings dispersion than the change in the worker skill distribution.