Rare Events: Asymptotics, Algorithms, Applications

Rare events have a significant impact on society ? recent examples are the UK flood, the financial crisis, and the India power outage. Understanding the frequency and nature of such events is important to enable prevention and mitigation. Their analysis is a central topic in probability, covered by large deviations and extreme value theory. The study of rare events can yield models where underlying random variables have a power law (heavy-tailed) distribution. Unfortunately, large deviations theory does not encompass this assumption; studies on rare events in heavy-tailed systems are still ad hoc. Another shortcoming is the asymptotic nature of the theory, which provides estimates of the rate at which probabilities converge to zero. These estimates are often poor for practical purposes. Statistical methods have similar shortcomings. The goals of this proposal are 1. Develop a general framework to analyze rare events in heavy-tailed systems 2. Design robust computation, estimation and simulation techniques 3. Apply the developed methods to several relevant problems I have recently been involved in the development of several promising ideas that make these goals within reach. Specifically, we propose to combine techniques from probability, optimization/control, and analysis in various innovative ways. For example, Goal 1 is attainable by systematizing a connection with discrete optimization, and control-theoretic techniques to develop various useful bounds. Goal 2 exploits a connection between perturbed Markov processes and optimization problems. Goal 3 is reachable by analysing stochastic dynamics driven by optimization problems, which arise in communication and electricity networks. This research leads to mathematical tools that enable the resolution of several longstanding open problems in applied probability. The algorithms increase the applicability of large deviations theory in other academic disciplines. The envisioned applications are urgent and form a useful testing ground of the methods developed in this proposal.