STRESS

At the core of scientific experimental studies, there is the question of testing a hypothesis: is a new medical treatment better than a placebo? Is a new algorithm more efficient on a computational task than the state of the art? Because of random fluctuations, it is necessary to repeat experiments over several trials (to gather several samples) before making a decision. For those applications, we need statistical methods for hypothesis testing that come with formal guarantees on their probability of error. Those methods should also allow for tests with as few trials as possible in order to be applicable to domains where gathering samples is long or costly. The goal of STRESS is to give sharp control on the minimal sample size needed to take a decision or to make a prediction. STRESS will answer a common problematic in sequential methods: to have a practical control on when the algorithm will stop collecting data.