Tabula Inscripta: Prior knowledge for deep learning

When a computer can intelligently see and perform visual tasks that are repetitive or require expert knowledge such as medical diagnosis, industrial inspection, autonomous vehicles, etc., our world will no longer be the same. We are now witnessing a global artificial intelligence arms race, and the engines in that race are powered by deep learning. Deep learning is a type of machine learning that learns from examples rather than instruction. Such deep networks are incredibly powerful machines that do not use physical models to design relevant image features, but instead use huge datasets to learn the relevant features. Excellent recent research investigates how to make deep networks more data-efficient, yet, to date, however, little work has addressed how to overcome the data dependency by adding prior knowledge to deep nets. As a consequence, all knowledge has to be (re-)learned implicitly from data requiring huge data and compute resources. My project Tabula Inscripta aims to remedy this gap by investigating *how to flexibly pre-wire deep networks with generic visual innate knowledge structures*, which allows to incorporate hard won existing generic knowledge from physics such as light reflection or geometry. To tackle this research question, I decompose the "visual" prior-knowledge into appearance and appearance-free models while the "flexibility" is decomposed into plasticity, which will allow a deep net to pass-by the knowledge priors if the model does not help, and in re-use, which aims for a plug-and-play deep network architecture where knowledge modules can easily be shared in the community. This proposal directly goes against the current trend to learn everything from data, and as such, it benefits mostly the long tail of companies and universities without massive data and compute.