One of the goals of artificial intelligence and machine learning is to allow computers to understand
our world. A crucial aspect to achieve this goal is to quantify uncertainty but - so far - that has been
been largely neglected in machine learning. Bayesian methodologies incorporate uncertainty in a
natural way. While they receive more and more attention, they still lack the accuracy or
performance that one has come to expect from deep learning systems.
The focus of this proposal is on Gaussian Processes (GPs). Stacking GPs (e.g., Deep GPs) and deep
kernel learning enable more accurate modeling of complex-structured data. In Deep GPs the output
of a higher level GP is fed as the input of a lower level GP. Deep GPs have shown some potential in
many applications but are restricted to smaller data sets due to scalability. This restriction has led to
the belief that Deep GPs are not applicable to very large data sets as used in other deep learning
methods. In this proposal, we explore the use of Deep GPs and, in general, Deep Bayesian Networks
for both discriminative as well as generative modeling of high-dimensional and complex-structured
data.
The key goals of this proposal are: (i) alleviate the computational complexity of the current methods
through the development of novel variational bounds and by further exploiting parallelism in the
model architecture, and (ii) using deep GPs or novel deep kernels to discover intricate patterns in
data.