Here is a collection of notes I have prepared in the past on various topics of interest.

Semi-supervised Expectation Maximization

This work considers the Expectation Maximization (EM) algorithm in the semi-supervised setting. First, the general form for semi-supervised version of maximum likelihood is derived from the Latent Variable Model (LVM). Since the involved integrals are usually intractable, a surrogate objective function based on the Evidence Lower Bound (ELBO) is introduced. Next, we derive the equations of the semi-supervised EM. Finally, the concrete equations for a fitting a Gaussian Mixture Model (GMM) using labeled and unlabeled data are deduced.
[pdf] [ipynb]

Variational Autoencoders

We seek to perform density estimation between a true (but unknown) probability distribution, and a set of proposal distributions. Formulated as minimization between distributions, this leads to the maximum (marginal) likelihood principle (see Section 2). To extend the expressiveness of our proposal distributions, we introduce latent variable models (see Section 3). The hidden variables raise issues in evaluating the marginal likelihood. We replace the marginal likelihood by a surrogate objective developed by Variational Inference (see Section 4). Combining these ideas gives us a powerful framework to estimate marginal, inference and joint distributions simultaneously. The Variational Autoencoder(see Section 5) is a specific application of these framework, that parametrizes the distributions to be optimized by neural networks.
[pdf] [ipynb]

Stochastic Optimization of Non-Differentiable Functions

We consider a gradient based parameter optimization of a stochastic computational graph consisting of random scene properties and a deterministic, non-differentiable render function. Our approach leverages ideas from Generative Adverserial Networks (GANs) and gradient estimators from reinforcement learning to jointly optimize all distributional and structural parameters based on generated images. This document should be regarded as an unfinished notebook to emphasize our main idea.
[pdf] [blendtorch]

Graph Neural Networks for Node-Level Predictions

This work aims to provide an overview of early and modern graph neural network based machine learning methods for node-level prediction tasks. Under the umbrella of taxonomies already established in the literature, we explain the core concepts and provide detailed explanations for convolutional methods that have had strong impact. In addition, we introduce common benchmarks and present selected applications from various areas. Finally, we discuss open problems for further research.
[pdf] [arXiv]