Variational AutoEncoders (VAE) and why they matters

Imagine you're a data scientist working on a project to generate realistic images of cats. You have a large dataset of cat images, but you want to be able to create new images that are not just copies of the existing images.

You could use a traditional generative model, such as a GAN, but GANs can be difficult to train and can sometimes generate unrealistic images.

Here comes the VAE. A variational autoencoder (VAE) is a type of generative model that can help you generate realistic images. VAEs work by first learning a latent representation of the data.

This latent representation is a high-dimensional space that captures the underlying features of the data. Then, VAEs can use this latent representation to generate new data that is similar to the original data.

Let's get into more details. The rest of the article is categorized in below sections.

• Introduction
• Types of VAEs
• How Variational Autoencoders (VAEs) Work?
• Why we should use VAEs?
• Comparison with other available solutions
• Conclusion

Introduction

VAEs are a powerful tool for generating realistic images. They are relatively easy to train and can generate images that are indistinguishable from real images. VAEs have been used to generate images of cats, dogs, faces, and even objects.

A VAE is a type of probabilistic generative model. This means that it learns a probability distribution over the data. The VAE consists of two parts: an encoder and a decoder. The encoder takes an input data point and maps it to a latent representation. The decoder takes the latent representation and maps it back to an output data point.

The encoder is a neural network that learns a function from the data space to the latent space. The decoder is another neural network that learns a function from the latent space to the data space.

The VAE is trained by minimizing a loss function that measures the distance between the generated data and the real data. The loss function also includes a regularization term that encourages the latent representation to be smooth and informative.

Types of VAEs:

There are two main types of VAEs: continuous VAEs and discrete VAEs.

Continuous VAEs use a continuous latent space, while discrete VAEs use a discrete latent space.

Continuous VAEs are more common, and they are typically used for generating images. Discrete VAEs are less common, but they can be used for generating text and other types of data.

How Variational Autoencoders (VAEs) Work?

VAEs are composed of two main parts: an encoder and a decoder. The encoder takes in the data and outputs a latent representation. The decoder takes in the latent representation and outputs a reconstruction of the data.

VAEs are trained to minimize two losses: the reconstruction loss and the KL-divergence loss. The reconstruction loss measures how similar the reconstruction is to the original data. The KL-divergence loss measures how different the latent distribution learned by the encoder is from a prior distribution.

The prior distribution is a distribution over the latent space that we want the encoder to learn. It is typically a simple distribution, such as a Gaussian distribution.

By minimizing the KL-divergence loss, we encourage the encoder to learn a latent distribution that is similar to the prior distribution. This helps to ensure that the latent space is smooth and continuous, which makes it easier to generate new data.

Here is a simple code example of a VAE in Python (do read the comments)

import tensorflow as tf

class VAE(tf.keras.Model):
    def __init__(self, latent_dim):
        super(VAE, self).__init__()

        # Encoder
        self.encoder = tf.keras.models.Sequential([
            tf.keras.layers.Flatten(input_shape=(28, 28)),
            tf.keras.layers.Dense(128, activation='relu'),
            tf.keras.layers.Dense(latent_dim * 2, activation='linear'),
        ])

        # Decoder
        self.decoder = tf.keras.models.Sequential([
            tf.keras.layers.Dense(128, activation='relu'),
            tf.keras.layers.Dense(784, activation='sigmoid'),
        ])

    def call(self, inputs):
        # Encode the inputs
        mean, log_var = tf.split(self.encoder(inputs), 2, axis=1)

        # Sample from the latent distribution
        z = tf.random.normal(shape=(tf.shape(inputs)[0], latent_dim), mean=mean, stddev=tf.exp(0.5 * log_var))

        # Decode the latent representation
        reconstruction = self.decoder(z)

        return reconstruction, mean, log_var

# Create a VAE with a latent dimension of 10
vae = VAE(latent_dim=10)

# Train the VAE on the MNIST dataset
vae.compile(optimizer='adam', loss='binary_crossentropy')
vae.fit(x_train, x_train, epochs=10)

# Generate a new image
new_image = vae.predict(tf.random.normal(shape=(1, 10)))

# Display the new image
import matplotlib.pyplot as plt
plt.imshow(new_image[0])
plt.show()

This code will train a VAE on the MNIST dataset, which is a dataset of handwritten digits. Once the VAE is trained, we can use it to generate new images by sampling from the latent distribution.

Why we should use VAEs?

VAEs vs Alternatives

VAEs are a popular type of generative model, but there are a number of other alternatives available. Some of the most common alternatives include:

• Generative Adversarial Networks (GANs): GANs are a type of generative model that uses two neural networks to compete with each other. One neural network, called the generator, tries to generate realistic data, while the other neural network, called the discriminator, tries to distinguish between real and generated data. GANs can generate very high-quality data, but they can be difficult to train.
• Autoregressive models: Autoregressive models generate data one point at a time, using the previously generated points to predict the next point. Autoregressive models are relatively easy to train, but they can be slow to generate data and can be difficult to control.
• Diffusion models: Diffusion models start with a random noise sample and gradually remove noise until a realistic image is produced. Diffusion models are relatively new, but they have shown promising results in generating high-quality images.
• VQ-VAEs (Vector Quantized Variational Autoencoders): VQ-VAEs are a type of VAE that uses a vector quantization layer to discretize the latent space. This makes the latent space more efficient to encode and decode, and also makes it easier to generate new data from the latent space.
• Flow-based models: Flow-based models are a type of generative model that learns to generate new data by learning a series of transformations that can be applied to random noise to produce new data.

Which one is better?

The best generative model for a particular task will depend on the specific requirements of the task.

VAEs are a good choice for tasks where it is important to be able to control the latent space. For example, VAEs can be used to generate new data that is similar to existing data, or to generate new data that has certain desired properties.

GANs are a good choice for tasks where it is important to generate high-quality data. GANs have been shown to be able to generate very realistic images and videos.

VQ-VAEs are a good choice for tasks where it is important to be able to generate data quickly and efficiently. VQ-VAEs are also a good choice for tasks where the latent space needs to be discretized.

Flow-based models are a good choice for tasks where it is important to be able to generate data that is similar to existing data. Flow-based models have also been shown to be able to generate very diverse data.

VAEs are used for a variety of tasks, including:

• Generating realistic images
• Inferring the latent representation of data
• Compressing data
• Denoising data
• Transferring learning between different tasks

VAEs are a powerful tool for unsupervised learning. They can learn the underlying features of data without any labeled data. This makes them a valuable tool for tasks such as image generation and data compression.

Is VAE worth using?

Whether or not VAE is worth using depends on the specific needs of the application. If you need a generative model that is easy to train and can generate high-quality data, then VAE is a good option. However, if you need a generative model that is very fast to train or that can generate very realistic data, then you may want to consider an alternative model, such as a GAN or a diffusion model.

Comparison with other available solutions

Here is a table that summarizes the key differences between VAEs and some of the most common alternatives:

Conclusion

VAEs are a powerful tool for generating realistic images. They are relatively easy to train and can generate images that are indistinguishable from real images. VAEs have been used to generate images of cats, dogs, faces, and even objects.

I hope this article has helped you understand what VAEs are and how they can be used. If you have any questions, please feel free to ask me.

I hope this article has given you a good understanding of VAE. If you enjoyed the article do consider checking more.