ailia Tech BLOG
Inpainting With Partial Conv: A machine learning model that predicts and fills in missing parts of an image.
This is an introduction to「Inpainting With Partial Conv」, a machine learning model that can be used with ailia SDK. You can easily use this model to create AI applications using ailia SDK as well as many other ready-to-use ailia MODELS.
Overview
Inpaining With Partial Conv is a machine learning model for Image Inpainting published by NVIDIA in December 2018. Given an input image and a mask image, the AI predicts and repair the missing parts in an image.
Source：https://arxiv.org/pdf/1804.07723.pdf
Model architecture
Inpainting With Partial Conv is based on PConvUNet .
In PConvUNet, instead of UNet’s Conv, Partial Conv is used, which determines whether a pixel is included in the convolution depending on the mask value.
Source：https://arxiv.org/pdf/1804.07723.pdf
In normal convolution, the input X is multiplied by the weight W. In this case, the missing pixels are also used for convolution, resulting in poor image quality. In Partial Convolutions, only the pixels with a mask value M of 1 are used for convolution, greatly improving the image quality.
Source：https://arxiv.org/pdf/1804.07723.pdf
In the mask update, if even one pixel is valid, the convolution is enabled.
Usage
The following command can be applied to any images located at the given path, the mask images will be read from a folder named masks.
python3 pytorch-inpainting-with-partial-conv --input IMAGE_PATH --savepath SAVE_IMAGE_PATH
You should get results as below, from the right is the input image, the mask image, the model result, and finally the ground truth image.
The model can also be used on drawn images.
Source：H2MD CHAN
ailia Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.
ailia Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.