ailia Tech BLOG
SRResNet : A Machine Learning Model to Increase Image Resolution
This is an introduction to「SRResNet」, a machine learning model that can be used with ailia SDK. You can easily use this model to create AI applications using ailia SDK as well as many other ready-to-use ailia MODELS.
Overview
SRResNet is a super-resolution model that increases image resolution with high quality. It takes an image of size (1,3,64,64) as input and outputs an image (1,3,256,256) enlarged by a factor of 4.
Conventional bilinear and bicubic enlargements have the problem of jaggy diagonal lines and blurry output. By using AI super-resolution, the enlarged image stays sharp.
Source: https://arxiv.org/pdf/1609.04802.pdf
By using PixelShuffler, SRResNet produces images with less noise than traditional AI super-resolution models using Deconvolution.
History of AI Super Resolution
SRCNN
Introduced in Image Super-Resolution Using Deep Convolutional Networksand used for example in Waifu2x, the image is first increased in resolution, then extracted features are convolved with multiple layers, and finally reconstructed to give the final high-resolution image.
Source: https://arxiv.org/pdf/1501.00092.pdf
FSRCNN
Fast Super-Resolution Convolutional Neural Network (FSRCNN) was proposed to speed up SRCNN. Deconvolution is used to increase the resolution at the end of the pipeline to speed up the process.
Source: http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html
ESPCN
Deconvolution has the issue to generate checkerboard noise, and ESPCN (SRResNet) is the solution to this problem. It uses PixelShuffler (Sub Pixel Convolution) to suppress this noise issue.
Source: https://nuit-blanche.blogspot.com/2016/09/real-time-single-image-and-video-super.html
SRGAN
There are two methods of optimization during training using the SRResNet model architecture: training based on PSNR and SSIM, and using GAN. With PSNR and SSIM, there is a problem that although the numerical performance is high, the result is not.
Using the SRResNet model, it is possible to build a model that performs high quality super-resolution by training the AI (the discriminator) to determine whether the image is generated by the AI (the generator)or the original image. This method is called SRGAN.
Usage
Use the following command to run SRResNet (SRGAN) with ailia SDK. By using the -p option, the input image can be enlarged to 4 times the resolution by tiling it in 64x64 pixel units.
$ python3 srresnet.py -i input.jpg -o output.jpg -p
Related product (ailia AI Refiner)
ailia Inc. and RADIUS5 Inc. published an AI super-resolution plug-in that can be used with Adobe After Effects. The product is a combination of a super-resolution model originally developed by RADIUS5 Inc. and ailia SDK originally developed by ailia Inc.
ailia Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.
ailia Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.