ailia Tech BLOG
M-LSD : Machine Learning Model for Detecting Wireframes
This is an introduction to「M-LSD」, a machine learning model that can be used with ailia SDK. You can easily use this model to create AI applications using ailia SDK as well as many other ready-to-use ailia MODELS.
Overview
M-LSD is a machine learning model developed by NAVER to detect wireframes of objects. Since it can accurately detect the contours of sheets of paper and books, it can be used for pre-processing of OCR.
Source: https://github.com/navervision/mlsd
Architecture
The classic approach to line detection is complex and made of with multiple modules, whereas M-LSD detects lines in a single shot, which allows for fast processing.
Source: https://arxiv.org/pdf/2106.00186.pdf
The model uses MobileNetV2 as backbone, with the addition of a block for generating heatmaps in the later stage.
Source: https://arxiv.org/pdf/2106.00186.pdf
Line segments are defined as Tri-Points (TP), as shown below. The line segment is defined by lc, which indicates the center point, ds which is the displacement vector to the start point, and de which is the displacement vector to the end point.
Source: https://arxiv.org/pdf/2106.00186.pdf
The output of the model is a (1,200,2) vector lc representing the center point of the line segments, a (1,200) vector which is the confidence of the line segments, and a displacement map (1,256,256,4) representing the displacement from the center point to the start and end points of the line segments. Line segments can then be calculated by adding the three vectors of center points, start points, and end points.
Source: https://arxiv.org/pdf/2106.00186.pdf
The data sets used for training are Wireframe and YorkUrban.
Source: https://arxiv.org/pdf/2106.00186.pdf
Usage
M-LSD can be used with ailia SDK 1.2.8 and later with the following command to detect wireframes from the webcam video stream.
$ python3 mlsd.py -v 0
ailia-models/line_segment_detection/mlsd at master · ailia-ai/ailia-models(Image from…github.com
Here is the result you can expect.
ailia Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.
ailia Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.