ailia Tech BLOG
3DObjectDetectionPytorch : 3D Object Detection Model
This is an introduction to「3DObjectDetectionPyrorch」, a machine learning model that can be used with ailia SDK. You can easily use this model to create AI applications using ailia SDK as well as many other ready-to-use ailia MODELS.
Overview
3DObjectDetectionPytorch is a machine learning model that calculates 3D bounding boxes of objects. Other object detection models such as YOLO generally computes 2D bounding boxes, but this new model returns bounding boxes with depth information.
Source: Objectron dataset
Architecture
3DObjectDetectionPytorch is capable of recognizing the following 9 classes.
OBJECTRON_CLASSES = (‘bike’, ‘book’, ‘bottle’, ‘cereal_box’, ‘camera’, ‘chair’, ‘cup’, ‘laptop’, ‘shoe’)
First, the 2D bounding box of the object is computed with MobileNetV2 SSD, and then the 3D bounding box is calculated with MobileNetV3 regression model. This regression model takes an image (1,3,224,224) as input and returns 9 keypoints (x,y) for each class (9,1,9,2).
3DObjectDetectionPytorch was trained on the Objectron dataset, which is publicly available from Google.
Source: https://github.com/google-research-datasets/Objectron
The Objectron dataset is a dataset developed for AR development and contains 15K annotated videos and 4M annotated images.
Source: https://github.com/google-research-datasets/Objectron
Usage
Use the following command to run 3DObjectDetectionPytorch with ailia SDK on a web camera video stream.
$ python3 3d-object-detection.pytorch.py -v 0
Here is the result you can expect.
Related topics
ailia MODELS also contains Google’s mediapipe_objectron which is a model also trained using the Objectron dataset.
ailia Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.
ailia Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.