Robot Vision Libraries¶
Smart manufacturing and Industrial 4.0 encourage users to introduce vision-guided robots for flexible and productive production lines. 3-D data processing is one of the key technologies of robotic vision. Point Cloud Library (PCL) and Fast Library for Approximate Nearest Neighbors (FLANN) provide a series of 3-D processing algorithms commonly used in industrial robots (for picking, welding, painting, gluing, and so on), mobile robots (for visual SLAM and navigation), and 3C manufacturing (for inspection, measurement, and so on)
According to customer feedback, algorithms from segmentation, sample consensus, surface, registration, k-dimensional tree (kdtree), octree are selected for acceleration on Intel® platforms. The acceleration is enabled with Intel® oneAPI Toolkits, a set of tools for deploying the computing across heterogeneous devices (for example, integrated GPU).
The following figure illustrates the high-level block diagram of oneAPI acceleration of RVL.
This section explains the following:
Procedure to build and install Robot Vision Libraries (RVL) on Intel® Edge Controls for Industrial (Intel® ECI or ECI) platform
Acceleration results with oneAPI Toolkits
Procedure to replicate the tests and demo, and details on where to find the example code
RVL Developer Reference¶
RVL Deb packages are built on packages-bullseye with Robotics Controller meta-packages selected, as shown in the following figure.
RVL Prerequisites¶
Recommended Platform¶
The robot vision libraries are enabled with oneAPI Toolkits which deploy the computing on acceleration devices like GPU. To observe the performance gain, a platform with Intel Gen8 or above integrated graphic devices is recommended. If you are not sure of the generation of graphic device you are using, check the Intel Graphics Processor Table. As a reference, the following platform was used for validation:
11th Gen Intel® Core™ i7-1185G7E @ 2.80 GHz
32GB DRAM DDR4 @ 2700 MHz
RVL Dependencies¶
As shown in the high-level diagram, the RVL packages depend on oneAPI Toolkits and Robot Operating System Software (ROS2).
oneAPI and GPU drivers
ROS2 Foxy
Install RVL¶
RVL components, oneAPI, and ROS2 are all built from the Robotics Controller meta-package. You can install the RVL from the ECI APT repository for Debian (Bullseye) ONLY. Setup the ECI APT repository, then perform the following command to install the RVL:
- Install from individual Deb packages
$ sudo apt install eci-robotics ros-foxy-moveit
Besides, you need to install GPU drivers which are not part of ECI in this release (follow instructions in GPU Installation Guides).
FLANN-DPCPP¶
FLANN is a 3-D processing library that PCL uses to search for nearest neighbors in three-dimensional or even higher dimensional space. FLANN builds indices to the input points and provides interfaces to search for the nearest K (number of) or nearest R (radius) neighbors.
The oneAPI DPC++ Compatibility Tool (DPCT) is used to migrate the CUDA implementation into Data Parallel C++ (DPC++) for building and searching the k-dimensional tree. Additionally, some CUDA thrust functions need manual migration. The dimension and size of index space, nd_range of SYCL are adapted according to the hardware device capability of the maximum work-group size. Unified Shared Memory (USM) is used to avoid additional memory copy.
FLANN Tests¶
The following section is applicable to:
Follow the FLANN-DPCPP section in /opt/intel/robotics/Readme.txt for FLANN tests.
Expected outputs (processing time varies according to the platform):
Note: It takes time for SYCL to load codes/data to GPU for the first time to create the kdtree.
PCL-oneAPI¶
PCL contains common algorithms for 3-D point cloud processing. For the PCL module with CUDA code, DPCT is used to migrate the CUDA code to DPC++ implementation. For the PCL module without CUDA code, Intel® VTune Profiler is used to identify the performance bottlenecks and OMP is used to enable multi-processing via compiler directives.
PCL Tests¶
The following section is applicable to:
Follow the PCL-oneAPI section in /opt/intel/robotics/Readme.txt for PCL tests.
Expected outputs (processing time varies according to the platform):
3-D Object Pose Detection Demonstration¶
Bolt pose detection application demonstrates traditional 3-D Object Pose Detection (OPD) using PCL-oneAPI. The application takes point cloud input from a PCD file, detects the 6-DOF pose of the bolt by invoking accelerated PCL registration algorithms, transforms the object pose into robot’s base frame, plans grasp pose for the bolt with necessary adjustment.
The following section is applicable to:
Follow the OPD section in /opt/intel/robotics/Readme.txt for OPD tests.
Expected outputs (processing time varies according to the platform):
Example Code¶
The following section is applicable to:
Example code is prebuilt into binaries as test cases and demo applications. Example code is also provided with the full source tree. You can modify and build the codes for customization.
FLANN¶
The FLANN-DPCPP example code is located at /opt/intel/robotics/flann_dpcpp.
$ cd /opt/intel/robotics/flann_dpcpp // Example codes for k-search and radius search: src/flanntest.cpp // Exmaple cmake file: CMakeLists.txt // Follow build and run steps in 'README.md', skipping the dependencies installation steps since they're already installed by ECI.
PCL¶
The PCL-oneAPI example code is located at /opt/intel/robotics/pcl_oneapi.
$ cd /opt/intel/robotics/pcl_oneapi // Example codes for using optimized PCL algorithms: test/oneapi // Example cmake files are available under each subfolder. // Follow build and run steps in 'oneapi/README.md', skipping the dependencies installation steps since they're already installed by ECI. // More tutorials are available in 'doc/tutorials/content/oneapi_*.rst'.
OPD¶
The OPD demo code is located at /opt/intel/robotics/ros2-opd/src.
$ cd /opt/intel/robotics/ros2-opd/src // Example codes for using pcl-oneapi for 3D object pose detection: grasp_ros2/src/grasp_detector_opd.cpp // Example cmake file: grasp_ros2/CMakeLists.txt // Build with colcon is not yet supported in bullseye.