Attention

You are viewing an older version of the documentation. The latest version is v3.3.

Robot Vision ROS2 Libraries¶

Smart manufacturing and Industrial 4.0 encourage users to introduce vision-guided robots for flexible and productive production lines. Point-Cloud Lirary (PCL) and Fast Library for Approximate Nearest Neighbors (FLANN) provide a series of 3-D processing algorithms commonly used in industrial robots (for picking, welding, painting, gluing, and so on), mobile robots (for visual SLAM and navigation), and 3C manufacturing (for inspection, measurement, and so on).

Based on customer feedback, algorithms from segmentation, sample consensus, surface, registration, k-dimensional tree (kdtree), octree are selected for GPU-acceleration on Intel® platforms. Acceleration is enabled with Intel® oneAPI Toolkit, a set of tools for deploying compute across heterogeneous devices (for example, integrated GPU).

Note

Intel® ECI aligns to the Debian maintained libpcl (1.13) and libflann (1.9) packages releases lifecycle. Debian source packages are available for user download from the ECI repository:

$ sudo apt-get source flann pcl

The following figure illustrates the high-level block diagram of oneAPI acceleration of RVL.

The following links provide useful benchmarks :

Launch 3-D Object Pose Detection tutorial¶

The ROS2 grasp planner tutorial demonstrates how a traditional 3-D Object Pose Detection (OPD) of a Point-Cloud Bolt object can be accelerated using Intel® oneAPI.

The application takes point cloud input from a PCD file, detects the 6-DOF pose of the Bolt by invoking PCL _registration_ algorithms GPU accelerated via Intel® oneAPI, transforms the object pose into robot’s base frame, and plans grasp pose for the bolt with necessary adjustment.

The following section is applicable to:

Enable Intel® oneAPI DPC++ and OpenMP Compute Runtimes to use acceleration on Intel® Graphics.
Verify that Point-Cloud message libraries are listed in repository packages:
```
$ sudo apt list | grep libpcl-.*segmentation*
```
In the following example, the libpcl-oneapi-segmentation1.13 corresponds to the libpcl-segmentation1.13 Debian release 1.13.1+dfsg-6 extended with Intel® oneAPI DPC++ and OpenMP runtime 2024.0.2:
```
libpcl-oneapi-segmentation1.13/unknown 1.13.1+dfsg-4eci1-oneapi2024.0.2 amd64
libpcl-segmentation1.13/unknown 1.13.1+dfsg-4eci1-oneapi2024.0.2 amd64
```
Verify that Fast Library for Approximate Nearest Neighbors libflann is listed in repository packages:
```
$ sudo apt list | grep libflann*
```
In the following example, the libflann1.9-dpcpp package corresponds to the libflann1.9 Debian release 1.9.2+dfsg-1 Extended for Intel® oneAPI 2024.0.2 DPC++ runtime compute:
```
libflann1.9-dpcpp/unknown 1.9.2+dfsg-1+deb12u1eci1-oneapi2024.0.2 amd64
libflann1.9/unknown 1.9.2+dfsg-1+deb12u1eci1-oneapi2024.0.2 amd64
```
Install ROS2 Desktop components, if not already completed.
Setup the Robotic node.

Install Grasp ROS2 packages:

$ sudo apt install ros-humble-grasp-ros2

Open a terminal and launch the Point-Cloud data publisher bold.pcd representing a _bold_ object:
```
$ cd /opt/ros/humble && ros2 run grasp_ros2 tgrasp_ros2
```
Open a second terminal and launch the 3-D Object Pose Detection algorithm for _bold_ object:
```
$ cd /opt/ros/humble/grasp_ros2 && ros2 run grasp_ros2 grasp_ros2 --ros-args --params-file cfg/opd.pcd.yaml
```
Expected Result (processing time varies according to the platform):

Open a third terminal and launch coordinates transformation (NOTE: redo hand-eye calibration for a new workspace setup):

$ ros2 run tf2_ros static_transform_publisher 0.12486974831501316 -0.5849824486260493 1.5341665864742178 0.9978993160964683 -0.0026913432432959052 -0.06430639616375905 0.007375569007019834 base camera_link

Open a fourth terminal and launch R-VIZ to visualize the scene:
```
$ cd /opt/ros/humble/grasp_ros2 && ros2 run rviz2 rviz2 -d rviz2/tests.rviz
```
Expected Result (processing time varies according to the platform):