roi_detected_object_fusion

Purpose

The roi_detected_object_fusion is a package to overwrite labels of detected objects with that of Region Of Interests (ROIs) by a 2D object detector.

Inner-workings / Algorithms

In what follows, we describe the algorithm utilized by roi_detected_object_fusion (the meaning of each parameter can be found in the Parameters section):

1. If the existence_probability of a detected object is greater than the threshold, it is accepted without any further processing and published in output.
2. The remaining detected objects are projected onto image planes, and if the resulting ROIs overlap with the ones from the 2D detector, they are published as fused objects in output. The Intersection over Union (IoU) is used to determine if there are overlaps between the detections from input and the ROIs from input/rois.

The DetectedObject has three possible shape choices/implementations, where the polygon's vertices for each case are defined as follows:

• BOUNDING_BOX: The 8 corners of a bounding box.
• CYLINDER: The circle is approximated by a hexagon.
• POLYGON: Not implemented yet.

Inputs / Outputs

Input

Name	Type	Description
input	autoware_perception_msgs::msg::DetectedObjects	input detected objects
input/camera_info[0-7]	sensor_msgs::msg::CameraInfo	camera information to project 3d points onto image planes.
input/rois[0-7]	tier4_perception_msgs::msg::DetectedObjectsWithFeature	ROIs from each image.
input/image_raw[0-7]	sensor_msgs::msg::Image	images for visualization.

Output

Name	Type	Description
output	autoware_perception_msgs::msg::DetectedObjects	detected objects
~/debug/image_raw[0-7]	sensor_msgs::msg::Image	images for visualization,
~/debug/fused_objects	autoware_perception_msgs::msg::DetectedObjects	fused detected objects
~/debug/ignored_objects	autoware_perception_msgs::msg::DetectedObjects	not fused detected objects

Parameters

Core Parameters

Name	Type	Description
rois_number	int	the number of input rois
debug_mode	bool	If set to true, the node subscribes to the image topic and publishes an image with debug drawings.
passthrough_lower_bound_probability_thresholds	vector[double]	If the existence_probability of a detected object is greater than the threshold, it is published in output.
trust_distances	vector[double]	If the distance of a detected object from the origin of frame_id is greater than the threshold, it is published in output.
min_iou_threshold	double	If the iou between detected objects and rois is greater than min_iou_threshold, the objects are classified as fused.
use_roi_probability	float	If set to true, the algorithm uses existence_probability of ROIs to match with the that of detected objects.
roi_probability_threshold	double	If the existence_probability of ROIs is greater than the threshold, matched detected objects are published in output.
can_assign_matrix	vector[int]	association matrix between rois and detected_objects to check that two rois on images can be match

Assumptions / Known limits

POLYGON, which is a shape of a detected object, isn't supported yet.