RGB-D Camera Based Target Detection and Active Tracking

In this project, we designed a perception-tracking framework to detected moving target with unknown future motion and then following that target with a TurtleBot robot.

We applied the YOLOv3 object detection network to detect the moving target in real-time with an RGB camera in ROS Gazebo Simulation Environment.

By utilizing the information from YOLOv3 network, we can obtained the distance and bearing measurements between the robot and the target based on the bouding-box location information provided by the YOLOv3 algorithm as well as depth information from the depth camera.

Implemented Extended Kalman filter (EKF) to estimate the target’s state based on the measurement from the RGB-D Camera.

Designed a control-based and a reinforcement learning-based target tracking algorithm to drive the robot to track and follow a moving target with unknown behavior. Both algorithm show a promising performance to track the target.