Vision Based Exploration in Fully Unknown Environment
I worked in Professor Yiannis Kantaros’ lab at WashU from August 2022 to May 2023. I applied autonomous exploration algorithm on our Turtlebot3 Waffle Pi robot by combining RRT naviation algorithm and Karto SLAM method. Besides this, I applied YOLOv7 on our robot for object detection and depth estimation to obtain distance of specific object. Lidar and YOLO data are combined to obtain position of pedestrian. The next step is predicting pedestrians trajectory by applying neural network.
Please check these demo videos.
Obtian person position by combining Lidar and YOLOv7 data. The closest Lidar distance in the YOLO framework is chosen as the person position. The code applies on 1 person, no person and mutiple people situation. depth and left or right of center represents x,y coordinate of the ground while Turtlebot is at the original point.
Object detection and autonomous navigation
depth estimation
The reason that I use Karto instead of Gmapping SLAM method is, Gmapping works pretty well in Gazebo but works bad on our Turtlebot. It somestimes messes the map up when twisting. When Turtlebot is stuck by something, Karto is able to know that robot is stuck, while gmapping method makes robot think it is still moving. When using gmapping SLAM, Turtlebot moves much faster than Karto SLAM.
Please check these 2 videos for comparison.
gmapping doesn’t know where it is.webm
Karto knows it is stuck by rod.webm
Computer Vision and Machine Learning with Self-driving Car project
I worked on this project from Aug 2019 to Nov 2020. I wrote my code of semantic segmantation for pixel-wise classification. The classification accuracy is increased from 34% to 87% on a small dataset by tuning upsample method and kernel size of CNN. The small dataset only contains 101 test 367 train images with their annotation images. The research is published in this paper.
The figure below shows comparison of input image, ground truth and segmentation outcome from different models. 
FCN-8s has the highest accuracy and mean Intersection over Union metric. 
Code