Vision for a forest robot that follows a person
Detection and tracking on an NVIDIA Jetson Nano, so a weatherproof rover can follow a person through the forest.
- Role
- AI developer, VOIAR (MWLC)
- Stack
- YOLOv8, TensorRT, Docker, NVIDIA Jetson Nano, Raspberry Pi 4
At VOIAR I worked on BYODR, short for "Build Your Own Delivery Robot". It is a weatherproof rover with a Raspberry Pi 4, an NVIDIA Jetson Nano, two pan-tilt-zoom cameras and a router, with its software split into Docker services. Rover segments can be linked together, like a caterpillar.
My main job was to make the rover follow a person through the forest.
The following service
The following service runs on the Jetson Nano:
- Detection. YOLOv8, running on the GPU through TensorRT, finds people in the camera image and keeps following the same person.
- Steering. The image is divided into steering zones. Where the tracked person stands decides how far the rover turns.
- Distance. The height of the person in the image is the distance signal. Above a threshold, the rover stops.
Driving it in the field shaped the details. I added a dead zone so the rover does not steer at every small movement, reverse steering, and a clean switch between following and manual driving.
Around the vision
The vision only works if the rest of the robot does, so I also worked on:
- the whole phone navigation, so the rover can be driven from a mobile phone, plus a third camera and a redesign of the operator interface;
- a fix for control delay that grew the longer the rover ran, and a heartbeat fix for motor stutter;
- adaptive IP handling, GPS readings over SNMP from the router, and GPIO relays;
- a log box backed by MongoDB, the upgrade from Python 2.7 to 3.10, a refactor with a command-line wrapper, and CI on GitHub Actions.
Open source from this work
To run YOLO with GPU acceleration on the Jetson, I built a TensorRT Docker image. It became part of the official Ultralytics YOLO repository. Read how I built it.