Flagship project · PhD research

A drone that finds the branch and measures the cut.

Pruning radiata pine by hand is dangerous work at height on steep ground. Existing machines lean on expensive LiDAR and stop at thick branches. I set out to do it with one low-cost stereo camera, down to branches 10 mm thin.

Part of the Drones with Tools programme, Victoria University of Wellington and University of Canterbury

A large pruning drone hovering beside a bare tree against a blue sky, with a cutting tool on a long carbon boom.
The programme's pruning platform closing on a tree, cutting tool out front on a carbon boom.
  1. Stereo pair
  2. YOLO / SAM3 masks
  3. DEFOM-Stereo disparity
  4. Per-branch filtering
  5. 3D branch position
  6. Branch-to-tool distance
01

Build the aircraft

I built up the UAV, calibrated and integrated the ZED Mini stereo rig on the airframe, and ran the field trials. Everything the perception stack needs flies with it.

Top view of a quadcopter with labelled flight controller, Jetson computer, microphone and speaker, M10 GPS and four brushless motors.
Compute
NVIDIA Jetson Orin NX Super, 16 GB
Flight control
Pixhawk, M10 GPS
Vision
ZED Mini stereo, 63 mm baseline
Audio
On-board microphone and speaker
The quadcopter on its landing gear in long grass with green hills behind.
Field test.
Flight test, with sound.
02

Simulate what can't be measured

No sensor gives clean ground truth on a thin branch, so I built it. UE5-Forest places 115 photogrammetry-scanned trees in Unreal Engine 5 and films them with a virtual rig matched to the real ZED Mini: 5,520 stereo pairs at 1920×1080 with pixel-exact disparity.

Paper Public benchmark

03

See the branch

Segmentation finds each branch and stereo matching gives it depth. YOLO reached 82.0% mask mAP50-95, ahead of Mask R-CNN, and I evaluated more than twenty configurations before settling on DEFOM-Stereo with SAM3.

False-colour disparity map of the tree: the trunk and branches stand out in red against a blue background. Camera view of a tree in the lab with each branch painted in its own colour. Branch masks Stereo depth
Drag to compare. Thin branches keep their shape in the disparity map.
04

Measure each branch

Raw disparity is too noisy for a saw. My progressive pipeline repairs mask edges with a skeleton-preserving erosion, validates colour in LAB space, then runs a five-stage depth filter. Per-branch depth deviation falls by 82%, leaving clean 3D point clouds to aim at.

Plot of six colour-coded branch point clouds in metres, with a small panel for each branch.

Springer LNCS arXiv

05

Run it on the aircraft

I trained five DEFOM-Stereo variants, designed three compressed ones for the drone's Jetson, and timed all of them on the flight hardware. The best reaches a 1.01 cm median depth error.

ModelParamsMedian depth errorJetson, 720p
ViT-S43.29 M1.01 cm0.19 FPS
ViT-L382.62 M1.04 cm0.08 FPS
PrunePlus32.81 M2.24 cm0.26 FPS
PruneStereo27.03 M5.87 cm0.72 FPS
PruneNano23.03 M13.22 cm1.91 FPS

Held-out UE5 test set, plain PyTorch FP32 on the Orin NX Super 16 GB. The target for a tight cut loop is 10 FPS, so these are operating points on the way there.

Code and checkpoints Paper

06

Hear over the rotors

A microphone on a quadrotor hears mostly its own motors, so spoken commands arrive below the noise. I trained keyword spotting on one quadrotor, tested on another, and measured false alarms over 4.4 hours of continuous rotor noise. The full pipeline costs 3 ms per 100 ms hop on the Jetson CPU.

Paper