VisRecon

A small multi-view reconstruction toolkit: capture RGB-D sequences from an Intel RealSense camera, segment the subject, run a scripted COLMAP pipeline from feature extraction through to a mesh, and inspect the result. It was built for reconstructing plants in an agricultural-robotics setting, where the point of a reconstruction is to measure something—leaf area, canopy shape—and kept general enough to point at anything else. Python, COLMAP and an Intel RealSense; 2025; MIT; repository.

Five small tools joined by a directory convention rather than a framework. The reason they are separate scripts is that dense reconstruction takes minutes to hours, and you almost always want to re-run it with different parameters against images you already have. Nothing here re-captures or re-segments unless asked.

Four stages—capture, segment, reconstruct, inspect—with the eight COLMAP steps inside the reconstruct stage: feature extraction, exhaustive matching, sparse mapping, best-model selection, undistortion, PatchMatch stereo, fusion, and Poisson plus Delaunay meshing
Four stages joined by a directory convention. Every stage-3 parameter is tuned the same way—more and better features from a thin, self-similar subject, at several times the extraction cost.

The problem

Plants are close to the worst case for structure-from-motion. Leaves are thin, so a surface seen from two viewpoints is often the same surface seen edge-on in one of them. Leaves are self-similar, so a descriptor on one is easily matched to a descriptor on its neighbor. There are few large flat textured regions, and the subject is green on a background that is also frequently green. The default COLMAP automatic reconstructor either refuses to initialize on a tight orbit or produces a cloud that is mostly background.

The toolkit exists to make one specific capture setup work reliably: a subject within half a meter of a hand-held RealSense, orbited deliberately, frame by frame. Every non-default choice below follows from that.

The pipeline

run_colmap_pipeline.sh runs eight COLMAP stages under set -e, with each call wrapped so a failure stops the run with a named stage rather than continuing on bad data. Three decisions carry most of the weight:

  • Feature extraction is set to "hard subject". estimate_affine_shape replaces SIFT's similarity-invariant patch with an affine-adapted one, which matters when the same leaf is seen from very different angles; domain_size_pooling pools descriptors across scales; and the per-image cap is raised to 16,384 features. All three point the same way—more and better features—and together they multiply extraction time several times over. That cost is the main reason this is not simply the automatic reconstructor.
  • The mapper is allowed to start narrow. init_min_tri_angle 2 lets the reconstruction initialize from a pair with only 2° of triangulation angle, far below the default. Small orbiting captures often have nothing wider, so without it the mapper simply refuses to begin—but a narrow-baseline initialization is poorly conditioned, so the result needs looking at rather than trusting.
  • Several models are allowed, and one is chosen. multiple_models 1 lets the mapper emit disconnected sub-models when part of the sequence fails to link—the honest outcome—and a loop after it runs model_analyzer on each and keeps the one with the most registered images. It is the most useful glue in the file and also the most fragile: it parses human-readable output that is not a stable interface.

After that: undistortion capped at 2,000 px (the single biggest lever on dense runtime), PatchMatch stereo with geometric consistency (halves the speckle at roughly double the cost, and requires a CUDA build), fusion, and both a Poisson and a Delaunay mesh. Running both is the point: where they disagree is exactly where the reconstruction is unsupported by data.

What the mask keeps

image_preparer.py writes masked copies of each frame alongside the raw ones, and the masked copies are what COLMAP reads. The mask is two tests ANDed together: 

depth_mask = (depth_image < 600).astype(np.uint8) * 255        # millimeters
color_mask = cv2.inRange(hsv, (30, 20, 20), (85, 255, 255))     # OpenCV hue 30–85: green
combined   = cv2.bitwise_and(depth_mask, color_mask)

So a pixel survives if it is within 600 mm of the camera and green; everything else is blacked out. That is exactly right for a green plant on a non-green background, and it is why a brown stem, a dry leaf, a pot or a fiducial marker is never reconstructed.

Results

A run produces two artifacts for each plant: the fused point cloud and the Poisson mesh, both viewable in the toolkit's own viewer. The mesh looks cleaner, and that is the trap—Poisson invents surface in unobserved regions, which for a plant means membrane between leaves that should be empty. For measurement, the point cloud and the Delaunay mesh are the trustworthy artifacts; the Poisson mesh is for pictures.

The repository also ships 40 sample input frames so the pipeline can be tried without a camera. They are tabletop objects rather than plants—enough to exercise every stage, though not the green mask.

Lineage

VisRecon grew out of an earlier repository of mine, Multiview Reconstruction.

Resources