NEF: Neural Edge Fields for 3D Parametric Curve Reconstruction from Multi-view Images

We study the problem of reconstructing 3D feature curves of an object from a set of calibrated multi-view images. To do so, we learn a neural implicit field representing the density distribution of 3D edges which we refer to as Neural Edge Field (NEF). Inspired by NeRF, NEF is optimized with a view-based rendering loss where a 2D edge map is rendered at a given view and is compared to the ground-truth edge map extracted from the image of that view. The rendering-based differentiable optimization of NEF fully exploits 2D edge detection, without needing a supervision of 3D edges, a 3D geometric operator or cross-view edge correspondence. Several technical designs are devised to ensure learning a range-limited and view-independent NEF for robust edge extraction. The final parametric 3D curves are extracted from NEF with an iterative optimization method. On our benchmark with synthetic data, we demonstrate that NEF outperforms existing state-of-the-art methods on all metrics.

We leverage 2D edge detection to directly acquire 3D edge points by learning a neural implicit field and further reconstructing 3D parametric curves that represent the geometrical shape of the object. We introduce several designs to train NEFs, and develop a coarse-to-fine optimization strategy to reconstruct parametric curves. The whole pipeline is self-supervised with 2D images (edge maps).

We show more reconstructed curves below. The videos shown are generated from the proposed ABC-NEF datasets. The input contains 50 views (or less) of posed images (a), and 2D edge maps (b) detected on them. Based on the trained NEF, the re-rendered 2D edge maps (c) can recover occluded edges. The 3D edge points (d) extracted from NEF are further applied to reconstruct 3D parametric curves (e).