Multiple Point Light Estimation from Low-Quality 3D Reconstructions

We address the problem of light source estimation in environments containing multiple, in-scene illuminants. While there have been several recent advances in this domain, current approaches are either ineffectual in the absence of specific visual cues or assume full knowledge of surface reflectance. We develop a more general framework, which works directly on noisy, colored 3D reconstructions, as produced by RGB-D video sequences. To model multiple light sources, we employ a large set of potential lights. Using physically-based rendering techniques, we calculate each light's capacity to illuminate the scene. From this assessment, we formulate a non-linear, least squares optimization problem to determine which lights to activate. In contrast with traditional approaches in which photometric error is minimized, our objective function resembles that found in intrinsic image decomposition. This obviates estimating reflectance as an intermediate step. We evaluate our framework on both real-world and synthetic datasets, illuminated by up to three light sources, and show it is capable of modeling complex lighting scenarios with high-fidelity.