Natural Light Uncalibrated Photometric Stereo (NaUPS) relieves the strict environment and light assumptions in classical Uncalibrated Photometric Stereo (UPS) methods. However, due to the intrinsic ill-posedness and high-dimensional ambiguities, addressing NaUPS is still an open question. Existing works impose strong assumptions on the environment lights and objects' material, restricting the effectiveness in more general scenarios. Alternatively, some methods leverage supervised learning with intricate models while lacking interpretability, resulting in a biased estimation. In this work, we proposed Spin Light Uncalibrated Photometric Stereo (Spin-UP), an unsupervised method to tackle NaUPS in various environment lights and objects. The proposed method uses a novel setup that captures the object's images on a rotatable platform, which mitigates NaUPS's ill-posedness by reducing unknowns and provides reliable priors to alleviate NaUPS's ambiguities. Leveraging neural inverse rendering and the proposed training strategies, Spin-UP recovers surface normals, environment light, and isotropic reflectance under complex natural light with low computational cost. Experiments have shown that Spin-UP outperforms other supervised / unsupervised NaUPS methods and achieves state-of-the-art performance on synthetic and real-world datasets.
The proposed Spin-UP handles natural light photometric stereo based on a new setup. We highlight our contribution as:
We record a video for an object under an environment light by rotating the object together with a linear, perspective camera in 360° on a rotatable platform. The relative positions and orientations between camera and object remain fixed during rotation.
We then extract around 50 images from the recorded video, regarding as our training data.
Based on the proposed setup, we derive a rough light map given the observed images.
We proposed two training strategies (top: interval sampling, bottom: shrinking range computing) to improve the training efficiency and alleviate aliasing issues.
@inproceedings{li2024spinup,
title={Spin-UP: Spin Light for Natural Light Uncalibrated Photometric Stereo},
author={Li, Zongrui and Lu, Zhan and Yan, Haojie and Shi, Boxin and Pan, Gang and Zheng, Qian and Jiang, Xudong},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year={2024}}