SlimmeRF: Slimmable Radiance Fields
3DVDec 15, 2023Best Paper
Neural Radiance Field (NeRF) and its variants have recently emerged as
successful methods for novel view synthesis and 3D scene reconstruction.
However, most current NeRF models either achieve high accuracy using large
model sizes, or achieve high memory-efficiency by trading off accuracy. This
limits the applicable scope of any single model, since high-accuracy models
might not fit in low-memory devices, and memory-efficient models might not
satisfy high-quality requirements. To this end, we present SlimmeRF, a model
that allows for instant test-time trade-offs between model size and accuracy
through slimming, thus making the model simultaneously suitable for scenarios
with different computing budgets. We achieve this through a newly proposed
algorithm named Tensorial Rank Incrementation (TRaIn) which increases the rank
of the model's tensorial representation gradually during training. We also
observe that our model allows for more effective trade-offs in sparse-view
scenarios, at times even achieving higher accuracy after being slimmed. We
credit this to the fact that erroneous information such as floaters tend to be
stored in components corresponding to higher ranks. Our implementation is
available at https://github.com/Shiran-Yuan/SlimmeRF.