`

Timezone: »

 
Poster
Meta-Learning Sparse Implicit Neural Representations
Jaeho Lee · Jihoon Tack · Namhoon Lee · Jinwoo Shin

Wed Dec 08 12:30 AM -- 02:00 AM (PST) @ None #None

Implicit neural representations are a promising new avenue of representing general signals by learning a continuous function that, parameterized as a neural network, maps the domain of a signal to its codomain; the mapping from spatial coordinates of an image to its pixel values, for example. Being capable of conveying fine details in a high dimensional signal, unboundedly of its domain, implicit neural representations ensure many advantages over conventional discrete representations. However, the current approach is difficult to scale for a large number of signals or a data set, since learning a neural representation---which is parameter heavy by itself---for each signal individually requires a lot of memory and computations. To address this issue, we propose to leverage a meta-learning approach in combination with network compression under a sparsity constraint, such that it renders a well-initialized sparse parameterization that evolves quickly to represent a set of unseen signals in the subsequent training. We empirically demonstrate that meta-learned sparse neural representations achieve a much smaller loss than dense meta-learned models with the same number of parameters, when trained to fit each signal using the same number of optimization steps.

Author Information

Jaeho Lee (KAIST)
Jihoon Tack (KAIST)
Namhoon Lee (University of Oxford)
Jinwoo Shin (KAIST)

More from the Same Authors