Timezone: »

 
Poster
Improving Local Identifiability in Probabilistic Box Embeddings
Shib Dasgupta · Michael Boratko · Dongxu Zhang · Luke Vilnis · Xiang Li · Andrew McCallum

Tue Dec 08 09:00 AM -- 11:00 AM (PST) @ Poster Session 1 #431
in Poster Session 2 »

Geometric embeddings have recently received attention for their natural ability to represent transitive asymmetric relations via containment. Box embeddings, where objects are represented by n-dimensional hyperrectangles, are a particularly promising example of such an embedding as they are closed under intersection and their volume can be calculated easily, allowing them to naturally represent calibrated probability distributions. The benefits of geometric embeddings also introduce a problem of local identifiability, however, where whole neighborhoods of parameters result in equivalent loss which impedes learning. Prior work addressed some of these issues by using an approximation to Gaussian convolution over the box parameters, however this intersection operation also increases the sparsity of the gradient. In this work we model the box parameters with min and max Gumbel distributions, which were chosen such that the space is still closed under the operation of intersection. The calculation of the expected intersection volume involves all parameters, and we demonstrate experimentally that this drastically improves the ability of such models to learn.

Author Information

Shib Dasgupta (University of Massachusetts Amherst)
Michael Boratko (UMass Amherst)
Dongxu Zhang (University of Massachusetts Amherst)
Luke Vilnis (University of Massachusetts, Amherst)
Xiang Li (UMass Amherst)
Andrew McCallum (UMass Amherst)

More from the Same Authors