Towards Automated Differentiable Logic Gate Networks: Temperature Selection and the Limits of Depth with Trainable Connectivity

Differentiable Logic Gate Networks (DLGNs) train circuits of logic gates end to end, discretizing them at inference time into hardware-synthesizable hard-gate circuits. The output of their Group-Sum layer must be scaled by a temperature $\tau$ before applying softmax; existing variants typically use a manually tuned constant that must be re-selected for each dataset and architecture. We propose two semi-automatic alternatives that remove this manual step. AutoTau generalizes a previously fixed divisor into an explicit, per-dataset hyperparameter $\sigma_T$. AdaVar uses an exponential-moving-average feedback controller to track the same $\sigma_T$ throughout training without backpropagating through $\tau$, avoiding the degeneracy observed when the temperature is learned directly. Across three datasets (MNIST, FashionMNIST, CIFAR-10) and three architectures per dataset, both methods achieve accuracy comparable to or better than manually tuned $\tau$. We further investigate the limited benefits of increasing depth in DLGNs with trainable connectivity and propose a hypothesis explaining why deeper architectures fail to outperform shallower ones.