How do Active Dendrite Networks Mitigate Catastrophic Forgetting?

We investigate the efficacy of Active Dendrite Networks (ADNs) in mitigating catastrophic forgetting in Continual Learning (CL). We consider Sparse Parity and Modular Addition to be our CL-task sequences. ADNs mitigate forgetting for Sparse Parity, but not for Modular Addition. For Sparse Parity, we perform an interpretability analysis to highlight the effectiveness of orthogonal (or uncorrelated) context vectors and task vectors in mitigation. We demonstrate that uncorrelated context vectors facilitate the creation of distinct subnetworks within ADNs, aiding in task separation. We also look at task uncorrelatedness to explain the difference in ADN's performance for Sparse Parity and Modular Addition.