I was wondering if anyone has implemented Elastic Weight Consolidation (EWC) as outlined in this paper? This algorithm allows for sequential/continuous learning without the model encountering catastrophic forgetting.
The main part of implementing this is calculating the Fisher information matrix. If anyone has any code they can share on this, that’d be great. Otherwise I’m happy to attempt it and share my code here.
Found a tensorflow implementation here: https://github.com/stokesj/EWC which we can use for reference.