We’ve just released our first Hugging Face model using our open source dendritic optimization repository. We would love to have some feedback from the community about how it works for you.
By perforating models with dendrites similar accuracy gains can be seen at a significantly improved parameter efficiency compared to increasing model size with increased channels and layers. Training on ImageNet, a single dendrite increases resnet-18 accuracy by 2.54% per million added parameters, while jumping up to a resnet34 results in only 0.35% per million.
This model is a pretrained perforated resnet-18, designed for transfer learning, that you should be able to just plug right into your current pipeline with a single copy-paste. On the flowers-101 dataset we saw the following results:
Model | Parameters | Accuracy
resnet-18 | 11.7 M | 87.127
resnet-34 | 21.8 M | 87.898
resnet-18-perforated | 12.3 M | 87.914
If you’re working with resnets please give it a try and let us know how you results compare between these 3 pretrained models.