I was going through the maml example in the higher library but it seems to me they are forgetting to divide by the meta_batch_size
, they have:
qry_loss.backward()
where they should have
meta_batch_size = qry_loss.size(0)
(qry_loss/meta_batch_size).backward()
is this correct? should they be dividing by the meta_batch_size
?
references:
- full code example working on omniglot: higher/maml-omniglot.py at master · facebookresearch/higher · GitHub
- crossposted: when do we divide by met_batch_size? · Issue #104 · facebookresearch/higher · GitHub