I make a summary of pros and cons of dynamic graph. List as follows:
pros:
- easy to debug
- flexible control flow,
cons:
- slower
- each forward pass defines a new computational graph.
- each backward pass for a new graph needs new building and new derivative computing.
- static graphs can optimize the graph up front while dynamic graph can not do it cite
some points I am not sure.
-
Does the dynamic graph need more GPU IO?
more reference
papers about DCG
How to think with dynamic graphs
keep updating
welcome to add more features that I have missed. and more dynamic graph examples, use case