I make a summary of pros and cons of dynamic graph. List as follows:
- each forward pass defines a new computational graph.
- each backward pass for a new graph needs new building and new derivative computing.
- static graphs can optimize the graph up front while dynamic graph can not do it cite
some points I am not sure.
Does the dynamic graph need more GPU IO?
welcome to add more features that I have missed. and more dynamic graph examples, use case