Pros and cons of dynamic computation graph (summary)

I make a summary of pros and cons of dynamic graph. List as follows:



  • slower
    • each forward pass defines a new computational graph.
    • each backward pass for a new graph needs new building and new derivative computing.
    • static graphs can optimize the graph up front while dynamic graph can not do it cite

some points I am not sure.

  1. Does the dynamic graph need more GPU IO?

  2. can we simulate all dynamic graph with static graph?

more reference

papers about DCG
How to think with dynamic graphs

keep updating
welcome to add more features that I have missed. and more dynamic graph examples, use case

Does dynamic graph makes it harder to translate to other platforms like iOS and android?