Data Structure coding for GDAS NAS

promach · May 25, 2021, 3:08pm

how to code the following graph structure ?
which kind of data structure is suitable in this case for both forward inference and backward propagation ?
Would Tutorial — NetworkX 3.2.1 documentation or PyG Documentation — pytorch_geometric documentation or https://www.dgl.ai/ be suitable ?

github.com/D-X-Y/AutoDL-Projects

Questions about DARTS

opened 04:47AM - 01 May 21 UTC

buttercutter

question

1. For DARTS complexity analysis, anyone have any idea how to derive the *(k+1)\…*k/2* expression ? Why 2 input nodes ? How will the calculated value change if graph isomorphism is considered ? Why "2+3+4+5" learnable edges ? If there is lack of connection, the paper should not add 1 which does not actually contribute to learnable edges configurations at all ? 2. Why need to train the weights for normal cells and reduction cells **separately** as shown in Figures 4 and 5 below ? 3. How to arrange the nodes such that the NAS search will actually converge with minimum error ? Note: Not all nodes are connected to each and every other nodes 4. Why is [GDAS](https://arxiv.org/pdf/1910.04465.pdf#page=7) 10 times faster than DARTS ? ![DARTS_normal_reduction_cells](https://user-images.githubusercontent.com/3324659/116770930-e97d4d80-aa79-11eb-8aae-ecf867aabe49.png) ![DARTS_complexity_analysis](https://user-images.githubusercontent.com/3324659/116770928-e2eed600-aa79-11eb-90b9-f00546958b12.png)

https://gist.github.com/promach/b6f526c56e20f029d68e6f9041c3f5c0#file-gdas-py-L106-L107