Forward pass computation for GDAS NAS coding

promach · June 7, 2021, 4:09am

For https://gist.github.com/promach/b6f526c56e20f029d68e6f9041c3f5c0#file-gdas-py-L143-L169 , how to do forward pass training computation for architecture weights of each and every edges ?

github.com/D-X-Y/AutoDL-Projects

Questions about DARTS

opened 04:47AM - 01 May 21 UTC

buttercutter

question

1. For DARTS complexity analysis, anyone have any idea how to derive the *(k+1)\…*k/2* expression ? Why 2 input nodes ? How will the calculated value change if graph isomorphism is considered ? Why "2+3+4+5" learnable edges ? If there is lack of connection, the paper should not add 1 which does not actually contribute to learnable edges configurations at all ? 2. Why need to train the weights for normal cells and reduction cells **separately** as shown in Figures 4 and 5 below ? 3. How to arrange the nodes such that the NAS search will actually converge with minimum error ? Note: Not all nodes are connected to each and every other nodes 4. Why is [GDAS](https://arxiv.org/pdf/1910.04465.pdf#page=7) 10 times faster than DARTS ? ![DARTS_normal_reduction_cells](https://user-images.githubusercontent.com/3324659/116770930-e97d4d80-aa79-11eb-8aae-ecf867aabe49.png) ![DARTS_complexity_analysis](https://user-images.githubusercontent.com/3324659/116770928-e2eed600-aa79-11eb-90b9-f00546958b12.png)