@mrshenli This question is not that closely related to pytorch, but it is really hard to find related resources on stackoverflow etc. My framework (mentioned in last post) development have finally reached a critical stage, I am testing the distributed part now.
So do you know any python libraries such as DEMi, fuzzowski which can simulate a connection layer and let users send & recv, shift, reorder & delay, poke & inspect messages? Since I need to simulate the RPC layer, these two libraries metioned above are not appropriate for this task. What are the development team of pytorch using to test your rpc service? Any idea would be great.
BTW, my library is hosted at https://github.com/iffiX/machin Come and have a look! the tutorial is not complete though, but api documentation and tests for most core functions should be close to complete now, will release the first milestone soon.
Hey @iffiX, thanks a lot for sharing the exciting project!! We will certainly study and share with other team members.
Regarding tests for delay/drop/retry messages, @osalpekar implemented a faulty agent for that purpose, and the majority of faulty tests can be found here.
So do you know any python libraries such as DEMi, fuzzowski which can simulate a connection layer and let users send & recv , shift, reorder & delay , poke & inspect messages? Since I need to simulate the RPC layer, these two libraries metioned above are not appropriate for this task.
If this is just for simulating RPCs, does mock work in this case? Sth like this.
Also cc TensorPipe expert @lcw, do you know any good tools for this purpose?
Mock is not sufficient, because I would like to implement tests based on fuzzy testing for machin.parallel.distributed.election.ElectionGroupStableBase and other higher level modules based on this core. Common white box testing and black box testing are not good enough for testing distributed core functions, so my testing idea comes from this github repo and the blog post from its “Fuzzing” section blog.
I think I will try to implement a simple framework myself today, maybe generate fuzzy data with boofuzz and write a rpc simulation layer. I am still looking forward to other ideas of your team!
Come and have a look! the tutorial is not complete though, but api documentation and tests for most core functions should be close to complete now, will release the first milestone soon.
