MapDataPipe support Mux / Demux

I noticed that there isn’t the capability to Mux/Demux a MapDataPipe. The use case I’m looking at is splitting a MapDataPipe into train, validation, etc… datapipes. I wasn’t sure if this would constitute creating an actual issue on their github, especially if it was already planned.

Thanks for taking interest. We are willing to review your PR for them.

Ok sounds good, would it be a good idea to make an issue with what I come up with first? Once we agree what the implementation objects look fine I can make a proper pr with tests etc. It would probably be this weekend that I would have something made.

We hava a Issue to track the missing MapDataPipe in PyTorch Core. [DataLoader] Implement Map-style functional DataPipe · Issue #57031 · pytorch/pytorch · GitHub
You can grab one and open a PR. If you want to discuss about the implementation, I am totally fine with either over issue or PR directly.

1 Like