Let's discuss **project architecture**

1

I spend a lot of time building projects in PyTorch and have not yet found a nice structure that encapsulates everything I want to do in a clean, decoupled way.

Mostly I want to decouple :

  • model definition
  • data management (preprocessing, dataloaders, I/O)
  • training
  • evaluation
  • logging

My current approach would be to use a class handling all data management issues, and a class containing the network definition.

Training and evaluation would be performed by different classes, let’s call them “engines”. Those engines would have the Data and Model class as attributes, and a logging as well.

How do you structure your own project ? Are there certain caveats to having an object as an attribute of multiple different objects ?

Let’s start a discussion !

2

Your approach sounds a bit like a higher-level API, such as Catalyst, Ignite, or Lightning, which uses layers of abstractions for these training, logging, checkponting etc. routines so you might want to take a look at them.

1 Like