Hi @ajhepburn, if you run layer conductance on a batch of documents it can definitely speed up things. Only you want to make sure that you have sufficient memory and perhaps create sub-batches combing multiple documents if you can’t fit everything into a memory. Note that the internal requirements of layer conductance is n_steps * #examples. n_steps is defaulted to 50 and you can change it if necessary.
As I can see, you are treating each document as one example, is that right ? If your documents aren’t too large you can definitely try to combine them.
Another option is to use DataParallels and GPUs.
More info about Dataparallels and Distributed Dataparallels can be found here: