@J_Johnson me again, I’m slowly working my way through your and @vdw 's options here to understand what will work best.
I’m still learning about Transformers, for the TransformerEncoder approach you mentioned, would you be able to share a rough example of what a TransformerEncoder model itself might look like in this instance? I’ve started a thread here as I’m having trouble adapting the PyTorch example to a classification task like this. Happy to reward you for your time. 