Hello, new user here and new to machine learning and pytorch framework. So, I have a sequence of vector embeddings that are generated by running input sequences through a GAN discriminator and capturing the output. I want to take this sequence and feed it back into another discriminator to generate embeddings over the sequence of the original embeddings.
The issue here is that the vector space over the potential combination of vector embeddings in a sequence of, say, even 5 vectors in sequence, is HUGE. Far more than can be held in my working environment, and I plan on doing this a 3rd time as well.
I was thinking that I could potentially override the nn.embedding functionality to implement a system other than a simple lookup table. Essentially, instead of trying to store every possible vector combination as another vector, I instead store mappings between input vectors and vector transformations. e.g. given one vector, the embeddings method looks up a transformation, applies to the given vector, and then spits out a new vector to represent the next vector in the sequence. The stored transformations essentially become the vector lookup table instead of trying to capture all possible vector combination sequences.
I asking here however because I’m very green at PyTorch. This is actually part of my Senior Project project at my university, and me being the overambitious little scrub that I am, chose a project that was… near the upper ends of what I have experience with, lol.
So, I really want to know: Is there a better way of doing this? My project entails using GAN networks to process nested sequences of information. I’m essentially using a first level GAN to processes the innermost sequences, but then I need to feed the data back and processes sequences of level 1 sequences, and then once again, where it is sequences of level 2 sequences of level 1 sequences.
Also, over the general topic of processing nested sequences, does anyone have any advice or information on that research topic in general? e.g. “Processing nested sequences of sequences” using GAN networks? This semester has been super information overload, haha, and I’m going to be needing all the help I can get in order to pull of my Senior Project.
Thank you very much for your time and in advance!