About step-by-step "Building,Training" a net and model for very easy example


#1

I had read lots of examples, most of them teach how to recognize photos or something more complex.
For a absolute novice, I just want to try a simple question like:
inputs: 100 of 1X1 matrix, like 1,2,3,…,100
and outputs: still 100 of 1X1 matrix, like 1,4,9,16,…,10000 , which is the input’s square
I tried to digest lots of information from every websites, but I still cannot understand what should I do to build a net, train a model, and at the end how can I have a well-trained model. At the end, I can open this model and try to give it an input and try to get an output from it.
I think only after I understand such a simple question, I can move forward to handle some difficult question.
(actually, I had tried this on matlab, and it is easy ,just have to type on excel, click some button and it is done.)


(Justus Schock) #2

I will create a small jupyter notebook containing an example for this (with some explanations inside). Are you familiar with the theoretic background? I will post it in a few hours.


#3

Thank you very much!
I had read so many examples and find out that it’s so difficult to make everything together(from the very beginning to the end). :disappointed_relieved:


#4

I understand the theoretic background , but It’s so difficult for me to write something out. especially most of the examples are so complex!


(Justus Schock) #5

Great, so I won’t spend much time on the theoretic background but explain the implementation. I’ll post here again, once it’s finished (probably in about 5-6 hours since I got some other stuff to do before ).


#6

okay!!
Thanks again!


(Justus Schock) #7

Here you go:

https://nbviewer.jupyter.org/gist/justusschock/63e8d3258d313e63b80ebbe218343532

If you execute it youself, you will also see a progressbar (unfortunately it is not rendered by nbviewer).

Note that this example is very minimalistic and far from complete, but it should give you a first impression.


#8

Thanks a lot!
I will try my best to understand every details of it!


#9

Hello,
I had tried my best to understand the codes, but there are still many questions.
at the beggining :
class CustomIntegerDataset(Dataset):
def init(self, transforms=None, length=100000, max_int=100, min_int=0):"
how did you decide that length? I find that the different length do affect the result and the running time very much!!!
and does that mean the data is filled with 1~100?
like 1,2,3,…,99,100,1,2,3…,99,100,1,2,3… to 100000 numbers?
or the rest filled with 0 like 1,2,3,…100,0,0,0,…
Thanks for replying!


(Justus Schock) #10

In this case the length is an arbitrary number I chose for simplicity. Usually the length of your dataset would give you the number of samples in it. Since 100 iterations are very few for a Network to train you could either use much more epochs or artificially increase the number of iterations per epoch (as I did).

To clarify:
Your dataset only contains max_int - min_int number of samples (one for each number between them). If the index is greater than this number of samples it will be mapped down to the range using the modulo operator:

true_index = index % len(self.data)
        data = self.data[true_index]

If you have 100 samples of data (meaning indices 0-99) and you would try to index sample 100 (which would be a valid operation since it is inside the range defined by your dataset’s length) you would usually get an index which is out of bound for your data. To prevent this, the modulo operator maps it down. Index 100 % 100 (which is the number of your data samples) would equal a real index of 0, Index 101 would equal a real index of 1 etc.

I hope this helps.
Let me know if there are other questions.


#11

Hello!
recently, i had tried to change some numbers in the code and have found many new things,
however, I failed to change the input and output form, which is difficult for me.
I think I need this example to learn how to Change these things.:
I want to make the dataset being more than 11
actually, I have many 7
4 tensors in txt file, i would like to let them be the input
and every file will correspond to a 51 tensor, it is also in another txt file, which will be the output
for example:
input:
1 1 1 1 1 1 1
2 2 2 3 4 3 2
4 3 2 2 3 4 3
4 4 3 2 6 2 3
output:
4 6 2 0 4
With lots of data like this
I think the only thing i have to change is
1.write codes to read these files, (and change them from number to tensor?)
2.change the input form from 1
1 to 74
3.change the layer structure to ensure the input is 28 and output is 5(or7
4 and 51?)
4.change the output form from 1
1 to 5*1
These are something that i’m struggling at now :disappointed_relieved::disappointed_relieved::disappointed_relieved:
Can you help me to change some numbers and structures from the first version you wrote for me to make these happen?
Thanks a lot!


(Justus Schock) #12

I won’t be able to do this today, but tomorrow it should be possible for me to do this.


#13

ofcourse!
please take your time!
And thanks again!!!


(Justus Schock) #14

Sorry, it took a bit longer.

It should work by changing the input from 1 to 74 and the output from 1 to 51.

Like this:

class SimpleCustomModel(torch.nn.Module):
    def __init__(self, num_layers=5, max_hidden_dim=512):
        
        super().__init__()
        
        layers = [torch.nn.Linear(71, 96),
                 torch.nn.ReLU()]
        
        curr_dim = 16
        for i in range(num_layers - 2):
            new_dim = min(curr_dim * 2, max_hidden_dim)
            layers += [torch.nn.Linear(curr_dim, new_dim),
                      torch.nn.ReLU()]
            curr_dim = new_dim
            
        layers.append(torch.nn.Linear(curr_dim, 51))
        
        self.model = torch.nn.Sequential(*layers)
        
    def forward(self, x):
        return self.model(x)

Note: I modified the internal structure a bit so that it should better suit your problem.

Unfortunately, I’m unable to help you with your data loading code without example files.


#15

Hi!
now I understand how to change the layer’s input and output! thanks a lot!
Sorry about not asking the question clearly.
And also, I will have some example files here.
If I want to input a 7X4(that means 7 cross 4 and have 28 numbers. I’m not really sure is it important to input 7X4 or its okay to input 28 numbers in a roll.)
And now I have some files on google drives:
https://drive.google.com/drive/folders/12Egi5jDaUnGRvVyddP2i5BgFS7gkjMSB?usp=sharing
There are 10 of 7X4 inputs files and 10 5X1 output files
can you teach me how to read these files (actually, if they are put in the same folder of this .py file), transfer them into matrixs or tersors that can be used as input and then train the model .
and how about the input and output layers? how can I modify the layers strucure to make an 7X4 matrix to be an input and 5X1 to be an output?

(By the way, each input files are almost the same, but I just want to make some example files so i think its fine!)