Hi there! Well I am trying to persist 500 million Torch vectors (1024 dim float dtype). Since it take up too much memory space, I was suggested to do a quantization (from Float to Int8). Being really sincere, I do not know how to do that and as usual I will keep researching. In the meanwhile some nice help would be really appreciated. Thanks so much
How much are you hoping to reduce storage by? Converting to int8 will cause you to lose information. Maybe you could try converting to float16 instead of float32 and see if that is enough first? That is likely to lose less information than converting to int8. For a tensor x, you can convert it to use float16 using x.half().