RuntimeError on running ALBERT for obtaining encoding vectors from text

nlp
1

I’m trying to get feature vectors from the encoder model using pre-trained ALBERT v2 weights. i have a nvidia 1650ti gpu (4 GB) , and plenty RAM but for some reason I’m getting Runtime error saying -
RuntimeError: [enforce fail at …\c10\core\CPUAllocator.cpp:75] data. DefaultCPUAllocator: not enough memory: you tried to allocate 491520000 bytes. Buy new RAM!

I’m really new to pytorch and deep learning in general. Can anyone please tell me what is wrong?

My entire code -
encoded_test_data = tokenized_test_values[‘input_ids’]
encoded_test_masks = tokenized_test_values[‘attention_mask’]

encoded_train_data = torch.from_numpy(encoded_train_data).to(device)
encoded_masks = torch.from_numpy(encoded_masks).to(device)

encoded_test_data = torch.from_numpy(encoded_test_data).to(device)
encoded_test_masks = torch.from_numpy(encoded_test_masks).to(device)

config = EncoderDecoderConfig.from_encoder_decoder_configs(BertConfig(),BertConfig())
EnD_model = EncoderDecoderModel.from_pretrained(‘albert-base-v2’,config=config)
feature_extractor = EnD_model.get_encoder()
feature_vector = feature_extractor.forward(input_ids=encoded_train_data,attention_mask = encoded_masks)
feature_test_vector = feature_extractor.forward(input_ids = encoded_test_data, attention_mask = encoded_test_masks)

Also 491520000 bytes is about 490 MB which should not be a problem.

2

How much host RAM does htop or any other tool report?
As the error message suggest, PyTorch cannot allocate the ~500MB of RAM, so I guess you might be indeed running out of memory or the process is not allowed to allocate more RAM for some reason.

3

Im using windows. I have around 2.8 GB available RAM. Pycharm IDE is using 600 MB. I doubt there is a cap set on the RAM usage.

4

2.8GB seems to be quite low, since the OS (and other processes) would also use host memory, so the OOM error seems to be valid.

5

Oh no. Its 2.8 GB memory of actually unused RAM. I have 8 GB RAM, and background process are using about 5 GB of it. And also, I tried reducing the number of training examples and also the length of the maximum padded input . The OOM error still exists even though the required space now is 153 MB , which should easily be managable.
I also have maxed out the RAM limit of the heap of pycharm software to 2048 MB. I really dont know what to do now… Anyways thank you for spending the time to help me!