Hi everyone, I’m new in this community and relative new in IA.
I downloaded the llama-2-7b-chat from meta and I want to run this in Java.
First I tried to load the module with:
Module mod = Module.load("llama-2-7b-chat/consolidated.00.pth", extraFiles, Device.CPU);
but I have theese errors:
Exception in thread "main" com.facebook.jni.CppException: PytorchStreamReader failed locating file constants.pkl: file not found
Exception raised from valid at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\caffe2\serialize\inline_container.cc:184 (most recent call first):
I read that is a problem with the model’s format and I should convert it into a pt (TorchScript model).
I’ve tried with a lot of code, and in my last attemp I wrote this:
import torch
from transformers import LlamaPreTrainedModel
#print(torch.__version__)
# Carga tu modelo entrenado en PyTorch
#model = torch.jit.load('consolidated.00.pth', map_location=torch.device('cpu'))
model = LlamaPreTrainedModel.from_pretrained(pretrained_model_name_or_path='consolidated.00.pth',
config='params.json',)
print(type(model))
# Convierte el modcelo a TorchScript
scripted_model = torch.jit.script(model)
print(type(scripted_model))
# Guarda el modelo en formato TorchScript
#torch.save(scripted_model, 'llama_chat.pt')
I can see only the lines of the error but not the exception, my terminal only displays:
StopIteration
Somebody managed to convert this model to torchscript?
Thanks for read!