Dear PyTorch Community,
I am reaching out to seek your guidance on an intellectual property concern that I have encountered while attempting to conduct a private deployment of the LLM for my customers. The issue pertains to the potential vulnerability of the model weights and associated server codes when they are deployed on the customers’ hardware.
As the customers possess root permissions to the hardware, I am uncertain about the level of protection that can be provided to prevent the model weights from being leaked or compromised. I am particularly concerned about the ease with which customers may be able to copy the model weights directly from the GPU memory.
I would greatly appreciate any insights or recommendations that you may have to address this concern.