I want to customize the pre-trained LLM model downloaded from HuggingFace. There may be operations like adding, removing, replacing or fusing operators.
One approach is to use torch.fx. However, most HuggingFace models take input as condition and torch.fx does not yet support dynamic control flow.
Will TorchScript supersede torch.fx in the foreseeable future? I don’t think TorchScript is an ideal tool to fulfill my goal because I wish to retain the PyTorch modules in code.
Is there any way that I can tweak models from HuggingFace in a flexible fashion?