Hi, is it possible to run a simple lora fine tuning job on my mac, no gpu, with torchtune.
I’ve followed the Fine-Tune Your First LLM — torchtune 0.6 documentation but I run out of memory and there’s possibly a hard requirement for a GPU?
Is there a recipe that can be used just to try this out locally on my mac?
Thanks
Macs use ‘metal plate shaders’ or ‘mps’ and unified memory, so unless your total system memory is shot you should be okay. Don’t forget to set the device to ‘mps’ instead of ‘cpu’ and of course don’t use ‘cuda’. There are some issues with non mps implemented things, so you should set a system var (export PYTORCH_ENABLE_MPS_FALLBACK=1) to enable fall back to CPU, you probably will get the relevant errors that point that out. If it’s just memory related, tough luck, you have to either reduce the inputs, or use a less demanding module.