Hi everybody,
I’ve recently been training a small language model of my own from scratch - no pre-trained models whatsoever. I know that language models predict the next character based on a text input, but I figure that I need to give my language model custom instructions (behind the scenes) to behave like an actual conversational chat. Otherwise, it just continues the encoded text sequence which I have started. Assuming I’m on the right track with custom instructions, what should they look like?
Currently, I have something like this:
input_string = f"You are MODEL_NAME, a conversational language model developed by DEVELOPER. The user said: 'USER_INPUT'. \nYou respond: "
Thanks for any advice!