- Model : The model parameters, including weights. Where applicable, these should include checkpoints from key intermediate stages of training as well as the final optimizer state.
The “model parameters” and the “model weights” are completely the same thing. Those are exactly what a “pre-trained” model is about. Those “parameters” or “weights” are automatically learned by the algorithm.
For instance, a “model parameter” can be the pre-trained Pythia 7B LLM model, dumped into the huggingface safetensors or pytorch state dictionary formats.
Do not confuse with “hyper-parameters”. Those are parameters controlling the model behavior through training or inference, but they are tuned manually by human or an “meta” algorithm (such as grid search) independent to the original algorithm.
For instance, a “hyper-parameter” can be the temperature for LLM during inference.