This is a training script I made so that I can fine-tune LLMs using my workstation with four 4090s ... Then, the data parallelism is automatically set so that all GPUs are used. For example with 8 ...

Many options present in other training scripts are not implemented. If you want more features added ... Start by reading through the config files in the examples directory. Almost everything is ...