Fine-tuning and LLM
Please work on this assignment in pairs.
If you need help, please ask Nyomi; she developed this assignment.
Overview
For this assignment you will try your hand at fine-tuning a large
language model. You will explore existing datasets and pre-trained
models and (hopefully) be able to produce a model that executes a
specific task.
Model Fine-tuning
Training a large language model to the level of commercially available
products such as ChatGPT, Bard, or others requires more computational
power than individuals such as ourselves could possibly have. That’s
why we’ll be doing fine-tuning! Informally, this essentially means
taking a smaller, pretrained version of the model and training on some
additional data for the model to be good at what we personally need it
to do.
For this, we will be using the LoRA (Low-Rank Adaptation of Large
Language Models) method which you might see a lot if you look at
fine-tuning tutorials made within the last few months. For our
purposes, we will treat this step of the process kind of like a black
box, but you can get deeper with the following resources:
Part 1: Let’s get something to work!
We will work with the TinyLlama model to do some fine-tuning. This
model is an instance that is theoretically accessible to common users
who want to do some smaller scale language tasks. Nyomi has created a
Colab
notebook to try fine-tuning the model to tell stories which is
inspired in combination with these three tutorials:
Please make a copy of the Colab notebook and take a look at the major
steps of the process. Try to run it and see if it works! DISCLAIMER:
The free version of Colab sometimes will not have GPUs available so
sometimes, you’ll have to delay running training-intensive processes
to another time. (One trick I’ve used outside of waiting is an
incognito tab or a different browser). You can explore other places to
host your computation if you need.
Things to note: We are training with 1000 rows from the dataset so
that this part of the assignment does not take too long! You might
have to adjust it though.
Create a lab manual and add both your names to it. Then provide the
following:
- Please provide a screenshot of the output from the model’s
inference.
- Discuss the performance of the model. Does the output make sense?
If not, what could make it better?
Part 2: Fine-tune your own model
Now that you’re a little familiar with the fine-tuning process, it’s
time to explore some datasets and fine-tune a model for your own
purpose. You can use the example from above, inspiration from an
online tutorial, or your own method to accomplish this since there are
many ways to do this. Please be sure to document the process and
successes/difficulties here in the lab notebook.
General Process Notes:
- Please provide a link to the dataset that you chose to fine-tune
with. You can search HuggingFace, Kaggle, or other sites for ones that
already exist. You can also consider using a large language model to
generate custom data to fine-tune your large language model if you
can’t find anything online.
- If you decide to fine-tune a different language model than
TinyLLama, or use any new and interesting methods please discuss them
here.
- What are the limitations of your model? What do you notice about
the performance during inference?
- Please provide some screenshots of a few prompts and responses from
your model.
Submission
Please submit a zipped copy of the following items to the appropriate
drop-box on Moodle.
- The lab manual.
- The code for fine-tuned and improved models.