CIS623 - Fine-Tuning LLMs

Course Description

In this course, you will discover how to adapt and refine large language models (LLMs) for tasks beyond their default capabilities by creating curated training sets, tweaking model parameters, and exploring cutting-edge approaches such as preference learning and low-rank adaptation (LoRA). You’ll start by fine-tuning a base model using the Hugging Face API and analyzing common optimization strategies, including learning rate selection and gradient-based methods like AdaGrad and ADAM.

As you progress, you will evaluate your models with metrics that highlight accuracy, precision, and recall, then you’ll extend your techniques to include pairwise preference optimization, which lets you incorporate direct user feedback into model improvements. Along the way, you’ll see how instruction-tuned chatbots are built, practice customizing LLM outputs for specific tasks, and examine how to set up robust evaluation loops to measure success.

By the end of this course, you’ll have a clear blueprint for building and honing specialized models that can handle diverse real-world applications.

You are required to have completed the following courses or have equivalent experience before taking this course:

• LLM Tools, Platforms, and Prompts
• Language Models and Next-Word Pronunciation

Faculty Author

Benefits to the Learner

• Gather and organize task-specific data to build a focused training set suited for targeted outcomes
• Duplicate and customize a pretrained model with the Hugging Face API to address specialized use cases
• Identify and predict the effects of optimization choices, such as learning rates, on model performance
• Determine and apply appropriate evaluation metrics for diverse model architectures and objectives
• Develop instruction-tuned chatbot capabilities, enabling interactive question-and-answer functionalities
• Fine-tune LLMs and critically assess their outputs across various contexts
• Use pairwise preference data to optimize models

Target Audience

• Engineers
• Developers
• Analysts
• Data scientists
• AI engineers
• Entrepreneurs
• Data journalists
• Product managers
• Researchers
• Policymakers
• Legal professionals

Applies Towards the Following Certificates

• Large Language Model Fundamentals : Required Courses