Hugging Face Transformers

Introduction

Hugging Face provides pre-trained models and easy-to-use APIs.

Basic Usage

from transformers import pipeline

# Sentiment analysis
classifier = pipeline("sentiment-analysis")
result = classifier("I love this movie!")
print(result)

# Question answering
qa = pipeline("question-answering")
result = qa(question="What is Python?", context="Python is a programming language.")

Using Specific Models

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
model = AutoModel.from_pretrained("distilbert-base-uncased")

inputs = tokenizer("Hello, world!", return_tensors="pt")
outputs = model(**inputs)

Fine-tuning

from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=16,
    learning_rate=5e-5
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset
)
trainer.train()

Practice Problems

1. Use pipeline for common tasks
2. Load specific pre-trained models
3. Tokenize text for models
4. Fine-tune on custom data
5. Extract embeddings from models