HashNotes
Hugging Face Transformers - An Introduction

Hugging Face Transformers - An Introduction

Mohamad Mahmood's photo
Mohamad Mahmood
·

3 min read

Hugging Face is a company that provides an open-source platform for natural language processing (NLP) called Transformers.

The Transformers library by Hugging Face is widely used in the machine learning community for working with pre-trained models in NLP tasks.

What is Hugging Face Transformers?

Hugging Face Transformers is an open-source library that provides a large collection of pre-trained models for various NLP tasks, including text classification, language translation, text generation, and more.

These models are based on state-of-the-art architectures like BERT, GPT, RoBERTa, and many others.

How to Use Hugging Face Transformers:

1. Install the Library:

Install the library using pip:

pip install transformers

2. Loading a Pre-trained Model:

Load a pre-trained model using the from_pretrained method:

from transformers import AutoModel, AutoTokenizer

# Replace 'bert-base-uncased' with the model of your choice
model_name = 'bert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

3. Tokenizing Text:

Before feeding text to a model, it needs to be tokenized.

Transformers library provides easy-to-use tokenizers:

text = "Hugging Face Transformers is awesome!"
tokens = tokenizer(text, return_tensors='pt')

4. Model Inference:

Once tokenized, the model can be used for inferences:

outputs = model(**tokens)

5. Extracting Results:

Depending on the downstream tasks, different outputs will be extracted from the model:

last_hidden_states = outputs.last_hidden_state

6. Fine-Tuning:

If fine-tuning a pre-trained model for specific tasks is needed, use the Trainer class provided by the library.

Example:

A simple example of using a pre-trained BERT model for text classification:

from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Step 1: Load Pre-trained Model and Tokenizer
model_name = 'bert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Step 2: Tokenize Your Input
text = "i like hugging face"
tokens = tokenizer(text, return_tensors='pt')

# Step 3: Perform Inference
outputs = model(**tokens)

# Step 4: Interpret the Results
logits = outputs.logits
predicted_class = logits.argmax().item()

print(f"Predicted Class: {predicted_class}")

This is just a basic guide.

Explore the documentation and fine-tune the models accordingly.

Summary of the steps:

  1. Load Pre-trained Model and Tokenizer:

    • AutoTokenizer.from_pretrained: Loads the tokenizer associated with the pre-trained model.

    • AutoModelForSequenceClassification.from_pretrained: Loads the pre-trained model for sequence classification.

  2. Tokenize Your Input:

    • Use the tokenizer to convert the input text into a format suitable for the model.

    • return_tensors='pt' ensures the output is in PyTorch tensors.

  3. Perform Inference:

    • Pass the tokenized input to the model and get the outputs.

    • In this example, model(**tokens) is used to pass the input tokens as keyword arguments.

  4. Interpret the Results:

    • Extract relevant information from the model's output. For classification tasks, often look at the logits or probabilities.

    • In this case, outputs.logits contains the model's predictions.

Adjust this example based on the specific requirements of NLP tasks and the model used.

Different tasks may require different model configurations and post-processing steps.

Always refer to the documentation of the specific model that are relevant to NPL objective.

Remember to check the Hugging Face Transformers documentation for more details and advanced use cases: Hugging Face Transformers Documentation.

Colab Notebook:

https://drive.google.com/file/d/1lUmp5Y8w6qY_szmyEJ4assl4yqHfdBXI/view?usp=sharing

huggingfacetransformers