Objective: The objective of this project is to build a text classification system that can accurately categorize customer support tickets based on their content. This system will help automate the ticket routing process, ensuring that each ticket is directed to the appropriate support team, improving efficiency and customer satisfaction.
Learning Outcomes: By completing this project, you will:
- Gain a deep understanding of prompt engineering and its role in shaping the behavior of language models.
- Learn how to preprocess and transform text data to create effective training datasets.
- Acquire knowledge and skills in building and fine-tuning a text classification model using the Hugging Face Transformers library.
- Develop expertise in evaluating the performance of a text classification model using appropriate metrics.
- Learn how to deploy a text classification model as a web service using Flask, allowing for real-time predictions.
Steps and Tasks:
- Set up the Project Environment
- Install the required libraries: transformers, pandas, scikit-learn, flask.
- Import the necessary modules and functions: AutoModelForSequenceClassification, AutoTokenizer, Trainer, TrainingArguments, pipeline, load_dataset, load_metric, Flask, request, jsonify.
- Load the customer support ticket dataset into a pandas DataFrame.
- Preprocess the Text Data
- Clean the text data by removing unnecessary characters, numbers, and special symbols.
- Split the dataset into training and testing sets.
- Encode the text labels into numerical values.
- Initialize the Text Classification Model
- Load a pre-trained language model suitable for text classification tasks.
- Define the text classification model architecture and configuration.
- Create a data processing function to prepare the text data for model training.
- Train the Text Classification Model
- Fine-tune the pre-trained model using the training dataset.
- Set up the training parameters, such as the number of epochs and batch size.
- Evaluate the model’s performance on the testing dataset using accuracy as the metric.
- Implement Ticket Classification Service
- Create a Flask web application.
- Define a route for the ticket classification service.
- Load the trained model for making predictions.
- Preprocess the incoming ticket text using the same data processing function as during training.
- Return the predicted category for the ticket.
- Deploy the Ticket Classification Service
- Run the Flask application locally.
- Test the ticket classification service using sample ticket texts.
Evaluation: For this project, your primary evaluation metric will be accuracy, which measures the percentage of correctly classified customer support tickets. Additionally, you can consider using other metrics like precision, recall, and F1-score to evaluate the model’s performance in handling different ticket categories.
Resources and Learning Materials:
- Prompt Engineering: OpenAI Cookbook
- Text Classification with Transformers: Hugging Face Tutorial
- Building a Web App with Flask: Flask Documentation
- Data Cleaning Techniques: Analytics Vidhya Article
- Introduction to Scikit-learn: Scikit-learn Documentation
Need a little extra help?
Step 1: Set up the Project Environment
- Install the required libraries: transformers, pandas, scikit-learn, flask.
- Import the necessary modules and functions: AutoModelForSequenceClassification, AutoTokenizer, Trainer, TrainingArguments, pipeline, load_dataset, load_metric, Flask, request, jsonify.
- Load the customer support ticket dataset into a pandas DataFrame.
To install the required libraries, you can use the following commands:
!pip install transformers
!pip install pandas
!pip install scikit-learn
!pip install flask
Here’s how you can import the necessary modules and load the dataset into a pandas DataFrame:
from transformers import AutoModelForSequenceClassification, AutoTokenizer, Trainer, TrainingArguments, pipeline
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
import pandas as pd
from flask import Flask, request, jsonify
# Load the dataset into a pandas DataFrame
df = pd.read_csv('customer_support_tickets.csv')
Step 2: Preprocess the Text Data
- Clean the text data by removing unnecessary characters, numbers, and special symbols.
- Split the dataset into training and testing sets.
- Encode the text labels into numerical values.
Here’s a code snippet that demonstrates how you can preprocess the text data:
import re
# Clean the text data
def clean_text(text):
text = text.lower() # Convert to lowercase
text = re.sub(r'\d+', '', text) # Remove numbers
text = re.sub(r'[^\w\s]', '', text) # Remove special characters
return text
df['cleaned_text'] = df['text'].apply(clean_text)
# Split the dataset into training and testing sets
train_texts, test_texts, train_labels, test_labels = train_test_split(df['cleaned_text'], df['label'], test_size=0.2, random_state=42)
# Encode the text labels into numerical values
label_encoder = LabelEncoder()
train_labels = label_encoder.fit_transform(train_labels)
test_labels = label_encoder.transform(test_labels)
Step 3: Initialize the Text Classification Model
- Load a pre-trained language model suitable for text classification tasks.
- Define the text classification model architecture and configuration.
- Create a data processing function to prepare the text data for model training.
Here’s how you can initialize the text classification model:
# Load a pre-trained language model suitable for text classification tasks
model_name = "bert-base-uncased"
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=len(label_encoder.classes_))
# Define the text classification model architecture and configuration
tokenizer = AutoTokenizer.from_pretrained(model_name)
# Create a data processing function to prepare the text data for model training
def preprocess_function(examples):
return tokenizer(examples['text'], padding=True, truncation=True)
Step 4: Train the Text Classification Model
- Fine-tune the pre-trained model using the training dataset.
- Set up the training parameters, such as the number of epochs and batch size.
- Evaluate the model’s performance on the testing dataset using accuracy as the metric.
Here’s an example of how you can train the text classification model:
# Fine-tune the pre-trained model using the training dataset
training_args = TrainingArguments(
output_dir='./results',
num_train_epochs=3,
per_device_train_batch_size=16,
per_device_eval_batch_size=64,
warmup_steps=500,
weight_decay=0.01,
logging_dir='./logs',
logging_steps=10,
evaluation_strategy='epoch'
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=test_dataset,
compute_metrics=compute_metrics
)
trainer.train()
# Evaluate the model's performance on the testing dataset using accuracy as the metric
results = trainer.evaluate()
print(f"Accuracy: {results['eval_accuracy']}")
Step 5: Implement Ticket Classification Service
- Create a Flask web application.
- Define a route for the ticket classification service.
- Load the trained model for making predictions.
- Preprocess the incoming ticket text using the same data processing function as during training.
- Return the predicted category for the ticket.
Here’s an example of how you can implement the ticket classification service:
app = Flask(__name__)
@app.route('/classify', methods=['POST'])
def classify_ticket():
ticket_text = request.form['text']
# Preprocess the incoming ticket text using the same data processing function as during training
processed_input = preprocess_function({'text': [ticket_text]})
# Make a prediction using the trained model
predictions = trainer.predict(processed_input)
predicted_label = label_encoder.inverse_transform(predictions.argmax(axis=1))[0]
return jsonify({'predicted_label': predicted_label})
if __name__ == '__main__':
app.run()
Step 6: Deploy the Ticket Classification Service
- Run the Flask application locally.
- Test the ticket classification service using sample ticket texts.
To run the Flask application locally, you can use the following code:
if __name__ == '__main__':
app.run()
You can test the ticket classification service using sample ticket texts by sending a POST request to the ‘/classify’ route with the text data. Here’s an example using the requests library:
import requests
text = "I'm having trouble with my internet connection."
response = requests.post('http://localhost:5000/classify', data={'text': text})
print(response.json())
Remember to replace ‘http://localhost:5000’ with the appropriate URL if you deploy the Flask application to a remote server.