Week 6
Things Learned:
-
Technical Area:
- Basics and usage of Colab
- Building a content based recommendation engine
- Classifying text using multiple classification algorithms: Naive Bayes, SVM, Logistic Regression, LightGBM, Decision Tree, BERT , etc.
- Making some more visual diagrams with the cleaned data
- Improving accuracies of classifiers
- Confusion matrices and classification reports
-
Tools:
- Google Colaboratory
- GitHub
- Python
- Trello
- VSCode
- Many libraries: pandas, texblob, nltk, numpy, sklearn, gensim, requests, matplotlib, collections, seaborn, simpletransformers, tarfile, os, etc.
-
Soft Skills:
- Watched presentations of other teams and learned many DOs and DON’Ts.
- Checked out the Thoughts/tips on presentations and interviews post and learned some important tips.
Three Achievement Highlights:
- Built multiple classifiers and recommenders
- Scraped urls
- Started our final presentation slides
Tasks completed:
- Completed the 4th part of processing textual data ( N-grams, TF-IDF, Bag of Words, Sentiment Analysis, and Word Embeddings) on Besart’s CSV file, as we chose to move on as a team with his file.
- Went back and scraped all urls of all posts from every Tapas category and uploaded the updated CSV files to github so that it can be used to output clickable titles for our final recommendation system.
- Built tested, and improved multiple classifiers and recommenders for the data: Naive Bayes, SVM, Logistic Regression, LightGBM, Decision Tree, and BERT classfiers; post title and original post based recommenders.
- Started making the final presentation for our team.
- Leads met and discussed about the final tasks, timeline, and asked to present next week on Wed.
- Attended our weekly team meeting and updated Trello.
Goals for Next Week:
- Put the best classifier in an app with the team
- Try to work on building a recommender that specifically solves our problem statement
- Finish, practice, and give our final presentation,