Overview of the STEM Away Project Experience:
Things I did in the past 5 weeks:
Week 1- Got acquaited with different web scraping libraries in python through reading the resources provided by project lead and researcing them. Also got familiarised with discourse forums.
Week 2- Selected a discourse forum to work on and implemented these web scraping tools to get the data (different posts) out of the decided forum. (atom discussion by github). Stored all the data in a csv file to implement Machine Learning on it. Cleaned the text data for the NLP algorithm.
Week 3- Studied different NLP algorithms to tokenise and vectorise the text. Implemented different algorithms and checked the accuarcy of all of these. Decided that BERT would be the most accurate model and implemented it on the test data set.
Week 4- The team decided to take the project further so we got divided into two groups and started to scrape a different forum (ketogenic) and performed the same tasks as we did for atom discussion. The end goal of the extended project was to create a recommendation system which would take a text input and be able to predict similar posts and which forums the text would be found.
Week 5- My group then performed several classification models to classify these texts such that the end goal would be realised.
Technical skills I learnt: Webscraping, NLP Algorithms, Classification models
Soft Skills I learnt: Team work, working remotely on a project