-
Technical Area: I learned how to web scrape and crawl on a forum for the very first time with python and beautifulsoup as well as uploading a pickle file.
-
Tools: I learned how to use GitHub, VS code,slack and Asana.
-
Soft Skills: I learned that effective communication is essential in order to get the tasks done. I also learned to work hard to achieve the goal.
Achievements -
- I was able to build a full web scraper with a forum through python.
-
- I was able to parse using beautifulsoup for the very first time.
-
- I built a web crawler for the first time.
-
Meetings: Colin’s Webinar, Malahi’s Webinar, Sara’s Webinar, Every Google Meet within our team (Mondays and Fridays at 8AM pacific)
-
Goal: To build a recommender system using BERT.
Tasks
- Each of us were assigned a forum to scrape and build a scraper using python. I used beautiful soup to parse the data and some of the difficulties I faced were not being able to decode the json file. It had come to my attention that not only I was struggling with this issue but some of my teammates had faced this issue and so our team lead called a meeting and troubleshooted the issue. Also, I myself did not know that we needed to install beautifulsoup in python and I thought it was a built in interface so I struggled with that but luckily I watched Maleeha’s webinar and searched up online how to install beautifulsoup. The other issues I faced such as the syntax errors in my codes and variable strings were solved by using the internet.