# Sihongy1 - Machine Learning (Level 1) Pathway

**URL:** <https://stemaway.com/t/sihongy1-machine-learning-level-1-pathway/8125>\
**Category:** Machine Learning Pipeline (NLP) (CE) 2021\
**Created:** [June 13, 2021, 9:57pm UTC](https://stemaway.com/t/sihongy1-machine-learning-level-1-pathway/8125 "2021-06-13T21:57:10Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sihongy1](https://d2nat51eddc6b4.cloudfront.net/user_avatar/stemaway.com/sihongy1/32/3567_2.png) [@sihongy1](https://stemaway.com/u/sihongy1)\
**Post date:** [June 13, 2021, 9:57pm UTC](https://stemaway.com/t/sihongy1-machine-learning-level-1-pathway/8125/1 "2021-06-13T21:57:10Z")

</div>

Technical Area: Basic machine Learning knowledge Basic natural Language Processing knowledge Web scraping and web crawing

Tools: Beautiful Soap Selenium Pytorch Git Github Trello BERT

Soft Skills: Project management Coding skill

Achievements: Pracitce with Beautiful Soap and Selenium Know the basic of ML and NLP Tried web scraping

Goal for the next week: Create Trello for the team and manage the team with tasks

Tasks: Went over the instructions Watched the webinars in the resources Got familiar with what ML is Learned what NLP is, how it is used nowadays

---

<div class="post-metadata">

**Author:** ![sihongy1](https://d2nat51eddc6b4.cloudfront.net/user_avatar/stemaway.com/sihongy1/32/3567_2.png) [@sihongy1](https://stemaway.com/u/sihongy1)\
**Post date:** [June 29, 2021, 8:31pm UTC](https://stemaway.com/t/sihongy1-machine-learning-level-1-pathway/8125/2 "2021-06-29T20:31:02Z")

</div>

Here is the link for my self-assessment 2: [Machine Learning - Level 1 Module 2 - Sihong Yuan](https://stemaway.com/t/machine-learning-level-1-module-2-sihong-yuan/8573)

---

<div class="post-metadata">

**Author:** ![sihongy1](https://d2nat51eddc6b4.cloudfront.net/user_avatar/stemaway.com/sihongy1/32/3567_2.png) [@sihongy1](https://stemaway.com/u/sihongy1)\
**Post date:** [July 13, 2021, 7:05pm UTC](https://stemaway.com/t/sihongy1-machine-learning-level-1-pathway/8125/3 "2021-07-13T19:05:32Z")

</div>

**Machine Learning Level 1 Module 2** (part2)

**Technical Area:**

- Learned how to use Beautiful Soup and Selenium

- Learned how to use webdriver

- Learned how to Web Scrape and save data into a csv file

- Learned how to do exploratory data analysis

**Tools** :

Beautiful Soap, Selenium, Jupyter Notebook, pandas, Tableau, chromedriver

**Soft Skills** :

- Web scraping

- Exploratory Data Analysis

**Achievements:**

- Successfully scraped data from Flowster Forum

- Cleaned the data by getting rid of the html

- Transfered the data into lowercase

- Removed stopwords of my data

**Tasks Completed:**

- Scraped data from Flowster Forum

- Explored my data after gathering it and analyzed it

- Performed basic cleaning to remove html tags from data

- Visualized my date

---

<div class="post-metadata">

**Author:** ![sihongy1](https://d2nat51eddc6b4.cloudfront.net/user_avatar/stemaway.com/sihongy1/32/3567_2.png) [@sihongy1](https://stemaway.com/u/sihongy1)\
**Post date:** [July 13, 2021, 9:05pm UTC](https://stemaway.com/t/sihongy1-machine-learning-level-1-pathway/8125/4 "2021-07-13T21:05:59Z")

</div>

Here is the link for my self-assessment 3: [Machine Learning - Level 1 Module 3 - Sihong Yuan](https://stemaway.com/t/machine-learning-level-1-module-3-sihong-yuan/8886)

---

<div class="post-metadata">

**Author:** ![sihongy1](https://d2nat51eddc6b4.cloudfront.net/user_avatar/stemaway.com/sihongy1/32/3567_2.png) [@sihongy1](https://stemaway.com/u/sihongy1)\
**Post date:** [July 21, 2021, 9:43pm UTC](https://stemaway.com/t/sihongy1-machine-learning-level-1-pathway/8125/5 "2021-07-21T21:43:40Z")

</div>

**Machine Learning Level 1 Module 3** (part 2)

**Technical Area**

- Vectorized my data using TF-IDF
- Calculated a distance metric using cosine similarity
- recommended 10 most similar posts based on the given post
- Identified simple machine learning models and trained them on our data

**Tools**

- sklearn
- matplotlib
- numpy

**Soft Skills**

- Vectorized words using TF-IDF
- Measured cosine similarity
- Word embeddings
- Simple classification model

**Three achievement highlights**

- Transformed my data into word vectors using TF-IDF
- Calculated cosine similarity
- Recommended 10 most similar posts based on a given post

**Goals for the upcoming week**

- start module 4
- Prepare for the next presentation

**hurdles**

Had trouble to train the classification model

---

<div class="post-metadata">

**Author:** ![sihongy1](https://d2nat51eddc6b4.cloudfront.net/user_avatar/stemaway.com/sihongy1/32/3567_2.png) [@sihongy1](https://stemaway.com/u/sihongy1)\
**Post date:** [July 28, 2021, 6:38pm UTC](https://stemaway.com/t/sihongy1-machine-learning-level-1-pathway/8125/6 "2021-07-28T18:38:33Z")

</div>

**Machine Learning Level 1 Module 4**

**Technical Area** :

- BERT
- Simple transformers
- Logistic Regression

**Tools** :

- Google Colab
- Simple transformers
- Sklearn

**Soft Skills** :

- Simple transformers library
- BERT model
- Logistics Regression model

**Achievements:**

- Used a pre-trained monolingual model to my dataset
- Used DistilBERT to embeded the flowster dataset
- Trained logistic regression model

**Goals for the upcoming week** :

- Continue to finish module 4

---

<div class="post-metadata">

**Author:** ![sihongy1](https://d2nat51eddc6b4.cloudfront.net/user_avatar/stemaway.com/sihongy1/32/3567_2.png) [@sihongy1](https://stemaway.com/u/sihongy1)\
**Post date:** [August 2, 2021, 6:40pm UTC](https://stemaway.com/t/sihongy1-machine-learning-level-1-pathway/8125/7 "2021-08-02T18:40:27Z")

</div>

**Machine Learning Level 1 Module 4** （part 2)

**Technical Area:**

- Learned how to use BERT and Roberta to update our word embeddings
- Learned how to tested different recommenders to decide which one is the best
- Learned how to built a classifier with group
- Learned how to put all the work to a website

**Tools:**

- Simple transformers
- tokenizers
- Google colab
- Flask
- Docker

**Soft Skills:**

- How to measure the effectiveness of the recommenders
- How to build a website using flask and Docker

**Achievements:**

- Tested different kinds of recommenders and decided which one is the best
- Put all our work into a web
- Built a classifier and measured the effectiveness

**Goals for the upcoming week:**

- Prepare for the final presentation
