introductory-presentation.rmd

---
title: "Machine Learning (ML) Module: Intro. Presentation" 
subtitle: "https://laser-institute.github.io/machine-learning/introductory-presentation.html#1"
author: "Joshua Rosenberg"
institute: "The LASER Institute"
date: "`r Sys.Date()`"
output:
    xaringan::moon_reader:
      css:
       - default
       - css/laser.css
       - css/laser-fonts.css
      lib_dir: libs                        # creates directory for libraries
      seal: false                          # false: custom title slide
      nature:
        highlightStyle: default         # highlighting syntax for code
        highlightLines: true               # true: enables code line highlighting 
        highlightLanguage: ["r"]           # languages to highlight
        countIncrementalSlides: false      # false: disables counting of incremental slides
        ratio: "16:9"                      # 4:3 for standard size,16:9
        slideNumberFormat: |
         <div class="progress-bar-container">
          <div class="progress-bar" style="width: calc(%current% / %total% * 100%);">
          </div>
         </div>
---
class: clear, title-slide, inverse, center, top, middle

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = FALSE)
```

```{r, echo=FALSE}
# then load all the relevant packages
pacman::p_load(pacman, knitr, tidyverse, readxl)
```

```{r xaringan-panelset, echo=FALSE}
xaringanExtra::use_panelset()
```

```{r xaringanExtra-clipboard, echo=FALSE}
# these allow any code snippets to be copied to the clipboard so they 
# can be pasted easily
htmltools::tagList(
  xaringanExtra::use_clipboard(
    button_text = "<i class=\"fa fa-clipboard\"></i>",
    success_text = "<i class=\"fa fa-check\" style=\"color: #90BE6D\"></i>",
  ),
  rmarkdown::html_dependency_font_awesome()
)
```

```{r xaringan-extras, echo=FALSE}
xaringanExtra::use_tile_view()
```

# `r rmarkdown::metadata$title`
----
### `r rmarkdown::metadata$author`
### `r rmarkdown::metadata$subtitle`
### `r format(Sys.time(), "%B %d, %Y")`

---

class: clear, inverse, center, middle

# Front Matter

---

class: clear, inverse, center, middle

background-image: url(https://imgs.xkcd.com/comics/machine_learning.png)
background-position: center
background-size: contain

---

# Goals for this module

### Over-arching goal:

- **Get started with applying machine learning methods in R** 

### Specific aims are to learn about:

- A small but important set of ideas about machine learning
- How these ideas are instantiated through the *tidymodels* R package

---

# One example of ML . . .

```{r, echo = FALSE, out.width="62%", fig.align='center'}
knitr::include_graphics("img/gpt-3.png")
```

https://beta.openai.com/playground

---

# How I came to ML

**A magical moment**
- Learned about supervised machine learning from a colleague
- Used these methods to discover patterns in students' embedded assessments, Rosenberg et al. (2021), *Journal of Science Education and Technology*, https://link.springer.com/article/10.1007/s10956-020-09862-4

**Using Twitter users profile descriptions to predict whether or not they were teachers**
- Manually coded > 500 profiles for whether they were teachers or non-teachers and then trained a ML model to predict 1000s of teachers' roles as teachers or non-teachers (with an accuracy of 85%), [Rosenberg et al., *AERA Open* (2021)](https://journals.sagepub.com/doi/full/10.1177/23328584211024261)

---

class: clear, inverse, center, middle

# Overview of Machine Learning (ML)

---

# Defining ML

- *Artificial Intelligence (AI)* (i.e., [GPT-3](https://openai.com/api/))
: Simulating human intelligence through the use of computers
- *Machine learning (ML)*: A subset of AI focused on how computers acquire new information/knowledge

This definition leaves a lot of space for a range of approaches to ML

---

# Supervised & unsupervised

## Supervised ML

- Requires coded data or data with a known outcome
- Uses coded/outcome data to train an algorithm
- Uses that algorithm to **predict the codes/outcomes for new data** (data not used during the training)
- Can take the form of a *classification* (predicting a dichotomous or categorical outcome) or a *regression* (predicting a continuous outcome)
- Algorithms include:
  - [Linear regression (really!)](https://web.stanford.edu/~hastie/ElemStatLearn/)
  - Logistic regression
  - Decision tree
  - Support Vector Machine

---

# What kind of coded data?

> Want to detect spam? Get samples of spam messages. Want to forecast stocks? Find the price history. Want to find out user preferences? Parse their activities on Facebook (no, Mark, stop collecting it, enough!) (from [ML for Everyone](https://vas3k.com/blog/machine_learning/))

In educational research:

- Assessment data (e.g., [1](https://link.springer.com/article/10.1007/s10956-020-09895-9))
- Data from log files ("trace data") (e.g., [1](https://www.tandfonline.com/doi/full/10.1080/10508406.2013.837391?casa_token=-8Fm2KCFJ30AAAAA%3Altbc8Y8ci_z-uLJx4se9tgvru9mzm3yqCTFi12ndJ5uM6RDl5YJGG6_4KpUgIK5BYa_Ildeh2qogoQ))
- Open-ended written responses (e.g., [1](https://link.springer.com/article/10.1007/s10956-020-09889-7), [2](https://doi.org/10.1007/s11423-020-09761-w))
- Achievement data (i.e., end-of-course grades) (e.g., [1](https://link.springer.com/article/10.1007/s10956-020-09888-8), [2](https://search.proquest.com/docview/2343734516?pq-origsite=gscholar&fromopenview=true))

---

# How is this different from regression?

The _aim_ is different, the algorithms and methods of estimation are not (or, are differences in degree, rather than in kind).

In a linear regression, our aim is to estimate parameters, such as $\beta_0$ (intercept) and $\beta_1$ (slope), and to make inferences about them that are not biased by our particular sample.

In an ML approach, we can use the same linear regression model, but with a goal other than making unbiased inferences about the $\beta$ parameters:

<h4><center>In supervised ML, our goal is to minimize the difference between a known $y$ and our predictions, $\hat{y}$</center></h3>

---

# So, how is this really different?

This _predictive goal_ means that we can do things differently:

- Multicollinearity is not an issue because we do not care to make inferences about parameters
- Because interpreting specific parameters is less of an interest, we can use a great deal more predictors
- We focus on how accurately a _trained_ model can predict the values in _test_ data
- We can make our models very complex!

---

# Okay, _really_ complex

- Neutral/deep networks
  - i.e., GPT-3 (175 B parameters), GPT-4 (>1 T parameters)
- And, some models can take a different form than familiar regressions:
  - *k*-nearest neighbors
  - Decision trees (and their extensions of bagged and random forests)
- Last, the modeling process can look different:
  - Ensemble models that combine or improve on ("boosting") the predictions of individual models

---

# Supervised & unsupervised

## Unsupervised ML

- Does not require coded data; one way to think about unsupervised ML is that its purpose is to discover codes/labels
- Can be used in an _exploratory mode_ (see [Nelson, 2020](https://journals.sagepub.com/doi/full/10.1177/0049124118769114?casa_token=EV5XH31qbyAAAAAA%3AFg09JQ1XHOOzlxYT2SSJ06vZv0jG-s4Qfz8oDIQwh2jrZ-jrHNr7xZYL2FwnZtZiokhPalvV1RL2Bw)) 
- **Warning**: The results of unsupervised ML _cannot_ directly be used to provide codes/outcomes for supervised ML techniques 
- Algorithms include:
  - Cluster analysis and Latent Profile Analysis
  - [Principle Components Analysis (really!)](https://web.stanford.edu/~hastie/ElemStatLearn/)

---

# What technique should I choose?

Do you have coded data or data with a known outcome -- let's say about K-12 students -- and, do you want to:

- _Predict_ how other students with similar data (but without a known outcome) perform?
- _Scale_ coding that you have done for a sample of data to a larger sample?
- _Provide timely or instantaneous feedback_, like in many learning analytics systems?

**Supervised methods may be your best bet**

---

# What technique should I choose?

Do you not yet have codes/outcomes -- and do you want to?

- _Achieve a starting point_ for qualitative coding, perhaps in a ["computational grounded theory"](https://journals.sagepub.com/doi/full/10.1177/0049124117729703) mode?
- _Discover groups or patterns in your data_ that may be of interest?
- _Reduce the number of variables in your dataset_ to a smaller, but perhaps nearly as explanatory/predictive - set of variables?

**Unsupervised methods may be helpful**

---

# Examples of machine learning in STEM Ed Research


.panelset[

.panel[.panel-name[Example 1]

**Using digital log-trace data and supervised ML**

Gobert, J. D., Sao Pedro, M., Raziuddin, J., & Baker, R. S. (2013). From log files to assessment metrics: Measuring students' science inquiry skills using educational data mining. *Journal of the Learning Sciences, 22*(4), 521-563.

- Utilized *replay tagging* to code sequences of students' activity within digital *log-trace* data
- Then trained a supervised ML algorithm to **automate** the prediction the presence or absence of students' engagement in inquiry
]

.panel[.panel-name[Example 2]

**Combining best practices in assessment with supervised ML**

Maestrales, S., Zhai, X., Touitou, I., Baker, Q., Schneider, B., & Krajcik, J. (2021). Using machine learning to score multi-dimensional assessments of chemistry and physics. Journal of Science Education and Technology, 30(2), 239-254.

- Carrying out a careful process of qualitative coding (and the establishment of validity and interrater reliability) of students' written constructed responses
- Training a supervised ML algorithm in advance of being able to **scale up** their coding to a larger number of responses
]

.panel[.panel-name[Example 3]

**Combining qualitative methods with unsupervised ML**

Sherin, B. (2013). A computational study of commonsense science: An exploration in the automated analysis of clinical interview data. *Journal of the Learning Sciences, 22*(4), 600-638.

- Used unsupervised machine learning methods to identify topics within students' interviews **from patterns in the data alone**
- Interpreted those topics in light of rigorous qualitative coding with the aim of boosting the validity of the use of both the machine learning and qualitative approaches
]
]

---

class: clear, inverse, center, middle

# Learning Labs!

---

# Learning Labs Overviews

.panelset[

.panel[.panel-name[Overview]

**What will I learn in this topic area?**

We'll work to answer these four questions:

- LL 1: Can we predict something we would have coded by hand?
- LL 2: How much do new predictors improve the prediction quality?
- LL 3: How much of a difference does a more complex model make?
- LL 4: What if we do not have training data?

]

.panel[.panel-name[LL 1]

**Machine Learning Learning Lab 1: Focusing on prediction**

We have some data, but we want to use a computer to **automate** or scale up the relationships between predictor (independent) and outcome (dependent) variables. Supervised machine learning is suited to this aim. In particular, in this learning lab, we explore how we can train a computer to learn and reproduce qualitative coding we carried out---though the principles extend to other types of variables.

]

.panel[.panel-name[LL 2]

**Machine Learning Learning Lab 2: Feature engineering**

Once we have trained a computer in the context of using supervised machine learning methods, we may realize we can improve it by adding new variables or changing how the variables we are using were prepared. In this learning lab, we consider **feature engineering** with a variety of types of data from many online science courses to predict students' success after only a few weeks of the course

]

.panel[.panel-name[LL 3]

**Machine Learning Learning Lab 3: Fine-tuning the model**

Even with optimal feature engineering, we may be able to specify an even more predictive model by selecting and *tuning* a more sophisticated algorithm. While in the first two learning labs we used logistic regression as the "engine" (or algorithm), in this learning lab, we use a random forest as the engine for our model. This more sophisticated type of model requires us to specify **tuning parameters**, specifications that determine how the model works.

]

.panel[.panel-name[LL 4]

**Machine Learning Learning Lab 4: Finding groups (or codes) in data**

The previous three learning labs involved the use of data with known outcome variables (coded for the substantive or transactional nature of the conversations taking place through #NGSSchat in learning labs 1 and 3 and students' grades in learning lab 2). Accordingly, we explored different aspects of supervised machine learning. What if we have data without something that we can consider to be a dependent variable? **Unsupervised machine learning methods** can be used in such cases.
]
]

---

# Thanks

I hope to see you in the ML topic area learning labs!

Slides [here](https://laser-institute.github.io/machine-learning/introductory-presentation.html#1)

## .font130[.center[**Thank you!**]]
<br/>
.center[<img style="border-radius: 80%;" src="img/jr-cycling.jpeg" height="200px"/><br/>**Dr. Joshua Rosenberg**<br/><mailto:jmrosenberg@utk.edu>]

Joshua Rosenberg  
jmrosenberg@utk.edu  
@jrosenberg6432    

Slides created via the R package [xaringan](https://github.com/yihui/xaringan)

Available: