note for data jobs

Confusion matrix -- way to remember:
                             Predicted Class 0                           Predicted Class 1
Actual Class 0             TN                                        FP(false alarm, may not harm - Innocent - Tpye I)
Actual Class 1              FN(Miss opportunity         TP
                                       two missed - Type II)      

• Recall = TP/(TP+FN), known as true positive rate, measures the ability of a classification model to identify all relevant instances(actual class 1) from the datase. (How many time recall due to FN)
  

• Precision = TP/(TP+FP) reflects the accuracy of the positive predictions made by a model.
• F1 score: 2× (Precision+Recall)/(Precision×Recall) --balance
  

Linear Regression

• the Least Squares (LS) method
• Maximum Likelihood Estimation (MLE)
  

1. **Least Squares (LS) Method:**
   - In the least squares approach, the goal is to minimize the sum of the squared differences between the observed values (actual outcomes) and the values predicted by the linear model. The objective function to minimize is the sum of squared residuals:
   
   \[ \text{Minimize} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2 \]

   - Here, \( y_i \) is the actual outcome for the \(i\)-th observation, and \( \hat{y}_i \) is the predicted value by the linear model.

2. **Maximum Likelihood Estimation (MLE):**
   - In the context of Linear Regression, MLE is used to estimate the parameters of the model by maximizing the likelihood function. The likelihood function represents the probability of observing the given set of outcomes given the model parameters. Assuming normally distributed errors, the likelihood function is given by:

   \[ \text{Likelihood} \propto \prod_{i=1}^{N} \frac{1}{\sigma \sqrt{2\pi}} \exp\left(-\frac{(y_i - \hat{y}_i)^2}{2\sigma^2}\right) \]

   - Here, \( \sigma \) is the standard deviation of the normally distributed errors.

3. **Relationship between LS and MLE:**
   - Interestingly, when you assume normally distributed errors and maximize the likelihood function, it turns out that maximizing the likelihood is equivalent to minimizing the sum of squared residuals. In other words, under the assumption of normally distributed errors, the LS method and MLE yield the same parameter estimates.

   - Mathematically, maximizing the likelihood is equivalent to minimizing the negative log-likelihood, which is proportional to the sum of squared residuals. Therefore, the LS estimate and the MLE estimate are the same.