# Model Evaluation ## Confusion Matrix | Actual\\Predicted | $C$ | $\neg C$ | | :---------------: | :------------------: | :------------------: | | $C$ | True Positives (TP) | False Negatives (FN) | | $\neg C$ | False Positives (FP) | True Negatives (TN) | - Accuracy = (TP + TN) / All - Error Rate = (FP + FN) / All - Sensitivity = TP / P - Specificity = TN / N - Precision = TP / (TP + FP) - Recall = TP / (TP + FN) = TP / P - F Measure / F Score = harmonic mean of precision and recall $ F = \frac{(\beta^2 + 1)PR}{\beta^2 P + R} $ - F1-measure = F score with $\beta = 1$ $ F = \frac{2PR}{P + R} $ ## ROC Curve - ROC = Receiver Operating Characteristics - See how a classifier performs with different threshold - Visualizes tradeoff between precision and recall - Procedure - Rank the test tuples with likelihood to be true in decreasing order - Horizontal axis as False Positive Rate, vertical as True - Interpretation - The area under ROC curve measures the accuracy of the model - Similarly, we have precision-recall curve ## MAE and RMSE - Mean Absolute Error (MAE) = $\sum_i|s_i - c_i| / n$ - Root Mean Squared Error (RMSE) = $\sqrt{\sum_i(s_i - c_i)^2 / n}$ ## Kendall's Tau - tau = (# concordant pairs - # discordant pairs) / number of pairs - Concordant pair means a positive tuple appears before a negative one in terms of prediction score ranking - Total number of pairs is $n \choose 2$