illustration ¹

• url https://framerusercontent.com/images/L1rUxpJ00yIfOffzFpQWSNpjEkA.png?width=1300&height=952 [20260504].

direction of change prediction ²

• In time series forecasting, the task of a machine learning model is to predict the future values of a time series.
• The forecast in one step ahead prediction is about the first future value in a series, where this value will be (more or less) different from the actual value, resulting in a forecast error.
• There is a case where we are more interested in predicting whether the difference in value of the next day will be positive or negative, the direction of change.
• In this case, we can use a confusion matrix.

classification metrics ³

• Most classification model evaluations begin with the construction of a confusion matrix.
• A confusion matrix is a summary of prediction results on a classification problem.
• The number of correct and incorrect predictions are summarized with count values and broken down by each class.

metrics from confusion matrix ⁴

• For a simple binary classifier there are true positive (TP), true negative (TN), false positive (FP), and false negative (FN) categories.
• From that classes accuracy, error rate, precision, recall, true negative rate (TNR), false positive rate (FPR), false negative rate (FNR), and F1-score.
• There are also receiver operating curve (ROC) and area under the curve (AUC) to consider.

the categories ⁵

• True Positive (TP): The model correctly predicted a positive outcome i.e the actual outcome was positive.
• True Negative (TN): The model correctly predicted a negative outcome i.e the actual outcome was negative.
• False Positive (FP): The model incorrectly predicted a positive outcome i.e the actual outcome was negative. It is also known as a Type I error.
• False Negative (FN): The model incorrectly predicted a negative outcome i.e the actual outcome was positive. It is also known as a Type II error.

tuning and metrics ^{6 7}

• Optimize model performance by tuning the decision threshold, improving data preprocessing, and addressing class imbalance; these factors can improve recall without necessarily reducing precision, depending on dataset characteristics.
• For medical datasets, high recall is generally prioritized to minimize false negatives, making it more critical than overall accuracy or precision.
• For applications such as spam detection or fraud detection, high precision is typically preferred to reduce false positives; this is achieved by threshold tuning while balancing the precision–recall tradeoff.
• F1-score is the preferred metric for imbalanced datasets when both precision and recall are equally important, particularly when neither false positives nor false negatives can be overlooked.

notes

• This note is used in 26a62 entry 10.

refs