Forecast Methods - Explanation

Created by Shyam Sayana, Modified on Mon, 3 Nov at 6:33 AM by Shyam Sayana

TABLE OF CONTENTS

AutoRegressive (AR) Model
Exponential Moving Average (EMA)
Moving Average (MA)
Simple Exponential Smoothing (SES)
Simple Moving Average (SMA)
ARIMA
AutoRegressive Moving Average
Holt-Winters
Theta Model
TSB (Teunter-Syntetos-Babai) Model
Seasonal ARIMA (SARIMA) Model
Average of History
Linear Model
KNeighborsRegressor (KNN for Regression)
N-BEATS (Neural Basis Expansion Analysis for Time Series)
LSTM (Long Short-Term Memory)
Prophet
XGBRegressor: Extreme Gradient Boosting for Regression
NeuralProphet: Advanced Time Series Forecasting

AutoRegressive (AR) Model

The AutoRegressive (AR) model is a forecasting technique in which future values in a time series are predicted as a linear combination of past values. It assumes that past observations influence future values, making it useful for datasets with strong autocorrelations.

Key Characteristics:

Assumes Stationarity – The mean and variance remain constant over time.
Captures Autocorrelation – Works well when past values influence future values.
Lag Order p – Determines how many past observations are included in the prediction.

Best Suitable For:

Time series with strong autocorrelation (e.g., stock prices, demand forecasting).
Short- to medium-term forecasting where past trends persist over time.

Exponential Moving Average (EMA)

The Exponential Moving Average (EMA) is a weighted moving average method that assigns greater importance to recent data points. This makes it more responsive to recent changes than a Simple Moving Average (SMA).

Key Parameter:

Smoothing Factor (α) – Controls the weight given to recent observations. A higher α\alphaα makes the model more reactive to recent changes, while a lower α\alphaα smooths out fluctuations.

Best Suitable For:

Short-term forecasting where recent trends matter.
Financial markets (stock price trends, currency exchange rates).
Demand forecasting with rapidly changing patterns.

Moving Average (MA)

The Moving Average (MA) method is a simple and widely used technique in time series analysis that helps smooth out short-term fluctuations and identify underlying trends. It works by taking the average of past observations over a fixed "window size" and using this average to estimate future values.

Formula for Moving Average

A simple moving average (SMA) of window size k at time t is given by:

SMAt=(Yt+Yt−1+Yt−2+...+Yt−n+1)/n

Where:

SMAt= Moving Average at time t
Yt = Actual value at time t
n = Number of periods used for averaging

Key Parameter:

Window Size (n) – Determines how many past values are averaged. A larger n provides more smoothing but may lag behind actual trends, while a smaller n responds more quickly to changes.

Best Suitable For:

Short-to-medium-term forecasting is when data has no strong seasonality or trend.
Smoothing demand patterns in inventory management.
Detecting long-term trends in stock markets and sales forecasting.

Simple Exponential Smoothing (SES)

Simple Exponential Smoothing (SES) is a forecasting method that applies exponentially decreasing weights to past observations. It is useful when data has no clear trend or seasonality but exhibits random fluctuations. SES prioritizes recent data points, making the forecast more responsive to recent changes.

The formula for SES:

Ft+1=αYt+(1−α)Ft

Where:

Ft+1 = Forecast for the next period
Yt = Actual value at time t
Ft = Forecasted value for time t
α\= Smoothing factor (0 ≤ α\alphaα ≤ 1)

Key Parameter:

A higher α\alphaα (e.g., 0.8) gives more weight to recent values, making the forecast highly responsive to changes.
A lower α\alphaα (e.g., 0.2) gives more weight to past values, making the forecast smoother and less sensitive to fluctuations.

Best Suitable For:

Short-term forecasting where data is relatively stable.
Sales forecasting for stable products with no strong trend/seasonality.
Inventory management for demand smoothing.

Simple Moving Average (SMA)

Simple Moving Average (SMA) is a basic time-series forecasting method that calculates the average of a fixed number of past observations. It smooths out short-term fluctuations and highlights long-term trends by averaging past values over a defined period.

The formula for SMA:

SMAt=(Yt+Yt−1+...+Yt−(n−1))/n

Where:

SMAt = Simple Moving Average at time t
Yt = Actual value at time t
n = Number of past periods used in the moving average

Key Parameter:

A smaller window (e.g., 3-5 periods) makes the forecast more sensitive to recent changes.
A larger window (e.g., 10-20 periods) provides a smoother trend but reacts more slowly to new data.

Best Suitable For:

Stable data with no strong trend or seasonality.
Stock market analysis for trend detection.
Sales forecasting for products with steady demand.
Inventory control to smooth demand fluctuations.

ARIMA

ARIMA is a powerful time-series forecasting model that combines three components: AutoRegression (AR), Differencing (I), and Moving Average (MA) to predict future values based on past observations. It is widely used for forecasting in financial, sales, and demand planning applications.

ARIMA Model Components:

AutoRegressive (AR) Component: Uses past values (lags) to predict future values. Defined by parameter p (number of lags).
Integrated (I) Component: Differencing is applied to make the data stationary (removing trends) and defined by parameter d (number of times differencing is used).
Moving Average (MA) Component: Uses past forecast errors to improve predictions. Defined by parameter q (number of lagged forecast errors).

ARIMA Notation:

ARIMA(p,d,q)

Where:

p = Number of lag observations in the autoregressive model
d = Number of times the data is differenced to make it stationary
q = Number of lagged forecast errors in the moving average model

Best suitable for

Sales and demand forecasting (with trend-based data).
Economic and business forecasting.

AutoRegressive Moving Average

The AutoRegressive Moving Average (ARMA) model is a time series forecasting method that combines two components:

AutoRegression (AR) - Uses past values (lags) to predict future values.
Moving Average (MA) - Uses past forecast errors to improve predictions.

Key Parameters of ARMA:

p (AutoRegression order) → Number of past observations used for forecasting.
q (Moving Average order) → Number of past forecast errors included in the model.

Best Suitable For:

Stationary Time Series (No long-term trend or seasonality)
Data with Short-Term Dependencies
Forecasting energy consumption, traffic, and weather patterns.

Holt-Winters

The Holt-Winters method is a time series forecasting technique that extends exponential smoothing by considering:

Level (Baseline value of the series)
Trend (Upward or downward movement over time)
Seasonality (Repeating patterns at fixed intervals)

Types of Holt-Winters Models:

Additive Model → Used when seasonal variations remain constant over time. Yt=(Lt+Tt)×St−m+et
Multiplicative Model → Used when seasonal variations increase or decrease proportionally with time. Yt=(Lt×Tt)×St−m+et

Key Parameters:

α (Smoothing factor for level) → Controls how much weight is given to the most recent observation.
β (Smoothing factor for trend) → Determines how the trend component is updated.
γ (Smoothing factor for seasonality) → Governs how seasonal patterns are adjusted.
m (Seasonal period) → Number of periods per season (e.g., 12 for monthly data with yearly seasonality).

Best Suitable For:

Time series data with trend and seasonality
Short-to-medium-term forecasting

Theta Model

The Theta Model is a time-series forecasting method known for its accuracy, particularly for forecasting demand and sales trends. It is an extension of simple exponential smoothing (SES) and is widely used in business forecasting and in competitions such as the M3 Forecasting Competition.

Key Parameters

Theta (θ) Coefficient → Determines the curvature of each transformed series.
Smoothing Parameter → Controls how much the data is smoothed.

Best Suitable For:

Demand forecasting
Inventory planning
Business and retail sales forecasting
Any dataset with trend and seasonality

TSB (Teunter-Syntetos-Babai) Model

The TSB Model is a forecasting method designed explicitly for intermittent demand—situations where demand occurs sporadically, with many periods of zero demand. Unlike traditional forecasting models, TSB does not assume that demand follows a normal distribution; instead, it models demand occurrence and demand size separately.

Key Parameters

α (Alpha) → Smoothing factor for the demand probability
β (Beta) → Smoothing factor for the demand size

These parameters help the model adjust dynamically based on recent demand patterns.

Best Suitable For:

Spare parts management
Retail items with low and sporadic sales
Inventory planning for slow-moving goods
Any scenario where demand is not continuous

Seasonal ARIMA (SARIMA) Model

Seasonal ARIMA (SARIMA) extends the ARIMA model by incorporating seasonal patterns into the forecasting process. It is ideal for time series data that exhibit repeating patterns over fixed intervals (e.g., monthly sales, quarterly revenues, or daily temperature variations).

SARIMA Notation

SARIMA(p,d,q)×(P,D,Q,m)

Where:

(p, d, q) → Regular ARIMA terms:
- p = Number of autoregressive (AR) terms
- d = Number of differencing steps to make data stationary
- q = Number of moving average (MA) terms
(P, D, Q, m) → Seasonal components:
- P = Seasonal autoregressive (SAR) terms
- D = Seasonal differencing steps
- Q = Seasonal moving average (SMA) terms
- m = Length of the season (e.g., 12 for monthly data with annual seasonality)

Best Suitable For:

Retail Sales Forecasting (monthly/weekly sales with seasonality)
Energy Consumption Prediction (electricity usage fluctuates seasonally)
Weather Forecasting (temperature, rainfall, etc.)
Tourism Demand Prediction (hotel bookings, airline passengers)

Average of History

The Average of History is a simple forecasting method that calculates the mean of past observations and uses it as the forecast for future periods.

Formula:

Forecast=∑Xt/N

Where:

Xt = Historical data points
N = Number of historical periods

Best Suitable For:

Stable Demand → No strong trends or seasonality.
Short-Term Forecasting → Works well for short forecasting horizons.
Baseline Forecasting → Used as a benchmark to compare with other models.

Linear Model

A Linear Model is a fundamental forecasting method that assumes a linear relationship between the independent variable (time) and the dependent variable (demand or sales). It predicts future values by fitting a straight line to historical data.

Formula (Simple Linear Regression):

Y=a+bX

Where:

Y = Forecasted value
a = Intercept (value of Y when X=0)
b = Slope (rate of change)
X = Time (or another independent variable)

Best Suitable For:

Data with a Clear Trend → Works well when demand increases or decreases at a constant rate.
Short to Medium-Term Forecasting → Effective for stable trend patterns.
Baseline Trend Analysis → Useful as a starting point for more complex models.

KNeighborsRegressor (KNN for Regression)

KNeighborsRegressor is a non-parametric supervised learning algorithm that predicts based on the K-nearest neighbors in the training data. Instead of fitting a mathematical function, it estimates the target value by averaging (or weighting) the values of the nearest data points.

Key Parameters

n_neighbors: The number of nearest neighbors to consider (default = 5).
weights: Determines how neighbors contribute to the prediction:
- "uniform" → Equal weight for all neighbors.
- "distance" → Closer neighbors have more influence.
Metric: Distance calculation method (e.g., "Euclidean," "Manhattan," "Minkowski").

Best Suitable For

Non-linear relationships (when a linear model isn’t a good fit).
Small to medium datasets (as KNN can be computationally expensive for large datasets).
Forecasting with minimal assumptions about the data’s distribution.

N-BEATS (Neural Basis Expansion Analysis for Time Series)

N-BEATS is a deep learning-based time series forecasting model that does not require domain-specific feature engineering. It uses deep feedforward neural networks to learn patterns from historical data and make accurate predictions.

Key Parameters

Stacked blocks: Number of layers that process historical data.
Forecasting horizon: The time horizon over which the model predicts.
Trend & Seasonality Basis Functions: Used for decomposing time series data.
Loss Function: Typically Mean Squared Error (MSE) or Mean Absolute Error (MAE).

Best Suitable For

Large datasets where deep learning can capture complex relationships.
Financial forecasting, demand forecasting, energy consumption prediction, and healthcare time series data.
Multi-horizon forecasting, where predictions need to be made for varying future time frames.

LSTM (Long Short-Term Memory)

LSTM is a deep learning-based time series forecasting model designed to handle long-range dependencies in sequential data. It is a type of Recurrent Neural Network (RNN) that overcomes the vanishing gradient problem, making it ideal for forecasting tasks with long-term dependencies.

Key Parameters

Number of LSTM layers: Defines how deep the network is.
Hidden units: Determines the number of neurons per LSTM layer.
Batch size & Epochs: Controls the training process and convergence.
Dropout rate: Prevents overfitting by randomly disabling neurons during training.
Learning rate: Adjusts how quickly the model updates weights.

Best Suitable For

Long-range forecasting (e.g., demand forecasting, stock price prediction, weather forecasting).
Complex sequential data where past values significantly impact future trends.
Multi-variate time series forecasting, where multiple variables influence the prediction.

Prophet

Prophet is an open-source forecasting tool developed by Facebook (Meta). It is specifically designed to handle time series data with strong seasonality, trends, and missing values. It provides an automated and flexible approach to forecasting, making it user-friendly for analysts and data scientists.

Formula

y(t)=g(t)+s(t)+h(t)+ϵt

g(t) = Trend function
s(t) = Seasonal component
h(t) = Holiday effects
ϵt= Error term

Key Parameters

changepoint_prior_scale: Controls flexibility of trend changes.
seasonality_prior_scale: Adjusts seasonality strength.
holidays_prior_scale: Importance given to holiday effects.
interval_width: Confidence interval size for predictions.

Best Suitable For

Business forecasting (sales, demand, revenue prediction).
Website traffic and engagement analysis.
Stock market and financial forecasting.
Energy consumption and weather forecasting.

XGBRegressor: Extreme Gradient Boosting for Regression

XGBRegressor is the regression variant of XGBoost (Extreme Gradient Boosting), an optimized machine-learning algorithm for high performance, speed, and accuracy. It is widely used for regression problems, including time series forecasting.

Key Parameters

n_estimators: Number of boosting rounds (trees).
learning_rate: Step size for adjusting predictions.
max_depth: Maximum depth of trees (controls complexity).
Subsample: Fraction of data used per boosting round.
colsample_bytree: Fraction of features used per tree.
Objective: Loss function to minimize (e.g., "reg: squared error" for regression).
Gamma: Controls model complexity (higher = more pruning).

Best Suitable For

Time series forecasting (sales, demand, financial data).
Regression problems with complex feature interactions.
Stock price prediction and market analysis.
Predicting customer behavior and trends.

NeuralProphet: Advanced Time Series Forecasting

NeuralProphet is a deep learning-based time series forecasting model that combines Facebook’s Prophet’s strengths with Neural Networks’s flexibility. It is designed for ease of use, scalability, and improved performance, especially for complex time series data with trends, seasonality, and external factors.

Key Parameters

trend_reg: Controls how flexible the trend model is.
seasonality_reg: Regularization for seasonal components.
n_lags: Number of past observations used for prediction.
learning_rate: Defines how fast the model learns.
Epochs: Number of training iterations.
seasonality_mode: Can be "additive" or "multiplicative".
loss_func: Loss function for training (e.g., MSE, MAE).

Best Suitable For

Sales and demand forecasting
Financial market predictions
Energy consumption forecasting
Traffic and web analytics
Weather forecasting
Inventory and supply chain planning