MG-TSD: Advancing time series analysis with multi-granularity guided diffusion model
Author: Chang Xu
Diffusion probabilistic models have the capacity to generate high-fidelity samples for generative time series forecasting. However, they also present issues of instability due to their stochastic nature. In order to tackle this challenge, researchers from Microsoft Research Asia introduce a novel approach called “MG-TSD”. The paper “MG-TSD: Multi-Granularity Time Series Diffusion Models with Guided Learning Process (opens in new tab)”, presented at ICLR 2024, capitalizes on the intrinsic granularity levels present in the data, utilizing them as predefined targets at various stages of the diffusion process. The aforementioned targets are employed to guide the learning trajectory of the diffusion models, thereby ensuring a more stable and accurate forecast.
It is noteworthy that the MG-TSD method yields remarkable outcomes without the necessity of additional data. In the field of long-term forecasting, researchers have established a new state-of-the-art methodology that demonstrates a notable relative improvement across six benchmarks with improvement ranging from 4.7% to 35.8%.
Guiding diffusion processes through intrinsic data granularities features in time series data
It can be observed that the forward process of the diffusion model, which sequentially corrupts the data distribution to a standard normal distribution, intuitively aligns with the process of smoothing fine-grained data into a coarser-grained representation. Both of these processes result in a gradual loss of finer distribution features. This suggests that intrinsic features within data granularities may also serve as a source of guidance.
The MG-TSD model employs multiple granularity levels within data to guide the learning process of diffusion models. The coarse-grained data at different granularity levels are utilized as targets to guide the learning of the denoising process. These targets serve as constraints for the intermediate latent states, ensuring a regularized sampling path that preserves the trends and patterns within the coarse-grained data. The introduction of inductive bias facilitates the generation of coarser features during intermediate steps and facilitates the recovery of finer features in subsequent diffusion steps. Each granularity level can guide the diffusion process through different steps. When implementing this, both the coarse-grained data and the finest-grained data share different percentages of the variance schedule (a hyperparameter of the diffusion model), referred to as the “share ratio.” Consequently, this design reduces variability and results in high-quality predictions.
MG-TSD achieves stable and outstanding prediction results
A comprehensive evaluation was conducted across six benchmarks and three performance metrics, in which nine baseline models were compared. The results demonstrate that the MG-TSD model achieves state-of-the-art (SOTA) status, with a substantial improvement ranging from 4.7% to 35.8% on the CRPS_sum metric across the six benchmarks. CRPS_sum indicates the similarity between two distributions; the smaller the value, the more similar they are.
Diffusion process aligns with data smoothing
The four subplots from Figure3(a) to Figure3(d) illustrate a gradual smoothing transformation of the distribution of increasingly coarser targets. The blue curve represents CRPS_sum values of coarse-grained targets and intermediate samples of the single-granularity diffusion model (1h) at each denoising step. As granularity transitions to coarser from left to right panels (4h→6h→12h→24h), targets progressively align with intermediate sample distributions at smaller denoising steps (approximately at steps 80→60→40→40). This comparison underscores the similarity between the diffusion process and the smoothing process, which transition from the finest-grained data to coarse-grained data. Both processes entail a gradual loss of finer characteristics from the finest-grained data through a smooth transformation.
Moreover, this observation is consistent with our selection of guiding steps for MG-TSD. The orange lines, which depict the performance of MG-TSD with different share ratios ranging from [0.2, 0.4, 0.6, 0.8, 1.0], and the blue lines, which represent the similarity of distributions, show a consistent trend. In other words, setting the guiding steps at a coarse granularity to match the steps where the diffusion intermediate samples have the closest distribution often achieves the best performance, as indicated by the grey region in the figure.
Coarse-grained samples demonstrate superior robustness in trend-capturing capabilities
Researchers visualize the ground truth and the predicted mean for both 1-hour and 4-hour granularity time series across four dimensions in the Solar dataset, as illustrated in Figure 4. In the MG-TSD model, the coarse-grained samples display a more robust capacity to capture the trends, subsequently guiding the generation of more precise fine-grained data.
For further information regarding MG-TSD, please refer to the project page at: https://github.com/Hundredl/MG-TSD (opens in new tab)