Can you use autoregressive diffusion to generate market data?

The piece examines whether autoregressive diffusion models can synthesize market-level event streams (orders, cancels, trades) including timing and price. An intern built an encoder-diffuser architecture: a causally masked transformer produces latent embeddings, a categorical head predicts event kind, and a diffusion head generates continuous features conditioned on the latent and sampled kind. Market data poses a hybrid challenge: discrete actions (e.g., pennying) coexist with high-cardinality or continuous-valued features and spiky timing distributions. Standard DDPM denoising proved unstable in this setting, producing exploding denoising trajectories; switching to rectified flow-matching avoided those instabilities and yielded much better out-of-the-box behavior.

The main technical work addressed discontinuities. Hand-engineering a 20-class categorical head to capture common “atoms” (zero interarrival times, price-direction changes) improved marginals but does not scale. The intern therefore developed “atom smoothing,” smoothing spiky distributions and re-sculpting point masses, and evaluated models by total variation of per-feature marginals plus a classifier distinguishing real from generated next events. Flow matching combined with atom smoothing produced substantially more realistic single-step samples, though autoregressive rollouts still degrade over time and synthetic spreads widen. The findings clarify which features to model categorically versus continuously and highlight representation/smoothing choices as critical for any practical market-data generator.