Particle Filter

Sequential Monte Carlo state estimation for nonlinear, non-Gaussian dynamics in prediction market tracking and jump detection.

Pro Feature. Requires a Pro or Ultra subscription. Get started at api.mathematicalcompany.com

What is this? A particle filter is like a Kalman filter that can handle jumps, fat tails, and non-linear dynamics. It tracks hidden state by simulating thousands of possible trajectories (‘particles’) and weighting them by how well they match observations. Use it when markets exhibit sudden jumps that Kalman filters smooth over too aggressively.

Particle Filter

Horizon provides a Sequential Monte Carlo (SMC) particle filter implemented in Rust. Unlike Kalman filters, particle filters handle arbitrary nonlinear dynamics and non-Gaussian noise, making them well-suited for jump-diffusion processes, fat-tailed markets, and multimodal state distributions.

Nonlinear & Non-Gaussian

No linearity or Gaussian assumptions. Handles jumps, fat tails, and multimodal posteriors.

Rust-Native SMC

All particle propagation, resampling, and weight computation runs in Rust.

Adaptive Resampling

Systematic resampling triggered when effective sample size drops below threshold.

Pipeline Integration

Drop hz.particle_tracker() into any pipeline for filtered state estimates in ctx.params.

---

ParticleFilter

The core particle filter class. Maintains a weighted set of particles representing the posterior distribution over the hidden state.

Constructor

import horizon as hz

pf = hz.ParticleFilter(
    n_particles=1000,
    initial_state=0.50,
    process_noise=0.01,
    measurement_noise=0.02,
    seed=42,
)

Parameter	Type	Default	Description
n_particles	int	1000	Number of particles. More particles = better approximation, higher cost.
initial_state	float	required	Initial state estimate (all particles start here with small jitter)
process_noise	float	0.01	Standard deviation of the process noise (state transition)
measurement_noise	float	0.02	Standard deviation of the measurement noise (observation model)
seed	int	None	Random seed for reproducibility. None = random.

For prediction markets, 500-2000 particles provide a good tradeoff between accuracy and speed. The filter processes one observation in ~10-50 microseconds in Rust depending on particle count.

update()

Process a new observation: propagate particles through the state transition, compute weights from the observation likelihood, and resample if needed.

state = pf.update(0.55)

Parameter	Type	Description
observation	float	New observed value (e.g., market price or probability)

Returns a PFState object with the filtered estimate.

PFState Type

Field	Type	Description
mean	float	Weighted mean of the particle cloud (point estimate)
variance	float	Weighted variance of the particle cloud
ess	float	Effective sample size (higher = more diverse particles)

State Access Methods

# Current filtered state estimate (weighted mean)
estimate = pf.state_estimate()

# Current state variance
var = pf.state_variance()

# Effective sample size (N_eff)
ess = pf.effective_sample_size()

# Raw particle positions
particles = pf.particles()

# Particle weights (sum to 1.0)
weights = pf.weights()

# Reset to initial state
pf.reset()

Method	Returns	Description
state_estimate()	float	Weighted mean of the particle cloud
state_variance()	float	Weighted variance of the particle cloud
effective_sample_size()	float	N_eff = 1 / sum(w_i^2). Ranges from 1 (degenerate) to N (uniform).
particles()	list[float]	Current particle positions
weights()	list[float]	Current particle weights (normalized, sum to 1.0)
reset()	None	Reset all particles to the initial state with uniform weights

---

Pipeline Integration

hz.particle_tracker

Creates a pipeline function that tracks a filtered price estimate using the particle filter. Injects filtered state into ctx.params.

import horizon as hz

def jump_detector(ctx):
    pf_state = ctx.params.get("pf")
    if pf_state is None:
        return []

    filtered = pf_state["mean"]
    variance = pf_state["variance"]
    raw = ctx.feed.price

    # Detect jumps: raw price far from filtered estimate
    deviation = abs(raw - filtered)
    jump_threshold = 3.0 * (variance ** 0.5)

    if deviation > jump_threshold:
        # Price jumped -- widen spread
        return hz.quotes(fair=raw, spread=0.10, size=2)
    else:
        return hz.quotes(fair=filtered, spread=0.04, size=10)

hz.run(
    name="pf_tracker",
    markets=["election-winner"],
    feeds={"book": hz.PolymarketBook("election-winner")},
    pipeline=[
        hz.particle_tracker(
            feed="book",
            n_particles=1000,
            process_noise=0.005,
            measurement_noise=0.02,
        ),
        jump_detector,
    ],
    risk=hz.Risk(max_position=100),
)

Parameter	Type	Default	Description
feed	str	None	Feed name to read prices from. None = first available.
n_particles	int	1000	Number of particles
process_noise	float	0.005	Process noise standard deviation
measurement_noise	float	0.02	Measurement noise standard deviation
seed	int	None	Random seed for reproducibility
param_name	str	"pf"	Key in ctx.params

Injected Parameters

Key	Type	Description
ctx.params["pf"]["mean"]	float	Filtered state estimate (weighted mean)
ctx.params["pf"]["variance"]	float	State variance from the particle cloud
ctx.params["pf"]["ess"]	float	Effective sample size
ctx.params["pf"]["ess_ratio"]	float	ESS / n_particles (1.0 = fully diverse, less than 0.5 = degenerate)

---

Examples

Offline State Tracking

Use the particle filter directly for research without a pipeline:

import horizon as hz

pf = hz.ParticleFilter(
    n_particles=2000,
    initial_state=0.50,
    process_noise=0.01,
    measurement_noise=0.03,
    seed=123,
)

prices = [0.50, 0.51, 0.49, 0.52, 0.70, 0.72, 0.71, 0.73]

for i, price in enumerate(prices):
    state = pf.update(price)
    print(f"t={i}: observed={price:.2f}, filtered={state.mean:.4f}, "
          f"var={state.variance:.6f}, ESS={state.ess:.0f}")

Particle Filter vs Kalman Filter

The particle filter excels when the state dynamics are nonlinear or noise is non-Gaussian:

import horizon as hz

# Kalman: assumes Gaussian noise, linear dynamics
kf = hz.KalmanFilter(state_dim=1, obs_dim=1)
kf.set_transition([[1.0]])
kf.set_observation([[1.0]])
kf.set_process_noise([[0.001]])
kf.set_measurement_noise([[0.01]])

# Particle: handles jumps and fat tails
pf = hz.ParticleFilter(
    n_particles=1000,
    initial_state=0.50,
    process_noise=0.01,
    measurement_noise=0.03,
)

# Simulate a jump
prices = [0.50, 0.51, 0.50, 0.49, 0.80, 0.81, 0.79]

for price in prices:
    kf.predict()
    kf.update([price])
    pf_state = pf.update(price)
    print(f"KF={kf.state()[0]:.4f}, PF={pf_state.mean:.4f}, raw={price:.2f}")

Combining with BOCPD

Use change point detection to reset the particle filter after regime shifts:

import horizon as hz

def pf_with_reset(ctx):
    bocpd = ctx.params.get("bocpd")
    pf_state = ctx.params.get("pf")
    if not pf_state:
        return []

    # If BOCPD detected a change point, the PF will naturally
    # adapt through resampling with high process noise
    if bocpd and bocpd["is_change_point"]:
        spread = 0.08  # Cautious during regime change
    else:
        spread = 0.04
    return hz.quotes(fair=pf_state["mean"], spread=spread, size=5)

hz.run(
    pipeline=[
        hz.bocpd_detector(feed="book", hazard_rate=200.0),
        hz.particle_tracker(feed="book", n_particles=1000),
        pf_with_reset,
    ],
    ...
)

---

Mathematical Background

Sequential Monte Carlo

A particle filter represents the posterior distribution of the hidden state given all observations, using a weighted set of N samples (particles). At each step:

1. Propagate: Draw each particle from the state transition distribution
2. Weight: Compute the likelihood of the observation given each particle
3. Normalize: Scale all weights to sum to 1
4. Resample: If the effective sample size is below the threshold, resample particles proportional to weights

The weighted mean sum(w_i * x_i) approximates the conditional expectation of the state.

Effective Sample Size

ESS = 1 / sum(w_i^2) measures the diversity of the particle cloud. When all weight concentrates on one particle, ESS equals 1 (degenerate). When weights are uniform, ESS equals N (maximum diversity). Horizon resamples when ESS drops below N/2.

Systematic Resampling

Horizon uses systematic resampling, which generates a single uniform random number and deterministically selects particles at evenly spaced intervals through the CDF. This has lower variance than multinomial resampling and O(N) cost.

Particle filters are stochastic: different seeds produce slightly different results. For reproducible backtests, always set the seed parameter. In live trading, the stochasticity is negligible for 1000+ particles.