Real-Time Forecasting Starts with Point-in-Time Correct Data

The Timestral release gives enterprise teams a forecasting model that emits 21 quantiles per time step and forecasts the variates of a series jointly. What it does not give them is any statement about how those context steps were assembled: Mistral's own model documentation describes zero-shot forecasting from a supplied historical context and makes no claim about streaming ingestion, event-time semantics, watermarking, out-of-order handling or point-in-time joins.

That gap is the core tension. The model produces forecasts; the platform decides what was knowable at the forecast origin. A quantile fan chart is only as trustworthy as the rows that fed it, and the model has no mechanism for auditing those rows.

This article compares two readiness postures. Model-centric readiness treats the forecast output as the deliverable and the data layer as an input. Platform-centric readiness treats the forecast origin as a governed artifact and the model as one consumer of it. The sections below give you an evaluation framework, a summary matrix, and an acceptance checklist you can run against your own stack, because real-time forecasting data readiness is decided before the model is called.

What Real-Time Forecasting Data Readiness Actually Means

Real-time forecasting data readiness is the property that every value a model sees at a forecast origin was actually available at that origin. The Timestral repository describes a model that forecasts variates jointly, accepts past-only and known-future covariates, and emits 21 quantiles per time step (mistralai, repository README, released 2026-10-15). The documented context cap is 32,768 steps, and the most recent steps are kept when the context exceeds it.

The boundary matters because covariates are conditioning rows, not forecasts: they pass information to targets but never read them, and only known-future covariates see the forecast horizon. That is a point-in-time constraint expressed in model API terms. The model documentation describes batch-style forecasting from a supplied context and does not claim to manage event-time or point-in-time data processing; Mistral's separate streaming-events documentation concerns publishing workflow state and token events, not time-series temporal semantics (Mistral AI, Timestral 1.0, released 2026-10-15).

Readiness, then, is not a slogan about latency. It is an assessable state with a named boundary: the data platform owns event time, availability time, point-in-time joins and refresh semantics, and the model owns nothing upstream of the context window. The 21-quantile output and covariate design are described as published by the vendor, not independently re-verified.

Event Time, Processing Time and the Forecast Origin

That boundary is where the forecast origin gets its meaning, and the engines define their time semantics in ways that do not map onto it directly. Flink defines event time as the time each individual event occurred on its producing device, and processing time as the system time of the machine executing the operation (Apache Flink documentation, Timely Stream Processing, retrieved 2026-11-05). The forecast origin is bounded by neither of those directly. It is bounded by availability time: the moment a value became queryable by the forecaster.

Spark Structured Streaming's own late-data example makes the gap concrete. A word generated at 12:04 in event time could be received by the application at 12:11, and the application should use 12:04 rather than 12:11 to update the counts for the window 12:00 to 12:10 (Apache Spark documentation, Structured Streaming, retrieved 2026-11-05). If your forecast origin is 12:10 and the record arrives at 12:11, the model at that origin never saw it, regardless of what its event timestamp says.

Kafka Streams advances a monotonic stream time, defined as the largest timestamp seen so far, which only moves forward (Confluent, Event Time and Stream Time in Kafka Streams, retrieved 2026-11-05). An out-of-order record earlier than stream time but still within the window plus grace period leaves stream time where it is.

The three engines declare completeness in three different ways, and the event time vs processing time distinction is not a single rule you can copy across stacks. Spark maintains window state and accepts late updates until max event time seen by the engine - late threshold > T (Apache Spark documentation, retrieved 2026-11-05). Flink's Watermark(t) declares that no further elements with a timestamp at or before t should arrive (Apache Flink documentation, retrieved 2026-11-05). Kafka Streams tolerates out-of-order records only within window plus grace period (Confluent, retrieved 2026-11-05). Flink also notes that in many real-world setups certain elements can be arbitrarily delayed, making it impossible to specify a time by which all elements of a given event timestamp will have occurred.

Platform-centric readiness wins this criterion because it names the origin explicitly and records which policy produced it. Model-centric readiness has no place to put that declaration.

Leakage-Free Snapshots and Point-in-Time Joins

Naming the origin is only half the job; the joins that populate it have to respect it. Databricks defines point-in-time correctness as creating a training dataset that reflects feature values as of the time each label observation was recorded, which prevents data leakage: using feature values for training that were not available when the label was recorded (Databricks, Point-in-time feature joins, 2026-09-11). The join matches the most recent value for a primary key that is not later than the timestamps_keys column.

The sources disagree on what the default does, and the disagreement is worth stating plainly. Feast's point-in-time join documentation says the default constrains only the feature's event timestamp, so a value backfilled or corrected after an entity dataframe timestamp can still be returned for it; filter_by_created_timestamp=True adds a created_timestamp <= entity_timestamp condition to keep backfilled values from leaking into training data (Feast, Point-in-time joins, retrieved 2026-11-05). Databricks frames its own default the other way, describing the training dataset as reflecting values as of the label timestamp.

These are not strictly contradictory. Feast documents a default and an opt-in; Databricks describes the intended semantics of its own store. But a reader would reasonably conclude the two behave differently out of the box, and Feast is the more precise source on its own default. The practical rule is the as-of join: for each entity and event timestamp, select only feature records whose availability timestamp is at or before that timestamp.

Databricks adds a second trap. Omitting timeseries_columns or timestamp_keys makes the join an exact-time match instead of a prior-value match, silently removing point-in-time logic (Databricks, 2026-09-11). Silent removal is the failure mode to test for, not an error message.

Leakage-free feature snapshots and point-in-time correct data win this criterion because the check is mechanical: you can count the rows an as-of join retrieves versus a latest-snapshot join on your own data and see the difference.

Late Arrivals, Incremental Refresh and Backfill Honesty

Those joins depend on data that may not have arrived yet, which is where refresh and backfill policy take over. Incremental refresh on a materialized view detects changes in the source and recomputes only the affected results, and the documentation is explicit that it is best-effort: some changes in data sources require a full refresh (Databricks, Incremental refresh for materialized views, 2026-10-02). Treating incremental refresh as a correctness guarantee is the mistake this criterion catches.

Backfill has a harder limit. A full refresh of a streaming table discards all existing data and metadata and restarts the stream from the beginning, removing all checkpoint data, and it does not reprocess data unless your source retains the full historical dataset (Databricks, Full refresh for streaming tables, 2026-09-11). The documented failure example is a Kafka topic with 24-hour retention: run a full refresh after that window and older messages are unrecoverable.

That limit interacts with the model's design. Timestral caps context at the trained length and keeps the most recent steps (mistralai, repository README, retrieved 2026-11-05). A pipeline that loses history cannot rebuild the long context the model is built to consume, and neither source says this on its own; it appears only when the two are read together.

The trade-off is stated officially. In batch, previous results are overwritten and corrected with late-arrival data, and results are always accurate and reflect all available data in the source; in streaming, results cannot always be accurate considering out-of-order and late-arrival data (Databricks, Batch vs. streaming data processing, retrieved 2026-11-05). That is a correctness caveat to design for, not an argument against streaming.

Late-arriving data handling and incremental refresh win this criterion because the platform posture states the retention requirement up front and the model posture does not.

Backtesting, Interval Calibration and Drift Evidence

Once the pipeline can state what it retained and when, the remaining question is whether the forecasts built on it can be scored honestly. The standard rolling-origin backtest described in Forecasting: Principles and Practice requires that the training set consist only of observations prior to the test observation, with no future observations used in constructing the forecast, and it notes that a model which fits training data well will not necessarily forecast well (Hyndman & Athanasopoulos, Forecasting: Principles and Practice, §5.10, retrieved 2026-11-05). A single held-out split does not satisfy this.

For probabilistic output, the evaluation metric has to be a proper scoring rule. A scoring rule is proper if the forecaster maximizes the expected score by issuing the true predictive distribution rather than a different one, and strictly proper if that maximum is unique (Gneiting & Raftery, Strictly Proper Scoring Rules, Prediction, and Estimation, March 2007). The paper also warns that skill scores are generally improper even when the underlying scoring rule is proper. Since the model emits 21 non-crossing quantiles per step (mistralai, repository README, retrieved 2026-11-05), backtesting and interval calibration should be built on quantile and interval scoring rules, and the sources here do not establish which metric the model's published benchmarks use.

Drift evidence needs more than one signal. Google Cloud's model-monitoring definitions distinguish training-serving skew, where the production feature distribution deviates from the training distribution, from inference drift, where the production feature distribution changes over time, and add output-distribution and feature-attribution drift (Google Cloud, Vertex AI Model Monitoring overview, retrieved 2026-11-05). AWS's drift-monitoring guidance separates data drift, a statistical change in the input data, from concept drift, a change in the relationship between inputs and outputs, and notes that concept drift often requires monitoring downstream business metrics rather than direct statistical tests (AWS Prescriptive Guidance, Detecting drift in production applications, retrieved 2026-11-05).

The regulatory frame has moved. The current US model-risk guidance is Revised Guidance on Model Risk Management, issued 17 April 2026 as SR 26-2 by the Federal Reserve, OCC and FDIC, and it supersedes and replaces SR 11-7 and SR 21-8; it states that a model's inherent risk reflects the quality of inputs and data constraints, and that testing includes out-of-sample and out-of-time testing with a critical assessment of data quality and inputs (Federal Reserve, OCC, FDIC, SR 26-2, 2026-04-17). The guidance says it does not set enforceable standards. NIST's 2026 monitoring report frames detecting performance degradation and drift as a barrier under functionality monitoring and cites the difficulty of detecting data-only distributional changes between pre- and post-deployment time points (NIST, AI 800-4, March 2026).

Forecast drift monitoring wins this criterion because the platform posture can produce the evidence these frameworks ask for: dated test reports, documented data selection, and monitored input distributions. The model posture produces a score.

Acceptance Checklist

The criteria above translate into thirteen checks you can run against your own stack.

1. Forecast origin is defined and recorded. Every forecast row carries the availability timestamp of its origin, not just the event timestamp of the latest observation.

2. Event time and processing time are separated in the schema. The pipeline stores both, and the windowing logic reads event time.

3. A completeness declaration exists per stream. Name the mechanism (watermark, stream time plus grace, late threshold) and the value, and record it alongside the data.

4. Point-in-time joins are verified, not assumed. Confirm whether your feature store's default constrains only the event timestamp, and enable the created-timestamp filter if it does.

5. Time-series keys are declared. A timestamp column used as a plain primary key silently removes point-in-time logic.

6. Retention is measured against the context window. Confirm the source retains enough history to rebuild the context length your model consumes.

7. Refresh semantics are documented. State whether incremental refresh is best-effort on your sources and what triggers a full recompute.

8. Backfill limits are written down. Record which sources can replay history and which cannot.

9. Backtests use a rolling origin. No future observations in any training fold.

10. Probabilistic metrics are proper scoring rules. Quantile and interval scores, not ad-hoc accuracy.

11. Empirical coverage is compared to nominal. Count the observations below each quantile level and compare to the level itself.

12. Drift monitoring covers inputs, outputs and concept. Input distribution tests plus business-metric monitoring for concept drift.

13. Validation evidence is dated and retained. Test reports and data-selection records exist in a form an auditor can retrieve.

Frequently Asked Questions

Is point-in-time correctness the same as time travel?

No. They solve different problems and the names invite the confusion. Databricks documents its point-in-time feature lookup as unrelated to Delta Lake time travel, which queries an older table version rather than reconstructing what a feature value was when a label was observed (Databricks, retrieved 2026-09-11). Time travel answers "what did the table look like then." A point-in-time join answers "what did this entity's features look like at the moment this outcome occurred," and only the second one protects a training set from leakage.

Can I use a foundation model without a feature store?

Yes, and plenty of teams do. Dropping the feature store removes the managed lookup service, not the requirement underneath it: some component still has to enforce the as-of join rule, so that each row pairs an observation with the feature values that existed before it. That logic can live in a transformation layer, a SQL view, or a notebook, but if it lives nowhere, the leakage it was meant to prevent comes back. The question is where the discipline is enforced, not whether you have a vendor product.

Does a watermark guarantee no data is lost?

No. A watermark declares an assumption about completeness, and the engines document what happens to records that break it. Spark accepts late updates until the watermark passes the window end, while Kafka Streams tolerates out-of-order records inside the window plus a grace period and drops later ones (Confluent, retrieved 2026-11-05). Late-arriving data handling therefore has two halves: the tolerance window you configure, and the path you build for everything that falls outside it.

Is SR 11-7 still the current model-risk guidance?

No. SR 26-2, issued 17 April 2026, supersedes and replaces SR 11-7, and it states that a model's inherent risk reflects the quality of its inputs and data constraints, that validation includes documenting data selection, and that testing includes out-of-sample and out-of-time testing (Federal Reserve, OCC, FDIC, 2026-04-17). It also states plainly that it does not set enforceable standards, so treat it as supervisory expectation rather than a compliance checklist. For forecast drift monitoring and point-in-time correct data, that expectation points the same direction the engineering does: document the data selection, test out of time, and keep the evidence.

Conclusion

The comparison resolves in one direction. Model-centric readiness assumes the data layer is correct and measures the forecast; platform-centric readiness defines what was knowable at the forecast origin and measures that. Every criterion in this article, from event-time handling to backfill limits to calibration evidence, is owned by the platform, and the model documentation makes no claim to any of it.

The evidence also has a boundary worth naming. No source in this review measures how often late arrivals occur in enterprise pipelines, and none empirically links point-in-time correctness to forecast error. The mechanism is documented; the effect size is not. Treat the framework as a set of testable properties rather than a proven performance claim.

The next step is the checklist. Run items 4, 5 and 6 against your own feature store and source retention this week: confirm whether your point-in-time join default constrains only the event timestamp, confirm your timestamp keys are declared as time-series keys, and confirm your sources retain enough history to rebuild the context your model consumes. Those three checks are mechanical, and each one produces a number you can put in front of a reviewer.