Evidence first, with a way out at every stage
We agree what success means before we fit anything, then test against simple methods, first principles and data the model has never seen.
-
Stage 1
Audit
Map your tags, check units and timestamps, and test whether the data can answer the question.
Gate Is the data fit for use?
-
Stage 2
Baseline
Agree success criteria up front, freeze a test period and score simple methods first.
Gate Is there room to beat the baseline?
-
Stage 3
Model
Fit the model and test it on data it has never seen, including its misses.
Gate Does it beat the baseline?
-
Stage 4
Shadow run
Run read-only forecasts alongside operations, with your engineers reviewing each one.
Gate Is it useful in practice?
-
Stage 5
Handover
Hand over the model, code, documentation and the conditions where it should not be trusted.
Gate Who owns it from here?
Validated against first principles
Fitting the data isn’t enough. Every data-driven model, surrogate and ROM has to pass these checks before you rely on it.
- Conservation
- Outputs must close mass and energy balances within an agreed tolerance.
- Physical limits
- Predictions must respect bounds such as temperatures, compositions and efficiencies that are physically possible.
- Correct response
- When an input moves, the output has to move the way the physics says it should.
- Full-model comparison
- Surrogates and ROMs are tested against the first-principles model on runs they never saw.
- Known range
- Every model records where it was tested, and flags cases outside that range.
Agreed before fitting
Choosing the pass mark after seeing the results would prove nothing, so we set it first.
- Target assets and operating modes
- The decision the model supports, and the lead time it needs
- The existing baseline or operating rule to beat
- Maximum missed events and false alarms
- Maximum useful uncertainty
- Number of independent test events required
Models that say when they don’t know
Outputs carry one of three states, and engineers keep ownership of every decision.
- Within validated conditions
- The plant is operating in a range the model was tested on.
- Review required
- Something looks unfamiliar. An engineer should check before relying on it.
- Insufficient evidence
- The model should not be used for this case.
What you keep
- A data dictionary and reproducible quality checks
- The model and its source code
- A comparison against simple baselines
- A failure atlas showing where forecasts went wrong
- A decision note: proceed, narrow the scope or stop
Data handling
- NDA first
- We sign before any data moves.
- Read-only
- We never need write access to a control system.
- Your data stays yours
- Nothing is published or reused without your written agreement.
- Engineers decide
- Models inform decisions. They don’t act on the plant.