Reading Optimization Results

How to interpret every component of the optimization results page -- summary cards, robustness score, overfitting probability, radar comparisons, heatmaps, return distribution, parameter sensitivity, correlations, and the parameter grid.

Reading Optimization Results

After an optimization finishes, Simula presents the results across several views: summary cards, a robustness score, a radar comparison of top combinations, a heatmap, distribution and sensitivity charts, and a sortable parameter grid. Each view answers a different question. This article walks through what to look for in each one.

Summary Cards

At the top of the results page, summary cards show the best-performing combination for key metrics like annualized return, Sharpe ratio, and max drawdown. These give you a quick overview of the best-case results across the parameter space.

Keep in mind that the best-case result for any single metric is often not the best overall choice. A combination that maximizes return might have an unacceptable drawdown. The summary cards are a starting point, not a final answer.

Robustness Score

The robustness score estimates how likely the best result is to hold up on data the optimizer never saw. It condenses the plateau-vs-spike question into a single number from 0 to 100, graded as Robust (70+), Moderate (40-69), or Fragile (below 40).

The score is built from three equal-weight components, each scored 0-100:

Performance Breadth

What fraction of all tested combinations achieved at least 50% of the best return. A high breadth score means many parameter settings produced good results -- the strategy is not sensitive to exact values. Simula treats 30% coverage as a perfect score and scales linearly from there.

Stability Ratio

How close the average return of the top 10% of combinations is to the single best return. If the top 10% average is nearly as good as the best, the best result is not an isolated spike -- it sits within a region of strong performance.

Neighborhood Consistency

How well the parameter combinations immediately adjacent to the best one perform. Simula identifies neighbors that differ by exactly one parameter step and averages their returns. A high score means the performance surface is smooth around the optimum -- small parameter changes do not cause large performance drops. For runs that used random sampling, the exact one-step neighbors may not have been drawn, so Simula measures the sampled combinations closest to the best one instead -- the same question, asked of the nearest tested points.

The overall score is the average of the three components. A score of 70 or above is encouraging, but it is not a guarantee. A high robustness score means the optimizer found a broad, stable region -- not that the region will persist into the future. Walk-forward analysis is the next step for testing whether it does.

Overfitting Probability

The overfitting probability card answers a sharper version of the same question: if you had only seen part of your data, would picking the best combination there have led you to a combination that also wins on the rest? The number it shows -- the probability of backtest overfitting (PBO) -- is the fraction of such experiments where the answer was no.

Simula runs the experiment like this. The date range is cut into equal time blocks (usually 16). Every possible way of assigning half the blocks to a "training" set and half to a "testing" set becomes one experiment -- with 16 blocks, that is 12,870 experiments. In each one, Simula finds the combination with the best total return on the training blocks, then checks where that same combination ranks among all combinations on the testing blocks. If the training winner lands in the bottom half of the testing ranking, that experiment counts as overfit. No extra backtests are run -- every experiment reuses the results your optimization already computed.

Two points on the scale are not arbitrary. A PBO of 50% is the mathematical baseline for pure noise: if the combinations carry no real information, the training winner's rank on the testing blocks is random, so it lands in the bottom half exactly half the time. And 5% is the rejection threshold recommended by the researchers who developed the method (Bailey, Borwein, Lopez de Prado, and Zhu) -- the same 5% convention used for statistical significance elsewhere. Simula grades the value into five bands around those anchors:

Below 5% -- Low

Under the method's own rejection threshold. The training winner kept winning on unseen blocks in almost every experiment; picking parameters by past performance genuinely worked on this data.

5% to 20% -- Moderate

Some experiments failed, but well under half. Strategies with a real but modest edge land here -- in the original paper's worked example, a strategy built on a genuine seasonal effect scored around 13%.

20% to 50% -- Elevated

A substantial share of experiments failed. The selection carries some information, but not enough to lean on -- treat the best combination's backtest numbers as optimistic.

50% to 70% -- High

At or worse than the 50% coin-flip baseline. The "best" combination's rank on unseen data is no better than random -- the optimization's pick tells you nothing about how it will rank next.

Above 70% -- Severe

The training winner systematically LOST on the testing blocks. The settings that looked best were fitted to noise specific to the training data, and choosing them is actively worse than choosing at random.

The boundaries between neighboring bands (20% and 70%) are judgment calls that follow the interpretation ranges used in the PBO literature -- the labels are orientation, not verdicts. The card always shows the exact percentage, and a 19% run is not meaningfully different from a 21% run.

A high PBO does not always mean no edge

When many combinations perform almost identically, no single one can reliably stay on top, so the PBO rises even if every combination is genuinely profitable. Read the PBO together with the return distribution below: tightly clustered positive returns with a high PBO means "the exact settings do not matter much", while widely spread returns with a high PBO means the top results are likely luck.

The card also shows the deflated Sharpe ratio of the best combination. Testing many combinations and keeping the best inflates the winner's Sharpe ratio -- even random strategies produce an impressive-looking best result if you try enough of them. The deflated Sharpe ratio corrects for how many combinations were tried and asks: what is the confidence that the winner's Sharpe ratio would still be positive after removing that selection advantage? A value above 0.95 is conventionally read as significant -- the performance is unlikely to be a side effect of trying many settings.

The third number, the performance degradation slope, fits a line through the experiments: each split contributes a point pairing the winner's training-blocks return with its testing-blocks return. A slope near 1 means training performance carried through at full strength; a slope near 0 means training performance said little about testing performance.

Why the slope is usually negative -- and why that alone is not damning

A combination's training half and testing half together always make up its full-period result -- a time block that boosts one half is absent from the other. So when the same combination wins most splits, the splits where its training return was highest are exactly the splits where its testing return was lowest, and the fitted slope is pulled toward -1 by that arithmetic alone. A negative slope therefore is not, by itself, evidence of overfitting. Judge the run by the PBO value and the rank distribution; treat the slope as context.

Why does the card say 'Not available for this run'?

The check needs enough raw material to be meaningful: at least 10 tested combinations to rank against each other, and enough candles to cut into at least 6 time blocks. Small grids or very short date ranges fall below those floors, and Simula omits the number rather than showing a misleading one. Widening the parameter ranges or extending the date range enables it.

Like the robustness score, this check reuses the single historical dataset your optimization ran on. It tells you whether selection worked WITHIN that history -- walk-forward analysis remains the test for whether re-optimizing over time would have kept up with changing markets.

Top Parameter Comparisons

The radar chart compares the top-ranked parameter combinations across all metrics at once. Each shape represents one combination, and each axis represents a metric (return, Sharpe, drawdown, win rate, and so on). You can select how many to compare (top 3, 5, or 10) and which metric to rank by.

When the shapes overlap closely, the top combinations are performing similarly across multiple dimensions -- this is a good sign. It means your best results are not a fluke of one metric; they hold up on risk-adjusted measures too. When the shapes diverge wildly, the top combinations are good in different ways, and you will need to decide which trade-offs matter most.

Normalization

Each axis is normalized against the full grid, not just the displayed entries. This means the top combinations will cluster near the outer edge of the chart, accurately showing they are strong relative to the entire parameter space. Values are inverted for metrics where lower is better (drawdown, VaR, CVaR), so the outer edge always means "better".

The Heatmap

The heatmap is the most informative view for understanding how performance varies across the parameter space. Its layout depends on how many parameters you optimized.

One Parameter

With a single parameter, the heatmap displays a bar chart. Each bar represents one tested value, and the bar height shows the performance for the selected metric. This gives you a direct view of the performance curve as the parameter changes.

Two Parameters

With two parameters, the heatmap becomes a color-coded grid where one parameter runs along the X axis and the other along the Y axis. Each cell is shaded by performance -- brighter or warmer colors indicate higher values. This is where you can visually identify plateaus and spikes.

Three or More Parameters

With three or more parameters, you can only display two dimensions at a time. Simula provides dropdown controls to select which two parameters form the grid axes and which values to hold fixed for the remaining parameters. By changing the fixed values, you can explore slices through the higher-dimensional parameter space.

Sampled runs aggregate instead of slicing

For runs that used random sampling, the fixed-value dropdowns are replaced by aggregation. A slice keeps only the combinations that exactly match the fixed values on every non-displayed parameter, and a random sample scattered across a large grid contains almost no exact matches for any single slice. Instead, each cell shows the average across every sampled combination that shares that cell's two axis values -- whatever the other parameters were -- and the tooltip shows how many samples the average is built from. Cells that received no samples stay empty.

Choosing a metric

The heatmap can display different metrics using the metric selector. Switch between return, Sharpe, drawdown, and other metrics to see whether the best region is consistent across measures. A parameter combination that ranks well on multiple metrics simultaneously is more trustworthy than one that excels on a single metric.

Plateau vs. Spike

The shape of the performance surface tells you more than the peak value. Look at the region around the best result, not just the result itself.

A plateau means many nearby parameter combinations produce similar performance. The strategy is not sensitive to exact parameter values -- it captures a broad pattern. This is a sign of robustness, not overfitting.

A spike means only one combination (or a tiny cluster) performs well, and everything around it is mediocre or negative. The strategy's success depends on hitting exact values. This is the signature of a curve-fitted result.

Return Distribution

The distribution chart is a histogram showing how results are spread across all tested combinations. Each bar represents a range of values for the selected metric, and the bar height shows how many combinations fell into that range. Mark lines show the mean and median.

A healthy distribution is left-skewed -- most combinations are profitable, with a long tail of weaker results. This means the strategy works across a wide range of parameter settings, not just a lucky few. If most combinations are negative and only a handful are profitable, the profitable results are likely overfit.

Use the metric selector to switch between return, Sharpe, drawdown, and other measures. A distribution that looks healthy on return but terrible on Sharpe suggests the profitable combinations achieved their returns through excessive risk.

Parameter Sensitivity

The sensitivity chart shows how much each parameter influences each metric. Simula uses eta-squared -- a statistical measure that captures what proportion of the total variation in a metric is explained by changing a specific parameter. A value of 0 means the parameter has no measurable effect on that metric. A value of 1 would mean the parameter completely determines the metric.

This chart helps you understand which parameters actually matter. If one parameter dominates the sensitivity chart across all metrics, the other parameters are having little effect -- you could simplify your strategy by fixing them at reasonable values and only optimizing the one that matters.

Conversely, if all parameters have roughly equal sensitivity, they are all contributing to performance. This is neither good nor bad on its own, but it means you should pay attention to all of them when selecting your final values.

Sensitivity requires 2+ parameters

The sensitivity chart only appears when you optimized two or more parameters. With a single parameter there is nothing to compare against.

Parameter-Metric Correlations

The correlation chart shows the direction and strength of each parameter's relationship with each metric using Pearson correlation. A positive value means increasing the parameter is associated with a higher metric value. A negative value means the opposite. Values near zero mean no linear relationship.

This is useful for understanding the direction of each parameter's effect. For example, if a moving average period shows a negative correlation with return but a positive correlation with Sharpe ratio, it means longer periods reduce raw return but improve risk-adjusted performance -- a classic smoothing trade-off.

Correlation is not causation

These correlations describe linear associations in the tested data. A parameter may show zero correlation overall but have a strong non-linear relationship -- for example, a moving average period that works well at 10 and at 50 but poorly at 30. The heatmap is better for spotting non-linear patterns.

The Parameter Grid

Below the heatmap, the parameter grid table lists every tested combination with its metrics. You can sort by any column -- return, Sharpe, drawdown, profit factor, win rate, or number of trades -- to find combinations that meet your criteria.

The grid supports CSV export for offline analysis. This is useful when you want to filter or compare results in a spreadsheet, or when you want to share results with someone who is not logged in to Simula.

Should I always pick the top-ranked combination?

Not necessarily. The top-ranked combination for any single metric is the one most likely to be overfit. Instead, sort by your preferred metric and then look at the neighborhood around the top results. If the top 10 combinations cluster around similar parameter values, that region is worth investigating. If they are scattered across wildly different values, the results are likely noise.

What to Do With Your Results

1. Identify the Stable Region

Use the heatmap to find the region where performance is broadly good -- not just the single best cell. This is your candidate region. The parameters you choose should come from within this region, ideally near its center rather than at its edge.

2. Check Multiple Metrics

Switch the heatmap metric and see whether the same region performs well across Sharpe, drawdown, and profit factor. A region that ranks well on return but poorly on risk metrics is less reliable than one that performs reasonably across the board.

3. Validate Before Deploying

The optimization results show what worked in the past. Before trusting these parameters for forward trading, validate them on data the optimizer never saw. Walk-forward analysis automates this by running the optimization-then-validate cycle across multiple rolling windows.

Optimization results are not predictions

The best combination in the grid is the best combination for that specific historical period. Markets change. Regimes shift. The combination that dominated 2020-2024 may underperform in 2025. Treat optimization as a filter that narrows the field, not as a crystal ball that predicts the future. Validation is the step that tells you whether the filtered results have staying power.

Where to Go Next

Walk-Forward Analysis -- validate your optimized parameters on unseen data.

Grid Search & Overfitting -- why the best result is not always the best choice.

Overfitting & Curve Fitting -- the broader problem of fitting noise instead of signal.

Sharpe Ratio -- the most common risk-adjusted metric for comparing combinations.