Comparing Bot Configurations with Relay's Compare Bots Mode

When you're running multiple bots, even bots on the same underlying symbol, small configuration differences can produce meaningfully different outcomes. The challenge is that those differences are usually spread across multiple views in the platform, making direct comparison slow and error-prone.

Relay's Compare Bots mode solves that. It pulls everything into a single, structured comparison: performance metrics and full configuration, side by side, delivered through a single request.

What compare bots does

When you ask Relay to compare your bots, it:

1. Lists all your active bots
2. Runs the comparison across performance and configuration data
3. Returns two outputs simultaneously: a card view for quick scanning and a written summary for context

The card view shows each bot as a structured panel with its key numbers and config settings laid out in parallel columns. The written summary presents the same information in plain language, useful for understanding what the differences actually mean, not just what they are.

Triggering a Comparison

From the capability chip or grid: Click "Compare bots" from the Relay welcome screen chips, or select "Compare Bots" from the full capability grid.

Through conversation:

"Compare my bots side by side." "How do my QQQ bots compare to each other?" "Compare QQQ UT (SHRT) and QQQ UT [LNG]." "Show me a comparison of all my SPY bots."

You can compare your entire bot portfolio at once, or narrow to a specific subset. If you have many bots, scoping the comparison to a symbol or strategy type usually produces more useful output than comparing everything at once.

Reading the card view

Each bot gets its own card in the comparison, with two sections:

Performance metrics:

• Total P&L for the period
• Win rate
• Total trades executed
• Average P&L per trade
• Average hold time

Configuration:

• Stop loss level
• Take profit levels and sizing (TP1, TP2, TP3 if applicable)
• Trailing stop setting
• Trail percentage
• Cooldown window
• Position sizing method and size
• Entry start/end time

The cards are designed for parallel scanning. Run your eye down the same row across multiple cards to spot differences quickly.

Reading the written summary

Beneath the card view, Relay delivers a written analysis. This is where the context lives. Rather than just stating the numbers, the written summary surfaces what the differences might mean:

• Which configuration settings differ between bots that are otherwise similar
• Whether the performance difference is consistent with what those configuration differences would predict
• Any patterns in the trade history that explain divergence beyond what the config alone would suggest

The written summary is also where Relay notes things the card view can't show, like whether one bot was affected by a specific market event, or whether a difference in trade count suggests a signal frequency issue rather than a strategy problem.

A real example: two bots, same symbol

Consider two bots running on QQQ with nearly identical configurations:

• Both use fixed dollar sizing at $1,000
• Both have a -40% stop loss
• Both have the same TP levels and cooldown
• The only meaningful difference: one holds an average of 8 minutes, the other 5 minutes

The 3-minute difference in average hold time is enough to show up in P&L divergence, not because the strategy is different, but because the exit timing catches different points in the move. A comparison surfaces this immediately. Without a side-by-side view, you might attribute the P&L difference to the bots' directions (short vs long) and miss that hold time is the more relevant variable to examine.

What to focus on

Not all configuration differences matter equally. When reading a comparison, prioritize in this order:

1. Stop loss and take profit levels These have the largest direct impact on trade outcomes. If two bots have different stop or TP settings, that's usually the primary driver of any P&L difference.

2. Position sizing A fixed dollar difference in position size changes the absolute P&L numbers even if the percentage outcomes are identical.

3. Hold time Significant differences in average hold time, even with identical configurations on paper, suggest that one bot's exits are firing differently. Worth investigating whether the exit conditions are actually equivalent.

4. Trade count If two bots have run for the same period but different trade counts, the bot with fewer trades may have stricter entry conditions or a longer cooldown. Higher trade count isn't inherently better, consistency matters more.

5. Cooldown settings A longer cooldown means fewer trades but more filtering between signals. If one bot's win rate is significantly higher, the cooldown difference is often a contributing factor.

Following up after a comparison

Compare Bots is a starting point. Once you've identified where two bots differ, the useful follow-up questions depend on what you found:

If one bot significantly outperformed:

"Why did QQQ UT (SHRT) outperform QQQ UT [LNG] this week?" "What configuration differences are most likely driving the P&L gap?"

If you want to test a change:

"What would QQQ UT [LNG]'s performance look like if I increased the hold time to match QQQ UT (SHRT)?"

This transitions the conversation to What-If Simulation mode, where Relay can model the impact of the change against historical data.

If the comparison raises questions about a specific bot:

"Run a health check on QQQ UT [LNG]." "Show me the individual trades from QQQ UT [LNG] this week."

Comparing across different symbols

Compare Bots also works across bots running on different symbols. This is useful for a portfolio-level view, understanding which symbol/strategy combinations are generating the most consistent results and which may need reconfiguration.

When comparing across symbols, focus less on absolute P&L numbers (different volatility environments produce different absolute moves) and more on win rate consistency, average hold time, and how the trade count aligns with your expected signal frequency for each symbol.