GigaLearn C++ Bot Training Guide

Welcome to the GigaLearn C++ PPO training framework for Rocket League! This guide provides a detailed, comprehensive overview of the core components, hyperparameter tuning, and advanced philosophies needed to train a competitive bot from scratch. This guide is tailored specifically for the C++ GGL framework.

Table of Contents (I will Fix this Someday)

The GigaLearn Training Loop
Core Components: The "Anatomy" of Your Bot
Key GigaLearn-Specific Features
Hyperparameter Tuning Guide
Interpreting Your Training Graphs
A Strategic Roadmap to a High-Level Bot
Advanced Reward Philosophy: Emergent Behavior
Reward Farming & The ZeroSum Illusion
Visualizing Your Bot
Fun Stuff & Handy Calculations

1. The GigaLearn Training Loop

Before tuning, it's critical to understand the terminology of the training loop.

Step: The smallest unit of gameplay. By default, this is 8 game ticks (at 120 ticks per second). The bot receives an observation and performs an action at each step.
Episode: A round of gameplay that ends when a TerminalCondition is met (ex: a goal is scored or a timer runs out).
Iteration: The main training cycle. It consists of two phases:
1. Collection: The learner runs many parallel games to collect a large batch of steps (experiences).
2. Learning: The learner uses the collected data to update the neural network via the PPO algorithm.

2. Core Components: The "Anatomy" of Your Bot

Your entire training environment is defined within the EnvCreateFunc function in your src/ExampleMain.cpp. This is where you assemble the pieces that define your bot's world.

Observation (Obs) Builders: The Bot's Eyes

The Obs Builder is arguably the most important component. It is the eyes of your bot. It takes the raw, complex GameState and translates it into a simple list of numbers (FList) that the neural network can understand. If crucial information is missing from this list, your bot will be blind to it and can never learn to react to it.

How to Improve an Obs Builder

The goal is to provide the most relevant information in the most useful format.

Relative Coordinates: Instead of just telling the bot its own global position and the ball's global position, it's often more effective to provide the ball's position relative to the car. This pre-processes the information into a more useful format. The AdvancedObs builder does this for you.
Game Context: You can add custom values to your observation. For example, adding the remaining game time or the current score differential can teach a bot to play more aggressively when losing and more defensively when winning.

// A conceptual custom Obs Builder that adds distance to the center of the field
class MyCustomObs : public DefaultObs {
public:
    virtual FList BuildObs(const Player& player, const GameState& state) override {
        // Start with the information from a base obs builder
        FList obs = DefaultObs::BuildObs(player, state);

        // Add our new custom feature: distance to the center of the field, normalized
        float distToCenter = player.pos.Length2D();
        obs += distToCenter / CommonValues::BACK_WALL_Y;

        return obs;
    }
};

Action Parsers: The Bot's Hands

The Action Parser does the opposite of an Obs Builder: it takes the network's output (a single integer representing a chosen action) and converts it into game controls.

DefaultAction: The standard parser for GigaLearn provides a pre-defined set of discrete actions for ground and air play. It also includes a crucial GigaLearn feature: Action Masking.

State Setters: The Training Scenarios

A State Setter resets the game at the start of each episode. The variety of your starting situations is critical for developing a well-rounded bot.

KickoffState: Resets to a standard kickoff.
RandomState: Places the ball and cars in random positions and with random velocities. Excellent for general training.

CombinedState: (Recommended) Allows you to mix multiple state setters with weights. This is the best way to ensure your bot trains in a variety of scenarios.

// Example: 70% random states, 30% kickoffs
#include <RLGymCPP/StateSetters/CombinedState.h>
#include <RLGymCPP/StateSetters/KickoffState.h>
#include <RLGymCPP/StateSetters/RandomState.h>

result.stateSetter = new CombinedState({
    { new RandomState(true, true, false), 0.7f },
    { new KickoffState(), 0.3f }
});

Reward Functions & Terminal Conditions

These define the bot's goals and the rules of the episodes. They are covered in detail in the sections below.

3. Key GigaLearn-Specific Features

GigaLearn introduces several powerful features designed to accelerate and stabilize training.

Shared Head Architecture

A highly efficient network design where the policy (chooses actions) and critic (predicts rewards) share initial layers. The game state is processed once by the shared layers, then the output is split into separate policy and critic heads. This reduces duplicate computation and allows you to build larger, more powerful networks with less performance impact.

Separate Networks:
- Critic: {1024, 1024, 1024, 1024}
- Policy: {1024, 1024, 1024, 1024}
Equivalent Shared Head:
- Shared Head: {1024, 1024, 1024} (with addOutputLayer = false)
- Critic: {1024}
- Policy: {1024}

This shared design combines the computational load, allowing for deeper networks that can learn more complex behaviors.

Action Masking

Action masking is one of GigaLearn's most powerful features and fundamentally changes how your bot learns, especially for aerial play. It is enabled by default in GigaLearn's DefaultAction parser, which sets it apart from nearly every other Rocket League training framework.

What is Action Masking?

Action masking prevents the bot from selecting impossible or invalid actions based on the current game state. For each step, GigaLearn generates a "mask" that marks which actions are valid, and the PPO algorithm is only allowed to choose from that subset.

Examples of masked actions:

Flipping when no flip is available: If the bot has already used its flip or hasn't jumped yet, all flip actions are masked out.
Dodging in invalid states: If the bot is on the ground, dodge actions are invalid.
Double jumping when already airborne: If the bot has already jumped, the second jump action becomes invalid after a short window.

By removing these impossible actions from consideration, the bot never wastes time learning "don't try to flip when you can't flip" it simply can't make that mistake in the first place.

The Difference: With vs. Without Action Masking

The presence or absence of action masking has a dramatic impact on your bot's behavior, especially regarding aerial play.

Without Action Masking (Traditional Frameworks)

When a bot tries to execute an impossible action (like flipping with no flip available), one of two things happens:

The action fails silently: The game ignores the input, and the bot receives no feedback that it did anything wrong.
The bot gets punished indirectly: The bot may lose positioning, miss the ball, or concede a goal as a consequence of the failed action.

The result: The bot learns that aerial actions are dangerous and unreliable. It discovers that:

Attempting flips in the air often results in nothing happening (when the flip isn't available)
Staying on the ground is safer and more predictable
Ground play leads to more consistent rewards

Outcome: Bots trained without action masking are notoriously ground heavy and avoid aerial play. They learn to stay grounded because it minimizes the risk of selecting invalid actions that lead to poor outcomes.

With Action Masking (GigaLearn Default)

With action masking enabled, the bot cannot select invalid actions. Every action it takes is guaranteed to execute as intended.

The result: The bot learns that aerial actions are safe and effective. It discovers that:

When it chooses to flip, it always flips (because it can only choose that action when a flip is available)
Aerial maneuvers are reliable tools for reaching the ball
The reward for successfully hitting the ball in the air is worth pursuing

Outcome: Bots trained with action masking are significantly more aggressive in the air. They are "flippy" and aerial oriented because they've learned that these actions are viable, consistent options. Additionally, Action masking just learns and performs better.

The 10x Air Action Difference

In practice, the difference is staggering. Bots trained with action masking can show a ~10x increase in the frequency of aerial actions compared to bots trained without it. This is because:

Valid actions only: The bot only ever experiences successful aerial maneuvers, so it associates them with positive outcomes.
Faster learning: The bot doesn't waste thousands of iterations learning to avoid invalid flip attempts it skips that phase entirely.
Encourages exploration: Since air actions are reliable, the bot is more willing to explore aerial plays early in training.

Important: Migrating to GigaLearn from Other Frameworks

If you are switching to GigaLearn from a Python based framework (like RLGym-PPO) or any other framework without action masking, you will immediately notice that your bot is MUCH more "flippy."

Why this happens:

Other frameworks train without action masking by default
GigaLearn has action masking enabled by default
Your reward structure was likely tuned for a bot that avoids the air

What you need to do:

Lower your AirReward weight significantly. Since the bot now naturally gravitates toward aerial play, you don't need a strong reward to encourage it. Start by cutting the weight in half or more.
Monitor for spam flipping: If your bot is constantly flipping without purpose, it's over incentivized for aerial actions. Reduce air related rewards.

Action Masking and Entropy

Action masking also has a significant effect on entropy, which measures the randomness/diversity of the bot's action choices. However, this interaction is actually highly beneficial for training efficiency and learning quality.

Without Action Masking:

The bot has access to the full action space at all times (90 possible actions)
Many of these actions are invalid in the current state
The entropy calculation includes all 90 actions, even the invalid ones
Result: Entropy appears artificially high, but much of it is meaningless noise from invalid actions

With Action Masking:

The bot only has access to valid actions in the current state (ex: 32 valid actions out of 90)
The entropy calculation only considers the valid subset/actions
Result: Entropy values are lower but more meaningful they reflect true behavioral diversity

Why Entropy Masking is Good

When action masking is enabled, You should apply entropy masking as well, which fundamentally improves the training process:

Meaningful Exploration: The entropy bonus (controlled by entropyScale) only applies to valid actions. This means the bot is rewarded for exploring useful behavioral diversity, not for randomly selecting impossible actions. Every bit of exploration actually teaches the bot something valuable about the game.
Faster Convergence: By eliminating entropy waste on invalid actions, the bot converges to effective strategies faster. It doesn't spend iterations randomly trying to flip without a flip available it only explores within the space of actions that can actually help it learn.
Stable Entropy Values: Without masking, the entropy value fluctuates depending on how many actions happen to be invalid in each state. With masking, entropy reflects the bot's actual decision making uncertainty, making it a more reliable metric for tuning entropyScale.
Better Synergy with Action Masking: Entropy masking and action masking work together perfectly. Action masking prevents invalid actions from being selected, while entropy masking prevents invalid actions from diluting the exploration bonus. Together, they create a "clean" action space where every action and every exploration choice matters.
Hyperparameter Consistency: Because entropy is calculated only over valid actions, your entropyScale setting remains effective even as the number of valid actions changes throughout the game (ex: when the bot has or doesn't have a flip available). This makes tuning more predictable.

Bottom Line: Entropy masking makes the exploration process smarter. Instead of encouraging random noise across all 90 actions (many of which are invalid), it encourages meaningful diversity within the actual valid actions. This accelerates learning and produces more consistent, reliable behavior.

How to Configure Action Masking

Action masking is controlled in your action parser. In GigaLearn's DefaultAction parser, it is enabled by default. If you are using a custom action parser it is up to you to include action masking or leave it out, If you want to disable action masking in DefaultAction you will also have to do this yourself by changing the action parser to not be masked.

// In your EnvCreateFunc (ExampleMain.cpp)
result.actionParser = new DefaultAction(); // Action masking is ON by default in GigaLearn

Recommendation: Keep action masking enabled. It is a powerful feature that accelerates learning and produces more reliable aerial play. The only reason to disable it would be if you wanted you to replicate the behavior of older, non-masked frameworks or you just can't figure out why your bot is spam flipping and how to avoid it.

Normalized Entropy Scale

The entropyScale hyperparameter rewards the bot for exploring different actions. Unlike the ent_coef in other frameworks, GigaLearn's entropyScale operates on a normalized entropy value. This means the setting remains consistent and effective even if you change the number of available actions in your parser, reducing the need for frequent retuning.

If you are migrating from a Python framework and want to convert your old ent_coef, you can use the following formula:

entropyScale = ent_coef * ln(number of actions)

For example, an ent_coef of 0.01 with 90 actions (LookupAction) so this would be 0.01 * ln(90) therefore, the entropyScale would aproximately be 0.045 in GigaLearn.

Model Configuration

GigaLearn offers fine grained control over your neural network architecture.

Activation Functions (activationType): These determine a neuron's output. GigaLearn supports RELU, TANH, LEAKY_RELU, and SIGMOID. LEAKY_RELU is a strong modern choice that helps prevent the "dying ReLU" problem, where neurons can become permanently inactive.
Layer Normalization (addLayerNorm): Enabled by default (true), this feature normalizes the inputs to each layer, which significantly stabilizes training and improves performance. It is highly recommended to keep it enabled.
Optimizers (optimType): ADAM is an excellent, all around default optimizer that adapts the learning rate for each network parameter individually.

4. Hyperparameter Tuning Guide

These settings in your LearnerConfig are the primary knobs for tuning the training process.

Parameter	What it Does	Recommended Value	Why
`policyLR` / `criticLR`	Learning Rate. Controls how large the network updates are.	Late: `< 0.8e-4`, Mid: `1e-4`, Early: `1.5e-4`	The most critical parameter for stability. Lower it as the bot gets better to allow for fine tuning.
`tsPerItr`	Timesteps per Iteration. The size of the experience batch.	Scoring: `200-300k+`, Trying To Score: `100k`, Touching: `50k`	A larger batch provides a more stable learning signal, crucial for advanced strategy.
`ppo.epochs`	Epochs. How many times to pass over the collected data.	1 to 3	See the detailed "Epochs Debate" section below. The modern consensus is that 1 epoch can be superior.
`ppo.entropyScale`	Entropy Scale. Encourages the bot to act more randomly (explore).	`~0.03 - 0.045`	Encourages exploration. Increase to escape a plateau; decrease to encourage refinement. See the GigaLearn features section for more details.
`ppo.gaeGamma` & `gaeLambda`	GAE Parameters. Control how future rewards are valued.	`gamma=0.99+`, `lambda=0.95`	`gamma` is the discount factor for future rewards. `lambda` controls the trade off between bias and variance. The defaults are well tested.
`numGames`	The number of parallel game instances to run.	As many games until your SPS stops increasing	More games means faster data collection (`Collection SPS`) and `Overall SPS`. Tune for maximum SPS. Note: After A certain amount of numgames there's no gains
`ppo.miniBatchSize`	The size of data chunks used during a learning update.	`25,000` - `100,000`	This is a performance/VRAM setting. It does not affect the learning outcome, only the speed. Tune for maximum SPS
`ppo.policy.layerSizes`	The architecture of your neural network.	`{1024, 1024, 1024, 1024}` or larger.	A larger network can learn more complex behaviors but trains slower.

The Epochs Debate: 1 vs. Many

This is one of the most debated topics in PPO tuning.

The Multi-Epoch Argument (High Sample Efficiency): Historically, data collection was the primary bottleneck. The logic was to use 2-3 epochs to learn as much as possible from each expensive batch of data. This maximizes sample efficiency. Furthermore, iterating multiple times over the same data increases the chances that the model will learn from rare, high value events (outliers) like a flip reset, preserving that information more effectively than a single pass would.
The 1 Epoch Argument (High Data Freshness): With modern hardware, data collection is incredibly fast. The new bottleneck is often the learning phase itself. PPO is an on-policy algorithm, meaning it learns best from data collected by the current version of the policy.
- Epoch 1: The policy update is calculated using perfectly on policy data.
- Epoch 2+: The policy has already changed from the first update. The training data is now stale (off policy). While PPO uses importance sampling to correct for this, the updates become less accurate and may lead to instability (very unlikely).

Conclusion: Since collection is now often faster than learning, it's frequently more effective to use 1 epoch. This prioritizes data freshness, ensuring every gradient update is based on the most relevant experience possible. This approach updates the policy more often in terms of real world time (since the learning phase is shorter), leading to more frequent cycles of fresh data collection. As Kaiyotech said: "if you can just learn on new stuff then you should probably do that." However, for some hardware configurations, multiple epochs may still be more "wall clock efficient", Kaiyotech uses 3 epochs.

The Horizon Problem: `tick_skip` and `gamma`

These two parameters are deeply linked and control how "far sighted" your bot is.

tick_skip: The number of game ticks (at 120 per second) that occur in a single simulation step. This directly determines your bot's reaction time. For reference, average human reaction time is ~200–250 ms.
- Reaction Time (milliseconds) = (tick_skip/120) * 1000
- Decisions per second = 120/tick_skip
- tick_skip=8 (Default): 67ms reaction time. The bot makes 15 decisions per second.
- tick_skip=4: 33ms reaction time. The bot makes 30 decisions per second.
gamma: The discount factor for future rewards. It determines how much the bot values immediate rewards versus rewards it could get in the future. A reward N steps away is only worth gamma^N of its original value. A value closer to 1.0 means the bot is more patient and will plan for longer term strategies.
The Horizon: The "horizon" is how far into the future the bot effectively plans. We can measure this with the half life horizon: the time it takes for a reward to lose half its initial value. Rewards decay gradually over time, so the farther in the future they are received, the less they are worth.

H1/2 (seconds) = (tick_skip / 120) * (ln(0.5) / ln(gamma))

The Critical Link: Adjusting Gamma with Tick Skip

When you change tick_skip, you change how many decisions (steps) happen per second. Because gamma is applied per step, you must adjust gamma when you change tick_skip to maintain the same planning horizon. Failing to do so will drastically alter your bot's behavior. Additionally, When changing tick skip, continuous rewards occur more or less often. For example, reducing tick skip from 8 to 4 doubles their frequency, so divide continuous rewards by 2. Event rewards stay the same.

Example: Let's say your current setup is tick_skip=8 and gamma=0.99. Your bot's horizon is ~4.6 seconds. If you switch to tick_skip=4 to get a faster reaction time but keep gamma=0.99, your new horizon would be (4/120) * (ln(0.5)/ln(0.99)) ≈ 2.3 seconds. You've halved your bot's planning ability, because you kept the same gamma value.

To find the equivalent gamma (New Gamma) for a new tick skip (New Tick Skip):

New Gamma = Old Gamma ^ (New Tick Skip/Old Tick Skip)

So, to maintain the ~4.6s horizon when moving from tick_skip=8 to tick_skip=4:

New Gamma = 0.99 ^ (4 / 8) = 0.99 ^ 0.5 which is approximately 0.995

The equivalent gamma for tick_skip=4 is 0.995.

Recommendation on Horizon

While it's possible to train a high level bot with a short horizon (gamma=0.99), it's significantly harder. The bot is incentivized to only find short term strategies, which hinders the natural discovery of complex mechanics like aerials, advanced boost management, and patient positioning.

A longer horizon is almost always better. As you increase gamma, you will notice your bot starting to attempt more complex, long term plays because it can finally "see" the value in them. A good strategy, as used by creators like Kaiyotech, is to start with a modest horizon (ex: ~8 seconds) and gradually increase it towards 20 seconds as the bot improves. Since gamma is very sensitive, increase it in small increments. A conservative approach is to add about 0.4 seconds to your horizon every billion timesteps, but feel free to experiment with your own values.

Training Against Old Versions

A powerful technique used by many high level bots is to periodically train against saved, older versions of itself.

Why use it?
- Prevents "Forgetting": A bot can develop a weakness such as a flawed kickoff and then exploit that same weakness in its opponent (itself). Training against an older version that doesn't have that weakness forces the bot to develop a more robust strategy.
- Reduces "Shadow Boxing": Continuously playing against itself can lead to a narrow, specialized playstyle that's brittle against outside opponents. Old versions provide a more diverse set of opponents, leading to a more generalized agent.
- Escapes Plateaus: It can help the bot break through training plateaus by reintroducing challenges it had previously "solved" against its current self.
How to enable it in GigaLearn: In your LearnerConfig, set the following:
- cfg.trainAgainstOldVersions = true;
- cfg.trainAgainstOldChance = 0.15f; // A 15% chance each iteration to play against an old version.
- This requires cfg.savePolicyVersions = true; to be enabled so the learner has a pool of old versions to draw from.

5. Interpreting Your Training Graphs

Remember: Watch your bot! A skill tracker or an ELO system is the only objective measure of skill. Graphs provide clues about training health, but they can be misleading.

Mean KL Divergence: Measures how much the policy's action distribution changes after an update. A larger value means the policy is changing more significantly.
SB3 Clip Fraction: Shows how often PPO's stability mechanism is being triggered. It's the best indicator of your learning rate's effectiveness.
- A healthy target is often cited as ~0.08 according to Zealan. However, this was later found out to be a target value for 10 epochs.
- Additionally, when using 1 epoch, the clip fraction will naturally be near zero. You should ignore this graph when using 1 epoch.

6. A Strategic Roadmap to a High-Level Bot (This Roadmap is from Zealans Guide)

This roadmap mirrors the structure of the section on "Making a Good Bot" in Zealans Guide.

The Early Stages (Bronze - Silver): Learning to Touch

The only goal is to make consistent contact with the ball.

Primary Goals: 1. Learn to touch the ball. 2. Don't forget how to jump. Bots often learn that staying grounded is safer, so an AirReward is crucial to prevent them from never jumping.

Example Rewards: Use simple, strong signals. Do not reward for goals yet, as it's just noise.

// Example Early-Game Rewards
{ new TouchBallReward(), 50.f },
{ new VelocityPlayerToBallReward(), 5.f },
{ new FaceBallReward(), 1.f },
{ new AirReward(), 0.15f } // Essential to prevent the bot from forgetting to jump

Note on Action Masking: If you are using GigaLearn's default action masking (recommended), you may find that your bot is naturally more inclined to go airborne compared to bots trained in other frameworks. You may need to use a lower AirReward weight (e.g., 0.05f to 0.1f) to prevent excessive spam flipping. Monitor your bot's behavior and adjust accordingly.

The Middle Stages (Gold - Plat): Learning Purpose

The goal is to learn to hit the ball with purpose towards the opponent's net and manage resources.

A Better Touch Reward: The simple TouchBallReward can be farmed. A better reward scales with the power of the hit. The StrongTouchReward is a perfect implementation of this. For a dribble flick bot, a TouchBallReward is sufficient just zerosum it when dribbling so it learns to flick.
A Good AirTouch Reward: Reward the bot for hitting the ball high in the air to encourage aerials. See Zealan's guide for the Python version of this reward; I don't use it and don't have a C++ version.
Boost Management: Teach the bot not to waste boost. A SaveBoostReward gives a continuous reward for having boost, and a PickupBoostReward incentivizes collecting it. Zealans guide also notes that bots tend to ignore small pads, so making a custom reward to make them more valuable can be very effective.
Don't Use Massive Goal Rewards!: The guide strongly warns against this. I wouldn't use his example of 20 for the goal reward, rewards are relative. I've used 150 with good results; it's even the default in Gigalearn despite his warning against such a high value.

The Later Stages (Diamond+): Learning to Win

This stage is about refinement, decision-making, and patience.

Natural Passiveness: Bots naturally tend towards passive play because it's less risky. A faster reaction time makes a defensive posture more viable.
Promoting Aggression: To counter passivity, you can reduce the penalty for conceding a goal. For example, new GoalReward(-0.8f) makes conceding less punishing than scoring is rewarding, encouraging more aggressive, risky plays.
Let it Cook: Progress will be slow. It's important to be patient and allow the bot to explore and refine its strategies on its own.

7. Advanced Reward Philosophy: Emergent Behavior

You don't need a specific reward for every single mechanic. By teaching a bot prerequisite skills, it can discover how to combine them in novel and effective ways on its own.

By creating rewards for Wavedashes and Fast Kickoffs, a bot might discover on its own how to do a Wavedash Kickoff—a behavior you never directly rewarded.

The most effective strategy is a hybrid approach:

Start with general rewards to build a solid foundation.
When the bot plateaus, introduce a targeted, specific reward to teach a new "tool."
Once the bot masters a skill, lower that reward's weight. It will keep using the skill if it's still useful for example, I removed speed to ball and faceball rewards, but the bot kept using them strategically.

8. Reward Farming & The ZeroSum Illusion

One of the most deceptive issues in reward tuning involves the ZeroSumReward wrapper, which can hide problematic reward farming behavior behind what appears to be a balanced reward signal.

The Problem: The "Illusion of Zero"

When you wrap a reward with ZeroSumReward (ex: ZeroSumReward(new VelocityPlayerToBallReward()), the final value for that reward in your training metrics will often show values near 0.0 or relatively small numbers. This creates an illusion that might lead you to believe the reward is balanced and not being exploited.

However, this is misleading.

What's Actually Happening

The bot is engaged in a competitive race with its opponent. Here's the reality behind the zero:

Large Raw Values: The bot is actually accumulating a large positive reward value by farming the behavior (ex: constantly chasing the ball to maximize VelocityPlayerToBallReward).
The Zero-Sum Calculation: ZeroSumReward subtracts the opponent's score from your bot's score: FinalReward = MyBotScore - OpponentScore.
The Race: Since both bots are trained with the same reward and are farming it equally hard, they both accumulate similar large positive values. When subtracted, this produces a result near zero.
The Hidden Motivation: The bot isn't ignoring the reward, it's farming it aggressively to avoid getting a large negative value. If your bot stopped farming while the opponent continued, the calculation would be something like MyScore (0) - OpponentScore (large positive) = large negative. The bot learns to chase just as hard as its opponent to prevent this penalty.

Why This Matters

The 0.0 you see is not a sign that the reward is unimportant or balanced it's the result of both agents farming equally hard. If the underlying weight is too high, the bot will still prioritize farming that reward above all else, even though the final ZeroSum value appears small or neutral.

How to Detect Hidden Farming

To see what your bot is really doing with ZeroSum rewards, you need to examine the raw, unwrapped reward values:

WandB "Rewards" Section: Navigate to your Weights & Biases (WandB) dashboard and check the "rewards" section. This will show you the raw reward values before the ZeroSum calculation is applied.
Look for Large Raw Values: If a reward you did or didn't zerosum shows large values compared to your other rewards in your metrics, that's a red flag. The bot is farming that reward heavily. You will need to decrease the weight.

Practical Example

{ new ZeroSumReward(new VelocityPlayerToBallReward() 0, 1), 5.0f }

(These are example values that hold no meaning it looks different for everyone):

**What you see: **: ~0.05 (looks fine)
What's in WandB raw rewards: 1.4 (farming the shit out the reward)
The reality: Both bots are constantly ball chasing. The 0.05 difference (1.8 - 1.75 = 0.05) hides the fact that this behavior is completely dominating their decision making.

Recommendations (Both for Zerosum and Without)

Always check WandB: When using rewards, regularly review the raw reward values in your WandB logs to ensure you're not inadvertently creating a farming incentive.
Lower weights if farming is detected: If you discover high raw values for a reward, reduce its weight significantly. The goal is for the reward to provide guidance, not dominate the bot's entire strategy.
Use ZeroSum deliberately: ZeroSum rewards are powerful for creating competitive behaviors, but they require careful monitoring. Use them for rewards where direct competition makes sense (ex: ball possession, touchball) rather than wrapping every reward.
Test without ZeroSum: Periodically test your reward structure with ZeroSum wrappers removed to get an unfiltered view of what behaviors are being reinforced.

The key insight: ZeroSum rewards don't eliminate farming, they just hide it in plain sight. Always verify the raw values.

9. Visualizing Your Bot

GigaLearn includes a RenderSender utility to watch your bot play.

Enabling Render Mode: In your LearnerConfig, setting cfg.renderMode = true; enables visualization. However, this also disables training, as the simulation is slowed down to a watchable speed.
Training and Rendering Simultaneously: The recommended solution is to use a command-line flag. This allows you to run one instance of your bot that trains at maximum speed, and a second instance that only renders.
- Step 1: Modify ExampleMain.cpp Add the following code to your main() function to parse for a --render flag.
```
int main(int argc, char* argv[]) {

    bool renderMode = false;
    for (int i = 1; i < argc; i++) {
        if (std::string(argv[i]) == "--render") {
            cfg.sendMetrics = false; // No need to log metrics from a render instance
            cfg.ppo.deterministic = true; // True: see bot at its best, False: see how it plays while learning
            renderMode = true;
            break;
        }
    }
}
```
- Step 2: Run Both Instances
  1. Run your bot executable normally (./GigaLearnBot.exe) to start training.
  2. Open a new command prompt, navigate to your executable's directory, and run it again with the flag: ./GigaLearnBot.exe --render.
  3. Open a visualizer like RocketSimVis to connect to the rendering instance.
Viewing New Checkpoints: The rendering instance loads the latest saved checkpoint when it starts. To see your bot's progress, you will need to occasionally close and relaunch the --render instance. Alternatively, you could write a script that automatically relaunches it when it detects a new checkpoint has been saved.

10. Fun Stuff & Handy Calculations

Here are a few useful tools and formulas for analysis and tuning.

Bot's Total Playtime

To find out how many real-world hours of Rocket League your bot has "played":

Hours = (total timesteps * tick_skip) / (120 * 3600)

Reward Half Life (Horizon) Calculator

Easy Mode (Desmos): GammaDesmos
- Simply plug in your tick_skip (t) and gamma (g) to find your reward half life in seconds.
Manual Formula:

H1/2 (seconds) = (tick_skip / 120) * (ln(0.5) / ln(gamma))

Bot Reaction Time

Calculate your bot's reaction time in milliseconds.

Reaction Time (milliseconds) = (tick_skip / 120) * 1000

Useful Unit Conversions

For creating rewards and analyzing game state values.

1 Unreal Unit (UU) = 1 Centimeter (cm)
1 Meter (m) = 100 Centimeters (cm)
Kilometers per Hour (km/h) = 0.036 * Unreal Units per second (uu/s)

Name		Name	Last commit message	Last commit date
README.md		README.md

Latest commit

History

Folders and files

Repository files navigation