Populated mode

Isolated and populated

In isolated mode every agent meets the same market to the bit, so a difference between two agents is a difference between their strategies. tf.evaluate and tf.rank have always worked this way, and a run without population= is the run it was before populated mode existed, on the same preset.

In populated mode the background traders react to each agent's trading, so the market an agent meets depends on what it does. The run is still reproducible: the same seed, universe, model and population give the same market. But two strategies in it no longer face identical markets, so a gap between them mixes strategy and reaction. tf.rank takes no population=, because its paired sign test needs every agent on the same market draw on each seed. Rank strategies in isolated mode and use populated mode to test one of them against other traders.

Under tf.evaluate each agent trades alone with the population in its own copy of the market, and the untraded baseline gets a copy with the population too. In a World with several agents, the agents and the population share one market.

A first populated run

The same two baselines on the same seed, once in isolated mode and twice with the crowded population:

import tradefloor as tf

def entrants():
    baselines = tf.reference_agents()
    return {"momentum": baselines["momentum"],
            "buy_and_hold": baselines["buy_and_hold"]}

universe = tf.Universe.random(20, seed=7)
crowded = tf.Population.crowded()
isolated = tf.evaluate(entrants(), seed=7, universe=universe, days=20)
populated = tf.evaluate(entrants(), seed=7, universe=universe, days=20,
                        population=crowded)
again = tf.evaluate(entrants(), seed=7, universe=universe, days=20,
                    population=crowded)

for name in ("momentum", "buy_and_hold"):
    print(f"{name:13} isolated {isolated[name].pnl:>9,.0f}  "
          f"populated {populated[name].pnl:>9,.0f}")
print("reproducible:", populated["momentum"].as_dict() == again["momentum"].as_dict())
print("population:", populated["momentum"].population_fingerprint)

momentum      isolated   -82,196  populated   -77,150
buy_and_hold  isolated    -1,194  populated    -1,217
reproducible: True
population: pop-ccdb13174be4

The background traders move prices, so even buy-and-hold, which trades once, ends with a different P&L. Running the populated evaluation a second time gives the same scorecards. This is one seed over 20 days, so the gap between the two modes says nothing on its own about whether momentum does better or worse beside other traders.

The shipped populations

Each participant sets a target position per name from what the market shows and trades toward it with market orders. Sizes are shares of the name's daily volume.

In the crowded population the reversal crowd checks the one-day reversal every 15 ticks, so it reaches a new loser before a rule that decides every 65 ticks. The momentum crowd trades the five-day momentum in the last 15 ticks before the close, so a rule that rebalances at the next open trades at the price the crowd left. The three momentum members carry loss limits spread from half to one and a half times stop, so a loss that stops one out can carry the others after it.

A participant acts at the start of a tick, before the market moves, and takes no random draw. Its orders take the market maker's levels and the latent depth, and can fill an agent's resting limit order, with population:<name> as the counterparty on the agent's fill. The background traders keep their own ledger, separate from any agent's.

The background traders' ledger

Engine.population_report() returns each participant's ledger: per name its position, cash, shares and dollars traded and P&L, then its totals. Here two baselines share one market with the standard population:

import tradefloor as tf

universe = tf.Universe.random(20, seed=7)
baselines = tf.reference_agents()
world = tf.World(seed=7, universe=universe,
                 agents={"momentum": baselines["momentum"],
                         "buy_and_hold": baselines["buy_and_hold"]},
                 population=tf.Population.standard())
world.run(20)

for row in world.engine.population_report():
    print(f"{row['name']:10} orders {int(row['orders']):>5}  "
          f"traded ${row['notional'] / 1e6:>7,.1f}m  P&L ${row['pnl'] / 1e6:>6,.2f}m")

manifest = tf.RunManifest.from_json(world.manifest().to_json())
print("manifest population:", manifest.population.fingerprint)
rebuilt = manifest.reproduce()                   # raises on any mismatch
print("reproduced:", rebuilt.state_hash() == world.engine.state_hash())

trend      orders   485  traded $1,257.6m  P&L $ -2.30m
reversion  orders  1654  traded $3,631.9m  P&L $ -8.96m
liquidity  orders  3023  traded $3,804.2m  P&L $ -4.03m
detector   orders     0  traded $    0.0m  P&L $  0.00m
manifest population: pop-2196192b43ca
reproduced: True

The trend follower, mean reverter and liquidity provider lost money over these 20 days. They trade with market orders, so each pays the spread on every trade. The detector sent no orders in this run. It trades only ahead of agent flow that recurs at the same minutes of each session, and only on names with a quoted spread narrow enough for max_spread.

Records and fingerprints

A population is immutable and has a fingerprint, pop- and twelve hex digits of a sha256 over its participants. The name is a label for people and is left out, so two populations with the same participants are the same market. A custom population is a list of participants:

import tradefloor as tf
from tradefloor.population import Participant

mine = tf.Population([Participant.trend(size=0.02), Participant.detector()],
                     name="mine")
print(mine)
print(mine == tf.Population(mine.participants, name="renamed"))

Population('mine', pop-c97362658f40, [trend:trend, detector:detector])
True

Every record of a populated run carries the population:

An engine's state snapshot carries the population's state under its fingerprint, so a snapshot restores only into an engine built with the same population. A World fork carries the population's state too.

Measured effects

tf.population.MEASURED holds what populated mode was measured to do, read with Population.crowded() on pt-v21. The checks are in tools/calibration/population_checks.py in the library repository, and each comparison is paired on the seed.

import tradefloor as tf

measured = tf.population.MEASURED
print(measured["model"], measured["population"], measured["programme_cost_excess"],
      measured["runtime_ratio"])

pt-v21 crowded 0.024 1.5

Impact in tradefloor is mostly transient, so a front-runner has less to trade ahead of than in a real market. Treat the cost of being front-run in populated mode as a floor. The participants all act at the same instant and post no resting orders, so populated mode has no latency or queue position. The realism checks in How it is measured were run in isolated mode.

Over MCP

evaluate_strategies, run_stress_scenario, open_session and start_job on the local MCP server take population, "standard" or "crowded". A populated result names the population and its fingerprint in provenance and adds a caveat with the measured figures above. rank_strategies refuses a population for the reason tf.rank does.

Next steps

• Compare strategies ranks agents across seeds in isolated mode.
• World runs several agents in one market.