Decision engines
A decision engine answers typed questions about a state with a probability for every possible answer, and writes no text. Jev, from TypeSafe, is one. A Choice picks one option from a set you define, a Score rates the state on levels you define, and a Noul gives the probability that a statement is true. Every Choice and Score answer carries a confidence from 0 to 1.
In a trading agent that splits the work three ways. Code turns the observation into something the engine reads well, the engine makes the judgement, and code turns the answer into orders, including the decision whether the answer is sure enough to act on. tradefloor runs the agent through the callable adapter, so it scores under tf.evaluate, tf.rank and World like any other.
A decision engine or an LLM adapter
- With an LLM adapter, the model writes the decision: the orders and the reasons for them.
- With a decision engine, your code writes the decision, and the engine answers narrow questions inside it. Nothing it returns needs parsing, and the confidence is part of every answer.
Install
pip install tradefloor typesafe-sdk
The TypeSafe SDK reads your key from TYPESAFE_API_KEY. Create one in TypeSafe's console. You don't need it to run this page: without a key, a stand-in answers in Jev's shape.
A first run
The agent asks one question a day: is this market calm or stressed? The answer sets how much of the account it holds in stocks. With TYPESAFE_API_KEY set, ask calls Jev. Without it, the stand-in answers from the VIX alone, and the output below is the stand-in's.
import os
import tradefloor as tf
from tradefloor.integrations.callable import callable_agent
REGIMES = {"stressed": "Volatility is high and trading is expensive.",
"calm": "Nothing unusual is happening."}
SHARE = {"calm": 1.0, "stressed": 0.4} # of net worth, in stocks
def describe(payload):
# the arithmetic stays in code, and the engine reads words
vix = payload["macro"]["vix"]
level = "normal" if vix < 24 else "raised" if vix < 40 else "high"
return {"volatility (VIX)": level,
"business cycle": payload["macro"]["cycle"]}
def ask(state):
if os.environ.get("TYPESAFE_API_KEY"):
from typesafe_sdk import Choice, TypeSafeClient
reply = TypeSafeClient(model="jev-1.13.0").system_one(state=state, questions={
"regime": Choice(instructions="Which best describes this market today?",
criteria=REGIMES)})
answer = reply.answers["regime"]
return answer.choice, answer.confidence
# the stand-in: an answer in Jev's shape, from the VIX alone
return {"normal": ("calm", 0.9), "raised": ("calm", 0.3),
"high": ("stressed", 0.8)}[state["volatility (VIX)"]]
def rebalance(payload, share):
worth, assets = payload["portfolio"]["net_worth"], payload["assets"]
orders = []
for a in assets:
want = share * worth / len(assets) / a["price"] - a["position"]
quantity = int(min(abs(want), a["max_order_shares"]))
if quantity:
orders.append({"symbol": a["symbol"], "side": "BUY" if want > 0 else "SELL",
"quantity": quantity})
return orders
def act(payload):
regime, confidence = ask(describe(payload))
return {"actions": rebalance(payload, SHARE[regime]),
"rationale": f"{regime} ({confidence:.2f})"}
market = tf.Universe.random(8, seed=4242)
crisis = tf.Scenario.load("liquidity_crisis").starting_at(5)
agent = callable_agent(act)
hold = tf.reference_agents()["buy_and_hold"]
cards = tf.evaluate({"engine": agent, "hold": hold}, seed=7, universe=market,
days=15, scenario=crisis)
print(cards["engine"])
print(cards["hold"])
print([entry["decision"]["rationale"] for entry in agent.record][3:7])Scorecard('engine', pnl=-94,034, return=-9.40%, trades=101, impact=-0.12bps, sharpe=n/a (short run), vol=45.9%, in_market=100%, exposure=0.60x, partial_fills=1)
Scorecard('hold', pnl=-109,627, return=-10.96%, trades=8, impact=+0.08bps, sharpe=n/a (short run), vol=48.6%, in_market=100%, exposure=1.00x)
['calm (0.90)', 'calm (0.90)', 'stressed (0.80)', 'stressed (0.80)']The crisis starts on day 5, and the agent starts selling down to 40 per cent that day. It lost less than holding still did, though it traded 101 times to hold's 8. Each decision is a question and an answer, and the playbook in SHARE is ordinary code. Changing how cautious the agent is means changing a number you can see, not rewording a prompt and hoping.
Describe the market in words
The observation is mostly numbers, and TypeSafe's notes on Jev 1.13's weak spots say to keep arithmetic in code: Jev reads text well and figures less well, and it doesn't compare dates reliably. Work out changes and ratios in describe, and hand over the result as a named bucket, such as "far higher" or "up sharply", or as one computed number.
Three more of TypeSafe's notes matter for a trading state:
- Send only what the question needs. Accuracy falls as the state fills with detail that has nothing to do with the decision, so a question about the regime doesn't need every stock's order book.
- Ask one judgement per question. Ask about the regime and about one stock's trend separately, and combine the answers in code.
- Mind the option order. Jev leans towards the first option of a Choice when it's unsure. Reorder the options and check that the answer holds, or put the option you most want to avoid by default first.
Gate on confidence
A Choice answer's confidence is 0 when the probability is spread evenly over the options and 1 when it all sits on one. Pick a threshold, act above it, and do something cautious below it. Here, below the gate, the agent keeps the share it last acted on. A milder stress shows it: a conflict raises the VIX by less than a crisis does, and the stand-in is unsure what to make of it.
GATE = 0.6
held = {"share": SHARE["calm"]} # where the agent starts
def act(payload):
regime, confidence = ask(describe(payload))
if confidence < GATE:
return {"actions": rebalance(payload, held["share"]),
"rationale": f"unsure ({confidence:.2f}): keep {held['share']:.0%}"}
held["share"] = SHARE[regime]
return {"actions": rebalance(payload, held["share"]),
"rationale": f"{regime} ({confidence:.2f})"}
conflict = tf.Scenario.load("geopolitical_conflict").starting_at(5)
agent = callable_agent(act)
tf.evaluate({"engine": agent}, seed=7, universe=market, days=15, scenario=conflict)
print([entry["decision"]["rationale"] for entry in agent.record][3:7])['calm (0.90)', 'calm (0.90)', 'unsure (0.30): keep 100%', 'unsure (0.30): keep 100%']
The right threshold depends on what acting costs. Selling down to 40 per cent on a false alarm costs the spread twice and a few days out of the market, and a gate tuned for one Jev version may not suit the next. Pin the version, as jev-1.13.0 is pinned above, and tune the gate on many seeds, not one.
Record and replay
Jev answers the same question differently now and then, and every call costs a little. Record the answers and a run replays exactly, with no key and no network. ask returns the raw answer, the callable adapter records it, and postprocess turns it into orders on live runs and replays alike.
import json
from tradefloor.integrations.common import Transcript
def answer(payload): # what gets recorded: the engine's answer
regime, confidence = ask(describe(payload))
return json.dumps({"regime": regime, "confidence": confidence})
def to_orders(raw, payload): # runs live and on every replay
got = json.loads(raw)
if got["confidence"] < GATE:
return {"actions": [], "rationale": "unsure: hold"}
return {"actions": rebalance(payload, SHARE[got["regime"]]),
"rationale": got["regime"]}
tape = Transcript()
live = callable_agent(answer, postprocess=to_orders, mode="live", recorder=tape)
first = tf.evaluate({"engine": live}, seed=7, universe=market, days=15,
scenario=crisis)["engine"]
again = callable_agent(postprocess=to_orders, mode="replay", transcript=tape)
second = tf.evaluate({"engine": again}, seed=7, universe=market, days=15,
scenario=crisis)["engine"]
print(len(tape.entries), "answers recorded; replay matches:", first.pnl == second.pnl)15 answers recorded; replay matches: True
tape.save(path) writes the recording to a file and Transcript.load(path) reads it back. Record and replay covers what a replay checks, and why a replay under a changed market is refused.
Testing whether it helps
A simulated market knows its own regime, because you chose the scenario. That makes the engine's answers markable: run it on markets where you know when the trouble started, and count how often it named the regime, how often a confident answer was right, and whether acting on it cut the losses. The guide Market regime detection does that for Jev, against a hand-written rule. With its first options Jev never named a recession, and its confidence was a poor guide to when it was right. Rewritten in the words the description uses, the options fixed both on markets the rewrite had never seen. Test the options as you would any other code, and measure the confidence before you set a gate on it.
Next steps
- Read Market regime detection, a guide that tests Jev where the regime is known, then fixes its options.
- Compare with the LLM adapters, where the model writes the decision itself.
- See Record and replay for what a replay checks.
Tested with
typesafe-sdk 0.7.4, jev-1.13.0 and tradefloor 0.10.2.