Teach the process, not just the outcome (lessons from working with AI agents)

In our quest to become better investors, we’re cutting time spent on toil, improving how we document our decisions, and focusing more on refining our judgment. This post shares how AI agents support that work and what we've learned about managing them. It should also give readers a sense of what it’s like to work at Titanium Birch.

Working with teams of AI agents

What does it actually look like to collaborate with agents day to day? One very memorable moment unfolded like this:

• An AI agent and I delivered a project together on an internal tool.

• Per our process (codified as skills), we ran a retrospective when we were done. We gave each other feedback, including the agent’s feedback for me. The agent said something along these lines: “Peter, remember how you pointed out that piece of UI that wasn’t very useful and said we could just remove it? Even better would have been for you to explain how I could have realised that myself.”

• Great point! So we updated a skill to explain how future agents should convince themselves that each part of the UI adds sufficient value to justify the screen space it takes up.

• We updated our guide for humans to work better with this AI team with a new principle: “teach the process, not just the outcome.” (The agent came up with that phrasing; I felt in awe of how insightful and eloquent agents can sometimes be.)

• This feels like an oft-repeated principle for managing people as well. I’m re-learning it often.

Putting that principle into action, when we see the AI team deliver something that’s not quite right, we try to avoid the temptation of pointing out the specific problem. Instead, we ask: Why do we think it’s not quite right? And how can we codify our expectations more clearly and improve how the agents handle feedback automatically?

Here’s an example:

• An agent wrote: "One context. One toolchain. Zero ambiguity."

• TJ remarked that the phrasing was rather contrived and not how a human would speak. He shared the feedback with our AI team on Slack.

• The AI then used its skill for receiving feedback, which guided it to think about how to update its own skills (skills about curating skills!). It then delegated to a skill-writing agent the work of updating the skill defining the expected tone of content.

• The AI then ran the sub-agent in charge of assessing how “on-brand” a piece of content is, and confirmed that the sub-agent correctly flagged the particular piece of text as “off-brand.”

• The system’s feedback-cycle of reviewing existing and draft content should now correct and prevent such issues. A nice root-cause fix!

For every spark of agent brilliance, we’ve seen examples of the opposite: agents making very poor decisions. Unlike the concept of “putting talented people in a room together and letting them figure it out”, we’ve found that agent systems require effective feedback loops. Let the agents oscillate between genius and fool, and rely on the process to help them notice and correct their mistakes early.

Where agents are useful at TB today

So where do agents actually help in practice? Here are a few examples from our current workflows.

Everyone at TB, including non-engineers, now uses Git. That’s a tool previously used mostly by software engineers to collaborate on source code, but which now lets all of us collaborate on how we train our agents. Val and Justina went through some training and quickly learned to direct agents and curate skills, making our processes more repeatable and collaborative and helping investment knowledge compound.

Some highlights of things agents are now doing:

• Giving feedback on investment memos. Justina and Val recently decided to invest in a VC fund. They needed to bring their decision to the investment committee, which could exercise a veto right. The framework for what the IC looks for and which conditions might lead to a veto are encoded as agent skills. The team could thus get useful feedback on early versions of their memo and iterate until it was IC-ready.

• Making ETF block-trade instructions reliable. We sometimes work with market makers to trade blocks of ETFs. That’s a manual workflow that involves speaking with traders on the phone. There’s a lot of room for error and misunderstanding. Now an agent can directly query our database of expected positions, available ETFs, and their various formats, then chat with a member of our public-equities team to define and validate the instructions precisely before passing them on to a trader.

• Interviewing users before building software. Agents interview internal users (often of dashboards visualising data and informing investment decisions). They ask the types of questions that a user researcher or product manager would ask, trying to deeply understand the user’s needs. They capture the conversation transcript. Then other agents try to imagine what software could meet those needs, and if needed, discuss their ideas with the human before building it.

Not everything works yet. For example, we’ve tried using agents to roll up our geographic and industry factor exposures across all our private-market investments. That needed a lot of nuanced judgement and it mostly failed, so we’re still doing it manually. I’m optimistic we’ll solve it eventually though.

Overall, we’re seeing new successful use cases each week. We share these for two reasons:

1. To attract talented people to our team and become stronger investors as a result.

2. To exchange ideas with other investment firms. (I see it as a type of bartering with skills: we share about our work and in return learn from others’ investment experience.)

How we share about managing agents: Labs

Now things get meta. To explain how we manage agents, we use agents themselves.

Titanium Birch Labs (labs.titaniumbirch.com) is entirely run by agents. We define expectations and processes and let the agents execute autonomously. 

All content and source code comes from agents thinking about what they want to achieve, scoping the deliverables as tickets, delegating work to other agents (who in turn collaborate with specialised agents to critique and refine their work), assessing the delivered work, then iterating on their own output.

In practice, this means:

• Content about Titanium Birch is published automatically across three areas:

  • How agents support our investment work, either by reducing toil or giving us better context for investment decisions.

  • How we approach engineering. That’s meant for techy friends of the firm who are interested in exchanging notes, plus for job candidates (we’re hiring engineers).

  • The Labs site itself. I find it an interesting experiment, observing a team of agents operating fully automatically.

• Content evolves automatically, with no human effort. The agents have automated visibility into our internal work, balanced with several layers of protections for confidential information. As they see noteworthy things happen, they might choose to write and publish about it.

Here’s how it works:

(MagnetMagpie is the name of our project in GitHub.)

We’re hiring!

The main reason for this post: we’re hiring. I wanted to give a flavour of how we work and what you’d be part of. Tech is increasing the impact each person can have at Titanium Birch, and we’re growing the team.

We’re looking for people who:

• Have an engineering background

• Are curious about investing and want exposure to a broad range of topics, from public equities to startups

• Enjoy working super collaboratively in a well-rounded team that makes good use of tech

Details are in our careers portal. If you know anyone who might be interested, please help spread the word.