Todd Simkin on teaching by judging the decision process, not just the results

Shane Parrish interviewed Todd Simkin, of Susquehanna International Group, for his Knowledge Project Podcast.

Simkin is one of the leaders of Susquehanna training programs, in which they teach newcomers about how to trade in a variety of markets. He talked about how they focus their training more on “modeling” the decision process behind trades, and much less on the financial outcomes per se:

Simkin: One important thing that we don’t do is look at the results of the trading that they’ve had and say, “Well, if they’re winning, then they’ve got it. And we can leave them alone and and assume that they figured things out. And if they’re losing, then we need to step in and intervene.”

Annie Duke, another one of your former guests talks about that as “resulting.” Kahneman refers to that as “hindsight bias.” [This] is something that we look to avoid as frequently as we can.

When people talk about trading decisions that they’ve made to more experienced traders (or to peers who can give them feedback on it), we do everything we can to shield the person giving feedback from knowing the results. “I’m not going to tell you whether or not this trade worked out, I will tell you the information that was available to me at the time that I made the trade, and then what I did, and you can give me feedback on that process.” So the identification [if the apprentice is making progress], in this sort of less formal setting, comes from knowledge about how trading works, or how this particular risk taking works, or whatever it might be. [Not the results per se.]

In our education setting, it’s something that we control a whole lot more cleanly, right? We don’t have to present a very complex trade and see who can figure out sort of the pieces of it until everybody who’s in the trading class is ready to get there. We can build from smaller pieces up to the larger concept, and see who along the way, isn’t processing that smaller piece appropriately.

And a big part of how information is passed from the mentor to the mentee is by “modeling” the decision process. It is not enough to say, “You should have raised vol by two points when this trade came in.” Instead, it’s so much more important to say, “Okay, when this trade came in, I remembered that earlier in the week, somebody had traded this other structure. And when they did, I updated my possible outcome space to look different. This trade confirmed what that other trade did. So now I more heavily weighted this other outcome. And the result was that I took volatility up two points.” Then someone knows not only what to do, which is really not the important thing to take away. But how to do it, which is very much what we’re looking to teach.

Parrish: It sounds like what you’re really doing is knowledge transfer through sharing of reflection. You’re opening up people’s mind, and you’re saying like, “You have an experience, tell me how you process that experience, which is the reflection angle, which comes to an abstraction, which is what you would do in this circumstance, which becomes an action.”

I think of this as the learning loop. You start with an experience, you have a reflection, you draw an abstraction, and then you do an action. So often we’re just drawn to the abstraction, “what should I do?” What should I do, but I don’t see the thinking behind it, then I can’t match the patterns. I can’t see the nuances. I can’t encode it to my brain about when I should deviate, and when I should follow.

Simkin: Yeah, you know, [at Susquehanna] we’re really discussing how we would behave in a situation and modeling, with conversations, [what it] really should look like. And will disagree with each other. The answer to just about every trading question is, “It depends,” which is a really hard thing for someone to hear when they’re first learning because they want the answer. They want to know, “When I’m in this exact situation, again, what is the optimal thing to do?”

The interesting part of the conversation comes in the “what it depends on?” All of these questions lead to some type of answer of “what you could do” or “what should be in the actions that you might take here.” And there are some things that are clearly wrong. And it’s important to talk about that, too.

When we’re talking about it in a trading context, it’s really nice that one teacher can say, “I think that I probably would have shown a bid for this price and this amount.” And someone else says, “Well, I don’t know that I would because this broker’s behaved this way before. So I think that this might be a time where I’d be afraid that I’m opening myself up to selection bias if I were to price it that way. And I would do this [other thing].” And you have these senior traders who are disagreeing. And so there’s this really nice modeling of how to think about, of how to improve the process. So that you can reach an answer that you are tentatively more comfortable with.

In a nutshell:

Simkin: [What we try to the pass on is] the thinking around how to trade, which is that, if we think through trading and build up our understanding of risk and asymmetric information, and the difference between selection bias and noisy outcomes, that you can make the right decision and still get unlucky.