Yomi Layer 3: Knowing the Mind of the Opponent
Sirlin
This is not really how Yomi works.Yomi is the Japanese word reading, as in reading the mind of the opponent. If you can condition your enemy to act in a certain way, you can then use his own instincts against him (a concept from the martial art of Judo). Paramount in the design of competitive games is the guarantee to the player that if he knows what his enemy will do, there is some way to counter it.
What happens, though, when your enemy knows that you know what he will do? He needs a way to counter you. He's said to be on another level than you, or another "Yomi Layer." You knew what he would do (yomi), but he knew that you knew (Yomi Layer 2). What happens when you know that he knows that you know what he will do (Yomi Layer 3)? You'll need a way to counter his counter. And what happens when he knows that you know....
Sound like a joke that could never happen in real gameplay of an actual game? Surprise: it's quite common in strategy games. The reason has to do with conditioning the opponent and the inequality of risk/reward in these guessing games (see my article on Rock, Paper, and Scissors in Strategy Games).
Before we get into how ordinary human minds can become entangled in complicated guessing games, let's look at what needs to be there to create these guessing games at all. The designer's tendency might be to create moves and counters. Then create counters to counters, then counters to counters to counters, then counters to those, and so on. Actually, a game need only support counters up to Yomi Layer 3, since Yomi Layer 4 can loop around back to Yomi Layer 0.
Let's say I have a move (we'll call it "m") that's really, really good. I want to do it all the time. (Here's where the inequality of risk/reward comes in. If all my moves are equally good, this whole thing falls apart.) The "level 0" case here is discovering how good that move is and doing it all the time. Then, you will catch on and know that I'm likely to do that move a lot (Yomi Layer 1), so you'll need a counter move (we'll call it "c1"). You've stopped me from doing m. You've shut me down. I need a way to stop you from doing c1. I need a counter to your counter, or "c2."
Now you don't know what to expect from me anymore. I might do m, or I might do c2. Interestingly, I probably want to do m, but I just do c2 to scare you into not doing c1 anymore. Then I can sneak in more m.
You don't have adequate choices yet. I can alternate between m and c2, but all you have is c1. You need a counter to c2, which we'll call c3. Now we each have two moves.
Me: m, c2
You: c1, c3.
Now I need a counter to c3. The tendency might be to create a c4 move, but it's not necessary. The move m can serve as my c4. Basically, if you expect me to do my counter to your counter (rather than my original good move m), then I don't need a counter that; I can just do go ahead and do the original move...if the game is designed that way. Basically, supporting moves up Yomi Layer 3 is the minimum set of counters needed have a complete set of options, assuming Yomi Layer 4 wraps around back to Layer 0.
This is surely sounding much more confusing than it is, so let's look at an actual example from Virtua Fighter 3 (which will almost certainly confuse you even more).
Example of Yomi Layer 3 from Virtua Fighter 3
Let's say Akira knocks down Pai. As Pai gets up, she can either do a rising attack (these attacks have the absolute highest priority in the game) or she can do nothing. A high rising attack will stop any attack that Akira does as she gets up, but if Akira expects this, he can block and retaliate with a guaranteed throw. Pai does the rising kick and Akira predicts this and blocks. Now the guessing game begins.
Akira would like to do his most damaging throw (that's his m), and be done with it. Even though the throw is guaranteed here, all throws can be escaped for zero damage if the defender expects the throw and enters the throw reverse command. The throw is guaranteed to start but Pai might reverse it. In fact, Pai is well aware that a throw is guaranteed here (it's common knowledge), and it's only obvious that Akria will do his most damaging throw. After all, this situation has happened a hundred times before against a hundred Akiras and they all do the same thing. It's really conditioning, not strategy, that tells Pai she needs to do a throw escape here (that's her c1). In fact, it will become her natural, unthinking reaction after a while.
Akira is tired of having his throw escaped again and again. He decides to be tricky by doing one of his very powerful moves such as a double palm, a reverse body check, a two fisted strike, or a shoulder ram (we'll just lump all those into c2). Why does a big, slow move work in this situation? First of all, if Pai does her throw escape and there is no throw to escape, the escape becomes a throw attempt. If her opponent is out of range or otherwise unthrowable for some reason, her throw attempt becomes a throw whiff. She grabs the air and is vulnerable for a moment. One important rule in VF is that you cannot throw an opponent during the startup phase or the hitting phase of a move. So if Akira does a big, powerful move, he is totally unthrowable until after the hitting phase of the move is over and he enters recovery (retracting his arm or leg).
Years after this article was written, a newer version of Akira still wants to do a throw ("m"), but instead opts for Double Palm ("c2").
Back to our story. Akira is tired of getting his throw escaped all day, so he does standard counter to any throw: a big, powerful move. This c2 move does a decent amount of damage, by the way. The next time this whole situation arises, Pai doesn't know what to do. Her instincts tell her to reverse the throw, but if she does, she is vulverable to Akira's powerful move. Rather than go for the standard reverse, Pai does her c3 move: she simply blocks. By blocking, she'll take no damage from the Akira's powerful move, and depending on exactly which move it was, she'll probably be able to retaliate.
So what does Akira do if he expects this? In fact, he needs no c4 move since his original throw (m) is the natural counter to a blocking opponent. A throw is a special kind of move that grabs an enemy and does damage regardless of whether they are blocking. It's specifically designed to be used against an opponent in block who is afraid of an attack.
In summary,
Akira has: throw; powerful move.
Pai has: throw escape; block.
As I tried to show, it's actually pretty reasonable to expect players to be thinking on Yomi Layer 3, 4 or even higher. It's because conditioning makes doing the throw escape an unthinking, natural reaction. But against a clever opponent, you'll have to think twice about doing a standard throw escape, or blocking. The Akira player will do the occasional powerful, slow move just to put his enemy off balance and abandon his instinct to escape the throw. Then Akira can go back to his original goal: land the throw.
Another very interesting property is "beginner's luck." Notice that a beginner Akira in this situation will go for the throw, since that works on other beginners who haven't learned to throw escape. The beginner Akira will never land the throw on an intermediate player, though, since the intermediate player knows to always throw escape. But strangely, the beginner will sometimes land the throw on the expert, since the expert is aware of the whole guessing game and might block rather than throw escape. Of course, the expert will soon learn that beginner is, in fact, a beginner and then he'll be able to yomi almost every move.
Just as a final note on Virtua Fighter to further demonstrate the complexity of its guessing games, I actually greatly simplified the example above. I left out, for example, that Pai could attack with a fast move rather than block. And Akira has another c2 move besides a slow, powerful move. He can also do what's called a "kick-guard cancel" or "kg." This means he can press kick, which will make him unthrowable until his kick reaches recovery phase. If Pai tries to throw, she'll whiff. But then Akira can cancel the kick before it even gets to the hitting phase. Now he's free to act and take advantage of Pai's whiffed throw vulnerability. Now, Akira has a guaranteed throw, putting him back in the exact same situation he began in.
This paper clip guy always seems to know exactly what people are thinking.
The catch is that if Akira does kg-cancel and then goes for the throw he originally wanted to do, Pai will probably not have time to react with a throw escape. It's just too fast. She'd have to be on the next yomi layer. She'd have to expect Akira to throw, enter a throw escape, see the kg-cancel, then immediately enter her next guess (probably an attack or throw escape). Any hesitation and she'd be thrown.
Suddenly forcing the opponent to think on the next yomi layer than were expecting is usually a way to blind them. Even if the opponent was ready for the guessing game I described above, if you do the "kg" trick and suddenly force a new guessing game they weren't ready for, they are likely to do an automatic response that's easy to predict becase they won't have the time or presence of mind to do anything else.
The point I'm making here is that despite Virtua Fighter's absurd complexity, players really are able to think on the levels I'm hinting at. Playing such a game and successfully landing a move because you knew he knew you knew he would do a particular move is the greatest feeling in the world. So design counters and counters-to-counters, and so on, but know that making Yomi Layer 4 the same as Layer 0 allows you to only design counters up to Yomi Layer 3.
Reader Comments (11)
Hi Sirlin. That was a nice read. I was wondering if any fighting game AI has been built upon this method of thinking? Not directly reacting to the game inputs of the player, but coming up with a set of solutions based on its situation and doing it's own guessing.
One solution computers can implement in a Yomi situation is the Nash equilibrium. This uses random numbers to make the computer's mind 100% impossible to read. For example, in rock-paper-scissors, the computer generates a random number - 1, 2, or 3.
1 ---> Rock
2 ---> Paper
3 ---> Scissors
What should the player do? It doesn't matter.
-----------------------------------------------------------
Or, you can try to Yomi them. The problem with computer-Yomi is it's always at one level. For example, the computer could make the rps-society style guess that the player will prefer:
Beginning ----> Rock
Winning -----> Scissors
Losing -----> Paper
Then it will play
Beginning -----> Paper
Losing ------> Rock
Winning ----> Scissors
And will be destroyed by
Begging -----> Scissors
Winning ----> Paper
Losing ------> Rock
--------------------------------
Computer Yomi is too easy to outsmart.
This is assuming that the computer makes the guess of the player playing a certain move through statistical data gathered from its play experiences thus far, correct? What is not to say that the computer can gather further data from its failures of a Yomi Layer 1 move to create a Yomi Layer 3 decision?
Because, there's still a code in their. Players can figure out what that is and predict it with 100% accuracy.
I'm not entirely convinced. I'm going to diverge away from RPS since my original question was about AI in fighting games and I don't think RPS is a good enough context. I will be using the notations from the article: m, c1, c2, c3.
Given the lifetime experiences of that particular computer AI, it will eventually come up with one set of Yomi Layer 1 decisions and one set of Yomi Layer 3 decisions for every situation. Using that information, if a player has a tendency to throw a lot of c2 moves in a given situation; let's say m (30%), c2 (70%), then the computer will use c1 (30%), c3 (70%) in those situations. Not entirely predictable, but the player "could" favor that statistic to his odds until it evens out to 50%, 50% in those situations.
And let's assume the RNG is good enough to meet our needs since we're in a real-time situation.
Play this:
m c2 m c2 m c2 m c2 m c2
Performance each round, scale of 0 to 1:
1/2
1
1/2
2/3
1/2
3/5
1/2
4/7
1/2
5/9
You just did better then 1/2. The computer could have done better by playing randomly.
If utilizing yomi is essentially guessing what your opponent is guessing you're guessing and so on then as far as AI goes you'd need to keep track of how often the player uses M, C2, C2 in response to C1, and M in response to C3 in each "yomi series". You can use these numbers to help create a probability of which response they player is likely to give. Of course the player can use those numbers to trick the AI into setting itself up to be exposed to an attack (which can happen even with a human opponent) so you can throw in a wild card, maybe a 25% possibility, that will cause the AI to use a response contradicting the probability numbers (i.e. there is a 25% chance it will choose C3 instead of C2 as its response, for example). Now the player has the threat of the AI "guessing" (randomly) that the player knows that it knows the player's probable actions and the AI might counter the player's counter-counter.
yomi could lead to some unnusual paradoxes.
quote: JWK5
If utilizing yomi is essentially guessing what your opponent is guessing you're guessing
it is not, it's deciding on what your opponent is going to do, based on logical reasoning, not guessing ^^,
Your statement about how players must guess which Yomi layer their opponent is acting on reminds me of a term used in the game Epic Mafia. Players created the term WIFOM, or "Wine in Front of Me" to describe infinite loops of reverse psychology, where a player may be telling the truth, or may be lying to appear to be telling the truth, or trying to appear as if they're lying so that people will then believe they're telling telling the truth, etc. Not sure if this is really relevant, but I found the similarity interesting.
Good to put a concrete name to the concept; thanks for sharing. Fencing has precisely this layered concept, codified in the "Tactical Wheel".
Attacks and their counters increase in complexity around the wheel until the final counter is a simple direct attack. In high level matches the tactics can whirl clockwise around the tactical wheel as each fencer tries to stay a step ahead of the their opponent's plan. Heady stuff.