Rock, Paper, and Scissors in Strategy Games
A simple rock, paper, scissors (RPS) system of direct counters is a perfectly solid and legitimate basis for a strategy game provided that the rock, paper, and scissors offer unequal risk/rewards.
Consider a strictly equal game of RPS. We'll play 10 rounds of the game, with a $1 bet on each round. Which move should you choose? It makes absolutely no difference whether you choose rock, paper, or scissors. You'll be playing a pure guess. Since your move will be a pure guess, I can't incorporate your expected move into my strategy, partly because I have no basis to expect you to play one move or another, and partly because I really can't have any strategy to begin with.
Now consider the same game of RPS with unequal payoffs. If you win with rock, you win $10. If you win with scissors, you win $3. If you win with paper, you win $1. Which move do you play? You clearly want to play rock, since it has the highest payoff. I know you want to play rock. You know I know you know, and so on. Playing rock is such an obvious thing to do, you must realize I'll counter it ever time. But I can't counter it (with paper) EVERY time, since then you could play scissors at will for a free $3. In fact, playing scissors is pretty darn sneaky. It counters paper---the weakest move. Why would you expect me to do the weakest move? Are you expecting me to play paper just to counter your powerful rock? Why wouldn't I just play rock myself and risk the tie? You're expecting me to be sneaky by playing paper, and you're being doubly sneaky by countering with scissors. What you don't realize is that I was triply sneaky and I played the original obvious move of rock to beat you.
That may have all sounded like double-talk, but it's game theory (in the mathematical sense) in action. And it had quite a curious property: playing rock was both the naive, obvious choice AND the triply sneaky choice. For much more on that concept, read my article on Yomi Layer 3.
Fighting games rely heavily on RPS. They have both overall games of RPS going on as well as many rapid fire situations of RPS. Virtua Fighter 3 games can even have 5 sets of RPS take place in a period of 2 seconds! No joke!
Virtua Fighter's overall system of RPS is as follows: attacking beats throwing, throwing beats blocking or reversing, and blocking and reversing beats attacking.
To be clear, let's define terms.
An attack is a move that deals damage. An attack has an initial startup phase where it can't yet do damage (a punch extending), a short phase where it actually can do damage (the sweet spot of the punch), and a recovery phase (the arm retracts). If the defender is blocking correctly, an attack will not damage him, but he can be thrown.
A throw is a special type of move that instantly grabs an opponent whether he's blocking or not and does damage. The catch is, a throw will not grab an opponent who attacking (specifically, a throw will fail if the opponent's move is in startup or hitting phase).
 |
|
Wolf performs his Twirl and Hurl (one of the biggest throws in Virtua Fighter 3) on Shun. |
A reversal is a special type of move that grabs an incoming attack. Reversals usually look like throws, but they work at the exact opposite times. A reversal only works when the opponent's move is in startup or hitting phase, which are, incidentally, the only times a throw would fail.
Even these explanations are simplified, but the RPS system is basically there. Attack the opponent. If they tried to throw you, you'll hit them. If they block or reverse your attack, they nullified your attack. If you expect them to block, you can throw. If they expect you to throw, they can attack.
Dead or Alive 2 basically uses this same system, except that the risk/reward for doing a reversal is much different. Reversals are difficult and relatively rare in Virtua Fighter, but they're incredibly easy and do a ridiculous amount of damage in DOA2. Reversals are so effective, in fact, that they can paralyze the enemy into not attacking for fear of being reversed. Of course, that's when you throw them....
 |
| Dead Or Alive 2's system of attacks, throws, and reversals is not pictured here. |
Real-time Strategy games are the other kings of the RPS system. Like fighting games there's the concept of RPS on large scale and a small scale. On the small scale, particular units are designed to counter each other in a RPS way. A marine dies to a guardian. A guardian dies to a corsair. A corsair dies to a marine. Abstractly, there are 6 categories of unit. Ground units can either attack 1) other ground units, 2) air units, or 3) both. Air units can attack 4) other air units, 5) ground units, or 6) both. Pure ground-to-ground units usually beat both other types of ground units, yet lose to both types of air units that can attack ground. Similarly, pure air-to-air units usually beat both other types of air units, but loose to both types of ground units than can attack air.
RPS is not limited purely to units countering each other though. Real-time strategy games also have the concept of trading off powerful units now for a strong economy now, which leads to even more powerful units later. So on one extreme, a Zerg player in Starcraft might sacrifice his entire economy to get a quick attack force ("6 pool" is the term). This will likely beat a player who chose the other extreme of playing for pure economy and no immediate attack force (by building double oven triple hatcheries). A moderate build (pool on 9th peon, one sunken colony) will likely defend against the early attacker's rush, though. Surviving the rush, the moderate build will have a much superior economy and win in the end. However, this moderate build will produce an inferior economy to the player who built 2 or 3 hatcheries and went for pure economy.
This is all very textbook and a number of other factors come into play in practice, but the underlying RPS is there, and it most certainly has unequal payoffs. In Starcraft, the early rush is a very, very risky strategy. It's all or nothing. You'll either win right away off it, or your rush will fail and you'll almost surely lose. Because of this, the early rush isn't all that common (depending on the map), but the very threat that the opponent might play the early rush is enough to stop you from playing for pure economy every time.
RPS Gone Horribly Wrong: Killer Instinct 2
Killer Instinct 2 boasted a rock, paper, and scissors system of moves. Every character had 3 moves assigned RPS designations. It was all rather arbitrary and artificial. Jago's "rock" move would beat any other character's "scissors" move. Jago's "scissors" move beat any other character's "paper" move. The entire system was so homogeneous, that there was little basis from which to choose rock over scissors. The gameplay was based on blind guessing, and felt hollow and devoid of strategy. RPS needs to be a natural part of the game, and it absolutely has to have unequal risk/rewards for each move.
For those interested in reading way too much about RPS strategy, I'll leave you with this link.
June 20th, 2006 at 5:51 am
[…] In a second Sirlin.net article on Rock-Paper-Scissors based combat systems, there’s an absolute gem of a quote that basically sums up the problem with the card game: […]
June 26th, 2006 at 7:18 pm
[…] It was an article on Sirlin.net that gave me the clue to what was missing (Thanks!). In the first attempt, the matrix underlying the card game was too symmetrical and not at all variable. Each ‘attack’ was its own perfect defense. Each attack (undefended) had precisely the same value as all others. […]
November 27th, 2006 at 3:02 am
“It makes absolutely no difference whether you choose rock, paper, or scissors. You’ll be playing a pure guess. Since your move will be a pure guess, I can’t incorporate your expected move into my strategy, partly because I have no basis to expect you to play one move or another, and partly because I really can’t have any strategy to begin with.”
You are incorrect, I suggest you learn more about RPS. This website may help: http://www.worldrps.com/
November 27th, 2006 at 5:34 am
Re lawson:
If you didn’t notice, Sirlin linked the exact same site at the end of the article, so you’re not really helping. Or pointing towards any direction.
When just playing 10-so matches against a random player, you have little to no information that might be useful, and the little bit of information there might be is the psychological reactions you see during the first matches, as far as I know.
November 27th, 2006 at 8:37 am
Woops, I’m sorry Sirlin. Anyways I think you can link their physical appearance, with their personality/ playing style; also most people play rock first.
November 27th, 2006 at 10:30 am
Speaking of that, to choose wich team goes first at gym for the past months, I’ve ALWAYS, and I mean ALWAYS chose Rock. And I gloated about it.
I’ve lost ONCE to another guy who kept choosing Rock, and after 8-so draw rounds, he knew I was just going to stick with Rock and chose paper. My streak ended ):
April 19th, 2007 at 2:44 am
The payoffs for real-life RPS can be considered unequal for at least three reasons.
First, as lawson says, you can typecast your opponent, and they’re very often aware of the impression they’re creating. Someone with multiple piercings and a t-shirt saying “I love Scissors” for example. You might expect them to throw Scissors, but they know you’re expecting them to do so, and you know they know that…
Second, you don’t get any extra points for playing a particular throw, but since Rock takes less physical effort/dexterity to throw than Scissors, and Scissors take less effort than Paper (assuming you have to throw Paper horizontally), there is a slight temptation for players to throw Rock, which unbalances the game just as unequal points would (admittedly not to the same extent).
Third, in certain in-game situations, there’s an imbalance. E.g. if I just threw two or more Scissors in a row, winning by throwing another Scissors could be considered to have extra (psychological) reward, because the other player is forced to admire my courage in sticking with Scissors yet again. Winning the next throw with Rock or Paper wouldn’t seem as impressive. If you assume that players play at least partly for those kind of psychological payoffs as well as winning the match, then that’s another source of imbalance.
April 28th, 2007 at 4:34 am
[…] Originally Posted by aliengenius To make RPS an actual interesting game all you have to do is assign disparate point values to the wins with different objects. For example, if you win with rock it’s worth 5, with scissors 3, with paper 1. Then race to predetermined number. This is game theory boiled down into its purest form. See this article on how most video fight games are based on Roshambo. […]
February 8th, 2008 at 5:31 am
The existence of different pay offs for winning in rock-paper-scissors with each throw does not create any basis on which to conclude a player is going to play a particular way.
For even RPS, you said that “your move will be a pure guess”, but if you are playing uneven RPS to win, you should also be making complete guessing–but not with equal chances. Consider the chance where winning with rock earns you $10, winning with paper earns you $3, and winning with scissors earns you $1. There exists an optimum mixed strategy for this game, in the sense that if I am using this strategy, you have no way to make more money than me in the long run. In fact, even if I tell you the details of my strategy before we begin, you can’t make more money than me. (Adopting the same strategy will enable you to earn the same amount as me in the long run, not more.)
To find the mixed strategy, I wrote a simple implementation of a genetic algorithm in PHP. I bet this blog is going to take away the indentation, but oh well.
What this program does is first generates 100 uneven RPS players who play rock with some chance, paper with some chance, and scissors with some chance — each player has his own chance of throwing each. Then, the players compete in a round robin tournament. The bottom 3/4 all die. The top 1/4 of the tournament gets to create the next generation — each one producing four offspring. When creating an offspring, there is a 5/11 chance that the offspring will be a mutant: rather than being a clone of his parent, his chances will be +-1% his parent’s, but they’ll still add up to 100% of course (by supplementary addition to the chances). Then, it finds the average chances among the top 1/4 of the just finishing generation and prints it out. Then it repeats. Eventually, evolution by natural selection should cause only the best uneven RPS players to be living.
/**
* Finds the optimal mixed strategy for the given uneven RPS game.
* — Colin Fitzpatrick
*/
// Constant size of community.
define(’INITIAL_POOL’, 100);
// How many offspring each winner produces.
define(’OFFSPRING’, 4);
// Rewards for winning with each token.
$g_reward = array(10, 3, 1);
class player {
var $m_p;
function player($p1 = 0, $p2 = 0, $p3 = 0) {
$this->m_p = array($p1, $p2, $p3);
}
function getChoice() {
$r = rand(0, 99);
if ($r < $this->m_p[0]) {
return 0;
}
if ($r < ($this->m_p[1] + $this->m_p[0])) {
return 1;
}
return 2;
}
function mutate() {
// Alter each gene by plus or minus 1.
for ($i = 0; $i < 3; ++$i) {
$this->m_p[$i] += rand(-1, 1);
if ($this->m_p[$i] < 0) {
$this->m_p[$i] = 0;
} else if ($this->m_p[$i] > 100) {
$this->m_p[$i] = 100;
}
}
$sum = $this->m_p[0] + $this->m_p[1] + $this->m_p[2];
$diff = $sum - 100;
if ($diff == 0) {
return;
}
$d = 1;
if ($diff > 0) {
$d = -1;
}
$diff = abs($diff);
for ($i = 0; $i < $diff; ++$i) {
while (true) {
$r = rand(0, 2);
$x = $this->m_p[$r] + $d;
if (($x < 0) || ($x > 100)) {
continue;
}
$this->m_p[$r] = $x;
break;
}
}
}
function getOffspring() {
$p = new player($this->m_p[0], $this->m_p[1], $this->m_p[2]);
if (rand(0, 10) < 6) {
$p->mutate();
}
return $p;
}
}
function getAverage($s) {
$v = array(0, 0, 0);
$l = count($s);
for ($i = 0; $i < $l; ++$i) {
for ($j = 0; $j < 3; ++$j) {
$v[$j] += $s[$i]->m_p[$j];
}
}
return array($v[0] / $l, $v[1] / $l, $v[2] / $l);
}
function playMatch($o) {
$s = -1;
$w = -1;
while (true) {
$t = array($o[0]->getChoice(), $o[1]->getChoice());
if ($t[0] == $t[1]) {
continue;
}
$s = 1;
if ((($t[0] == 0) && ($t[1] == 2))
|| (($t[0] == 1) && ($t[1] == 0))
|| (($t[0] == 2) && ($t[1] == 1))) {
$s = 0;
}
$w = $t[$s];
break;
}
return array($s, $w);
}
$g_player = array();
// Start off with 100 randomly generated RPS players.
for ($i = 0; $i < INITIAL_POOL; ++$i) {
$a = rand(0, 10000);
$b = rand(0, 10000);
$c = rand(0, 10000);
$t = $a + $b + $c;
$a = round($a / $t * 100);
$b = round($b / $t * 100);
$c = round($c / $t * 100);
$g_player[] = new player($a, $b, $c);
}
$generation = 0;
while (true) {
++$generation;
shuffle($g_player);
$pt = array();
$l = count($g_player);
for ($i = 0; $i < $l; ++$i) {
$pt[$i] = 0;
}
for ($i = 0; $i < $l; ++$i) {
for ($j = $i + 1; $j < $l; ++$j) {
$x = playMatch(array($g_player[$i], $g_player[$j]));
$pt[$x[0] ? $j : $i] += $g_reward[$x[1]];
}
}
arsort($pt);
$k = array_keys($pt);
// The top players get to reproduce.
$next = array();
$arr = array();
for ($i = 0; $i < $l / OFFSPRING; ++$i) {
$p = $g_player[$k[$i]];
$arr[] = $p;
for ($j = 0; $j < OFFSPRING; ++$j) {
$next[] = $p->getOffspring();
}
}
$av = getAverage($arr);
$g_player = $next;
echo “Generation $generation: (${av[0]}, ${av[1]}, ${av[2]}).\n”;
}
Anyway, if there were no stable strategy, we would expect constant fluctuation. This is not what we find.
Due to practical concerns (I didn’t want this to take long to find an equilibrium), It has a hard time finding a precise equilibrium, but it’s clear that if you want to make the most money possible, the optimum strategy is to play rock ~9% of the time, paper ~87% of the time, and scissors ~4% of the time, selected randomly.
So, uneven RPS is not anymore tactical than even RPS. Talk of “Yomi” is completely irrelevant because even if you know my optimum mixed strategy, you cannot do better than it.
February 9th, 2008 at 2:56 am
Sorry about posting twice - I don’t think you can edit.
The previous post’s result was a bit off. The exact equilibrium is ~7% rock, ~71% paper, ~21% scissors.
February 9th, 2008 at 3:50 am
Colin: You take mysteriously long to make your point. Mathematically, is there an optimum mixed strategy in the RPS described? Yes. A solution “by inspection” (no computer program needed) is to play rock 10/14th of the time, scissors 3/14th, and paper 1/14th (I think you mixed up which was which in your program, a typo perhaps).
People have trouble being random in an-equal odds game of RPS. I think you will find human players are very far from random in the 10-3-1 odds version, and that is why it’s possible to exploit their personalities.
It would be nice if I could link what is basically part 2 of this article now (I wrote the new one a year ago), but I’m holding off to publish it along with new information about my Yomi card game.
–Sirlin
February 9th, 2008 at 7:49 am
He reversed scissors and paper as compared to your example, Sirlin, which probably makes a huge difference, cause that makes the counter to the strongest option be the second best, rather than the weakest. However, he’s otherwise right, it’s rock 1/14th of the time as optimal in his version. (The program was still overkill… it’s actually trivial to see the odds when you simply imagine playing 100 games, where you’re opponent chooses rock 100 times in a row, just find the maximum score where you’re both still tied at the end)
You’re making two mistakes, though, Colin. The first is, as Sirlin said, humans can’t even properly randomize given the equal chances in normal RPS. It’s like trying to analyze how to play poker while completely ignoring how the best players spend as much time, if not more, reading each other, than they do counting cards.
The second mistake you’re making, is that RPS is being used as an example to illustrate teh design philosophy of more complicated games. It’s easy to see how, relying on luck, a completely non optimal strategy (such as constantly playing rock), will on average tie with your optimal strategy. However, over short time periods, it’s very possible for “non-optimal” strategies to win, and many human players will purposely choose greater risk/reward over a limited period of time. Someone who properly takes advantage of countering strategy changes like that (i.e. yomi), should always come out ahead of the completely optimal strategy you described. Yomi becomes even more powerful in a more complicated system (such as a fighting game), where sudden changes in strategies are extremely visible, rather than RPS, where it can take many turns to identify the difference between a “new strategy” and just random play.
February 9th, 2008 at 8:00 am
Oops… now I really wish I had an edit button, my description of how to find the optimal strategy mathematically may be the stupidest thing I’ve ever posted here;; However, I think if you consider three opponents, one who plays paper 100 times, one playing rock 100 times, one playing scissors 100 times, and try to maximize a tie score against all three players, using the same series of moves, then you will actually find the right ratios. Guess it’s a little more complicated that way. (Also, you may need more than 100 games, since the real values have decimal points… the integers you posted end with an all rock player losing 213 to 210)
February 9th, 2008 at 2:58 pm
Heh, yeah, the genetic solution is totally over the top.
However, if all we can do is analyse the opponent’s patterns then I would think even RPS is just as strategic as uneven RPS. It seems at a first glance that there would be no reason to pick one throw over the other. But either there *is* such a reason, or people can generate random numbers.
It seems that the main difference with uneven RPS is not the existence of a basis on which to make your move, but a more transparent basis. If somebody is “greedy” that is pretty apparent character attribute, whereas in even RPS you would have to look for something a lot more subtle, not directly related to the payoff of the game. But people must still be making their decisions on some grounds in even RPS (or else they are generating them randomly, which we’ve assumed they can’t do), grounds which a player with experienced “Yomi” can surely access.
Perhaps the lesson here is that a game should be designed so that choices presented to the player give them a chance to show off more obvious aspects of their personality (e.g. greediness) rather than more subtle ones (e.g. whatever is causing people to pick what they pick in even RPS).
Another interesting question would be whether people tend to deviate more from the equilibrium in uneven RPS than in even RPS. It’s pretty obvious which move is the best in uneven RPS, so play might actually come fairly close to the equilibrium–maybe even closer than in even RPS, since your most common choice in the uneven RPS game has a high optimum probability compared to the low optimum probabilities in even RPS (which are probably harder to simulate without a random number generator). This is an empirical question though. Perhaps some research has already been done on it.
By the way, Sirlin, I think this is a pretty great web site with a lot of intriguing game design topics. This is a pretty random comment but I may have come off hostile above and that was not my intent.
February 29th, 2008 at 3:31 am
to Colin Fitzpatrick:
Yomi is if I know you are going to play 7% rock, 71% paper, and 21% scissors, ( because I figured out you like to play the “optimal” versus random) I will simply play 100% scissors.
Out of 100 hands, I will make 213 and you will make 70. Of course, you will catch on that I am playing 100% scissors and so start to play more rock and deviate from your optimal strategy. But perhaps I already guessed you would play rock more frequently so I threw in 10 papers in a row to throw you off. etc, etc. Point is, yomi still applies because you aren’t playing a random generator.
Playing 100% paper, beats Sirlin’s R:10/14, S:3/14, P:1/14 in the long run by a small margin also. Though, throwing in a couple more scissors is enough negate that once my 100% paper is revealed. I can adjust but to do that I have to guess what strategy and level he is thinking on so I can counter. i.e. yomi.
February 29th, 2008 at 3:33 am
Yomi definitely still applies to RPS.
If I know Colin is going to play 7% rock, 71% paper, and 21% scissors, ( because I figured out he likes to play the “optimal” numbers) I will simply play 100% scissors.
Out of 100 hands, I will make 213 and you will make 70. Of course, you will catch on that I am playing 100% scissors and so start to play more rock and deviate from your optimal strategy. But perhaps I already guessed you would play rock more frequently so I threw in 10 papers in a row to throw you off. etc, etc. Point is, yomi still applies because you aren’t playing a random generator.
Playing 100% paper, beats Sirlin’s R:10/14, S:3/14, P:1/14 in the long run by a small margin also. Though, throwing in a couple more scissors is enough negate that once my 100% paper is revealed. I can adjust but to do that I have to guess what strategy and level he is thinking on so I can counter. i.e. yomi.
February 29th, 2008 at 6:04 am
Mike,
In Colin’s example, winning with scissors is only worth $1, so you would only make $71 vs. his $70 (difference being a result of decimal rounding, not a superior strategy).
March 4th, 2008 at 12:47 am
I see now. Thanks for pointing that out. I humbly retract my double posts.
March 4th, 2008 at 5:29 pm
The interesting thing about variable payoffs is when the payoffs aren’t easy to determine and change with the situation.
Static known variable payoffs are no more interesting than equal payoffs. The ability of people to properly randomize is not in itself a very exciting thing to test IMO.
April 1st, 2008 at 5:41 pm
Perhaps, rather than being an optimal strategy, it is just a strategy that is equally matched with any other strategy?
April 2nd, 2008 at 4:35 am
“Perhaps, rather than being an optimal strategy, it is just a strategy that is equally matched with any other strategy?”
According to the game theoretic definition of optimal, in any strict RPS variant the optimal strategy will be breakeven against any other strategy.
Also, random aside related to other points, there are top poker players who randomize decisions according to secret factors like the seconds number on their wristwatches.
May 18th, 2008 at 5:18 pm
[…] Sirlin - RPS […]
July 30th, 2008 at 4:56 pm
Well if both players think about which to choose or if they read your article there is no use in a strategy of choosing. Because it is an endless thought: rock brings most, so I choose paper since he will use it, but he might take scissor to counter it, I need rock to beat that, but than he uses paper and I need scissors, he will choose rock then, so I… and so on. It will be a endless mindknot which will never end so Its down to guessing and randomizing again.