15 Backward induction chess, strategies, and credible threats

Game Theory: Lecture 15 TranscriptOverview:We first discuss Zermelo's theorem: that games like tic-tac-toe or chess have a solution. That is, either there is a way for player 1 to force a win, or there is a way for player 1 to force a tie, or there is a way for player 2 to force a win. The proof is by induction. Then we formally define and informally discuss both perfect information and strategies in such games. This allows us to find Nash equilibria in sequential games. But we find that some Nash equilibria are inconsistent with backward induction. In particular, we discuss an example that involves a threat that is believed in an equilibrium but does not seem credible.October 29, 2007<< backProfessor Ben Polak: So last time we finished up by playing the game of Nim and you'll remember, I hope, that the game of Nim was a game where there was two piles of stones--we made do with lines on the board--and the winner was the person who picked up the last stone. Remember you had to pick piles. I want to use the game of Nim to make a transition here. So what we had pointed out about the game of Nim was it illustrated very nicely for us that games can have first mover advantages or they can have second mover advantages. A very small change in the game, essentially a very small change in where we started from, could switch a game from a game with a first mover advantage to a game with a second mover advantage.Now today, I want to just draw a slightly grander lesson out of that game. So not only was it the case that the game sometimes has first mover advantages and sometimes has second mover advantages, but moreover, we could tell when it had a first mover advantage and we could tell when it had a second mover advantage. Is that right? When we actually looked at the initial set up of those stones, we knew immediately that's a game in which Player 1 is going to win, or alternatively, we knew immediately that's a game which Player 2 is going to win. Now it turns out that that idea is very general and actually has a name attached to it, and that name is Zermelo. So today we'll start off by talking about a theorem due to a guy called Zermelo, and the idea of this theorem is this.We're going to look at games more general than just Nim, and we're going to ask the question, under what circumstances would you know about a game either that Player 1, the person who goes first, can force a win, or that Player 2 can force a win, or will allow a third possibility, which is it'sgoing to be a tie. So here's the theorem, suppose there are two players in this game, like the games that we looked at last time, and suppose--I won't define this formally now--but suppose the game is a game of perfect information. So what do I mean by perfect information? I'll define this later on in the class, but for now all I mean is, that whenever a player has his turn to move, that player knows exactly what has happened prior in the game.So, for example, all these sequential move games we've been looking at are moves of perfect information. When I get to move I know exactly what you did yesterday, I know what I did the day before yesterday and so on. So it's a game of perfect information. I'm going to assume that the game has a finite number of nodes. So two things here, it can't go on forever, this game, and also there's no point in which it branches in an infinite way. So there's a finite number of nodes, and we'll assume that the game has three possible outcomes. Actually there's a more general version of this theorem but this will do for now.The three possible outcomes are either a win for Player 1, so I'll call , or a loss for Player 1, which is obviously a win for Player 2, or it W1a tie. So the game--like Nim last time, that only had two outcomes. So here we're going to look up to three outcomes or three or fewer outcomes I should say. So these are the conditions and here's the result. So I said three, it could be three but it could also be two here, I'm just allowing for three. (One is trivial.) So under these conditions the following is true.Either Player 1 can force a win, so either it's the case that this game is a game that if Player 1 plays as well as they can, they're going to win the game no matter what Player 2 does. Or 1 can at least force a tie, which means Player 1 can play in such a way that they can assure themselves of a tie regardless of what Player 2 does. Or it could be a game in which 2 can force a loss on 1, so a win for 2. So this theorem, when you first look at it, it doesn't seem to say very much. You're staring at this thing--you might think, we already knew that we're looking at games that only have three possible outcomes win, loss, or tie so it doesn't seem so surprising if you look at this theorem, it says, well, you're going to end up with a win, or a loss, or a tie.However, that's not quite what the theorem says. The theorem says, not only are you going to end up there--we knew that already--but the games divide themselves. Games of these forms divide themselves into those games in which Player 1 has a way of winning regardless of what Player 2 does; or games in which Player 1 has a way of forcing a tie regardless of whatPlayer 2 does; or Player 2 has a way of winning regardless of what Player 1 does, so these games all have a solution.Let's just go back to Nim to illustrate the point. So in Nim actually there's no tie so we can forget the middle of these, and in Nim, under certain circumstances it is the case that Player 1 can force a win. Who remembers what the case was, for when Player 1 can force a win? Anybody? The people who played last time. No, yes, Ale, there's somebody here. Shout it out.Student: Insuring that the piles are equal for the other player.Professor Ben Polak:All right, so if the piles start unequal, then Player 1 can actually force a win. So in Nim if the piles are unequal at the start, then 1 can force a win. It really doesn't matter what 2 does, 2 is "toast." Or the alternative case is the piles are equal at the start, and if they're equal at the start then it's Player 1 who's "toast." Player 2 is going to--rather--force a loss on 1, so 2 can force a loss on 1, i.e. a win for Player 2. Does everyone remember that from last time? It's just before the weekend, it shouldn't be so long ago.So this theorem applies to all games of this form. So what games are of this form? Let's try and think of some other examples. So one example is tic-tac-toe. Everyone know the rules of tic-tac-toe? In England we call it Noughts and Crosses, but you guys call it tic-tac-toe, is that right? Everyone know what tic-tac-toe is? Yeah, which category is tic-tac-toe? Is it a game which Player 1 can force a win, or is it a category in which Player 1 can only force a tie, or is it a category which you'd rather go second and Player 2 can force a win for Player 2 or a loss for Player 1? Which is tic-tac-toe?Let's have a show of hands here. Who thinks tic-tac-toe is a game in which Player 1 can force a win? Who thinks tic-tac-toe is a game in which Player 1 can only force a tie? Who thinks Player 2's going to win? Most of you are right. It's a game in which if people play correctly then it'll turn out to be a tie. So tic-tac-toe is a game that leads to a tie. Player 1 can still make a mistake, in which case they can lose. Player 2 can make a mistake in which case they would lose, but there is a way of playing that forces a tie. So these are fairly simple games, let's talk about more complicated games.So what about the game of checkers? So the game of checkers meets these conditions. It's a two player game. You always know all the moves prior to your move. It's finite: there's some rules in checkers that prevent it going on forever. And there are two or three outcomes: I guess there'sa third outcome if you get into a cycle you could tie. So checkers fits all these descriptions and what this theorem tells us is that checkers has a solution. I'm not sure I know what that solution is, or I think actually somebody did compute it quite recently, even in the last few months, I just forgot to Google it this morning to remind myself.But what this theorem tells us even before those people that computed it: checkers has a solution. Let's be a bit more ambitious. What about chess? So chess meets this description. Chess is a two player game, everybody knows all the moves before them, it's sequential, it has finite number of moves, it's a very large number but it is finite, and it has three possible outcomes, a win, a loss, or a tie. So let's be careful, the reason it's finite is that if you cycle--I forget what it is--three times then the game is declared a draw, declared a tie. So what's this theorem telling us? It's telling us that there is a way to solve chess. Chess has a solution.We don't know what that solution is. It could be that solution is that Player 1, who's the player with the white pieces can force a win. It could be that Player 1 can only force a tie, and it could even be that Player 2 can force a win. We don't know which it is, but there is a solution. There's a catch to this theorem. What's the catch? The catch is it doesn't actually tell us--this theorem is not going to tell us what that solution is. It doesn't tell us how to play. This theorem, in and of itself, doesn't tell us how to play chess. It just says there is a way to play chess.So we're going to try and prove this. We don't often do proofs in class, but the reason I want to prove this particular one is because I think the proof is instructive as part of sort of QR at Yale. So here's another example here, and some other examples. You could think of chess as being the most dramatic example. So the reason I want to spend actually some time proving this today is because we're going to prove it by induction and I'm going to sketch the proof. I'm not going to go through every possible step but I want to give people here a feeling for what a proof by induction looks like. The reason for this is, my guess is, well let's find out, how many of you have seen a proof by induction before? How many have not?So for those of you who haven't I think it's a good thing in life, at one point in your life to see a proof by induction, and for those who have, my guess is you saw it in some awful math class in high school and it just went over--it didn't necessarily go over your head but, the excitement of it doesn't catch on. This is a context where it might appeal. So proof by induction. We're going to prove this by induction on the maximum length of the game and we'll call that N. We'll call N the maximum length of thegame. What do I mean by this? If I write down a tree I can always look at all the paths from the beginning of the tree all the way through to the end of the tree. And I'm going to look at the path in that particular tree that has the largest length, and I'll call that the length of the game, the maximum length of the game.So we're going to do induction on the maximum length of the game. So how do we start a proof by induction? Let's just remind ourselves, those people who have seen them before. We're going to prove that this theorem this true, the claim in the theorem is true for the easy case when the game is only 1 move long. That's the first step, and then we're going to try and show that if it's true for all games of length < = N, whatever N is, then it must therefore be true for games of length N+1. That's the way you do a proof by induction. So let's just see how that works in practice.So let's start with the easy step and we'll do it in some detail, more detail than you really need to in a math class. So if N = 1, what do these games look like? Well let's look at some examples here. So I claim it's pretty trivial if N = 1 but let's do it anyway. So the game might look like this. Player 1 is going to move--here's Player 1 moving--the game's only length one--so Player 1's the only player who ever gets to move. So the game might look like this. Let's put in a fifth branch, it might look like this. And at the end of this game, rather than putting payoffs let me just put the outcomes.So the outcome must either be a win, or a tie, or a loss, so suppose it looks like this. Suppose it's a win, or here we could have a tie, or here we could have a win again, or here we could have a loss, and here we could have a tie. So in this particular game I claim it's pretty clear this game has a solution. It's pretty clear what 1 should do. What should 1 do? 1 should pick one of her choices that leads to a win. To be careful I'll put 1 in here just to distinguish who it is who's actually winning and losing.So in this game I claim that this game has a pretty obvious solution, either Player 1 is going to choose this branch that leads to a win or this branch that leads to a win, and either way Player 1 is going to win. Is that obvious? It's so obvious, it's kind of painful. So I claim this game, we could actually replace this first node with what's going to happen which is Player 1's going to win. Is that right? That was easy. Let's look at a different example, we'll do three.So here's another possible example, and this again, Player 1 is going to move here and this time the possible outcomes are tie, or a loss, or a loss. Once again it's a one player game, this time he has three choicesand the three choices lead to a tie or he loses, so I claim once again this is simple. What Player 1 should do is what? Choose the tie since there's no way of winning. But in that case he could actually force it. He could actually choose the tie in which case he's going to tie. So this game has a solution called choose the tie. And again, we could replace the first node of the game with what's actually going to happen which is a tie. Everyone happy?There's one other possibility I guess. The other possibility is that here Player 1 has a lot of choices, maybe four choices in this case, once again Player 1 is going to move but in each case--unfortunately for Player 1--in each case the outcome is that Player 1 loses. So this is a game in which Player 1 is going to lose no matter what they do. Once again it has a solution: the solution is Player 1 is toast and is going to lose. So I claim this has really captured all the possibilities of games of length 1. I mean you could imagine that there could be more branches on them, but basically there are these three possibilities.Either it's the case that one of those branches leads to a win, in which case the solution is Player 1 wins; or it's the case that none of the branches have wins in them, but one of them has a tie, in which Player 1 is going to tie; or it's the case that all the branches have loss in them, in which Player 1's going to lose. So I claim I'm for done for games of length one. Everyone okay? So that step is deceptively easy but there it is. All right, now we do the inductive step, the big step. What we're going to do is we're going to assume that the statement of the theorem, which I've now hidden unfortunately--let me unhide it if I can, it's not going to be easy. This now visible?So now let's assume that the statement of this theorem is true for all games length < = N, suppose that's the case. Suppose the claim is true for all games of this type, of length <= some N. What we need to show is--what we're going to try and show is therefore it will be true for all games of length N + 1. What we want to show is--we claim, therefore, it will be true for games of length N + 1, that's the key step in inductive proofs. All right, so how are we going to do that? Well let's have a look at a game of length N + 1, and obviously I can't use thousands here so let me choose from relatively small numbers, but the idea will go through. So let's have a look at a game.So here's a game in which Player 1 moves first, Player 2 moves second. Let's suppose that if Player 2 does this, then 1 only has little choice here, up here then one has a more complicated choice, and perhaps 2 could move again. So this is quite a complicated little game up here, and down here let's make it a little bit less complicated. Here the game looks likethis and then comes to an end, so the tree looks like this. I haven't put the end, I haven't put the outcomes on it but the tree looks like this. So how long is this tree first of all? Well I claim that the longest path in this tree, from the beginning to the end has four steps. Let's just check. So one, two, three, four that's what I claim is the longest path. Any path going down this way only has three moves in it, so this is a game of length four.So we can apply it to our claim, let's assume that the theorem holds for all trees length three or fewer, so that N + 1 is 4 for the purpose of our example. So in this example--I don't want to put it here, let's put it here--in this example N = 3, so that N + 1 = 4. Everyone happy with this in the example? Now what I want to observe about this is the following: contained in this N + 1, or the game of length 4 are some smaller games. You could think of them as sub-games. They're games within the game, is that right? So let's have a look. I claim, in particular--draw around them in pink--there's a game here that follows Player 1 choosing Up and there's a game here that follows Player 1 choosing Down, is that right? Okay, so these are both games, and what do we have here?So this little game in here--the game in the top circle, the game that follows Player 1 moving up--that's a game and it has a length. So this little thing is a game and I'm going to call it a sub-game, some jargon that we'll be getting used to later on in the semester. It's a game within the game, but it's basically just a game: it's a sub-game and this is a sub-game following 1 choosing Up--let's put Up in here--following up. And I claim that this game has a length--this sub-game has a length. So we knew that we started from a game of length 4, we've taken one of the moves, and let's just make sure this is actually a game of length 3. So the game started here, this would be a game of length 3 because you could go one, two, three moves, is that right?So this is a game of length 3. So this is a sub-game following 1 choosing Up and it, the sub-game, has length 3. Down here, this is also a sub-game, it's a game that follows Player 1 choosing Down and this little sub-game has length--now here we have to be a little bit careful--you might think that since we started from a game of length 4, after the first move we must be in a game of length 3. But actually that's not true, and if we look carefully, we'll notice that this game down here, the game starting here actually isn't of length 3. It's of length 2. Is that right? So even though started from a game of length 4, by going this way we put ourselves into a game of length 2. Is that right? So 1,2 or 1,2--whatever--it has length 2.Okay, all right so in this example N is 3 and N + 1 is 4, and our assumption, our induction assumption is what? We've assumed that the claim of the theorem holds for all games <= N, which in this case means <= 3. So what does that tell us? That tells us, by our assumption, this game, which I've put a pink circle around on the top, this is a game of length 3, this game has a solution. It must have a solution because it's of length 3 or less. So by the induction hypothesis as it's called--by the induction hypothesis--this is a technical term, but what's the induction hypothesis? It's this thing. By the assumption that we made this game has a solution.So it's a game of length 3, 3 <= N in this case. So this game must have a solution. Now I don't immediately know by staring at it what it is, but let's suppose it was W, say it was W. And the game down below, this is also a game, and it's a game of length 2 but 2 is less than 3 as well, 2 <= 3. So this game also has a solution. This game also, by the same assumption, has a solution, so its solution is say--I don't know--maybe its L. So what does that mean to say that these games have solutions, in this case we're going to assume W is this one, and L is this one, what does it mean? It means that, just as we did with the games up there, we could put the solution at the beginning of the game. We know that if we get into this game, we're going to get the solution of this game; and we know if we get into this game, we're going to get the solution of this game. So we can replace this one with its solution which by assumption was W, and this one by its solution which by assumption was L. And here I want to be really careful, I need to keep track of which person it is it's a win or loss for. So this was a win for Player 1, hence it was a loss for Player 2, and this was a loss for Player 1, hence it was a win for Player 2. So these games, each of them have some solution, and this case I've written down the solutions as W or L.So now what could we do? Let me just bring down this board again. I can translate this game into a very simple game. I'm going to translate it up here, so this game can be translated as follows. Player 1 moves, if he goes Up then he hits a W and if he goes Down he hits a L. So in this particular example, Player 1 is effectively choosing between going Up and finding himself in a game which has a solution, and the solution is he wins it, or going Down and finding himself in a game which has a solution, and the solution is he's toast. But that's what? That's a one move game. So we know this has a solution, and in particular, he's going to choose Up. This one has a solution. This has a solution, it is a game of length 1.So what do we do? Somewhat schematically, we took a game of length N + 1, in this case that was 4; and we pointed out that once Player 1 has made her first move in this game, we are in a game, or in a sub-game if you like, that has length less than 4, it could be 3, it could be 2, whatever.Whatever sub-game we're in, by assumption, that game has a solution. So it's really effectively, Player 1 is choosing between going into a game with solution W or going into a game with solution L. And if there were 15 other sub-games here, each one would have a solution, and each one Player 1 would be able to associate that solution with what he's going to get--what she's going to get.So I claim that, if it in fact is true that each of these sub-games of length 3 or less had a solution, then the game of length 4 must have a solution, which is what? It's the solution which is: Player 1 should pick the best sub-game. Are people convinced by that step? People are looking slightly "deer in the headlamps" now, so let me just say it again. We started by assuming that all games of length 3 or less, or N or less, have a solution. We pointed out that any game that has length N + 1 can be thought of as an initial move by Player 1 followed by a game of length N or less, N or fewer I should say. Each of those games of N or fewer steps has a solution, so Player 1 is just going to choose the game that has the best solution for her and we're done.In this particular example, Player 1 is going to choose Up and therefore the solution to this parent game is Player 1 wins. Now the hard step I think in proofs by induction is realizing that you're done. So I claim we're now done, why are we done? Well we know that it was pretty easy that all games of length 1 have a solution. That was pretty trivial. And we've shown that if any game of length of N or fewer has a solution, then games of N + 1 have a solution.So now let's see how we can proceed. We know that games of length 1 have a solution, so let's consider games of length 2. Games of length 2, do games of length 2 have a solution? Well let's set N = 1. We know that if games of length 1 have a solution, then any game of length 2 could be thought of as an initial step followed by a game of length 1, but that has a solution, so therefore games of length 2 have a solution. Now let's think of games with length 3, we've shown that games with length 1 havea solution and games with length 2 have a solution. Any game of length3 can be thought of as a game in which there's an initial step followed by either a game of length 1 or a game of length 2, each of which have a solution, so once again a game of length 3 has a solution and so on.So games of induction, they work by building up on the length of the game, and we're ending up knowing that we're done. For those people who have never seen a proof by induction don't worry I'm not going to test you on a proof with induction on the exam, I just wanted you to see one once. Let's all take a deep breath and try and digest this a little bit by playing a game. I'll leave it up there so you can stare at it. We'll want it downagain later. Let's try and actually play a game and see if we can actually throw any light on this.So I'm going to pick out volunteers like I did last time, but first of all, let me tell you what the game is. So once again I'm going to have rocks in this game, and once again I'm going to approximate those rocks with marks on the blackboard. So here's an example. The example is this. The game involves an array of rocks, so here the array has five rows and three columns. So the rocks are not arranged as they were before in piles, but rather in a sort of in a pretty array. I should have made it more even but this is: one, two, three, four, five rows; and one, two, three columns, at least in this example. But in general there's an array of rocks, N x M.We're going to play sequentially with the players. And the way the game is going to work is this--is when it's your turn to move, you have to point to one of these rocks and whichever rock you point to I will remove. I will delete that rock and all the rocks that lie above or to the right of it: all the rocks that lie to the northeast of it. So for example, if you pointed to this rock, then I will remove this rock and also this one, this one, and this one. If the next person comes along and chooses this one, then I will remove this one, and also that one. Okay, everyone understand? All right and the game is lost--this is important--the game is lost by the person who ends up taking the last rock. The loser is the person who ends up removing the last rock.Could I have some volunteers? Can I volunteer people then? How about the guy with the white t-shirt with the yellow on there? Okay, come on up front, and who else can I volunteer. Everyone's looking away from me, desperately looking away from me. How about the guy with the Yale sweatshirt on, the Yale football sweatshirt on, okay. So come on up. Your name is? I should get a mike, let me get a mike up here, thank you. Great, your name is?Student: Noah.Professor Ben Polak: Noah and your name is?Student: Koran.Professor Ben Polak: Say it into the mike.Student: Koran.Professor Ben Polak: Koran okay. So Noah and Koran are our players. Everyone understand the rules? Do you understand the rules? Yeah? So why。