Showing posts with label Game Design. Show all posts
Showing posts with label Game Design. Show all posts

Friday, May 19, 2023

GameDevTV Jam 2023 Planning

 It has been a while since I have posted something here, but now that I have a tiny bit of time, I can finally work on my own things, so am entering the GameDevTV jam for this year. Why? Everyone who enters gets a free course so why not? I am going to be focusing my free time on writing some books on creating a game engine from scratch, so in preparation for the game jam I will spend the days I am able to work on it creating a game from scratch. Unfortunately, I still do have other obligations so will only be able to get in 30-50 hours during the 10 days so will have to take that into account. I am losing this first weekend and next Friday for sure and suspect there will be interruptions on other days so we will see how much I am able to do.

What do I mean by from scratch? There will be no libraries or frameworks other than the standard libraries and what DOM provides. I will write all the code. Ideally,I will extend this restriction to artwork and sound effects but that will depend on how much time I have at the end for polishing the game.

The theme of the game is Life in 2 Dimensions. The thoughtI immediately came up with is some variation on Conway’s Game of Life (CGL). Doing a straight CGL would not be much of a challenge, but if I did this more like a strategy game then there could be something interesting.

My idea is to use a hex grid where there would be 3 distinct types of cells which are either food or enemies. Red Cells eat Green cells which eat Blue cells which eat Red cells. Each round is broken into phases. Phase 1 would be the absorption phase where the cells around each cell is counted. If there are more enemy cells touching a cell then allies, the cell will be absorbed becoming an enemy cell otherwise it stays the same. The next phase is the movement phase.  

The rules for movement are easy. Head towards the nearest food cell (or, fake food cell when I add that later) though only using empty cells, so will stay still if cells closer to food are currently blocked.  Not sure what order the cells will move in, will be playing around with that aspect. 

The game will be an attempt to convert the entire board to your color. With the ability to place fake food on the board to guide the activity. I am hoping to have a crude version of the game by Tuesday or Wednesday as due to other obligations I am not going to be able to start until Monday. 

Monday, March 14, 2022

Making Santa-Tac-Toe part 3 of 4

 Part 3: Random Moves Meet Monty Carlo

Thanks in small part to AlphaGo, there is a huge interest in Monty Carlo Tree Search (MCTS). As the name implies, this is built off the idea of a Monty Carlo Simulation, where you run a large number of random simulations to help determine the best configuration of what is being simulated. Tic-Tac-Toe is too simple for MCTS, but the simulation aspect would be simple to implement and could be interesting so lets do that. We will then look at what would need to be done to turn the simulation into a proper MCTS AI. 

The heart of a simulation is the ability to play a random game. This is simply a matter of making random moves until the game is in a win or tie state, at which point we return the state of the game so we can keep track of the wins and losses.

playMockGame(board, curPlayer) {
   
let b = new TTTBoard(board.getTemplate());
    let
state = b.getGameState();
    let
p = curPlayer;
    while
(state.state === TTTConsts.PLAYING) {
        b.
setTileByIndex(super.getNextMove(b, p), p);
       
p = (p === TTTConsts.X) ? TTTConsts.O : TTTConsts.X;
       
state = b.getGameState();
   
}
   
return state;
}


We want to score each potential move that a player can make, so simply loop over all the moves adding them to our scores array which we will be using for the next move method that our API requires we implement.

buildScoreBoard(board, curPlayer) {
   
let scores = [];
    for
(let i = 0; i < 9; ++i)
        scores.
push(this.scoreTile(board, i, curPlayer));
    return
scores;
}

Scoring a tile is simply a matter of making sure the move is valid, followed by playing several mock games and counting how many of the games the AI wins versus the losses.

scoreTile(board, index, curPlayer) {
   
if (board.getTileByIndex(index) !== TTTConsts.EMPTY)
       
return -2 * this.trials;
    let
winState = (curPlayer === TTTConsts.X) ? TTTConsts.X_WON : TTTConsts.O_WON;
    let
loseState = (curPlayer === TTTConsts.X) ? TTTConsts.O_WON : TTTConsts.X_WON;
    let
score = 0;
    this
.testBoard.clone(board);
    this
.testBoard.setTileByIndex(index, curPlayer);
    let
opponent = (curPlayer === TTTConsts.X) ? TTTConsts.O : TTTConsts.X;
    for
(let i = 0; i < this.trials; ++i) {
       
let state = this.playMockGame(this.testBoard, opponent);
        if
(state.state === winState) score += 1;
        if
(state.state === loseState) score -= 1;
   
}
   
return score;
}

Finally, implementing getNextMove is simply a matter of calling buildScoreBoard and then picking the highest score. We add a random value to each score as a tiebreaker.

getNextMove(board, playerToMove= TTTConsts.EMPTY) {
   
let scores = this.buildScoreBoard(board,playerToMove);
    let
index = 0;
    let
high = scores[0] + Math.random();
    for
(let i = 1; i < 9; ++i) {
       
let temp = scores[i] + Math.random();
        if
(temp > high) {
            index = i
;
           
high = temp;
       
}
    }

   
return index;
}

The MCTS algorithm is a bit more involved than what we are doing, but with such a small tree, implementing a full MCTS is purely academic. We would need to track a bit more information for each node in the game to properly do MCTS. Lets quickly take a look at the algorithm. It is broken into 4 basic steps.

1 Selection. The idea here is that we pick a node that we want to explore further. This is usually done by looking at the win/loss ratio in combination with the times the node has been looked at for the nodes that are not fully expanded. We want to pick the most promising node but at the same time want to make sure that each node is explored enough to properly estimate it’s worthiness. There are many methods to do this though the simplest is to use threshold * (wins+1)/(plays+1) but there are better methods available.

2 Expansion is simply to add one of the unexplored nodes of the target onto our tree. Instead of just having a scoreboard for the current round, we would need to track all the nodes we have expanded in some type of tree structure and keep track of the win/loss ratio for all of them.

3 Simulation (playing random game), what we are doing already.

4 backpropagation. Taking the score the expanded node received and passing it through all the parent nodes adjusting their probability. Remember to consider who has won so that you invert the win/loss ratio from opponent nodes.

The big issue here is that it is easy to run out of memory as you expand. One solution to this is to cap the number of nodes allocated and when you run out of nodes go into a pruning stage where you remove nodes of poorly performing choices. A node pool is a good way of doing this as when the pool is empty, you do a pruning pass adding pruned nodes (including children) back into the pool.

Now that we have several AI agents, we can put together our Christmas game. This will be covered in the next part.


Friday, January 14, 2022

Making Santa-Tac-Toe part 1 of 4

 Part 1: Preparing for Santa and Krampus

Santa Tac Toe is a JavaScript port of a game I created when working as a teaching assistant for a course in AI. We had a team project where groups would create AI controlled players for the game of Amazons. Our summer terms are condensed to two months with double the lecture and lab time. To help students start their AI, I wrote a simple Tic Tac Toe game in Java to demonstrate recursive AIs. The advantage here is the search tree is so small that things like alpha-beta pruning and techniques for scoring a position are not covered so they would still need to figure those out themselves but would still have a starting point. This approach will be covered in part 2.  A popular technique for games, largely due to Alpha Go, is Monty Carlo Tree Search. While I didn’t want to do a full MCTS AI, in Part 3 we look at the starting point for this technique. Finally, in Part 4 we will make everything pretty by creating the GUI for the game.

 As most of my personal projects tend to be small, I tend to rely a bit too much on manual testing. This is not a bad thing, but as projects get larger and more people are involved, automated testing makes more sense. One of the trendy test driven development methods is red/green testing, which does sound like a good idea but has its detractors. The idea is you write a test first and after it fails, you write the code to make it pass the test. For code where it is easy to write automated tests, I am using Jasmine to write tests. For user interface and animation related stuff, I am still largely going to rely on manual testing, but as problems here will quickly be apparent this shouldn’t be too bad of a compromise. I do need to find a decent GUI testing suite but am not sure such a beast actually exists. For those of you interested in my test code, it is in the repository.


The board is simply an array of numbers, set up as a single dimensional array stored as shown in the figure above. Numbers were chosen instead of characters due to being faster. An alternative approach would be to store the board as a string, but strings are immutable in JavaScript. Altering a string creates a new one, and if you are replacing the old string then it gets garbage collected. To aid debugging, the constructor takes a string representation of the board and can create a string representation for logging or other debugging tasks.

Setting and getting tiles is trivial enough. The key method here is the getGameState() method which fills out the TTTState class with information about the current state of the board. Simply put, the game can be in progress, the X player (Santa) could have won, the O player (Krampus) could have won, or there are no more moves so the game ends in a tie (Cat). For win conditions, the line that forms the win is necessary so a way of checking lines on the board is needed.

The checkLine method assumes the starting location is correct for the direction being checked as this is meant for internal use. All that needs to be done is to check if three tiles all contain the same non-empty value and return true if they are. Because we are storing the value in an array, we can use steps to find the appropriate locations. Horizontal lines step at +1, vertical lines step at +3, diagonal step at +4 for right diagonals and +2 for left diagonals.

checkLine(x,y,direction) {
   
let index = x + y * 3;
    let
startTile = this.board[index];
    if
(startTile === TTTConsts.EMPTY)
       
return false;
    let
offset = TTTConsts.LINE_OFFSETS[direction];
    return
(this.board[index+offset] === startTile) &&
        (
this.board[index+offset+offset] === startTile);
}


With this we can work out the state of the game. We start by assuming the game is in progress. We then quickly check to see if the board is full. If so, we know there is either a win or a tie, so set the state to tie but continue checking to see if it is actually a win. Each of the possible horizontal and vertical wins are checked using the checkLine method above. Finally we check the two diagonal moves.

getGameState() {We now have the basics for the game but need players. As we are focusing on AI players, we will start with the easiest possible AI player, one that makes moves at random. When called, this agent simply picks a tile at random, then we see if that is a valid tile and if not move to the first open tile.
    this.gameState.state = TTTConsts.PLAYING;
   
// potential tie check     let tie = true;     for (let i = 0; i < 9; ++i)         if (this.board[i] === TTTConsts.EMPTY)             tie = false;
    if
(tie)         this.gameState.state = TTTConsts.TIE;
   
// win check     for(let i = 0; i < 3; ++i) {         if (this.checkLine(i,0,TTTConsts.VERTICAL))             this.gameState.setWinLine(this, i,0,TTTConsts.VERTICAL);
        if
(this.checkLine(0,i,TTTConsts.HORIZONTAL))             this.gameState.setWinLine(this, 0,i,TTTConsts.HORIZONTAL);     }
   
if (this.checkLine(0,0,TTTConsts.RIGHT_DIAGONAL))         this.gameState.setWinLine(this, 0,0,TTTConsts.RIGHT_DIAGONAL);
    if
(this.checkLine(2,0,TTTConsts.LEFT_DIAGONAL))         this.gameState.setWinLine(this, 2,0,TTTConsts.LEFT_DIAGONAL);
    return this
.gameState; }

We now have the basics for the game but need players. As we are focusing on AI players, we will start with the easiest possible AI player, one that makes moves at random. When called, this agent simply picks a tile at random, then we see if that is a valid tile and if not move to the first open tile.

class AIPlayer {
    constructor(playerID) {
        this.playerID = playerID;
    }
   
findNextFreeIndex(board, startIndex = 0) {         let index = -1;         let nextIndex = startIndex         do {             if (board.getTileByIndex(nextIndex) === TTTConsts.EMPTY)                 index = nextIndex;             nextIndex = (nextIndex+1)%9;         } while ((index === -1) && (nextIndex !== startIndex));
        return
index;     }
   
getNextMove(board, playerToMove= TTTConsts.EMPTY) {         return this.findNextFreeIndex(board, Math.floor(Math.random() * 9));     } }

This random player will be expanded upon for our Monty player in part 3, but next we want to look at the traditional recursive AI, which we will do in the next part.


Sunday, November 14, 2021

Making Knight's Tour part 3 of 4

 Heuristics

When you get right down to it, solving the knights tour is simply a search of really large tree.  The biggest problem is that if you take a wrong path then you end up wasting a large amount of time before you realize your mistake and come back track back up to where a viable solution can exist. The obvious solution to this is not to make bad mistakes, but how do you determine if a choice is a good choice or not? Human brains partially solve this problem by employing what is known as heuristics. The idea here is instead of doing complicated calculations the brain will look for shortcuts that usually, though not always, lead to the correct solution. With NP problems heuristics can be a very good way of solving the problems in a reasonable amount of time if you can find an appropriate heuristic.

Creating a heuristic is not a simple task. Usually several different techniques are tried to find out if any of them work with smaller versions of the problem set. The technique can then be applied to larger problems to see if they still work. Often we will start by looking at several versions of the problem and seeing if there's any pattern that can be found. The easiest way of finding a heuristic is to look at what other people have discovered and using their heuristic, but that requires an existing heuristic to already exist. Knight's tour is very popular with mathematicians so several heuristics have been devloped.

For Knights tour the Warnsdorff's heuristic is the heuristic that dominates. This is a quite simple heuristic. The idea is that for each potential move you look at the number of moves that tile will have once you have moved there and selecting the tile that has the least moves available. As you can see by the figure below, each of the tiles that we can move to has a certain number of moves attached to it, but there is also a tie. There are several approaches that can be taken for dealing with ties, so I went with a clockwise order of searching tiles and taking the clock position as the tie-breaker.


One of the things that continually came up when researching this heuristic was that when there was a tie that would quite often be where the problem with the solution was. This would mean that tide tiles would be a good candidate for a faster backtracking method. For this reason I updated the reset function to also include an indication of whether there was a tie at particular decision tree node. 

reset() {
    for (let i = 0; i < 64; ++i) {
        this.game.tiles[i].step = 0;
        this
.game.tiles[i].tiebreaker = false;     } }

 The heart of this Huristic is the counting method. This simply looks to see if the tiles currently occupied with the night and if so returns negative one. The negative one is very important as it is possible at the very end of the game that a valid tile will have zero moves as it will be the only tile available to be moved into. We simply count all the possible moves making sure that we do not count the move back to our source tile.

countTargetMoves(target, source) {
   
if ((target < 0) || (target >= 64))         return -1;  // NOTE invalid is -1 not zero as last move will be 0 moves!!!
   
if ( ! this.game.tiles[target].checkIfAllowedMove(target, source) )         return -1;
    if
(this.game.tiles[target].moveNumber !== KT.NO_MOVE)         return -1;
    let
moveCount = 0;     for (let t = 0; t < KT.knightTileOffsets.length; ++t) {         let knight = target + KT.knightTileOffsets[t];         if ((knight === source) || (knight < 0) || (knight >= 64))             continue;
        let
knightXAdjust = Math.abs( (knight % 8) - (target % 8) );
        let
knightYAdjust = Math.abs( Math.floor(knight / 8) - Math.floor(target / 8) );         if ( ((knightXAdjust+knightYAdjust) === 3) &&
            (
this.game.tiles[knight].moveNumber === KT.NO_MOVE) )
            ++moveCount
;     }     return moveCount }

 Sadly, my fast backtrack method was never tested but has been implemented here for reference. I know it was not used for reason that will become clear once the algorithm is ran. The idea behind fast backtracking is that you do not backtrack a single step but backtrack multiple steps to a decision point you know it is faulty. As we know that the tiles that have multiple tile entries are potential problem areas, this is a good point to do a fast backtrack too.

fastBacktrack() {
   
let curTileIndex = this.game.lastKnightTile;
    if
(curTileIndex === this.game.firstKnightTile) return;
    let
curTile = this.game.tiles[curTileIndex];
    this
.game.undoMove();
   
curTile.step = 0;
   
curTile.tiebreaker = false;
    if
( ! this.game.tiles[this.game.lastKnightTile].tiebreaker)         this.fastBacktrack();
}

 

The next move method seems a lot more complicated than it really is. The big problem that we have is that step is no longer linear as steps may be taken in totally different orders. To get around this problem, we are treating each bit of the step variable as one of the paths that we can take. If the bit is set then we know that path is either not available or has been tried already. To loop over the bits, we simply create a mask variable which we set to 1 to indicate the first bit. The left shift operator, which is represented using <<, is used to shift that one to the appropriate bit position. If the value of step is 255 then we know that all these steps have been tried and can do a backtrack. If moves are still available we simply loop over all the bits and find the lowest count from the moves that are remaining. The lowest move is then the move that we make. As it is simple enough to also do forward checking, we will call the is game still winnable method to do a forward checking step even though this is not required as part of the heuristic.

    nextMove() {
        
let curTile = this.game.lastKnightTile;
        let
step = this.game.tiles[curTile].step;
        let
tiebreaker = this.game.tiles[curTile].tiebreaker;
        if
(step === 255) {
//            this.game.undoMove();
//            step = 0;
           
return this.fastBacktrack();
       
} else {
           
let mask = 1;
            let
index = 0;
            let
lowest = -1;
            let
lowestCount = 9;
            let
lowestMask = 0;
            while
(mask < 256) {
               
if ((step & mask) === 0) {
                   
let target = curTile + KT.knightTileOffsets[index];
                    let
targetMoves = this.countTargetMoves(target, curTile);
//                    console.log(`target ${target} has ${targetMoves}`);
                   
if (targetMoves < 0)
                        step |= mask
;
                    else if
(targetMoves < lowestCount) {
                        lowest = target
;
                       
lowestCount = targetMoves;
                        
lowestMask = mask;
                   
} else if (targetMoves === lowestCount) {
                        tiebreaker =
true;
                   
}
                }
                mask <<=
1;
               
++index;
           
}
            
if (lowestMask > 0) {
                step |= lowestMask
;
                this
.game.addKnight(lowest);
                if
( ! this.game.isGameStillWinnable() ) {
                   
this.game.undoMove();
//                    console.log("Early backtracking!");
               
}
            }
else
               
console.log("No move found???")
        }
       
this.game.tiles[curTile].step = step;
        this
.game.tiles[curTile].tiebreaker = tiebreaker;
   
} 

Running with this Heuristic proved to be very efficient. as it turns out, the heuristic solver is able to find the solution in 63 steps for all of the possible starting locations. With the performance we are getting using this heuristic we are done, so why is there another solver in the game? My exploration of Knight’s Tour would not be complete without looking at circuits and larger boards.