人工智能知識及應用證書（兼讀制）

AI to Play Tic-Tac-Toe

Learning Objective

• This tutorial is to demonstrate how to build the reinforcement learning model in R
  
• After this tutorial, students will gain some insight of how AlphaGo was built.
• This is a low-code tutorial in R

Package Installation and Load

install.packages('ReinforcementLearning')

library('ReinforcementLearning')

Data Preparation

data("tictactoe")
print(nrow(tictactoe))

[1] 406541

head(tictactoe, 30)

the first five observations of a representative dataset containing game states of randomly sampled tic-tac-toe games. In this dataset, the first column contains a representation of the current board state in a match. The second column denotes the observed action of player X in this state, whereas the third column contains a representation of the resulting board state after performing the action. The fourth column specifies the resulting reward for player X. This dataset is thus sufficient as input for learning the agent.

Display TicTakToe Board

plot_board <- function(x,action="") {
  if (action!="") 
  {
    actionNumber=substr(action,2,2)
    substr(x, actionNumber , actionNumber) <- "X"
  }
  string=NULL
  for (i in 1:nchar(x))
  {
    if (substr(x,i,i)=='.') string=c(string,'0')
    if (substr(x,i,i)=='X') string=c(string,'1')
    if (substr(x,i,i)=='B') string=c(string,'-1')
  }
  pieced <- rep("", length(string))
  pieced[which(string == 1)] <- "x"
  pieced[which(string == -1)] <- "o"
  pieced[which(string == 0)] <- "*"
  board <- gsub(" \\|$", "", paste(pieced, "|", collapse = " "))
  board_lines <- gsub("(. \\| . \\| . )\\|( . \\| . \\| . )\\|( . \\| . \\| .)", 
                      "\n \\1\n-----------\n\\2\n-----------\n\\3",
                      board
  )
  return(writeLines(board_lines))
}

plot_board('......X.B')

 * | * | * 
-----------
 * | * | * 
-----------
 x | * | o

plot_board('......X.B', 'c4')

 * | * | * 
-----------
 x | * | * 
-----------
 x | * | o

Build the model

model <- ReinforcementLearning(data = tictactoe, 
                               s = "State", 
                               a = "Action", 
                               r = "Reward", 
                               s_new = "NextState", 
                               iter = 1)

Calculate optimal policy

pol = computePolicy(model)
# Print policy
head(pol)

.XXBB..XB XXBB.B.X. .XBB..BXX BXX...B.. ..XB..... XBXBXB... 
     "c1"      "c5"      "c5"      "c4"      "c5"      "c9" 

Let’s play tic-tak-toe

data_unseen <- data.frame(State ='B.....X..', stringsAsFactors = FALSE)
predict(model, data_unseen$State)

[1] "c5"

REF

• https://cran.r-project.org/web/packages/ReinforcementLearning/vignettes/ReinforcementLearning.html
• https://www.r-bloggers.com/2019/11/r-inforcement-learning-part-one-tic-tac-toe/