1. he bets 1 dollar each time (timid strategy).

For this, I will create probability matrix for each outcome.

Each outcome in the row is the total amount of dollar Smith holds before each bet. Each outcome in the column is the total amount of dollar Smith will hold after each bet.

In each outcome, you will either have 0, 0.4 or 0.6 which is the probability for each outcome to happen.

For example, At row1, column2, Smith holds 1 dollar at the beginning and earn extra 1 dollar upon successful bet with p = 0.4.

#Matrix in canonical form
P <- matrix(c(0,0.4,0,0,0,0,0,0.6,0,
              0.6,0,0.4,0,0,0,0,0,0,
              0,0.6,0,0.4,0,0,0,0,0,
              0,0,0.6,0,0.4,0,0,0,0,
              0,0,0,0.6,0,0.4,0,0,0,
              0,0,0,0,0.6,0,0.4,0,0,
              0,0,0,0,0,0.6,0,0,0.4,
              0,0,0,0,0,0,0,1,0,
              0,0,0,0,0,0,0,0,1), nrow=9, byrow=TRUE)
rownames(P) <- c("1","2","3","4","5","6","7","0","8")
colnames(P) <- c("1","2","3","4","5","6","7","0","8")
P
##     1   2   3   4   5   6   7   0   8
## 1 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.6 0.0
## 2 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0
## 3 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## 4 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## 5 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## 6 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## 7 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.4
## 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0
## 8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0
#Fundamental matrix
fm_1 <- P[1:7,1:7]

fm_2 <- diag(7)
fm_full <- solve(fm_2-fm_1)
fm_full
##           1        2         3         4         5         6          7
## 1 1.6328311 1.054718 0.6693101 0.4123711 0.2410785 0.1268834 0.05075337
## 2 1.5820777 2.636796 1.6732752 1.0309278 0.6026963 0.3172086 0.12688343
## 3 1.5059477 2.509913 3.1792228 1.9587629 1.1451229 0.6026963 0.24107851
## 4 1.3917526 2.319588 2.9381443 3.3505155 1.9587629 1.0309278 0.41237113
## 5 1.2204600 2.034100 2.5765266 2.9381443 3.1792228 1.6732752 0.66931007
## 6 0.9635210 1.605868 2.0340999 2.3195876 2.5099128 2.6367962 1.05471848
## 7 0.5781126 0.963521 1.2204600 1.3917526 1.5059477 1.5820777 1.63283109
#Absorption probablities
ap_1 <- P[1:7,8:9]
ap_full <- fm_full %*% ap_1
ap_full
##           0          8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243
#Let's find the probability of timid strategy
timid_p = ap_full["1","8"]
timid_p
## [1] 0.02030135
  1. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
bold_p = dbinom(3,3,0.4) 
bold_p
## [1] 0.064
  1. Which strategy gives Smith the better chance of getting out of jail?

Apparently, bold_p is bigger than timid_p. Bold strategy gives Smith the better chance of getting out of jail.