A feed-forward neural network approach to estimate parameters of models of COVID-19 Dynamics

class: left, bottom, my-title, title-slide

.title[
# A feed-forward neural network approach to estimate parameters of models of COVID-19 Dynamics
]
.author[
### Ziwei Ma and Jin Wang
]
.institute[
### University of Tennessee At Chattanooga
]
.date[
### 2022-10-15
]

---

<div>
<style type="text/css">.xaringan-extra-logo {
width: 55px;
height: 64px;
z-index: 0;
background-image: url(University_of_Tennessee_at_Chattanooga_athletics_logo.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
bottom:1em;left:1em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('div')
          logo.classList = 'xaringan-extra-logo'
          logo.href = null
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

</style>

# Outline

## Motivation

## DeepXDE package in Python

## DiffEqFlux package in Julia

## Future work

---
class: inverse, middle, center

# Motivation

---
class: inverse

# Motivation

- ## Modeling COVID-19 dynamics
  - ### Virous concentration
  - ### Long COVID

- ## Challenges
  - ### No reliable data
  - ### Limited data available for some components

---

##  Modeling COVID-19 dynamics

.panelset[
.panel[.panel-name[Model 1]

#### Considering the environmental coronavirus

`\begin{aligned}
\frac{d S}{d t} &=\Lambda-\beta_E(t) S E-\beta_I(t) S I-\beta_V(t) S V-\mu S \\
\frac{d E}{d t} &=\beta_E(t) S E+\beta_I(t) S I+\beta_V(t) S V-\left(\alpha+\gamma_1+\mu\right) E \\
\frac{d I}{d t} &=\alpha(1-p) E-\left(\gamma_2+\mu\right) I \\
\frac{d H}{d t} &=\alpha p E-\left(w+\gamma_3+\mu\right) H \\
\frac{d R}{d t} &=\gamma_1 E+\gamma_2 I+\gamma_3 H-\mu R \\
\frac{d V}{d t} &=\xi_1(t) E+\xi_2(t) I-\sigma(t) V
\end{aligned}`
]

.panel[.panel-name[Parameters]

- `\(\Lambda\)` : the population influx rate
- `\(\mu\)` : the the natural death rate for the human hosts
- `\(\alpha\)` : the incubation rate
- `\(p\)` : the portion of exposed individuals to be hospitalized
- `\(\omega\)` : the disease induced death rate
- `\(\gamma_1,\gamma_2,\gamma_3\)` : rates of recovery from the exposed, infected (non-hospitalized), and hospitalized individuals 
- `\(\beta_E\)` : direct, human-to-human transmission rates between the exposed and susceptible individuals
- `\(\beta_I\)` : direct, human-to-human transmission rates between the infected and susceptible individuals
- `\(\beta_V\)` : the indirect, environment-to-human transmission rate
- `\(\xi_1\)`  : rates of contributing the coronavirus to the environment from the exposed individuals
- `\(\xi_2\)` : rates of contributing the coronavirus to the environment from the infectd individuals
- `\(\sigma\)` : the removal rate of the coronavirus from the environment

]

.panel[.panel-name[Model 2]

#### Considering long COVID

`\begin{aligned}
\frac{d S}{d t} &=\Lambda-\beta(t) S I-[\phi(t)+\mu] S, \\
\frac{d V}{d t} &=\phi(t) S-\theta \beta(t) V I-\mu V \\
\frac{d I}{d t} &=\beta(t) I(S+\theta V)-[\alpha(t)+\gamma+w+\mu] I, \\
\frac{d I_L}{d t} &=\alpha(t) I-\left(\gamma_L+w_L+\mu\right) I_L, \\
\frac{d R}{d t} &=\gamma I+\gamma_L I_L-\mu R .
\end{aligned}`
]

.panel[.panel-name[Parameters]

- `\(\Lambda\)` : the population influx rate
- `\(\mu\)` : the the natural death rate for the human hosts
- `\(\beta\)` : transmission rate
- `\(\phi\)` : vaccination rate
- `\(\alpha\)` : the rate of short-term infected individuals transfer to long COVID
- `\(\theta\)` : the portion of breakthrough infections 
- `\(\omega, \omega_L\)` : the disease induced death rate
- `\(\gamma,\gamma_L\)` : rates of recovery

]

---
class: inverse middle center

# DeepXDE package in Python

---

## DeepXDE package in Python

.panelset[

.panel[.panel-name[Algorithm diagram]

![](PINN-aaa.png)

]

.panel[.panel-name[Algorithm description]

-   1 Construct a neural network `\(\hat{u}(x; \theta )\)` with parameters `\(\theta\)`. 
-   2 Specify the training sets for the equation and boundary/initial conditions. 
-   3 Specify a loss function by summing the weighted `\(L_2\)` norm of both the ODE equations and boundary condition residuals. 
-   4 Train the neural network to find the best parameters `\(\theta^*\)` by minimizing the loss function.

]

.panel[.panel-name[Usage of DeepXDE]

-   1 Specify the ODE using the grammar of `TensorFlow`. 
-   2 Specify the boundary and initial conditions. 
-   3 Combine the boundary/initial conditions together into `data.ODE`. To specify training data, we can either set the specific point locations, or only set the number of points and then DeepXDE will sample the required number of points on a grid or randomly. 
-   4 Construct a neural network using the `maps` module. 
-   5 Define a Model by combining the ODE problem in   4 and the neural net in   5. 
-   6 Call `Model.compile` to set the optimization hyperparameters.
-   7 Call `Model.train` to train the network from random initialization or a pretrained model using the argument model restore path. It is extremely flexible to monitor and modify the training behavior using callbacks. 
-   8 Call `Model.predict` to predict the ODE solution at different locations.

- [Example](https://colab.research.google.com/drive/1gh_jjgzlRbGRSPLKtVdzuyo_W2Ufmej7#scrollTo=tYQ9ru5QGtnX)

]

---
class: inverse middle center

# DiffEqFlux package in Julia

---

## DiffEqFlux package in Julia

.panelset[

.panel[.panel-name[Define ODE]

```
using Flux, DiffEqFlux, DifferentialEquations, Plots

# Define our equation systems

function long_covid!(du,u,p,t)
  s,v,i,i_l,r = u
  beta, phi, mu, theta, alpha, gamma, omega, gammaL, omegaL = p
  du[1] = ds = - beta*s*i-(phi + mu)*s
  du[2] = dv = phi*s-theta*beta*v*i-mu*v
  du[3] = di = beta*i*(s+theta*v) -(alpha+gamma+omega+mu)*i
  du[4] = di_l = alpha*i - (gammaL+omegaL+mu)*i_l
  du[5] = dr = gamma*i+gammaL*i_l - mu*r
end
```
]

.panel[.panel-name[Initial values]

```
# Initial values

tbegin1 = 1.0
tend1 = 60.0
tstep1 = 1.0
trange1 = tbegin1:tstep1:tend1
u01 = [300000.0, 10000.0, 10000.0, 3000.0, 1000.0]
tspan1 = (tbegin1, tend1)
#p1 = ones(9)
p1 = [1.77e-5, .065, 2.74e-5, .90, 0.3, 0.6, 0.002, 0.9, 0.0001]
# define problem and true values for parameters

true_params1 = [1.77e-6, .065, 2.74e-5, .90, 0.3, 0.6, 0.002, 0.9, 0.0001]
```

]

.panel[.panel-name[ODE solver]

```
# beta, ph=0.65, mu=2.74e-5, theta=0.9, alpha=0.3, gamma=0.6, omega=0.02, gammaL=0.9, omegaL=0.0001
prob01 = ODEProblem(long_covid!,u01,tspan1,true_params1)

# Numerical solve

sol = solve(prob01,saveat=trange1)

```

]

.panel[.panel-name[Define NNs]

```
prob1 = ODEProblem(long_covid!, u01, tspan1, p1)

function net1()
    solve(prob1, Tsit5(), p=p1, saveat=trange1)
end

```

]

.panel[.panel-name[Loss function]

```
# Define loss function

dataset_outs1 = reduce(hcat,sol.u)'
print(dataset_outs1)
function loss_func1()
  pred = net1()
  sum(abs2, dataset_outs1[:,1] .- pred[1,:]) +
  sum(abs2, dataset_outs1[:,2] .- pred[2,:]) +
  sum(abs2, dataset_outs1[:,3] .- pred[3,:]) +
  sum(abs2, dataset_outs1[:,4] .- pred[4,:]) +
  sum(abs2, dataset_outs1[:,5] .- pred[5,:])
end

```

]

.panel[.panel-name[Train model]

```
 Train a model

epochs = 100000
learning_rate = 1.0e-6
data1 = Iterators.repeated((), epochs)
opt = ADAM(learning_rate)
callback_func1 = function ()
  println("loss: ", loss_func1())
end
fparams1 = Flux.params(p1)
Flux.train!(loss_func1, fparams1, data1, opt, cb=callback_func1)

```

]

.panel[.panel-name[UK data]
![](uk-fit.png) 
]

]

---
class: inverse, middle, center

# Future Work and Acknowledgements

---

# Future work

- To explore deeper on applying DeepXDE and DiffEqFlux packages

- To write up our own packages on estimating parameters

# Acknowledgements

- National Institutes of Health under grant number 1R15GM131315

---
background-image: url(thks.jpg)
background-size: 50%