# 对于Neural ODE的小研究

●Midpoint method (or RK2) - 2nd order method方法只需

## 将神经网络与ODE求解器集成

We can use existing (and efficient) implementation of solvers to integrate NNs dynamics
The memory cost is O(1) , due to reversibility i.e. we don’t need to store all activations in the graph, we can easily recover them by backward integration (i.e. time reversed integration)
Complex dynamics can be modeled with fewer parameters
We can control accuracy/speed trade-off with adaptive solvers by setting lower/higher error tolerances
Hidden states can be accessed at any value of t - no discrete time steps as in RestNet skip connection

Adjoint method can be understand as a continuous version of chain rule
Chain rule: Consider following sequence of operations ( L is a scalar loss):

We can compute gradient of L w.r.t input state using chain rule