Over the years I have received a lot of emails in response to my post about neural ODEs where people ask for advice on a particular pitfall when applying neural ODEs on regression style problems. So here is a (long overdue) blog post to address that! Code can be found here. The first lesson of machine learning If you are a machine learning practitioner I’m sure you’ve been told to “start simple”. Read more

In a previous article we talked about how to put neural networks inside ODEs to learn their dynamics from data. Armed with that knowledge we created a powerful weather forecasting model. But learning the dynamics of a process is only one side of the neural ODE story, they can also be used as very flexible function approximators much like regular neural network. In this article we are going to create continuous neural network layers using neural ODEs and see how they can be used to classify the Fashion MNIST dataset. Read more

Weather forecasting is a tricky problem. Traditionally, it has been done by manually modelling weather dynamics using differential equations, but this approach is highly dependent on us getting the equations right. To avoid this problem, we can use machine learning to directly predict the weather, which let’s us make predictions without modelling the dynamics. However, this approach requires huge amounts of data to reach good performance. Fortunately, there is a middle ground: What if we instead use machine learning to model the dynamics of the weather? Read more