Building a virtual neuron – part 1

Image credit: Ionut Stefan

To paraphrase one of my former college professors, this post should’ve been called “differential equations”, but then none of you would’ve clicked on it. I do hope that I didn’t mention that term too early though and scared you all away. After all, you made the effort of opening this page, so why not read on for 30 more seconds?

I know, scary stuff. But I promise this won’t be one of those super-abstract, incomprehensible tutorials that assumes you already know the stuff that you’re here to learn. We’re going to start from some very basic notions and from truly simple examples. If you’re a physicist or a mathematician or basically anyone who has a formal education in this topic, you might wanna skip towards the end though. Otherwise you might find yourself screaming at the computer: “NO! NO! Absolutely not! It’s not a squiggly math formula, it’s an equation that relates one or more unknown functions and their derivatives! Everybody knows that!” I do get your frustration, but some people need to start from somewhere else to get to where you already are.

What are differential equations

So what’s a differential equation? Well, it’s a squiggly math formula that lets us describe a whole lot of things. Today we care about things changing over time. In the case of our virtual neuron, that would be the evolution of its membrane potential over time. But simulating that requires us to understand some basics, so we’ll start with something simpler. Much simpler.

Let’s say we’re watching a cat that’s moving around. And because we have literally nothing better to do in this imaginary world, we want to describe the distance that the cat is covering over a period of 30 seconds. At the beginning of this silly “experiment”, that is, at time , the distance covered is 0 centimeters, because the cat is just chilling for now.

But as it notices that we’re staring, it presumably gets a bit creeped out, so it starts to slowly inch away from us. That’s great news, because now we can proceed with our observations: we see that after one second (), the cat has moved 1 centimeter (we will denote that as ), after 2 seconds – 2 centimeters, and so on, until, after 30 seconds, it went 30 centimeters away from us. We can visualize that below.

In formal math, we would write that . Why? It’s because for every unit of time (for every second in this case), there is an increase in the distance of 1 (centimeter). In other words, the change in distance () per unit change in time () is always 1. is also called the rate of change of x.

While we wrapped our heads around that, the cat got even more weirded out and now it’s actively trying to outrun us, so it’s actually picking up speed the longer we look at it. Since, like I mentioned, we’re really bored, this is even more interesting, so we begin a new observation period. This time, the cat starts again at 0, but after 1 second, it’s 1 centimeter away, after 2 seconds – 4 centimeters, after 3 seconds – 9 centimeters, and so on, until, at the end of our standard 30 seconds, it’s a full 900 centimeters away!

Again, in math, the pattern we observed above could be written as , and the corresponding differential equation would be . Now, maybe you’re wondering how you’re supposed to guess what the differential equations of functions are. Well, for now, you aren’t. For the more basic functions, you can find tables online. You can also use WolframAlpha in a lot of cases. And with time, you might pick up other tricks.

The equation of an oscillating cat

Coming back to the cat, the poor thing seems to have realized there’s no escape from this imaginary hell, so now it’s just pacing up and down the room, in a sort of oscillating pattern.

In this case, we’re no longer looking at the distance covered within a given timeframe, but at the position of the cat in the room (which we’re still calling ). Watching this GIF is quite cute, no? Well, brace yourself, because I’m about to hit you with the math-speak describing this: .

At this point, I assume that you might be asking yourself some of the same questions I did when I first started learning these things, such as:

• Where in the freaking fresh hell did this entire thing come from? Am I supposed to just…look at squiggly lines and spit out such formulas?
• ? ? Who the heck are these? And how do you even pronounce this guy: ? (It’s “nu”, sounding kind of like “new”)
• hyperventilates in math
• I want to know who is, but in order to do that, I need to know who is? BUT WHO IS ??
• I just want to get my computer to show me this graph, how am I supposed to go from elvish to code?

I get it. Honestly, just breathe. If you wanna shed a few tears or snap a pen, that’s also ok. After that, bear with me. We will answer all of these one by one, so we will need to dwell in uncertainty for a bit longer. Let’s start with the easiest part: where did the equation come from? Well… from the internet. And no, you don’t need to come up with them yourself (at least not yet). As you learn more about the topic, you will see that there are established models for neurons, brain areas, even the whole brain. Of course, if you wanna be hardcore, you might eventually come up with your own model or at least tweak an existing one. But we’ll get to that later.

Alright, but who are the Greek letters? And for some reason A? I know you’re going to dislike this, but they are pretty much whomever we want them to be. “But how do we choose???”, you’re probably wondering, crushed under the weight of responsibility. Well, I already picked out some values that look nice, but truth be told, if that weren’t the case, you’d have absolutely no idea what would be good values. So you’d just have to try out a bunch of them, until you found something appropriate. In fact, I encourage you to do so anyway. It’s at this point that I’ll share a secret with you: all the Python code for generating the figures in this article can be found here. And it’s written in a beginner-friendly way, so hopefully everyone can follow along.

It’s important to note here that in your journey towards building virtual neurons, brains, or what-have-you, you will often come across such parameters. In some cases, you will know what values to assign to them based on experimental measures or on other people’s work or on intuition. More commonly, you will have to resort to good ol’ guessing (there are ways to do so automatically and systematically, without having to change each value manually, look at a plot, try another one etc, but that’s a story for another time).

Practical solution to the cat equation

Alright, the spiciest problem remains: how do begin to solve this beast? How do we go from the mathy stuff to the pretty stuff? I have to mention here that some differential equations have what’s called analytical solutions – that means that you grab a pen and a piece of paper and you (somehow) solve the equation, basically as if you were solving . For instance, the first two examples above can be easily solved analytically. But as equations become more complex, analytical solutions are more difficult, or even impossible to obtain. In that case, we need to employ what’s called numerical solutions. Here’s how.

First of all, let’s be clear on what our objective is: we want to find the values of for a range of time . As you saw above, in our “experiments”, starts from 0. In order to be able to numerically solve our equation, we need to know what value has at time (this is our initial starting condition). How do we know? Again, we guess. Here as well, we can play around with this. You’ll notice that, depending on the equation, different guesses might lead to different results. There is a mathematical reason for that, which is a topic in itself, but just to give you a bit of intuition: imagine a hill surrounded by a valley on the right and a lake on the left. If you place a ball on top of the hill (your initial starting condition), one of three things will happen: it will stay there if you place it exactly in the centre, roll towards the valley if you place it slightly towards the right, and roll towards the lake if you place it towards the left. The places where the ball stops moving are called fixed points and their number can be different from 3 depending on the system we’re studying.

But coming back to our numerical solution. We now have a value for . In our latest cat example, let’s say we set . To find all other values of , we use an iterative approach. Because we know , we can now calculate by simply plugging its value, 0.5, into the differential equation from above (of course, together with the values we set for our parameters , , , and A): , which is approximately 1.03.

Now, let’s take a step back. We said in the very beginning that is the rate of change, how much changes from one time point to another. If we know that the cat was at position 25 at time point and at position 16 at time point , then the rate of change is how much the cat changed position (25 – 16) divided by the amount of time in which it changed its position (in this case, 5 – 4) or . Because we are always increasing by the same amount of time, we can call that amount of time . In a more general form, we can then rewrite the equation from before as . And from there, becomes .

With this is hand, we just need to pick a and then we can find for any value of , assuming we know (or guess) . As a rule of thumb, we want to pick as small as possible (you can imagine that if we were to look at the cat now and then 10 minutes later, it would probably be very far away from where it started, but if we looked after only 0.1 milliseconds, there would hardly be any change). For the example above, we set . is then equal to . Hardly any change, right? But repeating the same process for many steps (10.000 steps, to be more exact) gives us the graph above. In practice, how many steps you want to simulate depends both on what you’re interested in observing and on what your computer can handle.

End of part 1 & small exercise

If you’ve made it through the entire section above, congratulations! Now it’s time for a well-deserved snack and a break. Let these memories consolidate until next time, when we’ll finally tackle that virtual neuron. And to get in shape for the big fight, you can try to implement a numerical solution to the equation below in the programming language of your choice. Feel free to share the plots with the world, either in the comments or on your favorite internet platform.

Update: part 2 is now up.

What did you think about this post? Let us know in the comments below. And if you’d like to support our work, feel free to share it with your friends, buy us a coffee here, or even both.

Subscribe to our RSS feed here.

You might also like: