Whenever you speak, you set off a pressure wave that ripples rudely through the air. Some fraction of this wave will eventually reach your victim’s pinnae—the flappy flesh things on either side of their head that everyone calls “ears.”
Remember those funnels cartoon characters stuck in their ears so they could hear better?
From the Looney Tunes cartoon Now Hear This.
Those cartoon funnels are, in effect, your pinnae, although nature has made yours fleshier. They work to try and capture as much of the soundwave as possible. They also reshape the soundwave and whatnot, which is pretty cool. But the important point is that they get sound where it needs to go: into your earhole.
Imagine, if you will, Dexter whispering into your ear. The resulting soundwaves will be shaped by your pinna and ear canal in interesting and good ways. The best ways, some say. Tremendous reshaping. Before they arrive at your eardrum. Your eardrum bends inwards and outwards, as one would expect, in response to the changing air pressure.
This is where our signal changes medium, and encompasses the story of the ear quite well. It is a series of transformations to the signal and the medium the signal uses for transmission. It was travelling through the air as a sound wave. Now it is percussion against your eardrum.
The pressure difference on each side of the eardrum pushes or pulls it. Its rim is fixed (kinda like a trampoline), so the membrane bows instead of moving freely. Its centre is attached to the malleus (seen below), which is moved along with it. Kind of like if you glued your feet to a trampoline.
In your middle ear is a series of bones designed to transform the high-travel, low-force movement of the eardrum to a lower-travel but higher-force movement at the stapes. These series of bones are called the ossicles or smth.
Higher force is required because the stapes pushes through the oval window into the cochlea, which is filled with fluid. Fluid does not compress as easily as air and thus requires more force to transfer the signal.
For the mathematically minded
Here Δp(t) is the pressure in the ear canal minus the pressure in the middle ear, measured at the same moment. A small mechanical model treats the eardrum and its load as a damped spring:
Mass resists acceleration, stiffness pulls the membrane back, and damping removes energy. Here xu(t) is displacement at the umbo: the point where the eardrum connects to the malleus. The displacement trace above comes from this simplified model.
The juice-filled cochlea is circular and kind of looks like a snail. Within the ear-snail lives the basilar membrane. Below you can see it in its natural rolled-up state. In the next panel I’ll unroll the ear-snail to illustrate what exactly its job is.
So, the stapes (you can see it all the way to the left below) pushes the juice. We have now transformed the signal medium again. The membrane will wiggle at a given point in response to the frequency the stapes is pushing. You can see this on the graph below. The membrane is primarily wiggling in the low tones section.
Very interesting! The membrane encodes the frequencies present in the signal. Evolution has conspired to develop hardware level compression.
Now we have a wiggly line, that wiggles in response to the frequencies present. How is that encoded and delivered to the brain?
We cannot send the wiggle itself to the brain. Nerves are not audio cables; they are telegraph wires, and rather poor ones. The only thing they can carry is a click, and every click is exactly like every other click. You can’t click louder. You can’t click longer. So your entire world of sound — every voice, every melody, every time your wife asks you to empty the dishwasher — must be spelled out in when the clicks arrive, which wires they arrive on, and how fast they come.
So: how do we go from wiggle-space to click-space? Holding on for dear life on the membrane at every point is a hair cell. So named for the tuft of stiff little hairs sticking out the top. The tips of neighbouring hairs are tied together by tiny strings, and each string is tied to a trapdoor.
When the membrane bounces, the hair tuft leans, the strings pull taut and yank the trapdoors open; on relaxation they slam shut.
The aforementioned juice is actually special juice — battery juice essentially. When the line wiggles and the trapdoor opens, the battery juice makes the voltage of the hair cell jump.
Our wiggles have transformed into voltage. Another transformation. It’s starting to seem inefficient. Evolutionary optimisation does not produce the most elegant of solutions, it seems.
Zooming into one hair cell (below) you can see how the voltage transforms into “clicks” — or “action potentials” as they’re called in the biz. Here we’re only looking at one hair cell. You can see the firing follows the voltage: the bigger the jump, the more clicks per second. It’s a probabilistic process, with each nerve having its own distribution over the voltage. Some fire at a whisper, others for a shout.
Zoom back out. Every place along the membrane has its own hair cell doing exactly this, each tuned to its own frequency — the cell you just watched writes a single row below. Watch the membrane’s movement print, moment by moment, into the picture the brain actually receives. Kind of. Not really. I simplified. I don’t understand it either, there’s feedback loops and shit. It’s all a big mess.
But this is good enough for a mental model!
It’s a Mel spectrogram baby. Who knew. Your brain has a hardware-level processing pipeline.
So — to recap the journey: