How Your Thoughts Are Built & How You Can Shape Them | Dr. Jennifer Groh

The conversation opens mid-argument rather than with the usual preamble. Asked to elaborate on a definition of thought, Jennifer Groh offers a theory: thinking may consist of running partial simulations on the same sensory-motor infrastructure the brain uses for perception and action. The modal framing matters - she says 'might be', and shortly after 'the theory is that maybe', so the claim is explicitly tentative rather than a reported result.

the mental instantiation of that, or the brain mechanism instantiation of having that thought, is to run a little simulation in the visual cortex that kind of includes what a cat looks like. A simulation in the auditory cortex, what does a cat sound like?

Groh then tests the claim live. She notes that she has just used the word 'cat' and asks what color cat her listener is imagining. Huberman answers 'I'm thinking of a gray cat, but I keep smelling kitty litter', and explains that his sister had cats and that the smell of kitty litter is 'just so aversive' to him. Groh treats the unprompted extra sensory detail as evidence: he had 'no hesitation in telling me the color and adding an additional sensory quality.'

She extends the idea into an everyday argument about shared cognitive resources. Why, she asks, would a driver merging into difficult traffic tell a passenger 'Okay, be quiet, I have to pay attention now'? Her answer is that speech would not impair a visual-motor task unless both drew on 'a kind of cognitive system that's in operation'; the demand is to 'shift some resources away from processing the conversation and towards actually dealing with the here-and-now sensory-motor task'. The reasoning is inferential and hedged with 'maybe', not a described experiment.

Only after the cold open does the standard introduction arrive. Huberman identifies himself as 'a professor of neurobiology and ophthalmology at Stanford School of Medicine' and introduces Dr. Jennifer Groh as 'a professor of psychology and neuroscience at Duke University' whose laboratory 'studies how our brain represents the world around us'. He frames the episode around how the senses are merged so 'we can focus and learn more effectively', including how eye movements 'dynamically control what our brain is capable of'.

Huberman makes a strong evaluative claim about the definition of thoughts to come, calling it 'perhaps the clearest and most useful definition of what thoughts are and how you can control them' and, after 'nearly three decades' in neuroscience, 'the most compelling and useful one I've ever come across' at the neural, psychological and experiential levels. He says the episode will show how to use experience and knowledge of how thoughts are built 'to become a better thinker and indeed smarter'. He also states the podcast is separate from his Stanford roles, is meant to bring 'zero-cost consumer information' to the public, and that 'today's episode does include sponsors'.

The substantive question is posed as a gap in the podcast's own coverage. Huberman notes they have discussed vision, 'a little bit about hearing, a little bit about touch, smell, and taste', but never 'how the senses come together', which he calls 'critical to everyday life, critical to perception'. He characterizes Groh as 'an auditory-visual integration person' despite her focus on the auditory system, and asks where eyes and ears 'first come together' - offering the whistle of a tea kettle and a knock on the door as cases where 'we know where the door is' and 'where the tea kettle ought to be'.

Rather than answer directly, Groh asks to 'start at the beginning of when I first got interested in this question' and recounts being a college student at a school with no neuroscience major, where she and a couple of peers 'talked a professor into offering a seminar in neuroethology'. She signals that she will 'begin with the neuroscience nerdy lingo, and then we'll unpack it' - a useful marker that the technical description is about to be translated into plain terms.

there's a brain structure called the superior colliculus that's responsive to both visual and auditory stimuli, and that the responses to auditory stimuli depended on where the eyes were looking. If you moved the eyes, the neurons' receptive field, the region in space where they were responsive to, would shift as the eyes moved.

Two features made the finding decisive for her. First, auditory responses in this structure were contingent on eye position - a cross-modal dependency, not a purely auditory code. Second, the finding 'blew my mind' and 'set me on the track that I've been on ever since then'. The appeal was a puzzle: computing where a sound lies relative to where the eyes point is, as she puts it, 'easy for us to do with a pencil and paper' and 'very simple math'.

If you know that the sound is located, say, 10 degrees to the right, and your eyes are looking 10 degrees to the left, then that tells you that the sound is 20 degrees to the right of where your eyes are.

The tension she identifies is between trivial arithmetic and unknown neural implementation. Given what she then knew about how the brain represents spatial information, it 'seemed a big puzzle for how the brain might actually create these kinds of moving representations of where the sound is located'. Huberman names the concept: 'what you're talking about are dynamic maps.'

Huberman supplies the contrast case: most people 'probably appreciate that we have a map of our body's surface, a so-called homunculus', where stimulating a region produces 'the illusion of being touched at that location on the body', and where more sensitive areas like 'the fingertips or lips or face or feet' have 'the larger the representation in the brain'. What Groh describes is different - 'shifting maps depending on where the eyes move, and the eyes move quite a lot'.

Groh confirms 'They move quite a lot, exactly, and mostly we're not aware of this'. The unnoticed fact is that 'every time your eyes move, the visual scene is shifting massively on the retina', yet 'we don't even notice this'. Her inference is that this is 'an indication that the brain is doing a ton of computation under the hood to give us that perceptual experience' - because 'if we were just representing reality, the reality would be these massively shifting, smeared visual scenes'.

Huberman adds a related observation about speed: what intrigues him about the auditory and visual systems is 'the extent to which they can contract and dilate so fast'. He illustrates with a commute - walking toward 'a light rail or a subway' with 'sound going by me', sitting down and opening 'a book or a computer', or going 'into the phone', so that 'our visual world and our auditory world just go into a small box'. The expectation is that 'whatever we're looking at' will 'relate to the sounds that we're hearing in that small box'.

The remapping is triggered by a social cue: if somebody says 'Excuse me, do you have a ticket?' you know to look up. Huberman stresses that 'we take this for granted' and that 'all of a sudden, we can remap our visual-auditory world and all the context in milliseconds'.

Huberman asks whether this all happens in 'the superior colliculus, below our neocortex', i.e. below 'our kind of conscious awareness'. Groh declines the premise: 'gosh, I wish we knew where conscious awareness was' and 'I think that's an open question'. She also qualifies the structure's role: it 'is important in this story because that's where the research began', but 'it's not that that's where the binding of visual and auditory space is necessarily fully contained there and only there'. She calls it 'a much bigger problem' involving 'shifting your resources around' that 'happens in a few different contexts'.

Screens are her central example. The sound 'was never coming directly from the screen where you're looking at the visual image'; it is 'coming from somewhere else', perhaps earbuds whose signal 'is simulating what the location should be if it was really coming from the screen, but it's a simulation. It's not actually reality'. Huberman's gloss: 'we merge what we see with what we hear if it makes sense to merge them'. The merger is conditional - a tiny temporal offset between a moving mouth and its sound 'looks weird', 'looks totally weird' and 'It's uncomfortable', as with a video 'ripped this video from the internet' carrying 'a time delay'.

In a theater or on a computer, 'the sound is not coming from the screen' but 'from a speaker, which is projecting vertically', yet 'the sound is jumping around in your perception as different people on the screen from different locations on the screen are speaking'. Crucially, 'Whatever means the sound is being delivered to you is not changing as the different people are speaking'. Huberman restates the puzzle: the physical source stays constant - 'whether it's a computer or speakers in the room or a movie theater or earbuds' - while 'we can quickly move the sound with our eyes, or our eyes move the sound with our ears?'

Groh then amends the claim, an explicit limitation: 'it depends a lot on how the sound is mixed. They can put in some spatial cues, but if they haven't done that, then what we just said applies'. Her preferred demonstration is 'videos of actual ventriloquists working with their puppets', where the puppeteer makes 'our perception switch back and forth from their own face to the puppet face'. Huberman's paraphrase - 'Hey, Cornelius, how are you?' followed by the same person saying 'I'm doing great' - captures the alternation.

Three craft details behind the ventriloquist illusion - First, performers 'try to speak like this without moving their lips too much'. Second, 'There are certain sounds that you just cannot make without closing your lips in front', and that is 'really hard to fool people about'. Third, for 'a word that begins with a B or has a B in it', a performer 'might subtly just cover their mouth a little bit while they're making that B sound', a 'kind of misdirection like a magician would do' meant to 'throw your attention over to the puppet'.

This is the most likely candidate for the source of this sound, so I'm going to override what my ears are telling me to perceive the sound as coming from here versus here.

On whether this is learned, Groh answers 'Partly, yes' and then, pressed on whether children 'come into the world understanding how to merge sight and sound', 'Absolutely'. Her full position is stronger: 'It has to be learned, and it has to be continuously updated during the course of development until you reach your adult body size'. The reason is anatomical, given later in the exchange: 'a baby's head is about half the width of an adult's head', so 'that half-millisecond for me is a quarter of a millisecond for a baby, and it's going to change as they grow. So that's why you have to do all this learning'.

The physical cues are described precisely. 'The way we tell where a sound is coming from is by the physics of the world causing differential delays for the sound to arrive at one ear versus the other': a sound from one side 'will get to this ear before it gets to this ear, and it'll be slightly louder in this ear than in this one', partly because 'it's just closer to that ear' but also because of 'a kind of acoustic shadow cast by the head'. The head shadow produces 'a little sort of dip in the sound intensity cast by the shadow of the head'.

Groh prefers timing cues because 'they're really easy to calculate': 'if you know how far apart your ears are, and you know what the speed of sound is, then you can figure out what the delay is'. She notes she will 'take off my glasses to measure the distance between my two ears that way', and gives the bound as 'something like about a half a millisecond, is the largest delay you can experience' - 'for the difference between a sound here versus a sound here', i.e. 'from your right versus from your left'. Yet 'We can obviously detect much smaller sound separations than just totally left versus totally right', which she calls 'an incredible feat of computational power by the brain'.

The puzzle is sharpened by a neural limit: 'half a millisecond is less than the duration of a single action potential'. Huberman defines these for listeners as 'the electrical signals that neurons use to communicate with one another' and 'the fundamental ways in which our brain works. Without these, we're dead'; Groh calls it 'the fundamental medium of communication in the nervous system'. Processing sensory information 'faster than the duration of that minimum increment of firing' 'would seem totally weird'. Her cited explanation stays hedged: 'there's some research about how exactly this can be done', involving 'lots of neurons firing together and really precise synapses that cause minimal delay and very high temporal precision as the signals are going from one neuron to the next'.

Huberman probes a self-generated sound: snapping his right fingers, 'intuitively, I know that it's my right hand because I did it', but 'my brain expects that sound to arrive more quickly to my right ear than my left ear'. For something 'directly in front of me, right at my nose' with eyes closed, 'I know to look in front of me' and 'I know to expect it in front of me once I open my eyes'.

Asked about real-world mislocalization, Groh answers 'Absolutely. So if you have hearing loss in one ear only, then it's very difficult to localize sound' - but immediately adds a qualification: 'It's not completely impossible'. It would be impossible 'if the hearing loss was complete and if this timing difference and level difference were the only cues that we use'. The escape route is spectral: 'the ear has these little folds in them, and the folds filter the sound as it comes in and in particular, it alters the frequency content of the sound'. Huberman's reaction - 'So these little dimples inside my ears are useful for something' - is confirmed: 'They're useful, and your ears are different than my ears', so each person expects 'a slightly different kind of fingerprint of what the sound sounds like as a function of location'.

The discussion turns to people whose outer ears are damaged - Huberman mentions 'the people I know that roll jiu-jitsu, the wrestlers, their ears are always beat up' and that 'They basically don't have these folds', with Groh supplying 'Flattened?' Groh is explicit about the evidential limit: 'I don't know of any studies, but I think we can predict from first principles that they would have an initial deficit, and that very likely they would learn to adapt, and they would kind of learn their new set of ears'. The 'very likely' and the appeal to first principles rather than data are the notable limitations here.

Huberman asks why, if 'the auditory system is so sensitive', he doesn't 'really hear my own voice', and why talking 'out one side of my mouth' 'doesn't throw me off', while 'most people have the experience of watching themselves or hearing themselves speak, and it feels awkward'. His summary metaphor: 'It's like we cancel ourselves out while we speak'; 'when it's coming at us from the front, it's odd'. Both agree the feeling is 'awkward' and 'uncomfortable'.

Groh bounds her answer: 'I can say more about why they sound weird and less about why we experience it as kind of unpleasant', speculating that 'we're just so used to the way we experience it, that to hear it recorded is going to be unfamiliar and strange'. She then enumerates 'three things'. First, 'the recording is not going to capture the full spectrum of frequency content of your voice'.