Sapere Aude
Tag: brain
Using the tools of interpretability, we give an unprecedented look into the rich visual concepts that exist within the weights of CLIP. Within CLIP, we discover high-level concepts that span a large subset of the human visual lexicon—geographical regions, facial expressions, religious iconography, famous people and more. By probing what each neuron affects downstream, we can get a glimpse into how CLIP performs its classification.
But it’s not just this flexible behavioral repertoire that’s so amazing. It’s not the fact that somehow, this dumb little spider with its crude compound optics has visual acuity to rival a cat’s (even though a cat’s got orders of magnitude more neurons in one retina than our spider has in her whole head). It’s not even the fact that this little beast can figure out a maze which entails recognizing prey, then figuring out an approach path along which that prey is not visible (i.e., the spider can’t just keep her eyes on the ball: she has to develop and remember a search image), then follow her best-laid plans by memory including recognizing when she’s made a wrong turn and retracing her steps, all the while out of sight of her target. No, the really amazing thing is how she does all this with a measly 600K neurons— how she pulls off cognitive feats that would challenge a mammal with 70M or more.
She does it like a Turing Machine, one laborious step at a time.
She’ll sit there for 2 hours, just watching. It takes that long to process the image: whereas a cat or a mouse would assimilate the whole hi-res vista in an instant, Portia’s poor underpowered graphics driver can only hold a fraction of the scene at any given time. So she scans, back and forth, back and forth, like some kind of hairy multilimbed Cylon centurion, scanning each little segment of the game board in turn. Then, when she synthesizes the relevant aspects of each, she figures out a plan, and puts it into motion: climbing down the branch, falling out of sight of the target, ignoring other branches that would only seem to provide a more direct route to payoff, homing in on that one critical fork in the road that leads back up to satiation. Portia won’t be deterred by the fact that she only has a few % of a real brain: she emulates the brain she needs, a few % at a time.
2022-06-24: Speaking of efficiency, the brain has a power saving mode.
When mice were deprived of sufficient food for weeks at a time — long enough for them to lose 15%-20% of their typical healthy weight — neurons in the visual cortex reduced the amount of ATP used at their synapses by 29%. Because the neurons in low-power mode processed visual signals less precisely, the food-restricted mice performed worse on a challenging visual task. The fact that these impairments in perception occurred long before the animal entered real starvation was unexpected.
A significant implication of the new findings is that much of what we know about how brains and neurons work may have been learned from brains that researchers unwittingly put into low-power mode. It is extremely common to restrict the amount of food available to mice and other experimental animals for weeks before and during neuroscience studies to motivate them to perform tasks in return for a food reward.
The question of whether single cells can learn led to much debate in the early 20th century. The view prevailed that they were capable of non-associative learning but not of associative learning, such as Pavlovian conditioning. Experiments indicating the contrary were considered either non-reproducible or subject to more acceptable interpretations. Recent developments suggest that the time is right to reconsider this consensus.
Last year, DiCarlo’s team published results that took on both the opacity of deep nets and their alleged inability to generalize. The researchers used a version of AlexNet to model the ventral visual stream of macaques and figured out the correspondences between the artificial neuron units and neural sites in the monkeys’ V4 area. Then, using the computational model, they synthesized images that they predicted would elicit unnaturally high levels of activity in the monkey neurons. In one experiment, when these “unnatural” images were shown to monkeys, they elevated the activity of 68% of the neural sites beyond their usual levels; in another, the images drove up activity in one neuron while suppressing it in nearby neurons. Both results were predicted by the neural-net model.
To the researchers, these results suggest that the deep nets do generalize to brains and are not entirely unfathomable. “However, we acknowledge that … many other notions of ‘understanding’ remain to be explored to see whether and how these models add value,” they wrote.
Eventually, the worms recoiled to the light alone. Then something interesting happened when he cut the worms in half. The head of one half of the worm grew a tail and, understandably, retained the memory of its training. Surprisingly, however, the tail, which grew a head and a brain, also retained the memory of its training. If a headless worm can regrow a memory, then where is the memory stored, McConnell wondered. And, if a memory can regenerate, could he transfer it? They had transferred a memory, vaguely but surely, from one animal to another, and they had strong evidence that RNA was the memory-transferring agent. Glanzman now believes that synapses are necessary for the activation of a memory, but that the memory is encoded in the nucleus of the neuron through epigenetic changes.
Earlier research had shown that these epigenetic changes can be inherited. Perhaps phobias are epigenetic?
Memories can be passed down to later generations through genetic switches that allow offspring to inherit the experience of their ancestors.
Mitochondria may hold keys to anxiety and mental health
Mitochondria seem to be central to the very existence of a stress response, serving both as mediators of it and targets for the damage it can do.
A patient at a British hospital played Mahler and Gershwin on the violin while a tumour was removed from her brain so that surgeons could preserve her ability to play music and honor her 40-year passion for the instrument. This takes “I’m fully awake during my surgery” to a new level.
individual neurons can do XOR
The march to specialized silicon, from CPU to GPU to FPGA to ASIC is now going further, to analog and quantum processing. At a high level, we are recapitulating our evolutionary computational march in silicon, and an ever-growing % of our compute will be massively parallel and in-memory processing, just like our cortex
Language is the scaffold of the mind
The lack of language affects even functions that do not seem to be intrinsically “linguistic,” such as math. Developmental research shows that keeping track of exact numbers above 4 requires knowing the words for these numbers. Imagine trying to tell the difference between 7 apples and 8 apples. The task becomes almost impossible if you can’t count them—and you can’t count them if you never learn that “7” is followed by “8.” As a result of this language-number interdependency, many deaf children in industrialized societies fall behind in math, precisely because they did not learn to count early on
Gregor J. Rothfuss 