Beyond the walls of the manicured garden that is Apple, you’ll find the wild world of Linux machines, which offer freedom and customizability at the cost of smoothness. If you are not the sort of person who likes endlessly tinkering with software and hardware, you are probably better off living in the comfort of Apple’s way of life.

Computational modeling lives in a similar space of possibilities: at one pole there are modular software packages that allow the working scientist to assemble models as one might assemble a lego house. At the other pole there is the wide open world of custom code, which requires something of the Linux user’s tinkering spirit.

So there’s a trade-off between convenience and flexibility. A convenient system is one that typically requires the user to sacrifice a certain amount of freedom, and a flexible system is one that typically requires the user to sacrifice a certain about of stability.

Subscribe now

In part 1 of this series, I laid out a bird’s eye picture of the hyperreal trajectory that neuroscience seems to be on, in which models proliferate and come to command more attention than the phenomena being modeled. In part 2, I drew attention to a growing tendency to blur the distinction between mechanistic and statistical modeling, and how artificial neural networks (ANNs) have accelerated this blurring.

In this post I’d like to talk about “walled garden thinking”. It’s not a business model: most of the popular tools used by modelers are free and open-source. The cognitive walled garden doesn’t monopolize a user’s tech budget. Instead, it monopolizes a user’s imagination.

The road to hyperreality is paved with walled garden models. The ride is smooth, so why bother off-roading? One answer is that walled garden models typically lack a key quality: extensibility.

Extensible models are like drawings on paper. You can transform a pencil sketch into a more refined drawing by systematically adding detail. Non-extensible models are like drawings made with an Etch A Sketch toy. It’s a very convenient way to draw, since it boils down to left-right and up-down movements. There’s no messy business of erasers. However, this convenience comes at a price. In principle you can draw curves (or more accurately, jagged approximations of curves), but in practice you are better off using straight lines. Also, you don’t really have the ability to use a rough sketch as scaffolding for a more detailed drawing.

Walled garden thinking offers a convenient framework for modeling and theory. But it is non-extensible in roughly the same sense that an Etch A Sketch is. Certain trajectories are much easier to navigate than others, given that only two basic strokes are available. By contrast, the number of ways you can mark paper is virtually infinite. This also means that paper is a relatively open medium. You can add ink or paint to a pencil sketch without too much difficulty.

The non-extensible nature of the artificial neuron

Nowadays, much of the non-extensible modeling in science takes the form of Python packages like Tensorflow, Torch and JAX1. Someone who is accustomed to these Python packages might think that I have it backwards: they might point out that these packages empower the user to make vast models that would be prohibitively laborious to set up using other frameworks. This is true as far as it goes, but scale is not the only form of extension that neuroscientists care about. A concrete example or two will illustrate what I mean.

Consider the basic unit of most neural network models: the neuron. ANNs are built up of artificial neurons, which are loosely inspired by the properties of real biological neurons.

An artificial neuron takes in a set of input quantities, sums them up in proportion to their connection strengths or “weights”, and performs a simple mathematical operation on the weighted sum. That final operation, defined by an “activation function”, determines the output that the neuron dispatches to neurons elsewhere in the network.

That’s it.

Zoom in on a state-of-the-art model like GPT or DALL-E, and eventually you’ll find simple artificial neurons, each performing a weighted sum of their inputs and then transforming that sum with an activation function. Most of the magic emerges from the weighting of inputs to each neuron in a network, which is gradually adjusted through exposure to vast quantities of data — this is the learning part of “deep learning”.

If you are looking at a biological neuron through a very blurry conceptual lens, you’ll discern some of the processes that artificial neurons perform. A biological neuron does sum up its synaptic inputs. Its output is causally related to the total input, and the activation function approximates that response.

When you switch to a sharper lens you’ll see that the biological neuron has a lot more going on. All the equivalents are partial. But, let’s focus on an aspect of neurobiological activity that has no equivalent in standard artificial neurons: temporal dynamics.

Artificial neurons have no temporal dynamics2. They sum their inputs and generate an output within a single computational time step, which essentially means the process is instantaneous. At each time step, the slate is wiped clean: artificial neurons do the same thing again with a new set of inputs. They show no trace of prior inputs.

In a biological neuron, input summation depends on the location and timing of the input. Unlike artificial neurons, which are abstract points living in mathematical spaces, biological neurons are three-dimensional physical objects. In most types of neurons, inputs arriving at closely-spaced synapses are more likely to sum together strongly than inputs arriving at distant synapses. Similarly, inputs arriving in close temporal proximity typically sum together more strongly than inputs that arrive at very different times.

This is textbook neuroscience, so here’s a diagram from an online textbook:

A more interesting and underappreciated aspect of temporal dynamics is history-dependence. The response of a biological neuron to its inputs can be strongly affected by its prior history. And this influence does not require any synaptic plasticity. One of my favorite examples of neural dynamics is rebound excitation (also known as rebound depolarization). Some types of neurons can produce firing as a result of inhibition. Specifically, after inhibitory input has been removed, the neuron’s activity transiently jumps up.

So the response of a neuron at a particular moment in time can depend on what it was “doing” at an earlier time.

Some neurons start firing in response to a positive input and continue to fire for a while after the input is shut off. Here’s an example of this sort of intrinsic persistent activity:

Rebound firing and persistent firing are just two examples from a vast zoo of complex temporal patterns shown by biological neurons, none of which can be directly captured by a typical artificial neuron. And because modern ANN infrastructure is based on artificial neurons, there is no straightforward way to add temporal dynamics without gumming up the works. In particular, the backpropagation algorithm used to modify the weights in an ANN requires that the artificial neurons behave in very simple ways. Backprop does not work well — or at all, really — with the kinds of temporal dynamics that I’ve described here.

Neuro-equivocation to the rescue?

Someone with good knowledge of ANNs may point out at this stage that there are excellent ways to model time series data using networks rather than individual neurons. Recurrent neural networks, including long short-term memory (LSTM) models and state space models (SSMs), are ANNs that can imitate many interesting temporal phenomena. So if you are primarily interested in creating models that mimic data, there is nothing wrong with saying that a single biological neuron is “equivalent” to a whole network of artificial neurons.

But now we are far removed from mechanistic modeling. In part 2 I said that one of the problems with the use of ANNs is a form of equivocation. ANNs are sometimes sold as statistical pattern-match devices and as models of real brains, but in an inconsistent way that depends on who the audience is and what the theoretical — or, arguably, polemical — goals are. This is not flexibility or epistemic pluralism3, but a two-card monte trick.

Neuroscientists don’t really need black-box models for individual neural activity: electrophysiologists and computational modelers have uncovered quite a bit about the mechanisms that produce these temporal dynamics. The diagram above about persistent firing indicates that sodium (Na) currents are involved in the phenomenon. Similarly, we know that many forms of rebound excitation involve calcium currents. Your ANN may be a perfect mimic of the dynamics of a single biological neuron, but it will tell you essentially nothing about the biochemistry of the phenomenon of interest.

“But maybe we don’t need all that biological complexity.”

This is the bizarre response one hears from the subset of ANN folks who want their models to be all things to all people4. A lot of work is being done by “we” and “need” here. The assumption seems to be that if ANNs can offer human-like performance on tasks, or even better-than-human performance, then the messy biological details might be irrelevant to “the” problem of understanding intelligence.

People who think in this way seem to have no idea that neuroscientists have goals other than emulating the alleged end-results of neural “computation”. There are all kinds of questions that neuroscientists might want to answer that go beyond imitating what biological networks do to inputs. For example, we may be interested in neuropsychiatric questions that turn on mechanistic details about ion channels in cells. Specific pharmaceuticals may affect sodium channels, so perhaps “we” really do need that particular instance of biological complexity.

Perhaps more importantly, we should be very skeptical of the notion that “what biological networks do” is a settled question. Viewed through the spectacles of machine learning, human behavior is reduced to a series of input-output transformations. In this framework, we take in “data” and spit out “answers” of some sort. But there are so many assumptions buried in the standard operating manual of ML, and asserting that they hold for brains leads to open questions in neuroscience and psychology.

Consider the example of visual pattern recognition. Both ANNs and humans can learn to classify animals as cats or dogs. In the ANN version of this task, the model is “shown” a very large number of static photographs of cats and dogs, each of which has been labeled as a cat or a dog. The images are randomized and then shown to the model, and over the course of multiple epochs of repetition with the same dataset, the ANN eventually learns to classify new images with human-level accuracy. This randomization is important for preventing forms of statistical bias.

How do humans learn to categorize visual experiences?

To start with, human vision does not involve processing static images. The signals sent from the eye to the brain are not like photographs: they form a stream of ever-varying neural activity. Laypeople and even some researchers liken this stream to a video, but this is highly misleading: a video is a discrete sequence of still images.

Humans can also choose where to look, which adds a whole level of complexity to the process of labeling what is seen. You might be told “that was a dog” several moments after your attention has moved elsewhere. Also, the history dependence we saw at the single-neuron level has analogues at the perceptual level: what you see can depend in striking ways on what you were looking at earlier. Afterimages are a simple example, but they’re just the tip of the iceberg5.

The way experiences are “sampled” is also very different in humans an in ANNs. Humans don’t experience randomly intermixed examples of dogs and cats. But this mixing is crucial to the ability of ANNs and other ML models to learn categories. Humans can continually learn new categories without having to revisit old ones, whereas state-of-the-art ANNs struggle to display this skill.

So if you want to assert that ANNs serve as a detailed mechanistic model of how humans learn to categorize visual patterns, you have to answer questions of the following sort:

1. How does the continuous stream of visual signals from the eye get transformed into quasi-static image patterns? And where exactly are these patterns to be found in the brain?

2. How does the brain enable a person to correctly assign a label to something, given that the label (“that’s a cat”) is not always experienced at the same time as the visual experience of the object (the cat) being labeled?

3. How does the brain learn new patterns continually without messing up older patterns? Does the brain somehow randomize its experiential “data”? If so, how?

It is normal for engineers to ignore such questions. Their goal is to perform specific tasks given specific constraints. Given that most of these tasks involve computer-readable data, it makes perfect sense to approach them as input-output transformations. Similarly, if your scientific goal is to create a statistical summary of data, or even a prediction machine for specific types of data, then you are perfectly justified in ignoring phenomena that leave no traces on your measuring devices.

But the field of neuroscience as a whole cannot ignore these sorts of questions. Regardless of how impressive the performance of an ANN is, neuroscientists should not lose sight of brain-specific phenomena — many of which are very hard to even articulate using a standard ML framework.

And that gets to the core of the problem with walled garden thinking. It dazzles you with impressive results, inviting you to adopt the methods that led to those results. But those methods are an awkward fit for many of the questions that the field needs to ask. A mind that has been “spoiled” by a convenient modeling framework may view certain questions as not merely difficult, but nonsensical. They become illegible from the perspective of the modeling framework.

General-purpose computational modeling, on the other hand, is far less restrictive at the conceptual level (though it may be inconvenient in various other ways). Once you know how to implement one sort of differential equation, adding others does not require a major leap in thinking. And you can bridge low-level molecular mechanisms with behavioral data without too much hassle, progressively adding interesting temporal dynamics as the need arises6. Hence the analogy with drawings on paper. You might not know exactly how to add water colors to your pencil sketches, but the medium, the paper, does not offer that many barriers to extension.

When you allow your sense of theoretical possibility to be constrained by computational expediency, your attention is actively drawn away from the places where your model does not align with the phenomenon being modeled. You color inside the lines drawn by software, rather than by scientific curiosity. In extreme situations — like the one captured by “maybe we don’t need all that biological complexity” — you decide that the real phenomenon fails to live up to the purity/clarity/optimality of the model, making negligence of it a sign of virtue.

In the next part of this series, I’d like to return to the analogy of sketching, using it to explore how computational scientists can use models to draw their attention closer to real phenomena in all their richness, detail and partial legibility.

This essay is part 3 of a series on neuroscience and the hyperreal.

Part 1 provides a bird’s eye perspective on the hyperreal tendency in neuroscience.

Part 2 explores the difference between forward and reverse modeling, and how some people use ANNs to blur this useful distinction.

Notes

The concept of hyperreality comes from Jean Baudrillard’s 1981 book Simulacra and Simulation, a collection of impressionistic vignettes about how society has become enamored of models.1

Baudrillard was diagnosing a cultural syndrome that prioritizes representations — theories, models, pictures, movies, stories, artforms — over the things being represented. He seemed to be saying that a threshold has been crossed: representations are no longer used for their traditional purpose, which is to allow people to refer to things, processes, people, phenomena, and in doing so contend with those things. In a hyperreal condition, representations become ends in themselves, enrapturing society so completely that they replace the things they represent.

So Baudrillard is saying we are in an era of representations without referent. Maps without territories. I'll grant that this sounds a bit hyperbolic, but sometimes one has to exaggerate a description of something to draw attention to it.

I believe there is a great deal of ambiguity and equivocation surrounding models, despite — or perhaps because of — their ubiquity in science. So in this post and the next one, I’d like to lay out a couple of useful lenses for viewing scientific models.

Forward and Reverse Modeling

A scientific model is a representation of a natural phenomenon. The idea of creating analogues of phenomena has been around for thousands of years. Ancient architects and sculptors probably used small material models to plan out larger works. And until fairly recently, tangible models were a standard part of many scientists' toolkits. Francis Crick and James Watson, for example, used ball-and-stick models of molecules to uncover the structure of DNA. Nowadays, models are made of mathematical rather than physical parts, and since the math is typically too complicated to calculate by hand, they are also made of computational parts.

It's handy to think of two broad classes of mathematical modeling: "forward" modeling and "reverse" modeling. The systems biologist and mathematician Jeremy Gunawardena defines these two categories in his paper "Models in biology: ‘accurate descriptions of our pathetic thinking’":

Reverse modeling starts from experimental data and seeks potential causalities suggested by the correlations in the data, captured in the structure of a mathematical model. Forward modeling starts from known, or suspected, causalities, expressed in the form of a model, from which predictions are made about what to expect.

The Ptolemaic model of the solar system is a good example of a reverse model. Over the course of millennia, stargazers in the Fertile Crescent accumulated a "database" that astronomers of classical antiquity used to create an efficient system for predicting the positions of the sun, the moon, and the planets. The model was very accurate, but it was not framed as a consequence of underlying physical phenomena. For this reason, there was no real way to use the Ptolemaic model to say something about phenomena other than the solar system.

A Newtonian account of planetary motion is an example of forward modeling: it starts with the universal law of gravitation, using it to derive the specific trajectories of two or more celestial bodies. But the applications of Newtonian thinking are far more general than this: it can be used to model terrestrial motion, as well as the motion of artificial objects such as projectiles or satellites.

So a forward model moves from causal hypotheses (or scientific laws) to observations, whereas a reverse model moves from observations to causal hypotheses.

In neuroscience, the Hodgkin-Huxley model is an example of a forward model. It represents the mechanism of action potential generation in terms of well-established physical and chemical laws pertaining to electrical phenomena, adding crucial phenomenological “placeholders” to capture the distinctly biological aspects of the phenomenon. The model has been so successful at capturing the voltage dynamics of cellular membranes that it is sometimes treated as the ground truth.

An example of a reverse model is the "Poisson neuron" model, which frames a stream of action potentials from a neuron as a series of random coin-flips. For many purposes, this is an adequate model: when cortical neurons are firing at a low rate, their activity does look a lot like a series of coin flips. If you start with the data — low-rate firing from cortical neurons — then the coin-flip model is the simplest way to capture the unpredictable nature of low-frequency spiking.

The coin-flip model of spiking is pretty much the bare minimum of a mathematical model: it posits a probabilistic phenomenon and nothing else. So it is purely descriptive, and doesn't really lead to causal hypotheses. Most reverse models do a little more, enabling researchers to find "statistically significant" differences in datasets. This kind of modeling is a mainstay of research in biology and psychology: it is how scientists decide whether there is a stable phenomenon in the first place.

For example, two groups of people who have different variants of a gene might be compared along some dimension, such as whether or not they have a psychiatric disorder. If the difference between the two groups crosses some threshold of significance, researchers tend to infer that the allele that is "associated" with the group that has the disorder might play a causal role in that condition.

As these examples suggest, many reverse models are also statistical models. In the previous post I mentioned the distinction between statistical and mechanistic models. Mechanistic models encode the mechanisms that the modeler thinks are instrumental in bringing the phenomenon about. So mechanistic modeling overlaps quite a bit with forward modeling.

Statistical models are mechanism-agnostic, seeking the simplest mathematical forms to characterize the data without mediation by physical and ostensibly "real" causes or tendencies. A good statistical model can help generate mechanistic hypotheses, which in principle can be used to create a forward model. In practice, however, this sort of feedback loop between forward and reverse modeling rarely happens.

One step forward, two steps back?

When I give talks2 about modeling, I often use the forward/reverse distinction, which I typically map onto the mechanistic/statistical distinction. But as I started putting this essay together, I realized that the alignment between these two distinctions is slipping. This is because artificial neural network (ANN) models, which are becoming more and more popular, are both reverse models and mechanistic models.

ANNs can be used to extract patterns from any sort of dataset. So in any field other than neuroscience, ANNs are unambiguously used as reverse models. But within neuroscience, it is tempting to view them as mechanistic, given that they embody highly abstracted versions of neural causality.

Earlier I suggested that a feedback loop between forward and reverse modeling might be a good thing, so perhaps a modeling framework that combines the two approaches is a good thing? A dialectical synthesis of the two? A tech-driven disruption of staid old patterns of thinking?

I doubt it. There are several reasons to avoid short-circuiting the separation between forward and reverse modeling in neuroscience. The general tendency with machine learning (ML) approaches is to make data the primary focus and demote mechanistic thinking to a secondary concern — if it is considered at all.

Neuroscientists who like to use ANNs as mechanistic models might respond with the idea that every model of a complex phenomenon involves simplifications. In any case, aren't simplifications the whole point of science? The fact that ANNs leave out certain empirical details might be a feature rather than a bug.3

I agree with the general argument — up to a point. Yes, all models need to simplify and abstract reality. But not all simplifications are the same. Some models simplify in a way that leaves open the possibility of progressively adding nuance and detail — they often provide pointers for how to do so4. Other models are structurally rigid, foreclosing this possibility. They typically compensate for a lack of flexibility with some combination of convenience and technical efficiency.5

Thanks to their widespread adoption by the tech industry, the software packages used to build ANNs are now as standardized and easy to use as Lego blocks. The word "pipeline" is often used by modelers in this context, conjuring up the invisible reliability of good plumbing systems.

Despite the fact that these software packages are typically free and open source, their effect on researchers bears a certain resemblance to that of the walled garden philosophy of many tech companies, most notably Apple: "Do things our way and you'll find that everything goes smoothly. But try to color outside the lines and you'll be sorry."

To a first approximation, the only mechanistic ideas embodied by ANNs are the ones that were already baked into their architectures back in the 1980s. Many mechanistic hypotheses are illegible from the perspective of ANNs and other data-centric approaches. Modelers have an incentive to avoid adding new mechanisms to ANNs beyond simple variations in connectivity, because such additions make it harder to avail of all the nice smooth pipelines.

So neuroscientists now have access to models that are powerful from a statistical point of view, but are not very mechanistically extensible. This might not seem like a bad trade-off, given that the field was already using black-box statistical models prior to the ML revolution of the 2010s.

But I don't think that convenience is adequate compensation for the rendering of certain scientific questions unaskable. Overreliance on non-extensible modeling may also contribute to the hyperreal proliferation of models, since a new model needs to be trained whenever a new experimental technique is invented, or even when data from an existing technique drift “out of distribution”6 . So in the next post I'll make the case that at least some computational neuroscientists should resist the siren call of reverse-yet-mechanistic chimeras and pay more attention to extensibility.

Notes

“… the logical evolution of a science is to distance itself increasingly from its object, until it dispenses with it entirely: its autonomy is only rendered even more fantastic – it attains its pure form.”

Jean Baudrillard, Simulacra and Simulation

When I think about the state of neuroscience these days, I can't shake the feeling that something is off.

The brain is becoming hard to discern in a lot of neuroscientific research. For a while I couldn't articulate how or why this might be happening. But then a single word popped into my head: hyperreality.

The French post-structuralist thinker Jean Baudrillard coined the term "hyperreal" to capture what he saw as a transformation underway in contemporary culture. In his 1981 book Simulacra and Simulation, Baudrillard defined the hyperreal as "the generation by models of a real without origin or reality".1

I would like to explore the idea that neuroscience is drifting towards a hyperreal epoch, when computational models may cease to be guides to reality. Instead, they will come to obscure and eventually "replace" reality.

Fleshing out this premonition requires explaining a few different moving parts, which I will do over the course of the next few posts.

In this introductory post, I'll sketch the broad contours of "neural hyperreality". In future installments, I'll pencil in some finer detail. I'll end the series with some suggestions for how the slide towards hyperreality can be countered by reframing the role of models in orienting scientific attention.

The rise of inexplicable models in neuroscience

To understand the road to the hyperreal we must trace the rising profile of models, which parallels the growth in the sizes of datasets available for modeling. These trajectories also mark the progressive blurring of useful distinctions between different types of model.

Each type of model in science has a different purpose and structure. In the next post I will explore a few different ways of contrasting model types, each of which draws out a different facet of the modeling process. Here I will introduce one of them: the contrast between statistical models and mechanistic models. Statistical models extract patterns from data, enabling the data to be summarized, classified, or fit using mathematical trend-lines. Mechanistic models simulate "mechanisms", which may be derived from well-established physics and chemistry theory, from coarse summaries of observations, or from speculative guesses about what is "actually" happening. Mechanistic models that don't have an obvious link to the physical sciences are often called "phenomenological models".

Ideally, statistical models tell you whether the data might be evidence of a repeatable phenomenon in the first place, while mechanistic models tell you how a phenomenon might arise from the interplay of real or hypothetical “forces”.

Mechanistic models have not been taken very seriously by experimentalists, who dominate the field of neuroscience. Experimental neuroscientists tend to treat mechanistic models as suspiciously speculative, if they are even aware of them. There is one very notable exception: the Hodgkin-Huxley model of the action potential, which is widely recognized as one of the most accurate and mechanistically insightful models in all of biology2. But for any scale larger than that of a single neuron, experimental neuroscientists have tended to rely exclusively on statistical models.

By the 2010s, datasets had started to grow rapidly in size and complexity, thanks to a crop of new experimental methods including fMRI and optogenetics. Many experimentalists found themselves confronting the limits of simple statistical models. More complex statistical models became feasible thanks to advances in computing power, but using these models correctly required a deep understanding of statistics.

To anyone who did have this level of understanding, it could often seem as if experimentalists were using statistical models as black boxes — data goes in, something mysterious happens, and publishable results come out. To add to the murk, every lab was using its own idiosyncratic mix of statistical models. Sometimes a research team would hop from model to model until they land on something "statistically significant"3.

And then deep learning burst on the scene, accelerating the proliferation of models.

Deep learning is a machine learning (ML) technique for training artificial neural networks (ANNs). ANNs have an unprecedented ability to perform pattern detection, classification, and function approximation. They seem to beat traditional statistical models at their own game4.

The success of ANNs doesn't come cheap, however. If you want the best possible results, you need to rely on the magic of scale. The performance of an ANN is often predicted by its size and by the size of the dataset used to train it. And the larger the model and dataset, the more computational power you need for training. This in turn makes state-of-the-art ANNs expensive and resource-intensive.

ANNs are also very hard to understand, even for the people who design them. There is no consensus about why they are so good at uncovering latent patterns, or why they fail in strange and unpredictable ways5. This has led to a whole cottage industry devoted to "explainability" and "interpretability" in AI and ML.

Inexplicable models are not really a new problem for experimental neuroscientists. The black box stance that many experimentalists were already adopting with respect to traditional statistical models is even more suited to ANNs, given that no one really knows how they work.

So to summarize, ANNs are efficacious, expensive, and enigmatic. And this combination of features generates an aura that makes the distinction between statistical and mechanistic modeling harder to recognize — particularly in a field where mechanistic modeling was never very popular in the first place.

Model muddle

Most ANNs used in ML are not really "neural" in a detailed mechanistic sense: they are based on networks of artificial neurons, and neither the neurons nor the networks they are embedded in bear a strong resemblance to their biological counterparts. And the techniques for training these models are even less biologically plausible than their architectures.

ANNs are nevertheless starting to be used in some circles as quasi-mechanistic models of the brain. The primary reason for this, apart from the fact that they are made up of artificial neurons, is that ANNs mimic human performance on a variety of tasks with far greater accuracy than previous generations of models that were more neurobiologically grounded. What this means is that for some neuroscientists, the primary metric to assess a model is statistical performance on a specific set of tasks, even if the model is otherwise implausible and/or impenetrable.

When ANN models match humans at the level of tasks, it is natural enough to ask if they match humans at the "neural" level too. For example, experimentalists have found correlations between activity patterns in image-processing ANNs and in the visual systems of the primate brain. There are various technical critiques that this approach has elicited6, but for our purposes the big problem is mechanistic-sounding extrapolation from such comparisons.

Researchers that are overly impressed with the quantitative metrics of statistical modeling — task accuracy and correlation with neural activity — may come to treat such metrics as a substitute for broader and more qualitative scientific goals, such as mechanistic understanding, explainability, and theoretical unification. These sorts of goals require a deep engagement with the experimental and theoretical literature.

Modern stats and AI tools nudge researchers in the opposite direction, making scholarly engagement seem old-fashioned and arduous. The tools are just so convenient to use! The popularity of ML methods in industry has led to a vast eco-system of software packages that are far more professional and user-friendly than the hacky tools that neuroscientists have come up with. These packages create a relatively frictionless experience that can induce researchers to search where there is available light, restricting their attention to the subset of experimental findings that are the most “machine readable”.

The hyperreal state will arrive if this process is taken to its logical conclusion: traditional conceptions of scientific understanding will then come to seem irrelevant in the face of the sheer power of the new models. Vast swathes of neuroscientific and psychological research will be ignored because they cannot easily be formatted as training data. In a hyperreal universe, many scientific questions will no longer be askable.

I think it is perfectly possible for neuroscience to avoid this fate. And this does not require rejecting any sort of model — so if you really love ANNs, rest assured that I am not insinuating that these models cannot be used productively in neuroscience. The risk of hyperreality does not come from any specific modeling approach, but from certain ways of using models to direct scientific attention. As I work my way to the end of this series of posts, I’ll offer some thoughts on how models can be used to enrich engagement with phenomena related to the brain and behavior, instead of displacing those phenomena in the minds of researchers.

Notes

Eric was a great teacher — especially of how to be skeptical of grand claims in neuroscience. He would interrupt his lectures on basic electrophysiology with wildly entertaining digressions, anecdotes3, and rants about the various “bogus” and “bodacious” claims made in neuroscience.

He condensed his accumulated exasperation into a list of informal “fallacies” in computational neuroscience. I’ve reproduced his text here (with a few small edits).

All this is to be taken with a sense of humor. I don’t mean any offense — though Eric probably did! This list serves as a handy set of tools for guarding against triviality and/or neurobabble, of which there seems to be an ever-increasing supply.

So here we go!

G.I.G.O.

Garbage in, garbage (or gospel) out. Well known problem with all computational models.

Unity Gain Simulation

You build an elaborate model of a simple phenomenon, including many redundant mechanisms to get out the phenomenon. If done well, you get the result that you wired in, but learn nothing, hence unity gain. If done poorly, reduces to G.I.G.O. True unity gain, although not interesting, is technically difficult, and requires some skill.

The Prayer Wheel

Building merit by turning cycles (on your computer). Your model is conceptually simple, and/or of little interest, but it required a supercomputer to run it. Legitimizing your model by emphasizing that it required the "world's largest computer" to run...

Two Card Monte

The street card game (shell game) played in academia: you sell the computer scientists that your model is an important contribution to biology, and you sell the biologists that your model is an important contribution to computer science. But it is neither. Works even better with more than two disciplines involved. The more remote they are, the better.

The Devil Made Me (Not) Do It

Your implausible and/or poorly constructed model (conveniently) can't be realistically tested due to computational complexity.

Pseudo Biological Detail (PBD)

Related to two card monte. Overload your model with irrelevant biological parameters and metaphors, in an attempt to direct attention away from conceptual weakness.

Bruno’s Lemma

When it is pointed out that the fundamental idea of your model strongly violates basic facts, you claim that your model can be fixed, since it is a model of the brain, and “the brain can do it”.

Proof By Sales Receipt

I'm a neuroscientist. I bought ten Macs (Suns, PCs,etc.) on my grant. Therefore, I'm a computational neuroscientist.

Klaus’s Transform

A colleague publishes a simple version of your complex mechanism. You then claim his simplifying idea as your own, and cite his simplification of your model as one of many complex models explained by "your" simplifying assumption.

Cargo Cult Neuroscience

Some island dwellers allegedly built plywood airplanes after WW2 in order to lure down cargo planes, which were no longer arriving after the war.4

"My network babbled like a baby while it was learning...." therefore my network is mimicking the learning process in a baby... See "totemic model" below.

Totemism

The totem is believed to (magically) take on properties of the object. The model is legitimized based on superficial and/or trivial resemblance to the system being modeled.5

The Hail Mary

After the football play of the same name: you have the ball, but no one is free downfield. So... close your eyes and hurl for the end zone.

You have a fragment of an idea, but don't know how to state, develop, or support it. But, if true, you will be famous. So, embed it in a far-fetched "model" and publish it, in the hope that your "model" will be proven "correct", somehow, someday.

Big Game Fishing in the Goldfish Bowl

You claim "predictions" for your model which are actually trivial and/or unavoidable properties of the nervous system: "A prediction of my model is... the existence of lateral inhibition in the pulvinar".

Neuro-Bagging

See carpet-bagging, as in the U.S. Civil War. (Thanks to Mark Rubin for this one.)

You assert that an area of physics or mathematics familiar to few neuroscientists solves a fundamental problem in their field. Example: "The cerebellum is a tensor of rank 10^12; sensory and motor activity are contravariant and covariant vectors". Related to 2-card monty (above), but distinguished by more extreme bodaciousness.

Scientific Pointillism

See Impressionist painting technique, e.g., used by Seurat, consisting of small dabs of paint creating the illusion of a figural painting.

Your "humanoid robot" is really a pair of active vision cameras and robot arms bolted together, with a flashy plastic body. The rhetoric describing it is laden with terms indicating its humanoid abilities: it "interacts" with humans (and even graduate students); it has emotions. However, when looked at more closely, the thing is supported by simple software that appears to be ten or fifteen years behind the state of the art. If done well, you can enjoy the feeling of metaphorically, as well as literally, being the "star of your own movie".

Are there any more fallacies to be added to the list in this golden age of Neural Networks?

“It’s hard to know what all this means.”

- Eric L. Schwartz, c. 2006 (Carefully transcribed by hand in my notebook.)

Notes

1

The pentagon of neuroscience. Here’s my version of Eric’s pentagon. I elaborate on the diagram in this essay.

Having said that, GPT, Midjourney and the other marvels of modern machine learning must give many neuroscientists — and particularly my fellow computational neuroscientists — a kind of envy. How is it that a bunch of engineers who probably can’t name more than four subcortical structures created software that emulates, and sometimes surpasses, the abilities of humans? Wasn’t that our job?

I think many of us will agree that the answer to that last question is no.

Given the tech and sci-fi bias of a lot of pop culture, it can seem as if imitation of human intelligence is the primary goal of computational neuroscience, if not of the entire field. Other closely-related goals are cognitive enhancement and technology-assisted immortality.

I think everyone interested in the brain — both researchers and the wider public — should pause to consider the sheer variety of interesting questions one could ask about brains and behavior. I think most people will find that only a fraction of these questions overlap with the ones being asked by researchers in AI and ML (or by people who want their favorite sci-fi concept to become a reality).

Large, Medium, and Small Questions

It is natural to assume that all neuroscientists are working to answer one big question: How Does the Brain Work?

I don’t deny this, but I think “how does it work” is far too vague a question. It skirts dangerously close to philosophical questions that tend not to get answered: they just get re-asked using the latest terminology. What is consciousness?2 Do we have free will? Can we upload our minds to the Cloud?3

At the other extreme from “How Does the Brain Work” we find a host of questions that are decidedly not vague. When you wander through the poster hall at your first Society for Neuroscience conference, you realize the sheer number of hyperspecific questions that can be asked. And at least partially answered, usually without any philosophical chin-scratching. Here are some parody poster titles4:

• “Effects of Boxytocin Antagonist Amatoran on the Grooming Behavior of Adolescent Mice Previously Administered Cyclocetraline-3”.

• “Optogenetic inactivation of FK4Tw7g-expressing neurons in the substantia nomenclatura pars ridiculata triggers aggressive chewing in prairie stoats”

• “Administration of anantamine to humans with nonspecific annoyance disorder leads to global aleph-nought synchronization”

Between the unanswerably broad and the squint-inducingly technical, there lies a rich zone of medium-sized questions that I think are both scientifically well-posed and of general interest. Also, they radiate outwards, spurring more specific experimental questions as well as philosophical speculations. Instead of moving in a “top-down” direction (a progressive increase of detail) or in a “bottom-up” direction (a progressive increase of abstraction), sometimes we can adopt a “middle-out” approach, widening and narrowing the scope of our questions with a flexible rhythm.

Sensory substitution — new senses for old

The BrainPort is a video camera attached to a multi-electrode “lollipop” that sits in the user’s mouth. With the help of this contraption, visually impaired people (or blindfolded journalists5) can learn to ‘see’ with their tongues.

Sensory substitution, also known as sensory remapping, seems to work quite well, but we don’t know how — which means we can zoom in with more specific questions. How do the signals in the tongue percolate through the brain to enable a visual experience? Which anatomical pathways are involved?6 What kinds of training are required? Can chemical or electrical manipulations modulate the effect?

The sensory substitution phenomenon can also point towards increasingly philosophical questions about perception. For example, do the signals from the tongue need to be routed to the visual cortex in order to trigger ‘seeing’? Or might these signals trigger ‘vision-manifesting” patterns wherever they are sent, as a result of their interrelationships rather than their location? Conversely, are there types of patterns we could send to the visual cortex that are not subjectively experienced as vision? Could we invent entirely new senses? And can we ever establish if tongue-generated vision is ‘like’ normal vision? What is it like to be a light-taster?

Fading percepts — do we perceive states, or changes?

The eye is often compared to a camera, but this creates a very misleading impression: that a sequence of static snapshots of the world is sent from the eyes to the brain. It has been known for decades that if we stabilize an image so that it is stationary with respect to the retina of the eye, then visual experience fades.

The exact mechanisms underlying this phenomenon are still not fully understood. One idea is that microsaccades — tiny eye movements that occur even when we think our gaze is fixed — evolved in humans and some other animals as a way to keep the world visually present. By contrast, it seems as if some animals only sense external changes in the world. Frogs are exceptionally good at flicking their tongues at passing flies, but they will starve to death when surrounded by dead flies — they simply don’t see them!7

But experimental work8 suggests that microsaccades might not be the whole story. In the lab, microsaccades do seem to allow images to reappear when they start to fade, but this occurs in conditions where the subject’s head is fixed, so they can’t move anything other than their eyes. But in natural conditions the head and the whole body can move. When the world lacks movement, we can just supply our own. But how are these movements calibrated? How do animals ‘decide’ when and how to move? These are testable, specific questions.9 But we can go in the other direction too.

Stabilization-induced image fading suggests to me that vision is a bit like the sense of touch: if you’re blindfolded, it is much easier to identify a texture by moving one’s hands over it rather than just leaving the hand stationary. Perhaps all senses work a bit like this. Perhaps consciousness is not about “snapshots” or “samples” of the state of the world, but about differentiating signals with respect to space and time, and then integrating the changes that happen to be useful in a given context. And perhaps raising attention to a very subtle process involves a form of targeted self-motion, or even virtual motion.

Hyperacuity — when the whole perceives more accurately than any of the parts

Any sensor used to pick up signals from the world has physical limits. The resolution of a CCD camera is constrained by the density and light sensitivity of its pixel sensors. To improve resolution, pixel density must be increased, so each pixel must be made smaller. If the pixels themselves are large, or sparsely arranged, there is not much that a standard10 digital camera can do downstream to improve the resolution of the final image.

But this is not the case with human vision. As with a camera, human vision is constrained by its low-level sensors: the rod and cone cells of the retina. The ability of these cells to discriminate adjacent patterns of light can be measured. This is known as acuity and is the biological analogue of pixel resolution: it determines the smallest visual object that we can see. But there is a type of visual information that humans can extract from the retinas that far exceeds the constraints of acuity: we can tell if two line segments are aligned with an accuracy that is up to ten times the acuity of individual cone cells in the retina. So the abilities of the organism as a whole exceed some of the limits one might expect if the visual system worked like a camera. This is known as visual hyperacuity: we can localize patterns of light that are separated by distances smaller than individual retinal cells can “distinguish”!

Visual hyperacuity is sometimes called vernier acuity, because it is the basis for our ability to use vernier callipers and similar devices to measure length: we are exceptionally good at telling if two edges are aligned or not.

We can measure how good humans are at this sort of thing, but we still don’t have a very good idea of how we achieve this. So the topic generates concrete questions for experimentalists and computational modelers. In what way do neurons cooperate to overcome their individual limitations? And where in the brain does this cooperation occur? Many related micro-questions are being asked; hyperacuity seems to occur in other sensory systems too.

And once again, we can go in the other direction. How is it that biology enables forms of synergy that differ so radically from our engineered solutions? The brain forms a loose hierarchy — so loose that the term “heterarchy” is probably more appropriate. Could the fuzziness of “levels” of processing in the brain somehow enhance performance, despite our experience that in tech, fuzziness tends to degrade performance? How is it that parts can get together to form wholes that display novel capabilities?

Meso-scale abstraction

For a few years I used to answer neuroscience questions on Quora. It was a great experience: sometimes people asked questions about the cognitive and microbiological benefits of very specific foods or activities. And of course there were countless permutations of the classic big questions about life, the universe and everything. But my favorite questions were medium-sized. I learned a great deal by attempting to answer them. Here are a couple of favorites:

• Why can most people identify a color without a reference but not a musical note?

• Is there any neurological basis for the belief that time seems to go by faster when you get older? Do kids really perceive the passage of time differently than adults?

I think asking questions like this requires a transformation of attention: you can’t be too focused on high-res details, nor can you just stop at rough sketches. Speaking of sketches, something like this seems to have happened at the dawn of impressionist painting. Artists like Claude Monet, Pierre-Auguste Renoir and even John Singer Sargent11, began to experiment with their attention, finding a spot between objective verisimilitude and solipsistic abstraction. They found medium-sized questions within the space of art.

As with painting, medium-sized scientific questions require that we find intermediate levels of abstraction: the fertile middle ground12 between hard-to-follow detail and metaphysical vagueness. This zone is a great starting point for exploration. I hope to visit more such questions in this newsletter.

Notes

Before long, the storytellers around the fire began to weave more complex narratives. Not satisfied with explaining how to set a trap or collect honey, our ancestors began to speculate. They told wild tales of animals and ancestors, of spirits and symbols, of wars and wisdom. Some of these stories snowballed into great myths: they were passed from generation to generation, first orally, and then using a startling new technology: writing. These myths posed big questions. How did the world begin? How did we get here? Why do suffering and death exist? And what should we do about it all?

Unfortunately, the most enchanting answers became tools to oppress people and stifle curiosity. After many centuries under the sway of myths, humans discovered a new way to engage with the world: science. Instead of narratives, science offered tools with which to predict and manipulate the world, and thereby to explain it. A cloud of unknowing was cleared by great intellectual revolutionaries. To this day, plucky bands of rationalists attempt to break the spell woven by the storytellers, hoping to usher in a golden era of peace, progress and prosperity.

The paragraphs above tell a myth about the origin of myths and the birth of science. But by this I do not mean that it is wholly untrue. I suspect that most people who valorize science believe in some aspects of this narrative, though they would be justified in calling it a gross oversimplification1.

Why would I use the term “myth” to refer to a story that I myself put some credence in? Isn’t “myth” a shorthand for a popular misconception? This pejorative connotation has always bothered me — I like mythology2. I view traditional myths as a kind of memory technology: they braid history, explanation, ethical instruction, and entertainment into a grand whole.

Rather than viewing myths as popular misconceptions, perhaps we can just think of them as popular conceptions. And if we can’t shake the negative connotations, maybe we can use the term “mythos” instead. This has somewhat more positive connotations, since it conveys creative world-building or mythopoesis.

A mythology/mythos is marked less by whether it is true or false than by its cultural and psychological uses. A myth serves as a map for navigating a vast cognitive space. This space could be the accumulated legends and beliefs of a culture, the fantastical visions of a single creator, or the sum total of all human knowledge3.

Reductionism as Science Fiction

Reductionism is the idea that in order to understand any phenomenon, we just break it up into the tiniest measurable pieces, and then understand the pieces. The pieces are the only “real” entities in the universe, and everything else is just the sum of whatever the pieces do individually. This worldview — that subatomic particles are the lego blocks and everything larger is “built up” from them — is so common that it may seem synonymous with science itself.

There is a loose sense in which the central reductionist claim is true: whenever we break a thing down, whether it is a rock, a tree or a person, we typically find that the amorphous remains are mixtures of the elements in the periodic table. And these in turn can be broken down into a handful of subatomic particles. But if I give you the appropriate quantities of carbon, hydrogen, oxygen, nitrogen, phosphorus, iron and the other elements central to life, you cannot make a person. You cannot even make most biological molecules, let alone cells or organs.

Here is the stage at which a reductionist often claims that one could build up (or simulate) a person “in principle” at some point in the future4. But that concedes the essential point: right now, the idea that humans are “built” or “made” from atoms is akin to science fiction, since by the epistemological standards of science, no one has actually demonstrated this process of building. We don’t even know how to sketch a blueprint!5

This implies that the reductionist narrative does not acquire value in the same way quantum mechanics or general relativity does6. In a narrow scientific context, narratives don’t really help much: we need theories, which tell us about the key processes underlying phenomena, and models, which allow us to predict and control phenomena. The bare assertion that everything is “just chemicals” does not help anyone develop a cure for cancer.

An earlier version of yours truly would have argued that this lack of utility means we should abandon reductionism. But that logic might be skipping a few steps. I still think reductionism is a dodgy worldview, but not because it is an inaccurate scientific theory or mathematical model. We should instead assess reductionism in its role as a science-inspired mythos. In other words, we have to examine it outside of narrow scientific contexts, with an eye on how it confers meaning and context to people’s lives.

The Not-Yet-Discarded Image

CS Lewis knew a thing or two about world-building. And not just because he created the Chronicles of Narnia. He also wrote a little masterpiece called The Discarded Image, which examines the world-image that developed in Europe during the Middle Ages. In this image, the earth sits at the center of a series of embedded spheres, one for each of the classical planets: the Moon, Mercury, Venus, the Sun, Mars, Jupiter, and Saturn.

Here Lewis captures a key aspect of medieval thinking:

In medieval science the fundamental concept was that of certain sympathies, antipathies, and strivings inherent in matter itself. Everything has its right place, its home, the region that suits it, and, if not forcibly restrained, moves thither by a sort of homing instinct:

Every kindly thing that is

Hath a kindly stede ther he

May best in hit conserved be;

Unto which place every thing

Through his kindly enclyning

Moveth for to come to.

(Chaucer, Hous of Fame, II, 730 sq.)

The book describes how this conception arose, evaluating it as an artistic creation and source of inspiration, as opposed to a failed scientific model. The medieval cosmos arose from a harmonizing of Greco-Roman mythology with Christian theology, mediated by astronomy and astrology. The synthesis was by no means straightforward, and took centuries to stabilize into an all-encompassing tapestry.

Most importantly, the medieval synthesis provided a framing narrative for individuals on earth. Their lives participated7 in processes that stretched out to the visible cosmos. This vision was particularly influential among artists and poets. In fact, CS Lewis points out that mathematicians, astronomers and astrologers were probably well aware that their framings were metaphorical, as are our own uses of “law” in the context of science:

On the common medieval view there were four grades of terrestrial reality : mere existence (as in stones), existence with growth (as in vegetables), existence and growth with sensation (as in beasts), and all these with reason (as in men). Stones, by definition, could not literally strive or desire.

If we could ask the medieval scientist 'Why, then, do you talk as if they did,' he might (for he was always a dialectician) retort with the counter-question, 'But do you intend your language about laws and obedience any more literally than I intend mine about kindly enclyning? Do you really believe that a falling stone is aware of a directive issued to it by some legislator and feels either a moral or a prudential obligation to conform?'

If we are dealing with mere metaphors, why should we care about which ones populate our framing myths? Lewis has a good answer.

On the imaginative and emotional level it makes a great difference whether, with the medievals, we project upon the universe our strivings and desires, or with the moderns, our police-system and our traffic regulations. The old language continually suggests a sort of continuity between merely physical events and our most spiritual aspirations.

I think of reductionism as the Not-Yet-Discarded Image. Interestingly, its metaphors of “building up” rhyme with older theological ideas: the reductionist world is composed of logically coherent atoms (which may be ‘material things’ or ‘bits of information’) that could have been assembled by a cosmic clockmaker, architect, or programmer. Such personifications are circumvented by careful use of the passive voice: the world “is built up” from leptons and quarks, but not by anyone.

Given that we cannot yet emulate the great Passive Maker except in very restricted circumstances, what use does the image serve? We have to examine “the imaginative and emotional level”. Reductionists do not live in a universe where every kindly thing “enclynes” towards its rightful place8. But their mythos offers other consolations.

One of them is an intuitively satisfying explanation for the intelligibility of the universe. If the universe is made up of parts that are finite in number, simple, and non-capricious, then we can feel confident that whenever we break unfamiliar things down we will eventually find familiar things. But there is more to reductionism than an expression of faith in rational analysis.

The Reductionist Image — especially when drawn with Deterministic Ink — seems to ward off magical spirits and divinities. The hope is that a worldview without these unaccountable causal interlopers can inoculate us against vain hope in miracles — especially given how such hopes are exploited by unscrupulous miracle-merchants. “Hard-nosed” reductionist heroes view themselves as waging a heroic war against sentimentality, superstition, and stupidity.

Now imagine what emergentism sounds like to the faithful reductionist. A whole that is different from the sum of its parts? An affront to the elegance and simplicity of the lego universe! Phenomena that cannot be predicted from first principles? A backdoor to let in inexplicable spirits! The Warriors Against Woo cannot allow any backsliding.

I sympathize with some of this. As with the medieval image, the reductionist image is not without aesthetic and ethical merit — as long as you value intelligibility and non-miraculousness.

Why then would I want people to discard the reductionist image? I think the answer is twofold: it does a disservice to the actual practice of science, and perhaps more importantly, it seems to devalue human agency and dignity, at least for some people.

Through a Glass, Darkly

Reductionism concocts a worldview out of the successes of analysis, which means “breaking up”, “loosening” or “splitting”. But this leaves out the complementary process: synthesis — putting things together. The breakthroughs of chemistry involved tight alignment between analysis and synthesis9. If we only had analysis, we’d be in a Humpty Dumpty situation: all the king’s horses and all the king’s men would not be able to reassemble the phenomena that we break apart. This is the state of play in most scientific fields. Geologists can chip away at a mountain, but they do not have the ability to create one. Molecular biologists can tell us the elemental composition of a cell, but that doesn’t mean they are able to start with those elements and create a viable cell. Our synthesis abilities only get worse as we move to more complex phenomena.

One might counter that we don’t really need to make phenomena from scratch if we “know” what they reduce to when smashed to smithereens. Maybe theoretical understanding takes other forms? Maybe it involves perfect prediction or simulation of properties?

Consider water — ordinary, ubiquitous, life-giving water. It remains the case in the 21st century that several important properties of water cannot be perfectly predicted using models that we “know” to be correct from analysis. In other words, the most accurate way to determine many crucial properties of water is to actually measure them. There are no theoretical shortcuts yet10.

This turns out to be rather common in physics and chemistry. Even approximate solutions are not guaranteed to be accurate (relative to the capabilities of our measuring devices). As the physicists Robert Laughlin and David Pines say in their paper “The Theory of Everything”:

… the schemes for approximating are not first-principle deductions but are rather art keyed to experiment, and thus tend to be the least reliable precisely when reliability is most needed, i.e., when experimental information is scarce, the physical behavior has no precedent, and the key questions have not yet been identified. There are many notorious failures of alleged ab initio computation methods, including the phase diagram of liquid 3He and the entire phenomenology of high-temperature superconductors (8–10). Predicting protein functionality or the behavior of the human brain from these equations is patently absurd. So the triumph of the reductionism of the Greeks is a pyrrhic victory: We have succeeded in reducing all of ordinary physical behavior to a simple, correct Theory of Everything only to discover that it has revealed exactly nothing about many things of great importance.

So reductionism misleads us about how theory relates to experiment, even within the domain touted as reductionism’s greatest success: physics. This can create deep misunderstandings about the history of science and the relations among disciplines, and can therefore bias both the people who do research and the people who fund it11.

Most practicing scientists with an eye on empirical findings are aware of the pragmatic limits of reductionism, so we might be tempted to say that the narrative is harmless. Let the heroes on the frontlines of the war on ignorance employ caricatures and half-truths. After all, isn’t all this for a good cause?

I’m not so sure. One’s sense of what is possible and what is important derive in large from the myths one subscribes to, rather than the Methods section of a technical paper. If one thinks that the microscopic is always more fundamental and more important than the macroscopic, one may miss out on revealing aspects of a phenomenon of interest. As Laughlin and Pines point out, science involves “art keyed to experiment” rather than a reductionist instruction booklet. And art requires imagination. If we are open to the possibility that some collective phenomena are not linear combinations of their parts, we may theorize, model and experiment differently.

The role of reductionism as an imagination-constraining myth is not the only problem though. I think the main problem is how it ties into value systems. If only subatomic particles are “real”, then are scientists who study other phenomena less worthy of respect? If everything I perceive is “unreal”, what meaning does my life have? If “real” causality occurs at microscopic levels below consciousness, is my feeling of volition a mere epiphenomenon? Some might choose (!) to soften the blow of reductionism with the help of Stoicism or Buddhism, but there are limits to the diffusion of such subtleties. I have encountered members of the general public who experience severe anxiety as a result of the suggestion that Science-with-a-capital-S has “disproved” free will12. I doubt saying "there is no self" would offer them much succor. Reductionism in the natural sciences can also give intellectual cover to social reductionism, which induces us to see everything in terms of ruthless individualist utility-maximization. Does anyone really believe that climate change, exploitation, and other systemic crises can be solved by just tweaking individual behavior, leaving larger institutions unchanged?13

So it seems clear to me that the reductionist myth is far from harmless: first, its a misleading map of most scientific fields, and for this reason it may stifle the imagination; second, it can trigger despair in people who look to science for some source of hope or meaning; and third, it can be used to justify apathy in the face of problems that require collective agency. But let me be clear: the problem with reductionism is not that it is a myth, but that it is not a very good myth. It has a few interesting elements — intelligibility and skepticism of miracles — but I think we can recover them in a modified narrative14.

I’ve suggested here that we should see myths as potentially useful fictions that map cognitive spaces and contribute to emotional and “imaginal” well-being. We do not discard mythic images in order to see the world “face to face” (which is impossible15). Instead, we create new images that, if not less distorted, are at least different in aesthetically and ethically enriching ways. Mythic images are lenses — we do not just see them, we see through them.

Notes

This sort of assertion is uncontroversial among the majority of scientists and philosophers, who probably see it as a restatement of physicalism or a pledge of allegiance to scientific methods. But when we investigate what exactly a ‘fact’ is, the situation soon becomes murky.

A fact is something that is known to be true. If there are no knowers, how can anything be true or false?

The roots of this way of thinking can be traced back to the dawn of western philosophy. Since the era of Plato and Aristotle, if not earlier, it has been routine to separate knowledge into a study of what exists (“ontology”), and a study of how we know what exists (“epistemology”).

I have always found this division of labor unnatural and bizarre. We could pause here to speculate about why this state of affairs arose and why it persists to the present day1. But my goal is to suggest that we now have a more integrated way of looking at knowledge: the invariance perspective.

Invariance means immunity to a possible change. It is also known as symmetry. The mathematicians’ framing might seem strange at first, but I think it helps us make explicit our unstated intuitions about symmetry. I sometimes think this framework is so convincing that it can displace earlier notions. So maybe you should write down what you think symmetry is, because you might not even remember your ‘pre-invariance’ perspective.

As is suggested by the name, invariance conveys some kind of resistance to variation. Mathematicians and physicists typically use the word ‘transformation’ for a process that causes a specific variation. Consider a square. It is a paragon of symmetry. But can we say something more specific?

A square is invariant with respect to several transformations. It looks the same if we rotate it by 90 degrees, 180 degrees, 270 degrees, or 360 degrees. We can also flip the square around the two perpendicular bisectors, or around the two diagonals passing through the center. These modifications to the square are all transformations, and they are the keys to the chamber of symmetry. The fewer transformations there are that render a shape the same as it was initially, the more asymmetrical it looks.

Here’s a little exercise: how would you describe the symmetries of a yin-yang diagram? What transformations leave it looking unchanged? Sometimes a combination of distinct transformations is required.

Any object or process that we can identify, however abstract, displays some degree of symmetry or invariance. Rigid objects retain their shape when we move them around. In fact we can use transformations to define rigidity. Rigidity means invariance with respect to forces — up to a certain magnitude. (Any real physical object will break, losing it’s rigidity, if you subject it to a large enough force.)

Analogies are a type of invariant. “Dog is to puppy as kitten is to cat.” Or “Sailor is to navy as soldier is to army”. These analogies are often written like this “A: B :: C : D” (“A is to B as C is to D”). We can say that the relationship, which isn’t explicitly named or described, is invariant to swapping out pairs (A,B) with (C,D). However complicated an analogy may be, you can boil it down to some relationship among concepts that is invariant with respect to replacing certain specific details. In a deck of cards, every card in each suit has a counterpart in each of the other 3 suits. So the concept of a flush is invariant with respect to the specific deck: in poker you can make a flush with spades, hearts, diamonds, or clubs.

Every scientific discipline involves the hunt for invariants: the stable patterns that show up, often fleetingly, against the incessant transformations of nature. My own field, computational neuroscience, might be described as a second-order quest for invariants: we seek not only invariants in neural and behavioral data, but also a mechanistic understanding of how organisms recognize and construct invariants in the first place.

Language and music reveal some very interesting invariants. For example, we can comprehend a sentence even when spoken by very different voices, or at very different speeds. And we can recognize the same tune played on different instruments, in different keys, and even with jazzy interpolations and syncopation. We still don’t have particularly good neuroscientific models of this kind of behavior2.

One of my favorite human invariants involves how we implicitly represent shapes. You can trace out the shape of the letter ‘A’ with your elbow, or your foot, even if you have never done this before. This shows that your implicit motor representation of the letter ‘A’ is invariant to the effector, which is the specific body part or device you employ as the tip of your virtual pen. This example also reveals how important transformations are: when we do things that are outside our usual habits, we might uncover new aspects of our implicit representations.

The human ability to construct invariants is not perfect: if you speed up or slow down an utterance too much, it becomes incomprehensible. It is safe to assume that natural invariants have limits. An ideal sphere is invariant to rotations around its center, but a physical sphere will have tiny scratches and indentations, so a careful observer might be able to tell that it has been rotated.3

The lens of invariance and transformation can thus be used to magnify the gap between idealized theory and practical know-how. By keeping track of not just the invariants but the specific transformations that accompany them, we get a sense of where the bounds of our claims are. And by learning how to actually perform these transformations, we move from the armchair to the laboratory (or factory, or kitchen, or studio).

So what does any of this have to do with “mind-independent reality”?

I think the word “independence”, unlike “invariance”, connotes disconnection or separability. When we pluck a fruit from a tree, it is independent of the tree: even if the tree were destroyed, the fruit would retain its identity. When a country declares independence from another, it is (in principle) breaking bonds of political and economic dependency, so that the former ruler’s existence ceases to affect it. When we say a machine has a certain number of independent moving parts, we mean that these parts can be detached from each other and examined individually.4

From this perspective, “mind-independence” is a meaningless concept: there is no way, even in principle, that we can slice off the mind from reality and examine the two separately. No one would be ‘there’ to verify our story! Mind-independent reality is often described as the “view from nowhere.” Perhaps it could also be called a “non-view from everywhere”?5

Without the concept of mind-independence, does scientific objectivity go out the window? Are we giving intellectual shelter to every flat-earther and conspiracy theorist?

Not really, because we have a concept that is far better than mind-independence: observer-invariance. Objective truths are invariant with respect to swapping out observers. The sun is not “real because it’s there even when no one is looking”, but rather it is “real because almost everyone can observe it and agree on at least some of its properties.” Instead of painting a picture of objectivity that demands the (impossible!) removal of minds from reality, invariance allows us to recognize that minds are the very foundation of objectivity.

I sometimes think that declarations of faith in “mind-independent reality” or “cold hard facts” are a way to avoid the hard work of understanding how the invariants of reality are discovered. It is convenient to imagine that facts are just floating out there independent of us — one can then savor the trappings of objectivity without any real awareness of the transformations that establish invariance.

Perhaps I am being hard on people who simply don’t have the time to get into the nitty-gritty of how we know, for instance, that masks help curtail the spread of COVID-19, or that global warming is real. But surely something about our public conversations needs to change if we are to improve trust in scientific institutions — and more fundamentally, (re)build an objective world in which we agree on basic facts. The alternative is a return to rigid social hierarchy, where truths are stipulated by our “betters”, and the rest of us accept them as baffling revelations.

No doubt, each person has a limited ability to understand how an invariant was discovered. But there has never been a better time for people to learn about the human transformations that engender objective knowledge: YouTube, for example, has excellent videos on scientific topics at various levels of detail and rigor6. Perhaps this is naive idealism, but I think that even a taste of ‘applied epistemology’ can make us simultaneously more humble about the limits of our individual know-how and more curious about how collective expertise is gradually constructed.

We live on the precipice of an epistemological apocalypse. Faced with crises that require coordinated global action, vast sections of humanity seek out excuses to do the opposite. They are succumbing to nihilistic skepticism, rejecting the painstaking work of experts in favor of self-serving nonsense. But we should refrain from simply dismissing such people as ‘stupid’. There are no simple explanations for how we got to this point, but one thing I’ve noticed is that imperiously distributing ‘facts’ from on high can rub people the wrong way. What we need is some degree of epistemological participation. We need people to see for themselves that their minds are not superfluous to objectivity, but central to it. This is precisely what “observer-invariance” can convey, and what “mind-independence” does not.

Notes

On November 30th, the machine learning company OpenAI unleashed a chatbot service called ChatGPT on an unsuspecting world. ChatGPT can produce impressive responses to user ‘prompts’.

Let’s ask it how it works. In the excerpts, the text in boldface is my prompt, and the rest is the reply from ChatGPT.

Explain in a list how chatbots based on large language models work.

1. A chatbot based on a large language model is a type of artificial intelligence (AI) that uses a natural language processing (NLP) system to generate responses to user input.

2. The chatbot's responses are generated using a pre-trained language model, which is a machine learning algorithm that has been trained on a large dataset of text. This allows the chatbot to understand the context and meaning of the user's input.

3. When a user types a message to the chatbot, the message is first passed through the pre-trained language model. This allows the chatbot to understand the meaning and intent of the user's message.

4. The chatbot then uses its knowledge and understanding of the user's input to generate a response. This response is typically in the form of a natural-sounding text message, but it could also be in the form of an image, a video, or some other form of media.

5. The chatbot's response is then sent back to the user, completing the conversation. This process can be repeated indefinitely, allowing the chatbot to engage in conversations with multiple users at the same time.

As you can see, the response is relevant and coherent. Non-experts will of course wonder whether it is correct. I am not an expert on machine learning but I know enough to recognize that this is a perfectly adequate explanation. I would not have been able to tell if it was written by a human or not. What is debatable is whether terms like “understand” and “knowledge” are appropriate for describing a glorified statistical model. We’ll get to this debate in a moment, and why I think there are more interesting conversations to be had.

ChatGPT is a way for users to interact with GPT-3.5, the latest version of OpenAI’s General Pretrained Transformer. It’s a type of artificial neural network trained on vast quantities of text. It’s job during training is to predict the next word, given its history of ‘experience’ with text. The way it retains a trace of this history is by using prediction errors to tweak its billions of connection weights (which are loosely inspired by the synapses linking real neurons in brains).

Is it impressive? Yes.

ChatGPT represents a qualitative leap forward for large language models (LLMs). To appreciate this, it helps to see how a previous iteration of GPT did. When GPT-2 came out, my friends and I were obsessed with it — largely because of how bizarre the outputs were1. It produced grammatical sentences with little coherence or structure. Even with scare quotes, no one would have used “understanding” to describe what it did.

Here’s GPT-2 attempting to explain how Hide and Seek works:

ChatGPT had no trouble:

ChatGPT is not just spitting out remembered text, however. It can generate sentences that have never been written before (as far as I can tell from Google string searches). It also maintains very impressive long-range order: it can write stories that are quite coherent, even though they tend to be limp and clichéd.

Amazingly, you can also get it to write explanations of technical concepts… in verse. The rhymes are often quite bad, and the scansion is terrible, but the GPT doggerel is still startlingly impressive. Here is one explaining the backpropagation algorithm, which is used to train artificial neural networks such as GPT itself:

And here is an explanation of Newton’s Law of Gravitation in the style of a pirate.

How about that joke at the end? Definitely not just a copy-paste from the internet.

Is it dangerous? Yes.

A lot of commentary about ChatGPT centers around the various ways it can be harmful. It is capable of producing accurate summaries of scientific concepts, but it can also drift into nonsense, and even make-up non-existent citations. Anyone using a model like this has to be extra vigilant to check facts and watch out for logical or ethical lapses. Vigilance is perennially in low supply, so I don’t blame people for thinking of this as another portent of the impending epistemic apocalypse. But ChatGPT can crank out acceptable essays on almost any mainstream topic, so teachers are worried that students will use it to cheat on homework2.

Also, despite some improvements in safety filtering, it is still easy to get it to generate text that is racist, sexist or bigoted towards marginalized people. So if — or more likely, when — such services become widely available, people will suddenly be able to produce textual horrors at industrial scale: harassment, spam, fake news, and propaganda. In quantity, such text can even alter search engine results, so Google (whose results have been declining in quality anyhow) will have its work cut out for it. And it’s important to note that many of the people who report problems with LLMs are unpaid volunteers3.

GPT can also generate computer code, which means it could serve as a tool for software engineers. Or, more worryingly, it could serve as a replacement for software engineers. Here I asked ChatGPT to generate python code which can produce a surface plot. Note the explanation of the code. Clearly this kind of technology has the potential to be used to teach people how to code.

I ran it in Google Colaboratory and it worked without needing any corrections:

Is it conscious? Is it sentient? Maybe this doesn’t matter.

Not too long ago, an employee of Google triggered a major controversy for suggesting that one of Google’s large language models, LAMDA, was conscious. His thoughts on this were larded with idiosyncratic religious ideas, tickling technophiles, technophobes and the philosophically circumspect in just the right zones.

With ChatGPT, however, I have not yet come across anyone claiming that the software is conscious. Instead, people are arguing about whether GPT’s demonstrably stunning abilities illustrate “real” understanding and knowledge, or “mere” statistical correlation. In my interactions with ChatGPT, I see no evidence of a personality or emotion, and only a thin veneer of common sense. The skills on display are both human-like and incredibly alien. It is the latter aspect that fascinates me.4

The sheer number of free parameters in a GPT-style model, combined with the nature of the training (predict the next word) suggests to me that there is no substantive commonality between how ChatGPT produces coherent text and how a human does. Humans do not learn to produce language by completing other peoples’ sentences: our language use is embedded in a wider web of nonverbal actions, reactions and experiences. Yes, humans are capable of confabulation, overconfidence, and mindless imitation.

But just because something walks like a duck and talks like a duck does not mean it’s a duck.

I think what is interesting about GPT and other large language models is not what they imply about artificial ‘minds’, but what they suggest about the nature of the human textual corpus.

A second Parthenon of words?

In Ardor, Roberto Calasso explores the strange world of Ancient Sanskrit texts. He points out that the anonymous authors of these texts left almost no material traces: no ruins, no temples. Instead, they left a “Parthenon of words”. For centuries, scholars have wandered through this Parthenon, trying to unearth old meanings and detect resonances with new ones.

Large language models may offer us radically new ways to explore the Parthenon of text that humanity has collectively produced 5. No single human being can read even a tiny fraction of the training data baked into the connection weights of GPT. Whether these weights constitute sentience or consciousness is a discussion that can wait until we have intelligent machines that seem to have agency: the set of quasi-stable goals and quirks that develop through embedded interaction in the actual world6. But I suspect that no mere software tool, however skilled it becomes at churning out text, will ever convince me of its sentience. And it is unlikely that companies with the resources to create such tools have any interest in machines that display agency: independent motivation might interfere with profit-maximization.

While we wait for our favorite sentient-AI movie plots to bleed into reality, we might be able to use models like GPT to gain new perspectives on the emergent structures latent in our corpus of texts. Human faculties find their funhouse mirror reflections in the LLMs. Lurking in these statistical pattern-detectors that share none of our evolutionary, cultural or experiential history, we find warped and cracked versions of memory, imagination, free-association, poesy, logic, and analogy, all stitched together with the ability to generate grammatical sentences. In my experience, ChatGPT seems capable of creating structural templates: this suggests that somewhere in its weights we may find implicit representations of abstract concepts. How else could it write roughly accurate descriptions of backprop (or Hebbian learning) in a variety of styles?

These implicit representations need not resemble our own explicit concepts or our unconscious neural representations7 at all — in fact, we may learn more from them if they don’t. It may be that GPT’s statistical ‘knowledge’, despite superficial resemblances to our own skill-set, is orthogonal to natural intelligence: the ‘machine unconscious’ may be a completely different path through the high-dimensional space that is our textual jungle. For this reason it may stimulate new abstractions in us. Future GPT-whisperers may be able to discern novel patterns in complex data, harmonizing two modes of intelligence to produce a new 'cyborg stereo vision'.8

For now, it may be easier to get a taste of unanticipated connections by examining the outputs of AI image generators. Tools like DALL-E and MidJourney produce art that can remix styles and subjects in dizzying combinations. These tools often use visual elements to “illustrate” common features of seemingly disparate ideas provided in the prompt. In the image below, the prompt I provided was “Polish Star Wars poster featuring an elephant”. In three of the generated images, it seems to have landed on a latent similarity between elephants and the AT-AT walkers from The Empire Strikes Back. I had no inkling of this when I wrote the prompt.

The Tower of Babble is a byzantine complex of ungrounded symbols9 that is too vast to navigate on foot. But with LLMs we can teleport into the labyrinth at random; perhaps we will find unexpected chambers and secret passages. ‘Prompt engineering’ — the skilled crafting of inputs into LLMs and other machine learning models — may become a sought-after skill. The best prompt engineers will be more like prompt poets, supplying the vitality and caprice required to find hidden structures in the labyrinth.

(Hopefully the Tower doesn’t come crashing down on our heads before we can learn something useful from it10.)

Notes

10

Bonus! I generated a story based on the Tower of Babble idea.

As it turns out, the very last post on my ‘personal’ blog was about this experience. Here’s a slightly edited version:

I just attended a symposium on music and the brain at Harvard called “Crossing the Corpus Callosum II”. The website subtitle is “Healing our Community through Music”, so naturally I had my reservations. I tend to steer clear of medical neuroscience and therapy-oriented psychology: the touchy-feely sides of my field. But music and neuroscience and a friend’s request for company got me past the snowbanks, the puddles, and the desire to sleep till 1pm and then order buffalo wings. And I’m glad I went. The talks were stimulating and informative — not once did I want to slink off. In addition to the neuroscience of music, for the first time I heard about the role of music in helping people recover from traumatic brain injury and in treating language disorders.

The brain data was more or less familiar to me, but the therapy research was impressive. There are stroke victims who develop aphasia — the loss of speech. They can still comprehend language though, so it’s a frustrating disorder — both for them and for their families and care-givers. A technique called melodic intonation therapy helps them start recovering speech by asking them to sing instead of speak. (If you’ve seen The King’s Speech you’ll know that people with speech disorders can often sing perfectly well. This has to do with fascinating right-brain left-brain asymmetries that I can’t get into here.) There was also a talk by someone outside the world of science and medicine. (He had never used PowerPoint. Ever!) A Juilliard– trained musician who has worked with Def Jam, he started a project call Musicorps, which helps US war veterans deal with their injuries by teaching them to play music. Music helps bring these severely traumatized soldiers back from the brink of suicidal despair. Quite inspiring.

Even more moving: the stories and videos of music treatment for autistic children. (Just found a Boston Globe article about some of today’s speakers.) Many autistic children never acquire the ability to speak. One of the conference speakers talked about an experimental therapy method that uses musical tones to get these children to start using words, again by singing them. It’s amazingly effective — in two months a 9 year old autistic kid was able to start saying a few words, like “hello”. Normally, if children do not develop language by this age, they never will. During the question session a woman asked if the treatment was available for non-autistic children with speech disorders. She broke down while asking the question — her four year old daughter has an unrelated brain disorder and cannot speak. She said that she wanted to hear her child say “mommy”. As scientists we often forget that people pin their hopes on new research. The panel had a few words of hope for her, although the therapy in question is still in the testing phase, and not available for the general public. A music therapist in the audience piped in at this point, saying she would get the woman in touch with someone who might be able to help. Applause all round.

~

An experience like today’s always increases my respect for the patients and their families, the researchers, the doctors, the therapists, the nurses, the counselors. Their patience and dedication should inspire us in our careers as well as in our relationships. And there’s another take-home lesson, at least for me. We should try to appreciate the abilities we take for granted. There are people who live under a veil of silence. There are people who cannot comprehend even their own emotions, let alone anyone else’s. And there are people from whom these abilities have been stripped away — by war, accident or disease. Those of us who are “whole” are given the opportunity every day to connect with the people around us — to talk, to empathize, to understand. Sometimes that last one seems all but impossible. But we have to keep talking, keep listening, and keep trying.

There was actually more to this experience. While at the symposium, I was overcome by a feeling that I was not helping anyone through my work. My research at the time — modeling the neural basis of timed behavior — was conducted in a spirit of nerd curiosity, with little or no thought for whether it would contribute to individual or social well-being. I felt vaguely guilty: I was not offering anything back to society (which, via taxes, was funding my curiosity). The musical therapy I was learning about seemed so much more rooted in real human concerns than my little network simulations.

I decided to do something. I felt as though it might be a little late to become a clinical researcher or therapist (and in any case I doubted my skill set would fit with that world), so I decided to engage in science communication. I wanted to do more than just report on existing research. I wanted to share my experiences as an explorer of interstices: the gaps between academic fields; the negative spaces surrounding positive knowledge; the elusive patterns of thought that crisscross disciplinary boundaries. My rovings in cognitive hinterlands had been useful to me, and perhaps they would be useful to others.

So during the symposium, I flipped to the back page of my notebook and wrote down a list of topics that I felt I might be able to say something interesting about. Here’s the list:

Over the course of the next several years, I worked my way through most of these topics (and others that cropped up along the way), initially on my own blog, and then as a columnist for the 3 Quarks Daily blog/aggregator. I also did a little neuroscience blogging, and a whole lot of neuroscience question-answering on the site Quora.

I called my new blog Axis, Praxis. I explained the meaning of this term in the first post:

I have a habit of juxtaposing words in nonsensical ways, for the “sound-sex” of it, as Stephen Fry invites us to do. For instance, well before I had an inkling of what “dialectical materialism” meant, I twisted the phrase into “analectic immaterialism”. Half the fun is then deciding what such a collision of words might mean. (Analectical immaterialism, if you’re curious, came to suggest a quasi-mystical political philosophy that combines Confucius with postmodernism.) “Axis, Praxis” was another lexical copulation of this sort. (For the full effect, introduce a gentle pause between the words, half way between colon and semicolon, and emphasize “Praxis” with the zeal of a Marxist and/or Evangelical preacher. I’ll explain why in a moment.)

“Axis, Praxis” refers to the relationship between the ways we arrange our information (axis), and the ways we then use this information (praxis). […]

So “Axis, Praxis” is about ordering the world in order to act in the world. The two processes are intertwined, because our actions determine what we know and how we (can) represent it, and our knowledge determines what we (can) do. This is vague enough to sounds a tad ambitious: it just might entail talking about all of human knowledge and behaviour (?!). But my goal is not to explain (or even list) all our axes of knowing and our praxes of change. I want to survey the space, look at illustrative examples, and see if anything interesting emerges from a view that is by necessity wide and fuzzy. In parallel, I want to see if nontechnical discussions have anything to gain from scientific language and metaphor. Can science offer axes around which to orient our day-to-day praxis?

These were the first few Axis, Praxis essays:

• Truth, Validity and Usefulness — on epistemology, logic and problem-solving

• Vision, the Master Metaphor — on how metaphors grounded in seeing capture various aspects of knowledge, emergence, and alignment among theories

• The Great Red Spot (or, When Can a Thing be Said to Exist?) — on why processes may be more important than ‘things’

• Science Fiction – The Shadows cast by Modernity — an essay that was requested by a friend working at Down To Earth magazine, where it was published. I used three visual metaphors to classify themes in science fiction: SF as lens, SF as mirror, and SF as kaleidoscope.

• Metaphor: the Alchemy of Thought — some musings on the centrality of metaphor in human thought, and in particular in scientific thought

In 2013, when the 3 Quarks Daily editors put out a call for a new set of Monday columnists, I submitted the metaphor essay. I was quite excited when I heard that I had been chosen. If I recall correctly, I happened to be at the Society for Neuroscience conference at the time, and I was in the midst of an extreme bout of imposter syndrome. Finding out that my writing was appreciated was a much-needed confidence booster.

So my Axis, Praxis ‘project’ shifted over to 3 Quarks Daily. Having a monthly deadline was a great discipline (thought I did occasionally and guiltily email the editors to say my piece was not ready yet). In retrospect I probably took the assignment — an unpaid one at that — more seriously than absolutely necessary: I did a fair amount of research for each post. I’m very proud of these essays, even though they all have flaws and are perhaps too long.

After a few years I felt as though I had nothing more to say — or perhaps I needed to learn new ways of saying things. Either way, I stopped writing for 3QD, and also cut down on my Quora answering (which was becoming a strange addiction).

The time spent away from consistent blogging was relatively productive. I organized a discussion group that has been very stimulating: it has enabled a little community to persist and even grow (thanks to Zoom). The emergence journal club was, unfortunately, not recorded, but the subsequent three series — (1) on dynamical systems in neuroscience, (2) on Stephen Grossberg’s book Conscious Mind, Resonant Brain, and (3) on AA Cavia’s Logiciel: Six Seminars on Computational Reason — are available on my youtube channel.

I’ve also had many interesting interactions on twitter since the pandemic started. Twitter is widely (and accurately) described as a factor in the increased fragmentation and superficiality of public thinking, but one particular series of debates actually induced my to flesh out my thoughts on neural representations. The representation tweetstorm even landed me an invitation to go to Dublin for a workshop on the topic, where I contributed my two cents and had a great series of conversations.

The ephemeral nature of twitter does make me miss longform writing, of course. Composing aphoristic witticisms is great fun, but eventually one needs to construct some kind of larger edifice of thought. Which brings me back to the Axis, Praxis project. I’ve been meaning to republish my 3QD essays on my homepage. This is partly because there are lots of typos that I cannot easily fix on the 3QD site, but also because I’d like to add some notes and signposts so that readers have a better sense of the eclectic potpourri of ideas and references found in each piece. I delayed doing this because it initially seemed too backward-looking. But I now think that revisiting these essays does constitute a new project: I have by no means exhausted the topics I’ve written about, and new links continually suggest themselves. More specifically, I think that the potential relationships between theory and therapy that I partially discerned in 2011 can be explored from fresh angles.

So stay tuned for new old ideas and old new ideas!

Subscribe now

A believer in the magpie larceny story might find fault with the evidence. Perhaps magpies get bad vibes from nosy scientists? My preferred possibility is that magpie’s are drawn to things that humans seem to value. In other words, it is the human interest in an object that catches the magpie’s eye, rather than its lustre per se. Which, if you think about it, is how humans often decide what is valuable.

Anyway, I will stick with the name ‘Magpie’s Nest’ for these collections. Each section is independent, so feel free to skim or jump around.

TV gold

That leads me to my first (or is it second?) shiny object: the TV show Detectorists, which I finished watching earlier this month. It’s the charming tale of a pair of bumbling treasure-seekers in rural England. They scour the countryside with metal detectors looking for Saxon gold and suchlike. Mostly they find buttons and bits of soda cans.

The final series of Detectorists features a magpie subplot, which is why the bird was in my head in the first place.

What I found particularly appealing about the show was its gentleness: events tickle the boundaries of cringe comedy, but always return to warmth and optimism. The show’s characters really do love hunting for elusive gold, but they also value history, venerable old trees, and last night’s episode of Q.I. The show may have led to a boom in metal detecting as a hobby; its earnest portrayal of male friendship seems to have struck a chord. The pristine cinematography — lingering shots of meadows, burbling brooks, and insects browsing in flowers — is part of the show’s charm, as is the folky soundtrack supplied by Johnny Flynn1. The theme song is lovely.

Music: It’s Black Country Out There!

This week saw the release of For the First Time, the debut album by Black Country, New Road — a band that jumped into my list of favorites. Like those hapless detectorists, they’re from England — but an England that might as well be a parallel dimension. Their sound is harrowing, abrasive, and cathartic. They’ve been compared to the dour post-rock sludge of Slint, but the glint of brass puts me in mind of jazz, prog, and avant-garde classical music — think King Crimson and Third Ear Band, but with lyrics about Gen Z existential dread.

I love this city, despite the burden of preferences
What a time to be alive, oh
I know where I'm going, it's black country out there

This ‘city’ from ‘Science Fair’ could well be the internet.

Their cover of MGMT’s ‘Time to Pretend’ gives a reasonable sense of what they are up to right now — I imagine their sound will have evolved in six months though.

BC,NR has made me return with new ears to old music I didn’t much care for (specifically Slint). This phenomenon — the ability of the new to recontextualize the old — is one of the unsung miracles of human intelligence. Our memories are not at all like static files in computer folders — anything we do in the present can transform the traces of what we did before, creating new patterns. Most of the time these changes accrue imperceptibly, like subterranean rhizomes spreading. So when the shock of the new makes us uproot old experiences and consciously replant them, we know we have encountered something radical. Everything revolutionary is a prism.

Philosophy of mind: Is predictive coding old wine in a shiny new bottle?

Twitter led me to a recent and very readable philosophy paper (no small achievement) that casts a critical eye on predictive coding models of the brain. If you are an observer of neuroscientific fashions, you may have come across the predictive coding word-cloud, which includes “active inference”, “Bayesian brains” and the “Free Energy Principle”.

In ‘New Labels for Old Ideas: Predictive Processing and the Interpretation of Neural Signals’, the Berkeley philosopher Rosa Cao argues that talk of ‘predictions’ in predictive coding models can always be replaced by less mental-sounding terms (like “top-down modulation”), without any change in the underlying function of the model, or its ability to explain neural data. In other words, predictive coding just attaches mentalistic labels to already-known physical processes, adding no new insight into how those physical process cause mind to emerge2. It’s like trying to explain the brain with lego blocks by giving the lego blocks suggestive names like “the visual perception block” or “the cognition block”. Anthropomorphizing our modeling blocks is a temptation that is hard to resist (and one I like to both criticize and play with), but it is particularly useless when trying to understand the anthropos in the first place.

Cao builds her case methodically, ending with a clear message: words like “prediction” and “inference” are shiny ornaments rather than theoretical contribution.

It’s true that the world seems richer when we describe it in more colorful language – just look at our readiness to anthropomorphize the simplest of systems. But the color should be theoretically justified, and it should carry some empirical weight; in this case, it’s not clear that it does.

The following point she raises is subtle:

The casual use of the terms active, passive, error, contents, contributes to the uneasiness – these terms do not have clear empirical import, and it seems very easy to slide between personal-level versions of terms like “prediction” or “expectation” to the subpersonal ones employed in theorizing.

When a model is presented to us using words that resonate with intuition and naive introspection — particularly our common-sense notions of interpersonal interaction — we may in fact experience a mere mirage of comprehension. The satisfaction such models offer to their fans may arise from recognition rather than understanding. They draw the eye with reflected light, rather than illumination.

There is a lot in the paper that careful neuro-enthusiasts will find interesting. And in a way it might also help neuroscientists see that their mode of answering questions — always provisional, tentative, instrumental — is ill-suited to certain age-old questions about mind, self and consciousness, and therefore may never satisfy philosophers. Or members of the general public, for that matter, who sometimes share with philosophers a desire to know what the mind is, rather than what it seems to be like3.

Mysticism: Rise above the Administrative System

Out of the blue this week the brilliance of Nusrat Fateh Ali Khan resurfaced in my mind. While listening to ‘Dam Mast Qalandar’ I began digging into the song’s history. Apparently Nusrat’s song is inspired by an older tune called ‘Dama Dam Mast Qalandar’ — a qawwali that may have been written in the 1300s by Amir Khusrow, and later modified by Bulleh Shah. Here is a rousing version of ‘Dama Dam’ by the Nooran sisters.

After a lifetime of being an Urdu-deficient South Indian, I finally sought out what the word ‘qalandar’ means. The Wikipedia definition is quite something.

A Qalandar is a person who has excelled in seeing things and advances stage by stage into the Being. He even rises above the Administrative System and witnesses the core of Oneness in detail and after enjoying the Unity of the Being returns without losing his grades and then reaches back into his humanly status, so much so that his rise and fall becomes one and the same thing for him.

Somehow it makes me happy that the analytic philosophers’ precision-tooled approach to mind and being (they seek a rather administrative system of ideas, don’t they?), exists in the same world as that of Jhulelal and the transcendent Qalandars.

Bits and bobs

• This year is shaping up to be a bumper year for quality memes. Already we’ve seen sea shanties on tiktok, the Nooran sisters having quite the effect, the Handforth parish council, and of course Bernie’s mittens everywhere.

• In India a dog got trapped in a bathroom with a leopard for several hours. And survived, unharmed. I used to love reading these kinds of quirky wildlife stories in The Hindu.

• The Public Domain Review shared some gorgeous drawings of the nervous system by Ramon y Cajal and Golgi.

Two amygdalar chambers can be traced by the anatomist. One chamber, the basolateral, houses corpulent pyramidal neurons, cousins of the celebrated pyramids that dominate the cortical landscape. Another chamber, the central, houses spiny projection neurons, which show markings of kinship with residents of the striatum — the ancient market town where, to this day, transactions are mediated in the currency of dopamine. With one chamber resembling the cortical manufactories and another the striatal emporia, the amygdala seems a telencephalon in miniature. A premonition.

But these two chambers are not the only rooms in the amygdala. A slender hallway both divides and joins them; a band of cells know as the intercalated masses. To this day, some brain atlases simply ignore these borderline neurons. Perhaps things that help mark boundaries are confused with the boundaries themselves.

I first came in contact with the sprawling (and still-incomplete) saga of brain evolution when I joined my current lab, right after grad school. It’s a neuroanatomy lab, carrying on the grand tradition of Santiago Ramón y Cajal, the father of neuroscience. But in stark contrast to my collaborators, I’m a computational modeler — far removed from lab coats, protocols, rubber gloves, pipettes, microscopes. I use code and equation as an artist does canvas and pigment — to create impressionist portraits of neural phenomena. The kind of modeling I do remains a somewhat niche pursuit in neuroscience, precisely because of its impressionistic quality. The task of modeling is ostensibly to draw lines connecting the dots that are experimental findings — and perhaps anticipate a few dots before they are produced. But the dots are innumerable, and every other day someone from Stanford seems to develop a new way to generate more of them.

My first project in the lab was to help make sense of the intercalated masses — the neurons of the amygdalar gaps. They are a small and not-particularly-famous group of neurons; few would even think to correlate their chatter with grand ponderables like ‘cognition’, ‘empathy’, or ‘consciousness’. But they receive targeted input from brain areas that do get associated with those words — crucial parts of the prefrontal cortex. My little model of the intercalated masses may well contribute to the multigenerational quest to understand emotion and motivation, and to one day tame mental illness. This is of course unlikely, given the deluge of data-dots.

But even if the world doesn’t learn much from my model, I may have learned a thing or two while developing it. For instance, I could see very clearly, even with my broad brushstrokes, that one cannot judge the influence of something by its size or its apparently marginal position. The tiny intercalated masses (or at the very least, my simulation-paintings of them) appear to be a pivot of the emotional system, enabling swings from caution to recklessness. From another perspective, the neurons in this liminal zone are functionaries of the cortex, manipulating ‘facts on the ground’ to promote — or to suppress — emotional tumult.

The word ‘intercalated’ is related to the word ‘intercalary’. An intercalary day (or week, or month) is inserted into a calendar to keep it in sync with larger cosmic events. A leap day is a classic example. It is inserted once every four years to keep our otherwise-365-day calendars aligned with the solar year, which is closer to 365-and-a-quarter days long.

Intercalation is a reminder that rational constructions are simultaneously useful and incomplete. In antiquity, rationality was about ratio. The cosmos tantalized the early numerate humans with the possibility of whole number proportions: almost 30 days in a month (or ‘moonth’), almost 12 moons in a year, almost 13 cycles of Venus in 8 years… the list goes on. The ‘perfect’ versions of some of these almost-ratios are, coincidentally, used to tune musical instruments. But the key word here is almost. Without intercalation, our ‘rational’ cycles would fall out of harmony with the heavenly spheres. It is as if the gods are jazz musicians, and mortals are simply trying to keep up with their intricate — yet strangely loose — sense of rhythm.

So intercalation is a name for the filling of holes in our understanding. It is the reasonable amendment of our rational models, in the interest of coherence with the world. It is how we can take unforeseen circumstances in stride; we mind the gaps without getting tripped up.

My life is a series of interwoven attempts at intercalation. Disciplinary boundary-crossings. Silk routes through intellectual hinterlands. Marginalia. My quest is to mine these gaps — and I confess, I do occasionally trip on them.

Notes

More on the Mayan calendar here: The Classic Maya Calendar and Day Numbering System

I find myself frequently revisiting the idea of ‘jazz cosmology’. Here’s the latest iteration: Sailing the Seas of Thought

Image sources

Janak, Patricia H., and Kay M. Tye. "From circuits to behaviour in the amygdala." Nature 517, no. 7534 (2015): 284-292.

Zikopoulos, Basilis, Yohan J. John, Miguel Ángel García-Cabezas, Jamie G. Bunce, and Helen Barbas. "The intercalated nuclear complex of the primate amygdala." Neuroscience 330 (2016): 267-290.

Cajal’s Purkinje cells

Aztec (or Mayan?) calendar

Share

Leave a comment

I’m a computational neuroscientist at Boston University. I investigate the neural circuitry mediating the interplay between cognition, emotion, and motivation.

This newsletter will color outside traditional disciplinary lines, folding in philosophy of mind, metaphysical speculation, art, music, and literature.

For more of my writings, check out my home page.

If you like what you’ve seen (or expect to see) so far, tell your friends!

Subscribe by email using the box below:

Subscribe now