Pain is a curious thing. Why do we have painful pain instead of just a more neutral painless pain, when it can backfire so easily as chronic pain, among other problems? Why do we have pain at all instead of regular learning processes or experiencing rewards as we follow plans?
Can we understand pain as another two-level learning process, where a slow but ground-truth outer loss “central governor” constrains a fast but unreliable inner loss? I would suggest that pain itself is not an outer loss, but the painfulness of pain, its intrusive motivational aspects, is what makes it an outer loss. There is no logical necessity for pain to be pain but this would not be adaptive or practical because it would too easily let the inner loss lead to damaging behavior.
So let’s consider the possibilities when it comes to pain. There isn’t just “pain”. There is (at the least):
useless painful pain (chronic pain, exercise)
useful painful pain (the normal sort)
useless nonpainful nonpain (dead nerves in diabetes or leprosy or congenital pain insensitivity ; bed sores and RSI are everyday versions demonstrating that even the most harmless activities like ‘lying on a bed’ are in fact constantly causing damage)
useful nonpainful nonpain (adrenaline rushes during combat)
useless nonpainful pain (pain asymbolia where they maim & kill themselves, possibly also Lesch-Nyhan syndrome);
-
and intermediate cases: like the Marsili family who have a genetic mutation (Habib et al 2018) which partially damages pain perception. The Marsilis do feel useful painful pain but only briefly, and incur substantial bodily damage (broken bones, scars) but avoid the most horrific anecdotes of those with deadened nerves or pain asymbolia.
Another interesting case is the Scotswoman Jo Cameron, who has a different set of mutations to her endocannabinoid system (FAAH & FAAH-OUT): while not as bad as neuropathy, she still exhibits similar symptoms—aside from odd symptoms like being unusually forgetful, her father may also have been a carrier and died peculiarly, she regularly burns or cuts herself in household chores, she broke her arm roller-skating as a child but didn’t seek treatment until it reset at a “funny angle”, delayed treatment of a damaged hip & then a hand damaged by arthritis until almost too late (and may have damaged her joints so badly because of the lack of pain), took in foster children who stole her savings, etc. Her son carries one of her 2 mutations, and “frequently scalds his mouth with hot drinks and food”. (Biologist Matthew Hill describes the most common FAAH mutation as causing “low levels of anxiety, forgetfulness, a happy-go-lucky demeanor”, and “Since the paper was published, Matthew Hill has heard from half a dozen people with pain insensitivity, and he told me that many of them seemed nuts” compared to Jo Cameron.) A possible case is the absurdly prolific fantasy writer Brandon Sanderson, who routinely puts in 8-hour days of typing (on a couch, to boot), and who turns out to be almost entirely insensitive to pain.
but—is there ‘useful painless pain’ or ‘useless painful nonpain’?
It turns out there is ‘painless pain’: lobotomized people experience that, and “reactive dissociation” is the phrase used to describe the effects sometimes of analgesics like morphine when administered after pain has begun, and the patient reports, to quote Dennett1978 (emphasis in original), that “After receiving the analgesic subjects commonly report not that the pain has disappeared or diminished (as with aspirin) but that the pain is as intense as ever though they no longer mind it…if it is administered before the onset of pain…the subjects claim to not feel any pain subsequently (though they are not numb or anesthetized—they have sensation in the relevant parts of their bodies); while if the morphine is administered after the pain has commenced, the subjects report that the pain continues (and continues to be pain), though they no longer mind it…Lobotomized subjects similarly report feeling intense pain but not minding it, and in other ways the manifestations of lobotomy and morphine are similar enough to lead some researchers to describe the action of morphine (and some barbiturates) as ‘reversible pharmacological leucotomy [lobotomy]’.23”
And we can find examples of what appears to be ‘painful nonpain’: Grahek2001 highlights a case-study, Ploner et al 1999, where the German patient’s somatosensory cortices suffered a lesion from a stroke, leading to an inability to feel heat normally on one side of his body or feel any spots of heat or pain from heat; despite this, when sufficient heat was applied to a single spot on the arm, the patient became increasingly agitated, describing an “clearly unpleasant” feeling associated with his whole arm, but also denied any description of it involving crawling skin sensations or words like “slight pain” or “burning”.
A table might help lay out the possibilities:
A taxonomy of possible kinds of ‘pain’, split by organismal consequences, motivational effects, and reported subjective (non)experience.
useless |
painful |
pain |
chronic pain; exercise? |
useful |
painful |
pain |
normal/injuries |
useless |
nonpainful |
pain |
asymbolia |
useful |
nonpainful |
pain |
reactive dissociation, lobotomies; exercise? |
useless |
painful |
nonpain |
unconscious processes such as anesthesia awareness. Itches or tickles, anterograde amnesia? |
useful |
painful |
nonpain |
cold/heat perception, as in the somatosensory cortex lesion case-study |
useless |
nonpainful |
nonpain |
deadened nerves from diseases (diabetes, leprosy), injury, drugs (anesthetics) |
useful |
nonpainful |
nonpain |
adrenaline rush/accidents/combat |
Pain serves a clear purpose (stopping us from doing things which may cause damage to our bodies), but in an oddly unrelenting way which we cannot disable and which increasingly often backfires on our long-term interests in the form of ‘chronic pain’ and other problems. Why doesn’t pain operate more like a warning, or like hunger or thirst? They interrupt our minds, but like a computer popup dialogue, after due consideration of our plans and knowledge, we can generally dismiss them. Pain is the interruption which doesn’t go away, although we could easily imagine pains which did go away or were in fact ‘pleasurable’ (Morsella2005):
Theoretically, nervous mechanisms could have evolved to solve the need for this particular kind of interaction otherwise. Apart from automata, which act like humans but have no phenomenal experience, a conscious nervous system that operates as humans do but does not suffer any internal strife. In such a system, knowledge guiding skeletomotor action would be isomorphic to, and never at odds with, the nature of the phenomenal state—running across the hot desert sand in order to reach water would actually feel good, because performing the action is deemed adaptive. Why our nervous system does not operate with such harmony is perhaps a question that only evolutionary biology can answer. Certainly one can imagine such integration occurring without anything like phenomenal states, but from the present standpoint, this reflects more one’s powers of imagination than what has occurred in the course of evolutionary history.
In the reinforcement learning context, one could ask: does it make a difference whether one has ‘negative’ or ‘positive’ rewards? Any reward function with both negative and positive rewards could be turned into all-positive rewards simply by adding a large constant. Is that a difference which makes a difference? Or instead of maximizing positive ‘rewards’, one could speak of minimizing ‘losses’, and one often does in economics or decision theory or control theory.
“Do Artificial Reinforcement-Learning Agents Matter Morally?”, Tomasik2014, debates the relationship of rewards to considerations of “suffering” or “pain”, given the duality between costs-losses/rewards:
Perhaps the more urgent form of refinement than algorithm selection is to replace punishment with rewards within a given algorithm. RL systems vary in whether they use positive, negative, or both types of rewards:
In certain RL problems, such as maze-navigation tasks discussed in Sutton & Barto1998, the rewards are only positive (if the agent reaches a goal) or zero (for non-goal states).
Sometimes a mix between positive and negative rewards6 is used. For instance, McCallum1993 put a simulated mouse in a maze, with a reward of 1 for reaching the goal, −1 for hitting a wall, and −0.1 for any other action.
In other situations, the rewards are always negative or zero. For instance, in the cart-pole balancing system of Barto et al 1990, the agent receives reward of 0 until the pole falls over, at which point the reward is −1. In Koppejan & Whiteson2011’s neuroevolutionary RL approach to helicopter control, the RL agent is punished either a little bit, with the negative sum of squared deviations of the helicopter’s positions from its target positions, or a lot if the helicopter crashes.
Just as animal-welfare concerns may motivate incorporation of rewards rather than punishments in training dogs [Hiby et al 2004] and horses [Warren-Smith & McGreevy2007, Innes & McBride2008], so too RL-agent welfare can motivate more positive forms of training for artificial learners. Pearce2007 envisions a future in which agents are driven by ‘gradients of well-being’ (ie. positive experiences that are more or less intense) rather than by the distinction between pleasure versus pain. However, it’s not entirely clear where the moral boundary lies between positive versus negative welfare for simple RL systems. We might think that just the sign of the agent’s reward value r would distinguish the cases, but the sign alone may not be enough, as the following section explains.
What’s the boundary between positive and negative welfare?
Consider an RL agent with a fixed life of T time steps. At each time t, the agent receives a non-positive reward rt ≤ 0 as a function of the action at that it takes, such as in the pole-balancing example. The agent chooses its action sequence (at) t = 1…T with the goal of maximising the sum of future rewards:
T∑t=1rt(at)
Now suppose we rewrite the rewards by adding a huge positive constant c to each of them, r′t = rt + c, big enough that all of the r′t are positive. The agent now acts so as to optimize
T∑t=1r′t(at)=T∑t=1((rt)at+c)=Tc+T∑t=1rt(at)
So the optimal action sequence is the same in either case, since additive constants don’t matter to the agent’s behavior.7 But if behavior is identical, the only thing that changed was the sign and numerical magnitude of the reward numbers. Yet it seems absurd that the difference between happiness and suffering would depend on whether the numbers used by the algorithm happened to have negative signs in front. After all, in computer binary, negative numbers have no minus sign but are just another sequence of 0s and 1s, and at the level of computer hardware, they look different still. Moreover, if the agent was previously reacting aversively to harmful stimuli, it would continue to do so. As Lenhart K. Schubert explains:8 [This quotation comes from spring 201412ya lecture notes (accessed March 201412ya) for a course called “Machines and Consciousness”.]
If the shift in origin [to make negative rewards positive] causes no behavioral change, then the robot (analogously, a person) would still behave as if suffering, yelling for help, etc., when injured or otherwise in trouble, so it seems that the pain would not have been banished after all!
So then what distinguishes pleasure from pain?
…A more plausible account is that the difference relates to ‘avoiding’ versus ‘seeking.’ A negative experience is one that the agent tries to get out of and do less of in the future. For instance, injury should be an inherently negative experience, because if repairing injury was rewarding for an agent, the agent would seek to injure itself so as to do repairs more often. If we tried to reward avoidance of injury, the agent would seek dangerous situations so that it could enjoy returning to safety.10 [This example comes from Lenhart K. Schubert’s spring 201412ya lecture notes (accessed March 201412ya), for a course called ‘Machines and Consciousness.’ These thought experiments are not purely academic. We can see an example of maladaptive behavior resulting from an association of pleasure with injury when people become addicted to the endorphin release of self-harm.] Injury needs to be something the agent wants to get as far away from as possible. So, for example, even if vomiting due to food poisoning is the best response you can take given your current situation, the experience should be negative in order to dissuade you from eating spoiled foods again. Still, the distinction between avoiding and seeking isn’t always clear. We experience pleasure due to seeking and consuming food but also pain that motivates us to avoid hunger. Seeking one thing is often equivalent to avoiding another. Likewise with the pole-balancing agent: Is it seeking a balanced pole, or avoiding a pole that falls over?
…Where does all of this leave our pole-balancing agent? Does it suffer constantly, or is it enjoying its efforts? Likewise, is an RL agent that aims to accumulate positive rewards having fun, or is it suffering when its reward is suboptimal?
So with all that for background, what is the purpose of pain?
The purpose of pain, I would say, is as a ground truth or outer loss. (This is a motivational theory of pain with a more sophisticated RL/psychiatric grounding.)
The pain reward/loss cannot be removed entirely for the reasons demonstrated by the diabetics/lepers/congenital insensitives: the unnoticed injuries and the poor planning are ultimately fatal. Without any pain qualia to make pain feel painful, we will do harmful things like run on a broken leg or jump off a roof to impress our friends, or just move in a not-quite-right fashion and a few years later wind up paraplegics. (An intrinsic curiosity drive alone would interact badly with a total absence of painful pain: after all, what is more novel or harder to predict than the strange and unique states which can be reached by self-injury or recklessness?)
If pain couldn’t be removed, could pain be turned into a reward, then? Could we be the equivalent of Morsella’s mind that doesn’t experience pain, as it infers plans and then executes them, experiencing only more or less rewards? It only experience positive rewards (pleasure) as it runs across burning-hot sands, as this is the optimal action for it to be taking according to whatever grand plan it has thought of.
Perhaps we could… but what stops Morsella’s mind from enjoying rewards by literally running in circles on those sands until it dies or is crippled? Morsella’s mind may make a plan and define a reward function which avoids the need for any pain or negative rewards, but what happens if there is any flaw in the computed plan or the reward estimates? Or if the plan is based on mistaken premises? What if the sands are hotter than expected, or if the distance is much further than expected, or if the final goal (perhaps an oasis of water) is not there? Such a mind raises serious questions about learning and dealing with errors: what does such a mind experience when a plan fails? Does it experience nothing? Does it experience a kind of “meta-pain”?
Consider what Brand (The Gift of Pain again, pg191–197) describes as the ultimate cause of the failure of years of research into creating ‘pain prosthetics’, computerized gloves & socks that would measure heat & pressure in real-time in order to warn those without pain like lepers or diabetics: the patients would just ignore the warnings, because stopping to prevent future problems was inconvenient while continuing paid off now. And when electrical shockers were added to the system to stop them from doing a dangerous thing, Brand observed patients simply disabling it to do the dangerous thing & re-enabling it afterwards! Less exotically, consider professional athletes: promising careers end every day due to injuries, and one of the most important playing skills of a top athlete in the NBA or NFL is being able to not play.
What pain provides is a constant, ongoing feedback which anchors all the estimates of future rewards based on planning or bootstrapping. It anchors our intelligence in a concrete estimation of bodily integrity: the intactness of skin, the health of skin cells, the lack of damage to muscles, joints sliding and moving as they ought to, and so on. If we are planning well and acting efficiently in the world, we will, in the long run, on average, experience higher levels of bodily integrity and physical health; if we are learning and choosing and planning poorly, then… we won’t. The badness will gradually catch up with us and we may find ourselves blind scarred paraplegics missing fingers and soon to die. A pain that was not painful would not serve this purpose, as it would merely be another kind of “tickling” sensation. (Some might find it interesting or enjoyable or it could accidentally become sexually-linked.) The perceptions in question are simply more ordinary tactile, kinesthetic, thermoreceptor, or other standard categories of perception; without painful pain, a fire burning your hand simply feels warm (before the thermal-perceptive nerves are destroyed and nothing further is felt), and a knife cutting flesh might feel like a rippling stretching rubbing movement.
We might say that a painful pain is a pain which forcibly inserts itself into the planning/optimization process, as a cost or lack of reward to be optimized. A pain which was not motivating is not what we mean by ‘pain’ at all. The motivation itself is the qualia of pain, much like an itch is an ordinary sensation coupled with a motivational urge to scratch. Any mental quality or emotion or sensation which is not accompanied by a demandingness, an involuntary taking-into-consideration, is not pain. The rest of our mind can force its way through pain, if it is sufficiently convinced that there is enough reason to incur the costs of pain because the long-term reward is so great, and we do this all the time: we can convince ourselves to go to the gym, or withstand the vaccination needle, or, in the utmost extremity, saw off a trapped hand to save our life. And if we are mistaken, and the predicted rewards do not arrive, eventually the noisy constant feedback of pain will override the decisions leading to pain, and whatever incorrect beliefs or models led to the incorrect decisions will be adjusted to do better in the future.
But the pain cannot and must not be overridden: human organisms can’t be trusted to simply ‘turn off’ pain and indulge an idle curiosity about cutting off hands. Note that we can kill ourselves by starvation or thirst, but we cannot kill ourselves by refusing to sleep, or have a heart beat, or breathe—unless one suffers from the (extremely lethal) central hypoventilation syndrome, that is. We are insufficiently intelligent, our priors insufficiently strong, our reasoning and planning too poor, and we must do too much learning within each life to do without pain.
A similar argument might apply to the puzzle of ‘willpower’, ‘procrastination’. Why do we have such problems, particularly in a modern context, doing aught we know we should and doing naught we oughtn’t?
On the grave of the ‘blood glucose’ level theory, Kurzban et al 2013 (see later Shenhav et al 2017) erects an opportunity cost theory of willpower. Since objective physical measurements like blood glucose levels fail to mechanically explain poorer brain functionality or why strenuous activities like sports are ‘restful’ & reduce ‘burnout’, similar to the failure of objective physical measurements like lactate levels to explain why people are able to physically exercise only a certain amount (despite being able to exercise far more if properly motivated or if tricked), the reason for willpower running out must be subjective.
To explain the sugar-related observations, Kurzban et al 2013 suggest that the aversiveness of long focus and cognitive effort is a simple heuristic which creates a baseline cost to focusing for ‘too long’ on any one task, to the potential neglect of other opportunities, with the sugar interventions (such as merely tasting sugar water) which appear to boost willpower actually serving as proximate reward signals (signals, because the actual energetic content is nil, and cognitive effort doesn’t meaningfully burns calories in the first place), which justify to the underlying heuristic that further effort on the same task is worthwhile and the opportunity cost is minimal.
The lack of willpower is a heuristic which doesn’t require the brain to explicitly track & prioritize & schedule all possible tasks, by forcing it to regularly halt tasks—“like a timer that says, ‘Okay you’re done now.’” If one could override fatigue at will, the consequences can be bad. Users of dopaminergic drugs like amphetamines often note issues with channeling the reduced fatigue into useful tasks rather than alphabetizing one’s bookcase. In more extreme cases, if one could ignore fatigue entirely, then analogous to lack of pain, the consequences could be severe or fatal: ultra-endurance cyclist Jure Robič would cycle for thousands of kilometers, ignoring such problems as elaborate hallucinations, and was eventually killed while cycling. The ‘timer’ is implemented, among other things, as a gradual buildup of adenosine, which creates sleep homeostatic drive pressure and possibly physical fatigue during exercise (Noakes2012, Martin et al 2018), leading to a gradually increasing subjectively perceived ‘cost’ of continuing with a task/staying awake/continuing athletic activities, which resets when one stops/sleeps/rests. (Glucose might work by gradually dropping over perceived time without rewards.) Since the human mind is too limited in its planning and monitoring ability, it cannot be allowed to ‘turn off’ opportunity cost warnings and engage in hyperfocus on potentially useless things at the neglect of all other things; procrastination here represents a psychic version of pain.
From this perspective, it is not surprising that so many stimulants are adenosinergic or dopaminergic, or that small children might especially struggle with mental fatigue (there is a world full of novel opportunities tempting them away), or that many anti-procrastination strategies (like Getting Things Done or the Procrastination Equation) boil down to optimizing for more rewards or more frequent rewards (eg. breaking tasks down into many smaller tasks, which can be completed individually & receive smaller but more frequent rewards, or thinking more clearly about whether something is worth doing): all of these would affect the reward perception itself, and reduce the baseline opportunity cost ‘pain’. This perspective may also shed light on depression, or on occupational burnout and why restorative hobbies are ideally maximally different from jobs and more miscellaneous observations like the lower rate of ‘hobbies’ outside the West: burnout may be a long-term homeostatic reaction to spending ‘too much’ time too frequently on a difficult not-immediately rewarding task despite earlier attempts to pursue other opportunities (perhaps tasks which would never be rewarding), which were always overridden, ultimately resulting in a total collapse; and hobbies ought to be as different in location and physical activity and social structure (eg. a solitary programmer indoors should pursue a social physical activity outdoors as soulcraft) to ensure that it feels completely different for the mind than the regular occupation; and in places with less job specialization or fewer work-hours, the regular flow of a variety of tasks and opportunities means that no such special activity as a ‘hobby’ is necessary. A further analogy might be to communication: email and chat message overload, compared to ‘worse’ communication technologies, may reflect a lack of ‘pain’ to the sender—resulting in ‘damage’ to the overloaded recipient.
Perhaps if we were superintelligent AIs who could trivially plan flawless humanoid locomotion at 1000Hz taking into account all possible damages, or if we were emulated brains sculpted by endless evolutionary procedures to execute perfectly adaptive plans by pure instinct, or if we were simple amoeba in a Petri dish who had no real choices to make, there would be no need for a pain which was painful. And likewise, were we endlessly planning and replanning to the end of days, we should never experience akrasia, we should merely do what is necessary (perhaps not even experiencing any qualia of effort or deliberation, merely seeing events endlessly unfold as they always had to). But we are not. The pain keeps us honest. In the end, pain is our only teacher.