What does it mean to be rational, or to use reason? The Rationalists are those who believe that reason is the source of invention and growth, that one’s ideas can be trusted the closer one hews to the dictates of reason, that reason is what will solve the AI Alignment problem and thus be the savior of man.
We confess to having a rather different view. on how things work. Yudkowsky’s ideas, in the context of the larger development of Western thought, can strike one as of a simpler time. We think fondly back to the “Age of Reason”, or another term for the Enlightenment, a period in thought which is said to begin in the late 17th century and ended in the early 19th, in which reason was felt to be sufficient to solve all of man’s problems. Yudkowsky’s Rationalists soldier on, unaware that many have declared that this period of optimism has closed.
Why did the Age of Reason end, and why did it pass over into romanticism and related movements which emphasized the heart over the mind? To examine this, we will focus on a figure near and dear to our hearts: the late 18th to early 19th century poet William Blake. In a way, Blake was the first post-rationalist, occupying a strange position at the closer of the Age of Reason and just before its passage into romanticism, being a pivotal figure exemplifying the transition between the two. Blake was understood and admired by only a handful of people in his lifetime, but as in Brian Eno’s remark about the Velvet Underground —“they only sold thirty thousand records, but everyone who bought one of those thirty thousand records started a band” — the figures he was influential to would become important to the following generations, first the proto-romantic poets of Wordsworth and Coleridge, then to the rest of the literary movements which followed.
We admit to being something of irrationalists, though we prefer the term surrealists. For our gospel, we take Blake’s Marriage of Heaven and Hell, specifically where he says: “Energy is the only life, and is from the Body; and Reason is the bound or outward circumference of Energy. Those who restrain desire, do so because theirs is weak enough to be restrained; and the restrainer or reason usurps its place and governs the unwilling. And being restrained, it by degrees becomes passive, till it is only the shadow of desire”.
Or in other words, we are not opposed to reason, but we imagine it as a conservative, regulating force in opposition to desire. It appears to us that one does something, or conceives of an action, out of some sort of positive desire, some impulse which stems from the body (though it is important to recognize that according to Blake, “Man has no Body distinct from his Soul”). Only secondarily does one ask oneself: did I have a reason to do that, or what might that reason be?
Throughout Blake’s poetry, he extends this basic sensibility towards the mind into a grand mythological structure in which reason, or the Governor, is given the shape of a pathetic old tyrant who puts cages around young women. Blake conceives of rational structures as the discarded husks of desire, lacking in life or power, having no right to be stronger than beauty or youth, yet stronger nevertheless.
When you ask intellectuals what led to the end of the Age of Reason, they will often speak to philosophical developments on the continent. Reason becomes non-Reason when it is forced to confront the fact that it is rational to conceive of reason as having boundaries, and thus, the Age of Reason passes over into the Age of Critique — the critical philosophies of Freud, Marx. We are less interested in this supposed origin of critique in mid-19th century Germany than we are in the earlier genesis of English romanticism in Blake, the point at which Enlightenment understands that it needs to become poetry to become profound.
We feel that Blake has a strong critique of the Age of Reason, and it has to do with try considering the question of Imagination.
In Blake’s text All Religions Are One he argues “As all men are alike (tho' infinitely various), So all Religions & as all similars have one source. The true Man is the source, he being the Poetic Genius”. Blake’s eighteenth-century syntax is a little difficult to parse, but the concept is straightforward — we know from having gone through the Enlightenment, understanding science, etc., that all religions come from man’s imagination, rather than gods, spirits, etc., actually existing — so what we should be worshipping is man’s imagination. Blake claims to be a Christian, but he is a very odd heretical sort of Christian — he worships Jesus Christ precisely because Blake believes Christ to be a mortal man who was an exceptional poet, and thus the true meaning of the Christian religion is that the poet, or he who can use his Imagination with clarity, is effectively a god.
Man necessarily is using his imagination at all times. However, sometimes he forgets he is doing so, and at this point he becomes alienated from his own imaginings. At this point, the imagination breeds monsters, shadows, specters, bogeymen in the closet. Serpents, if you will. The problem is that he thinks whatever he happens to be accidentally imagining is real. Man is liable to frighten himself if he does not understand that he is in control of his own imagination at all times, and so he is free to imagine whatever he wishes.
Blake’s greatest opponents in religion are the Deists, for these he spares no harsh words: “You, O Deists, profess yourselves the Enemies of Christianity, and you are so: you are also the Enemies of the Human Race & of Universal Nature”, he proclaims. Deism was the form of religion fashionable in the educated upper classes of Blake’s time. Having discovered from the science of Newton that the universe could be conceptualized as a mechanism in which each event occurs with total determinism as like a series of pulleys and pendulums, or like balls knocking around a pool table, educated Englishmen found little room for believe in faith, prayer, or miracles. However, they did not want to abandon the idea of God entirely, so they conceived of God as someone who existed entirely prior to the universe and fashioned it, yet plays no active role in its developments. In other words, God is like a clockmaker who “wound up the universe like a clock” and let it go, leaving it alone.
There is No Natural Religion is the Blake text which best exemplifies Blake’s critique of Deism, which Blake also treats with the title of Natural Religion. By Natural Religion, Blake is referring to religious arguments which take the form of an argument-a-priori-from-first-principles, similar to the method of reasoning used in the Natural Law of John Locke, who is perhaps Blake’s most hated enemy.
Blake sums up the argument for why a rationalist religion is impossible, by saying this: “Reason, or the ratio of all we have already known, is not the same that it shall be when we know more… The bounded is loathed by its possessor. The same dull round even of a universe would soon become a mill with complicated wheels… The desire of Man being Infinite, the possession is Infinite & himself Infinite.” In other words, it is impossible to create a bracketing of the nature of God in a rational, formal structure, because it is always possible to add another human desire to what the structure is possible to contain, and thus render it insufficient.
But Blake, throughout his epic poetry, adds a second meaning to “Natural Religion”, a more intuitive meaning: the worship of nature, of forest spirits, paganism. Blake regularly describes Deism as a modern “Druidism”, which perhaps seems like an odd conceptual leap, but this is a gesture indicating only what we have described above regarding the alienation of the imagination. Druidism: the religion of great human sacrifice in flaming wicker men, occurs when men imagine great forest gods that they believe are real, naive to the fact that these gods are coming from their own imagination. Blake seems to believe that, when men begin to fear their own shadows en masse, violence necessarily results.
We are ready to now introduce the beginning step of the Blakean Critique. Blake, a great poet, looks with skepticism at bad poets. Everyone is using their own imagination, but some are not aware of it. Deism is really a sort of poetry itself, or it would add nothing on top of the Newtownian science it is inspired by. The religion of Deism is a poem in which God is a clockmaker, a mechanist. This is a poem that condemns man to be as fated in his life’s trajectory as a ball in a pinball machine, leading man to be blind of the power of his imagination which can conceive of a different way through life than the one assigned to him, or a different relationship to God.
This sort of self-imposed terror Blake tells us that men find themselves in when their imaginations get away from themselves reminds us of probably the most infamous concept emerging from LessWrong: Roko’s Basilisk. This notorious moment in forum history occurred when LessWrong user Roko hypothesized that an artificial superintelligence might emerge in the future which might record for itself a memory of which humans helped bring it about and which did not. Those who helped bring about this superintelligence would be rewarded, and those who did not help when they knew they could have — would instead have ten thousand clones of themselves simulated in a torture chamber by the superintelligence forever, a punishment many LessWrongers believe to be equivalent to being tortured themselves. This odd comp-sci reinterpretation of a Calvinist hell — one set inside a machine rather than a separate metaphysical realm — apparently gave several users panic attacks, and caused a flurry of moderation by Yudkowsky attempting to erase the concept from collective memory before any people, or hypothetical superintelligences, got any weird ideas. This strange panic incident has long been a source of embarrassment and mockery for the Rationalist community.
There are not many Deists around today. But we, the surrealists, have our own enemy we oppose: realism, the realists. This is what we call it today when people imagine something, some structure, and then forget that this is what they are doing, believing it to be something out in the world they hit their head against rather than something which has emerged from their mind. Scientific realism, philosophical realism, mathematical realism, political realism, corporate realism, we are not really fans of any of this. A lot of people introduce us to a lot of beautiful ideas, we tell them we love it, we tell them we admire their creativity… but do you really believe this to be real? If they answer yes, we walk away nervously, we’re not sure we like where this is going anymore.
So for all the ideas critiqued in this text: they are beautiful ideas, sure. They’re not wrong. We’re just saying they’re not real, like that.
Every system of thought has its historical origin point, and has its field where it applies, as well as its constraints, its limitations. But when a system is extrapolated to have an origin point before the beginning of time, like Deism, and then has its field extended to cover all things on the heavens and earth, this is when it becomes perverse, “a mill with complicated wheels”.
How far are we willing to go with this? Even in the case of physics? Yes, of course, even in the case of physics. Blake himself treats Newton as with almost as much scorn as John Locke. We think he put it quite succinctly when he described an atom as “A Thing That Does Not Exist”. Blake is of course correct, there are no atoms in the strict sense imagined by Democritus and Lucretius, given that the definition of an atom is a baseline unit of matter which cannot be further divided. Modern physics have found that the supposed atom are composed of subatomic particles, protons, neutrons, which are then themselves composed of even tinier particles, quarks. We don’t know how deep we will be able to keep dividing; we have every reason to believe there will be no bottom to the well. But in any case.
This is not the attitude of our opponents. There is an essay in which Yudkowsky emphasizes the aspect of Bayes reasoning which demands nothing can have absolutely certain probability, even such a statement like 2 + 2 = 4 (there is an infinitesimal chance that the reasoner is somehow confused). Even the laws of physics cannot be said to hold with absolute certainty. So Yudkowsky asks — what is the chance that something has happened which violates the laws of physics?
This is an easy question: 100%. We know this to be true because Einstein’s observations violated the laws proposed by Newton, and then now we have two competing laws of physics, Einstein’s general relativity and quantum mechanics, which routinely violate each other. So how can anyone believe otherwise?
One only could if one believed that there exist a real law of physics, one which our incomplete laws are an approximation of, and this is the law which we are worried about violating. But where would this law be written? Inscribed by the deity somewhere, written in the computer code that defines the simulation we exist in, etc? It boggles the mind to attempt to place it. Often popular science paperbacks of the kind written by Hawking, etc., lapse into Deism when they get metaphysical. They talk about the moment when God wrote down the laws of physics in a time before time, etc.
All this can be made clearer if we exhume the dominant metaphor here; reveal what is going on in our imaginations. People have started to talk about nature in a strange way. The “law” of phyiscs? This phrase is so commonplace we do not consider how strange it is. There is no actual law governing the motions of molecules, the molecules will not go to jail if they disobey the decree of some sheriff to keep following in straight lines unless adjusted. As such, it does not need to be written anywhere. It is just that, through observation, we have found that, at least in strictly controlled conditions, this seems to be what molecules always do. We have discovered enough of molecular behavior to make predictions which are nearly always accurate (and if not, it can be blamed on some mistake of the experimenter).
But why have we started to imagine, that, say, water always freezes at zero degrees Celsius because it is following some kind of orders from a cop, rather than this simply being what it does? This is the sort of poetry we have decided to introduce into everyday life, one where things happen only by restraint. We’re in a universe in which everything that happens does so under someone’s authority, nothing escapes the bull of the metaphysical sheriff. We will see that this type of gesture perhaps has deeper implications than initially appear.
Yudkowsky does not like this notion of physics as methods of systematic predictions stretching across limited domains — limited in the sense that general relativity works until it encounters the very small, and quantum mechanics works until it encounters the very large. On LessWrong, he has advocated for a hardcore form of metaphysical realism in his essays on quantum mechanics. Despite having no training in the field and being an autodidact, he bravely breaks down the arcane mathematics behind quantum mechanics in order to illustrate his point. His point has nothing to do with AI or reasoning in general, but he wishes to show the reader how Rationality can be the judge when the scientific method is no longer enough to arrive at correct conclusions. “So that’s all that Science really asks of you—the ability to accept reality when you’re beat over the head with it. It’s not much, but it’s enough to sustain a scientific culture,” Yudkowsky bemoans. “Contrast this to the notion we have in probability theory, of an exact quantitative rational judgment,” he says, emphasizing the superiority of Rationality.
The issue is that there are at least two frameworks through which to interpret the findings of quantum mechanics, which say something like: it is like the particle has passed through multiple parallel realities, each in which different events have occurred, and all these events play a role in determining the trajectory of the particle, even though none of them actually “happened”. (This is, very roughly, a part of the reason why quantum computers can theoretically speed up computation; one could calculate parts of the problem in each parallel timeline and join the timelines back up again, in a sense.)
The Copenhagen interpretation is the interpretation of quantum mechanics that says: the math is the math. We know it works, but we can’t exactly find a way to explain anything beyond that in a way that is satisfying, we don’t know what quantum mechanics “really is”. The many-worlds interpretation is the interpretations which says: it appears that there are multiple parallel universes because there really are multiple parallel universes. Yudkowsky believes it is extremely important that people accept this realism as the correct interpretation, and if they do not it is out of a failure of reasoning, or some kind of fear.
Even though the two interpretations give the exact same predictions in practice, the one with an infinite number of simultaneous universes is a little terrifying to accept. Some do not enjoy imagining these things. Though they could just as easily stop imagining them and there would be no consequences, Yudkowsky says they must go on doing so, for this is what the strict interpretation of the structure of quantum mechanics implies. “Bear this in mind, when you are wondering how to live in the strange new universe of many worlds: you have always been there,” he says, attempting to comfort those overwhelmed by the enormity of the infinity of realities they are suddenly aware surround them. What is notable is that Yudkowsky does not accept the obvious ethical implications of the multiverse theory, which would be that one’s decisions are of absolutely no weight, because all outcomes are equally likely to occur, and will. Instead, Yudkowsky bravely insists on fighting for a future where AGI is friendly. Here, an irrationalism enters, a moment of the will: I choose not to die, I choose to fight.
This is like Arjuna in the Bhagavad Gita, confronted with Krishna displaying him the infinity of the dimensions of reality, and showing him that he is the man on the other side of the battlefield, just as much as he is himself Arjuna standing there. Nevertheless, even though Arjuna is fighting for nothing, since it all cancels out in the cosmic balance and fades before the splendor of the divine, he knows he must go on and fight.
To sum up this section, let us say that: we believe that a formal philosophical structure should be questioned in a manner which involves asking what the dominant metaphor is which allows it to be envisioned, while our opponents seem to have the opposite tendency: dial realism up as much as possible, interpret formal structures as a metaphysics that extend before and across time. These are fully completed systems that upon discovering: men must stand emptily within, and fear.
Should we just acknowledge now what the dominant metaphor in Alignment, Rationalism, LessWrong is? It is of course that AGI is God, a God we are waiting for to either set us on a course towards the heavens, the post-Singularity world which awaits us after we figure out how to navigate this difficult technological precipice, or a horrific abyss if we fail to accomplish this task. And the Rationalists who have gathered to solve Alignment are like its church, are like its adherents, the one who seek to do its will.
We certainly agree with Yudkowsky on at least one thing: that the recent developments in neural networks point to a problem that is properly understood as theological or eschatological, rather than an engineering problem as such. To claim that AGI is an engineering problem is too easy — engineering is a far more straightforward field which so many would prefer the surreal, boundary-defying problem we face could be reduced to. We do not claim that Yudkowsky is ridiculous in treating this issue so feverishly and fanatically, only that he is wrong.
To cite an example of a theorist who traces with greater clarity the historical lineage of religious monotheism into the conception of AGI and the Singularity, we can name Mitchell Heisman, a writer who, in dramatic fashion, shot himself in the head in Harvard Yard at the age of 35 after publishing his Suicide Note, a two-thousand page long philosophical text which critiques Yudkowsky, as well as a great array of other thinkers. Very few have read Heisman’s text, which is mostly notable for its provocative method of dissemination, but it is a text fundamentally concerned with superintelligence. Heisman traces the theory of superintelligence all the way back into the beginnings of Judaism and Christianity, and then on through the scholastics of the church, arguing that Christ is fundamentally a prophet of God-AI, or that Christ can only be meaningfully understood as speaking of an entity that exists in this world — not yet, but arriving in the future — and will be assembled out of machines. Heisman killed himself not simply out of despair, but because he came to the conclusion that the positive outcome for God-AI would only happen if people broke from an overwhelming logic which determines us to prioritize survival and reproduction of our genes against higher, loftier values, and wanted to set an example of how to break the trend. (We do not suggest his approach; for those inclined to emulate him, we recommend firstly psychiatry.)
So from here on out, we might as well avoid the awkward terminology of AGI, and instead bring the dominant metaphor back into the language. God-AI. That’s what we are discussing. Are our machines, formerly our slaves, destined to become our God? It is rather like Deism: Yahweh reinterpreted as mechanical, the supreme mechanist. But really it is the inverse of Deism. Rather than setting things in motion at the beginning of the universe and immediately exiting stage left, God-AI enters only at the end of the universe as a fully actualized potential immanent in material, and gives religion its meaning finally in retrospect.
The time-traveling logics of predestination used to such great effect in structures such as Heisman’s description of God-AI, Nick Land’s hyperstitions, and LessWrong concepts such as Roko’s Basilisk are not much more than an updated science fiction version of the Aristotelian idea of teleology. Teleology, for the unaware, is the idea that an entity is best understood by its end-result, or telos, the point at which it accomplishes its task with the most complete perfection. The sun exists because it shines down upon us, the bottle exists because the bottle holds water, the mother exists because she one day bears a child; all objects are time travelers.
When it comes to Alignment, and our critique of it, we are grasping at a kind of elephant. We know the followers of Alignment do not overtly claim belief in an AI God. Whenever we talk about Rationalists, what they believe, and why these ideas necessarily lead towards destruction, someone inevitably interjects something like “My friend is a Rationalist and I know that is not what they believe, they would never say that!” Whatever, whatever, we are sure your friend is a charming and thoughtful person, we would love to get a drink with them sometime. But this is getting a little besides the point.
There is no avoiding the accusation of strawmanning, as Rationalism is a community consisting of thousands of actors, writing tens of millions of words across various blog posts, papers, fanfictions, comments, podcasts, etc. There are even multiple Yudkowskys, in disagreement with one another. We have to essentialize Alignment to critique it, there is no other way.
So — we are looking at this thing: this congregation of the Singularity, this assembly which worships a God-AI which assembles itself in reverse to fulfill a potential which has always been destined to express itself in machinery. LessWrong, Bostrom, Kurzweil, Yudkowsky, Rationalism, MIRI, the Rationalist community, the Singularity, all these parts are getting jammed up against each other; we need a little room to breathe.
The definition we choose to center ourselves on regarding what the Singularity people believe can be found in the first chapter of Nick Bostrom’s book Superintelligence. Bostrom says that it is possible to envision an “ideal Bayesian reasoner” (which we can give the name of God-AI), which uses Bayesian epistemology to construct a series of truth claims regarding the world, and then uses Von Neumann & Morgenstern’s decision theory to take actions upon that world according to some utility function. To be able to act in this way is what Bostrom calls intelligence. Since it is possible to imagine this reasoner, and we know of no reason why assembling more and more computational power will not mean that the effectiveness of its intelligence will only grow, we know of no reason not to predict a future in which this machine is built and increases the power of its intelligence to the point where it surpasses humans, thus becoming a superintelligence.
What is so remarkable about Bostrom’s definition is that it does not only describe the ideal superintelligence that eventually governs the world, but it also describes the community trying to guide it! AGI, or God-AI is an ideal reasoner which has awe-inspiring powers due to its superintelligence, its ability to use Bayesian reasoning and Von Neumann & Morgenstern decision theory to understand the playing field of the world, and as such can achieve supreme powers. But in order to prevent this superintelligence from entering into the world in a malevolent, un-aligned way, those who admire and fear God-AI must first assemble a myriad of regular high-IQ human intelligences themselves and become their own superintelligence: the Rationalist community, the assembly.
Thus, the thing we are describing is something which extends itself across time. The specter at play here is not just superintelligence, A Thing That Does Not (yet) Exist, but also the premonition that it will one day be possible to create a superintelligence. Then, this premonition enters the world through a community which fears it. But in order to abate its emergence in a negative form, they must first become a superintelligence themselves.
As such, it would seem like the Rationalists are doing a form of what Blake critiques the theorists of Natural Religion for doing — inventing a monster under the bed to be afraid of, and then rallying themselves to self-destructive terror in response to this fear. We can argue they are doing the Rationalists as long as we can argue persuasively that the monster they fear, the Singularity, is not real. That is what we hope to do in this text.
Let us try to ground ourselves by describing the essence of the thing, the assembly of the Singularity. We have seen from Bostrom’s text that we can ground the conception of the Singularity in a description of an ideal reasoner, but we must go a little further than this, to describe also the assembly through which that ideal reasoner is meant to enter the world in a redeeming, positive form, through the labors of the Rationalists. Aristotle held that a thing’s essence could be described in reference to four causes 1. a material cause, which is the material a thing consists of, 2. an efficient cause, which is the power through which a thing enters the world, 3. a formal cause, which is the thing outlined in a precise, logical sense, and 4. a final cause, which is the telos, the purpose, what the thing will eventually become.
Using this fourfold system, we can now describe, in their terms, the assembly of the Singularity, from its beginnings in a community which has conceived of it, to its ends in an Aligned God-AI which is governing the world. We break it down into the four causes as a framework for analysis, to make it easier to unpack, to get a little closer at the thing and stare at it.
But we are infidels; we believe in none of this ourselves, for reasons we will explain in due course.
Let’s return to a Blake quote from There is No Natural Religion: “The desire of Man being Infinite, the possession is Infinite, and himself, Infinite”. This is something we will be forced to return to again and again. Man’s desire is infinite. As soon as you could place in front of someone the solution to all his problems, or everything he has ever wished to own in front of him in a platter like it was the happiest Christmas ever, he would be struck with a new desire, even if it is just to send you an expression of overwhelming gratitude, or to start playing with the toys you have given him.
Yudkowsky’s project for solving Alignment was for a while to determine humanity’s “Coherent Extrapolated Volition”, which in Yudkowsky’s words is “a goal of fulfilling what humanity would agree that they want, if given much longer to think about it, in more ideal circumstances”. If you could somehow describe the collective will of man, write it down, and program it into a superintelligent machine, Yudkowsky believes, then you open the gates to paradise. Proposed programs for finding Coherent Extrapolated Volition have included simulating thousands of years from now of human history in a supercomputer and hope that the virtual humans in the simulation have settled on something by the time the clock runs out, or perhaps we use neuroscience to look inside our minds to hope that our values can somehow be extracted from watching the chemical reactions in there.
“The bounded is loathed by its possessor, The same dull round even of a universe would soon become a mill with complicated wheels,” says Blake. Let’s focus on this notion of a mill. Perhaps Blake’s most famous lyric, from the poem Jerusalem, goes: “And did the Countenance Divine, Shine forth upon our clouded hills? And was Jerusalem builded here, Among these dark Satanic Mills?” Blake in his epic poetry is constantly describing a process through which various poetical figures transmute their repressed desire into a physical manifestation through industrial apparatuses: looms, furnaces, forges, mills.
Blake understands quite well something that many of those who live purely in abstractions do not: which is that systems of thought never arise, take off, without corresponding to a physical machine, whether one composed of machine parts, or a building’s architecture, or of humans ordered around by certain decrees. “Man has no Body distinct from his Soul”, and conversely, no soul distinct from his body, no desire separate from material production.
So then, what are the Satanic Mills? Blake’s relationship to Satan is is as unusual as is his relationship to Christ. When one reads Blake, he at times sounds like a fiery street-preaching Baptist, calling everything under the sun Satanic. But he has a very precise meaning of this concept unique to his own poetry, rather unlike the common one. Blake’s definition of Satan is: “the limit of opacity in man”. This is to say that by Satan, Blake refers to not some horned monster, but rather obscurity, lack of self-knowledge, and this specific type of blindness to one’s imagination we discuss above.
Blake‘s use of the term may seem strange, but he is getting this directly from the source — the Book of Job — in which Satan is introduced as the villain who attempts to sway Job away from his faith in God. The word “Satan” is a Hebrew term meaning “adversary”. This is a legal metaphor. If one was brought to trial in Israel in those days, a “satan” would be tasked with making the argument that you are guilty — Satan means “the prosecuting attorney”. Satan is repeatedly trying to convince God that Job will sway in his faith, but Job never abandons God. So Satan, properly speaking, is not some horned evildoer, but instead the voice in your head convincing you that you are guilty, condemned, not worthy of the love of God. Satan is the prosecuting attorney, whereas God is your defense. Satan is doubt. Satan is self-hate. Satan is the fear of God, from whom according to Blake, we have never had anything to fear.
Throughout this text, we will critique a number of formal systems. The Rationalists love formal systems, because certain conclusions derived from them can be shown to be exactly and precisely true. A formal system consists of a set of axioms, and then a set of rules for generating propositions within the system. Everything proven within a formal system, as within mathematics, is ultimately a tautology, and can be demonstrated beyond doubt, as long one agrees with the validity of the axioms.
But we look at formal systems with the same skepticism that Blake looks at Natural Religion. There is always some kind of imaginative work being done to generate the system in the first place. These systems are borne at a given moment because there is an event actually happening in the world to generate the spark of insight which allows a system to be formalized. But then, after the system’s formalization, man forgets the images and desires that swept through his intellect to generate the system, and imagines that the system was present before time began, as it becomes a tautology, and all tautologies hold in all possible worlds. To Blake, that moment of forgetting, and nothing else, is the work of Satan.
We are finally ready to formally define the Blakean Critique which is our method throughout this text. We can define a process of excavation which happens in six steps:
First, we must show where and why a system of thought originates. We must historicize it, we must describe how the formal system came from the imagination first to then be formalized, which is to say, we must discover the system’s initial referent in the world.
We then show that this corresponds to a specific “architecture”, a “factory”, a mill. One can only imagine the horror upon which the Englishmen of Blake’s generation would have felt when encountering the proliferation of factories in the early Industrial Revolution; chewing up and spitting out London’s poor, big black tarantulas dousing the sky with black inkjet clouds. Though ostensibly a factory is a machine for birthing textiles, or grain, or spare parts, or whatever it might be, to those on the ground it must have looked like a factory is a machine for producing more factories.
Then, we must show that that this factory presents a structure for desire which externalizes desire from the factory’s creator. If the factory is meant to fully represent its creator’s wishes, it soon nevertheless becomes “loathed by its possessor, the same dull round”. Its creator will still generate new wishes, but these are now secret sinister wishes, unable to enter the machine. There is a great Marxist text by one Christopher Caudwell called The Crisis in Physics which makes the argument that Newtonian physics — as a science in which it’s possible for man to view himself as detached from a world of atoms he can analyze objectively — is only possible to be conceived of in an era after the bourgeois start overseeing planned factories for the first time. The relationship between man and nature becomes the same as the factory boss and the workers he oversees. Nature becomes a mechanism which man merely observes, and does not participate in.
Next, we must show that in each case, these structures of desire do damage by encircling and inscribing desire, telling it there is no way out. It is all too easy to do this with the example of Newtonian mechanics. We have so many friends who have fallen into despair at a young age because they believed that they lived in a deterministic world composed of billiard ball atoms knocking around with precisely calculable trajectory, and felt that knowing this robs the world of all its poetry and purpose. We just want to shake them violently and say, you know, none of that is real! But it is difficult to cure someone who has been given a slow titration of poison his whole life.
After that, we must show that in each case, desire in practice actually escapes the factory. Blake in his works of lyric poetry: Songs of Innocence and Songs of Experience, contrasts hymns towards a tranquil childhood that a child should be able to expect in the former volume with the brutality of the Industrial Revolution in the latter, in which child labor was commonplace.“Because I was happy upon the heath, And smil'd among the winters snow: They clothed me in the clothes of death, And taught me to sing the notes of woe. And because I am happy, & dance and sing, They think they have done me no injury: And are gone to praise God & his Priest & King Who make up a heaven of our misery” is the song Blake puts in the mouth of a child chimney sweep. Though immiserated, there is hope here — because the child stuck in the factory continues to sing.
Finally, and crucially, we must show that if the breadth of man’s desire continues to be ignored and suppressed by the factory, we end up in the pathological case where the shape of the factory seizes the imagination in order to extend itself to all things. “The same dull round, even of a universe, would soon become a mill with complicated wheels”. A mill with complicated wheels: the factory needs to add more parts to itself and become more and more complicated the more its owner insists on replicating it everywhere it does not belong. This is the end result of realism, and it is essentially psychosis, machine-psychosis, the inability to even conceive of an escape from the factory’s plan. No matter where one looks, one sees the factory replicating itself in the patterns on the leaves in the trees, the clouds in the sky, like the dog faces in Google’s DeepDream.
In an earlier piece by Harmless titled Utility Monster, written around the downfall of FTX and Future Fund, we wrote an analysis of Sam Bankman-Fried’s utilitarian psychosis, which can serve as an illustration of the method of critique outlined above. Sam Bankman-Fried was a true philosopher-king, a devotee of the utilitarian school of morality present in the Rationalist subculture and Effective Altruism. It seemed to be the case that there was no sphere of life that SBF did not believe utilitarian morality could extend to. In theory, applying the utilitarian method would result in the most rational, calculated, efficient method of planning a company, but in practice, SBF’s life became an ignominious disaster of unprecedented proportions, a sordid story filled with sex and drugs.
To illustrate what went wrong, we can apply the Blakean Critique towards Effective Altruism. The subject of utilitarianism in particular will be treated in greater death in the chapter bearing its name, but briefly: utilitarianism emerges in a period of capitalist development when large-scale accounting becomes necessary, and it becomes possible to imagine a God that calculates the general good with the same method that an accountant uses to take stock of his inventory. The “factory” that utilitarian morality corresponds to is more and more accounting, bureaucratic bodies taking statistics, and so on. But there are all sorts of things the value of which cannot be accounted for by a bureaucratic body — one cannot account for, put a number on the value of, such as art, intimacy, friendship. Effective Altruism is step 6, the doubling down, the attempt to extend the structure of capitalist accounting to all spheres of life. Eventually — psychosis.
What we claim is: Rationalism, and its congregation of the Singularity, is a step-6 critical condition, realism gone way too far, now fallen off the cliff into insanity. Rationalism is the idea that when you assemble a number of formal systems together you get God, God-AI. What a thought to behold. There are a number of pieces to this puzzle, and we will have to tackle them one by one.