Continued from yesterday. Here is Part 2 of my new essay on writing and publishing. Please remember that, if at any point you decide that you would like to have all of this essay available in one lump, you can go here and download the whole thing as a PDF file. You can then print it out, pass it to friends, et cetera.
Part 2: The experiment with rats
The experiment described
Taleb is under contract to produce a book on black swans; and, at the time of writing this essay, he has posted a draft chapter from the book on his web site. The present title of the chapter is ‘On the Invisibility of the Drowned Worshippers’, which is a reference to the work of Francis Bacon in the seventeenth century.
Bacon, it seems, was once shown a set of portraits of men who had survived shipwrecks; these portraits had been commissioned by the Church authorities. The subjects of the portraits were all good Christians: before embarking on a voyage they had taken the steps recommended by the Church for those in peril from the sea; these preparations no doubt included going to communion, spending a great deal of time praying, and, I imagine, making a substantial contribution to Church funds.
The result of these Christian preparations was that, when their ship sank, lo and behold, God rewarded them by saving their lives: hence their portraits, which were presented by the Church authorities as an example to others.
It was Francis Bacon who asked one of those questions that you’re not supposed to ask. Where, he enquired, were the portraits of those mariners who, before their voyage, had also gone to communion, said their prayers, and made a contribution to Church funds, but had nevertheless drowned? These were the ‘drowned worshippers’ who had become invisible. The Church, through some oversight, had not mentioned them, or commissioned their portraits.
The drowned worshippers constitute a phenomenon which will often be mentioned in this essay: survivorship bias. We human beings are fallible creatures, and we have a habit of seeing only the survivors of a set of experiences. This, Taleb tells us, is an error in thinking which can get us into serious trouble.
In the course of his draft chapter from the book on black swans, Taleb sets out to describe several other kinds of erroneous thinking. In order to illustrate these errors, he asks us to imagine an experiment with rats. (And since this is a hypothetical experiment I can give an absolute assurance that no animals were harmed during the writing of this essay.)
Suppose, Taleb says, that we have access to a city full of rats: rats of all kinds, fat, thin, sickly, strong, well proportioned, et cetera. In order to determine which of these rats are the strongest, we select a random sample, one that is truly representative of the rat population as a whole. We then put the sample group into a large vat and subject the rats to increasingly high levels of radiation.
As the levels of radiation increase, many of the rats will die. By the end of the experiment (unless you take it too far and kill them all) you will be left with a small number of survivors.
Taleb uses this hypothetical experiment, and its results, to illustrate a number of errors in thinking.
Flaws in the methodology
First, we need to think about the experimental procedure itself. Alert readers will already have noticed that the methodology of the experiment, as described for the purposes of this essay, is flawed.
The intention is to select the ‘strongest’ rats. But while the experiment will certainly reduce the numbers of rats, there is no guarantee that the survivors will be the strongest.
The surviving rats would only be the ‘strongest’ in the limited sense that they were the ones best able to withstand increasing doses of radiation. They might not be the strongest in terms of ability to survive without water, or ability to climb fences. The ability to withstand radiation might or might not be a useful characteristic in the real world.
Second, at least some of the survivor rats may have survived by pure chance. At the moment when the next blast of radiation was administered, a ‘weak’ rat may have been shielded from radiation by a ‘strong’ rat. Furthermore, there might be some variations in the way in which the radiation was distributed around the vat: in some spots (perhaps towards the rim) the rats might absorb less than in other spots.
In short, the design of the procedure leaves much to be desired; and this, we shall see, is the case with some procedures in publishing.
Survivorship bias
We have already noted the phenomenon which is known in statistics as survivorship bias; and history suggests that it is all too easy to fall prey to this lax way of thinking.
Survivorship bias involves mistaking what you see for what is really there. The tendency is for human beings to see only the survivors of some set of circumstances, and to ignore those who, for one reason or another, disappeared or dropped out as events proceeded. We often find ourselves earnestly discussing the traits in a cohort of survivors when, in truth, those traits are no different from those in a much larger population; if you consider the circumstances carefully it may be apparent that the survivors emerged as a result of sheer randomness, rather than through the possession of some special qualities.
It may be, if clear thinking is applied to any set of events, that those who dropped out, voluntarily, or were eliminated, perhaps as a result of chance, have at least as much to teach us about what is important and relevant as those who survived.
Nietzsche’s error
Nietzsche is responsible for the aphorism ‘What does not kill me makes me stronger.’ If repeated, in a suitably solemn tone of voice, in front of a group of people who are aware that Nietzsche is a Big Name in Philosophy, this dictum may well induce nods of agreement. And you may sometimes hear people say, after a young person has had some kind of setback, ‘Well, he will be all the better for the experience.’ Once in a while it might even be true.
In general, however, Nietzsche’s aphorism is nonsense. On the physical level, a car crash which brings you close to the point of death may leave you paralysed for life. So, although you are not actually dead, you are certainly not stronger than before. And in an emotional context, a bereavement which causes you seriously to contemplate suicide may, even if you do not succumb to the temptation, leave you lonely and depressed.
So it is with our rats. The rats which survived our experiment are by no means necessarily stronger. In reality, there is a good chance that they will be weaker. Radiation is not often good for you.
Taleb quotes a newspaper article about the Russian Mafia, which referred to the new generation of gangsters as being ‘hardened by their Gulag experiences’. But, if any modern gangsters have indeed survived the Gulag, they are hardly likely to have been ‘hardened’; the camps were not famous for providing fitness-training courses.
Despite these readily apparent flaws in Nietzsche’s aphorism, there are circumstances in which people behave as if it were true. We often assume that survivors of some intense selection process are stronger than those who were eliminated. We assume that survivors are necessarily the best of the cohort; whereas in reality the procedure may have been flawed and they may simply have emerged by chance.
The dead rats, you see, are no longer around to steal our cheese, or to give us Weil’s disease. Whatever their strengths or weaknesses, virtues or vices, they are gone, and are mourned by nobody. We forget them. But it is a mistake to overlook them because some of them, at least, might well have had characteristics which would be valuable outside the context of our flawed selection procedures.
Another point to note is that the survivors, the chosen few, will themselves tend to conclude, falsely, that they are necessarily superior to those who died. Usually, the nature of rats being what it is, they will conclude that they are infinitely superior to the dead. Some humans share this characteristic.
The swimmer’s body
Another mistake in thinking is described by Taleb as ‘the swimmer’s body’ error.
It is observable that athletes who participate in different events have differing physiques: rugby forwards are big and beefy; high jumpers are tall and slim; and swimmers often have rather beautifully proportioned bodies with ‘elongated muscles’.
Observers who are fuzzy thinkers sometimes conclude from this that if they want to have a beautifully proportioned body they should take up swimming. But this is what is known, in popular parlance, as getting things arse over tip.
Swimmers do not end up looking beautiful because they took up swimming; they excel at swimming because they have the kind of physique which lends itself to fast progress through water, and which is itself aesthetically pleasing, the more so when developed by exercise.
The swimmer’s body error often involves the false attribution of a particular outcome to a particular set of traits. In the business world, it is not unusual to find books which purport to identify the common characteristics of successful entrepreneurs. The analysis is usually based on the careers and backgrounds of those who have got to the top, and it generally yields a list of such ‘success factors’ as optimism, confidence, and a willingness to take risks. Yet had the researchers interviewed those who failed in business, going bankrupt within two or three years, they would undoubtedly have found the same characteristics present.
By the way, it does not necessarily follow from this (should you be tempted to think it) that the successful entrepreneurs are successful only as a result of chance, or random events; but it is a mistake to attribute their success to qualities which they share with many of those who failed.
Casanova: a case history
Casanova’s famous memoirs tell of a life of continual setbacks and escapes from dangerous situations, followed by periods of prosperity and advancement. His was a roller-coaster ride, and it seems that, when it came to overcoming difficulties, Casanova was an unusually resourceful man.
This impression is false. In the above paragraph, the words ‘it seems’ are the crucial ones. We know about Casanova simply because, through a series of accidents of fate – random events – he happened to survive long enough to write a ten-volume set of memoirs about his escapades.
Other adventurers, thousands of them, doubtless got into similar scrapes and difficulties, but they ended their days on a dueller’s sword or died in a debtors’ prison. Casanova was not unusually talented or resourceful: there was nothing to differentiate him from any other self-serving layabout except that Madame Randomness happened to take a shine to him.
Next step
It would be possible to spend some time discussing Taleb’s arguments and examples, which are not without their own weaknesses. That, however, is not the point of this essay. The point here is to learn to think clearly about writing and publishing; and to that end we will now apply some of Taleb’s ideas to the present-day circumstances of writers and publishers, and see what emerges.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment