On the Survival of Rats in the Slush Pile -- Part 1

Continuing from yesterday, here is Part 1 of my new essay on writing and publishing. Please remember that, if at any point you decide that you would like to have all of this essay available in one lump, you can go here and download the whole thing as a PDF file. You can then print it out, email a copy to friends, have it framed to hang on the wall, et cetera.

Part 1: The concept of black swans

A definition

Before we proceed further, I need to introduce you to the concept of black swans.

In his book Fooled by Randomness, Taleb defined a black swan, in the context of investment management, as an unexpected and catastrophic event which could destroy even a so-called ‘master of the universe’.

Subsequently, Taleb wrote an essay entitled Fooled by Success: the Black Swan and the Arts; and in relation to the arts Taleb defines a black swan as ‘a piece of work that, unexpectedly, captivates interests, spreads like wildfire, and dwarfs other contributions.’ As examples of these massively successful phenomena he quotes the Harry Potter books, Mel Gibson’s movie The Passion of the Christ, and the success of the Beatles. (Taleb, you see, is not an intellectual snob.)

Black swans in the arts are distinguished by the fact that they occur extremely rarely (when compared with the total amount of work which is offered to the public), and yet they have enormous impact. They provide, in short, exactly the kind of success that every writer (or publisher, producer, actor, et cetera) yearns for.

Taleb argues that these black swans are random events. After they occur, many observers claim to be able to see that their success was inevitable, for reasons which they then proceed to define; most of these reasons have to do with the innate qualities of the work in question. But Taleb maintains that these post-event explanations are essentially false and unreliable. They are highly influenced by hindsight bias, which makes use of ‘posterior information’. Observers of black swans tend to overestimate the analysable and underestimate the non-explainable.

Above all, those who claim to understand black swans (but only after they have come into view) are neglecting the ‘silent evidence’. Taleb maintains that if we are to understand the factors behind the huge success of Harry Potter (to mention but one convenient example) we need to do more than recognise the qualities that are present in the Harry books; we also need to consider the qualities which were present in the thousands of manuscripts which were rejected by agents and publishers and which never even made it into print.

Similarly, Taleb suggests, if we were to try to pin down the reasons for the rise to stardom of some young actor, we would need to consider the qualities present in other young actors – those who were not, for some reason, given a part for which 400 men auditioned. We would need to identify what it was that caused the producer to choose our future star for a part in his film rather than one of the 399 other candidates; and we would be likely to find, Taleb avers, that the producer’s choice had nothing to do with ‘talent’, however defined. Or, to put it another way, we would find that many of the rejected applicants had just as much talent as the future star.

An example: Harry Potter

Just in case you don’t know the much-told story, it is worth recording that the first Harry Potter book was rejected by every major publisher in London (some sources say as many as 20); and when it was eventually bought by Bloomsbury, the one publisher who showed the smallest degree of interest, they paid but a small sum of money for it (sources say between £2,000 and £3,000). Clearly, none of the ‘experts’ who read the book in manuscript, and rejected it, had the slightest inkling of the massive money-making machine which they held in their hands.

Implications

The Harry Potter case is an all-too-typical example of the failure to identify a black swan at an early stage. And yet it is highly desirable to identify them, if possible, because of their massive power to generate income. The black swans dominate their competitors in a way which distorts the rewards available: they are part of, and may be the cause of, a winner-take-all mechanism.

In the book world, what this means is that bestsellers tend to become massive, while sales of ‘ordinary’ books are minuscule. It is not that bestsellers sell twice as many copies as the average novel: they sell hundreds of times as many.

This circumstance is observable in most of the arts: in other words, you are either overwhelmingly successful, in terms of money, fame, and reputation, or you are nothing.

Interestingly, the same clustering effect can be found in other contexts, outside the arts: similar concentrations can be found in the academic-citation system, and it doesn’t matter whether the academic field is physics or social science.

It is also important to note that the concentration effect becomes more marked, not less, as the size of the pool of works on offer increases. The more product that is available, the more the big hits dominate and stand out.

How black swans come about

The appearance of a black swan is influenced by, among other factors, the ‘tipping-point mechanism’. Contagious diseases spread furiously above a certain minimum level (the tipping point), but die down below that level.

In the arts, the mechanisms of contagion are accelerated by the media, and, of course, by word-of-mouth recommendation. Thomas Gilbert and his colleagues at the University of California have used some statistical methods which are normally applied to phenomena such as the spread of diseases, or earthquake aftershocks, in order to analyse the spread of information about books. They distinguish between exonogous (external) and endogonous (internal) stimuli. Publishers use exonogous methods of generating awareness of a book when they give it a large advertising budget; endogonous shock is what occurs through one person recommending the book to another.

Both exonogous and endogonous stimuli play a part in turning ugly ducklings into black swans. A large advertising budget may generate some initial awareness of the product, but it does not inevitably create a black swan; it may evoke nothing more than yawns. Endogonous effects, by contrast, are absolutely essential to the emergence of a black swan, whether it has a large publicity budget or not, and they cannot always be created, no matter how much money is spent; they either occur spontaneously, or they don’t.

Taleb’s principal conclusion about the black-swan phenomenon in the arts is that the process is ‘far less fair than it seems to participants’. The randomness of the system is greatly underestimated. Furthermore, people involved in the arts tend to suffer from overconfidence, and overestimate the chances of their own success. This, believe me, is particularly dangerous for writers, but it can also be catastrophic for publishers who commit massive resources to books which flop. Example: the Dorling Kindersley collapse which occurred as a result of overprinting Star Wars books.

Two observers reaching the same conclusion

One of the reasons why I find Taleb’s paper on black swans in the arts so intriguing is that it echoes, with added scientific and intellectual underpinning, my own conclusions, reached earlier and independently. My views on the ‘secret of success’ for writers were set out in Chapter 9 of The Truth about Writing: there I argued that success for writers is determined by circumstance.

Circumstance, I said, is a factor which some might call chance, fate, luck, serendipity, or karma. But the true definition of circumstance, for my purposes, is everything that you cannot control, or even influence.

Here is an example of circumstance, drawn from the film industry. In the 1950s, the actor Montgomery Clift turned down the lead parts in four films. He declined (1) the part in Sunset Boulevard which was later played by William Holden; (2) the Marlon Brando part in On the Waterfront; (3) the James Dean part in East of Eden; and (4) the Paul Newman part in Somebody Up There Likes Me. As you will already have noticed, if you know anything about the history of the cinema, each of the actors who picked up a part that had been rejected by Montgomery Clift used that opportunity to establish his own name; and they all became stars as a result. None of which would have happened if Clift had decided to play any of the parts himself.

I even compressed my idea about circumstance into a mathematical formula, or expression (of sorts):

S :: C

This formula holds true, I suggested, where S = Success (however defined), and C = Circumstance (as defined above). The symbol :: was introduced by William Oughtred in 1631, and it means ‘varies as to’. S :: C is therefore a compact way of saying that Success varies according to Circumstance.

Both Taleb and I, therefore, approaching matters from wholly different directions, have concluded that success in the arts, and particularly success as experienced by writers, is a random event. It is not determined by hard work, who you know, or talent (not, at any rate, above a certain level).

You, the reader, will probably resist the Taleb/Allen conclusion at this point; but you at least are thoughtful enough to be reading this essay, so please reserve final judgement on the causes of the black-swan phenomenon until you have read the rest of the argument.

Taleb and I are not alone in reaching our conclusion about the effects of randomness aka circumstance.

In his autobiography Nudity in a Public Place, the actor John Nettles quoted a friend of his whom he described as ‘a great literary figure and a major celebrity’. This individual remarked to Nettles: ‘Nothing is more common today than successful men with no talent…. Success and celebrity do not necessarily depend on talent in these dog days and it is a good thing you never ever believe they do, otherwise you might miss out on the joke of the century.’

On a less elevated level than John Nettles’s friend, one of the former Spice girls recently spoke with some awe about the popular-music business; she was amazed, she said, that ‘so many people with so little talent are making so much money.’

The remainder of this essay will enlarge on the idea that huge literary and/or commercial success for writers, who are not already famous names, comes (if it ever does) in the form of a black swan, or a random event. The discussion will then be used as the basis for generating strategies which might be adopted by those who work in the book trade. The strategies will, however, be of particular importance to writers – especially if they wish to avoid lasting psychological, and hence physical, damage; and if they wish to avoid allocating scarce resources (e.g. time and energy) to an almost certainly futile project.