Continued from yesterday. Here is the second half of Part 3 of my new essay on writing and publishing. Please remember that, if at any point you decide that you would like to have all of this essay available in one lump, you can go here and download the whole thing as a PDF file. You can then print it out, pass it to friends, et cetera.
Survivorship bias
Indulgence in the faulty thinking known as survivorship bias is universal throughout publishing.
Slush-pile readers, editors, publishers in general, agents, critics, media commentators, readers, and (published) writers – all have a marked tendency to assume that the slush-pile procedure actually works, and that the survivors are indeed the best.
If pushed up against a wall, with a loaded gun inserted into a nostril, most publishing professionals will admit that the selection of the ‘best’ books is subject to occasional errors. (We have encountered plenty already.) But the next day, the same person will fall back into the old (and incorrect) mode of thinking, and the slush-pile procedure will continue to operate, unchanged.
This is probably the place to point out that the most dramatic illustration of survivorship bias in the book world occurs following the award of a prize. Let us consider, for instance, the Booker Prize (properly the Man Booker Prize), which is currently the most prestigious literary award in the UK.
If you and I are presented with a piece of string, and are asked to guess its length, you may say that it is 15 inches long, and I may say that it’s 18 inches. In order to resolve our disagreement, we can measure its length against a ruler and come to a conclusion which all sane parties will accept as correct.
But when you and I are faced with a novel, and asked to say whether it is a masterpiece or an overblown piece of self-indulgent nonsense, there is no universally recognised scale against which we can measure the book and come to a clear conclusion. Judging a novel is a matter of taste and sensibility, and you are likely to maintain that your taste and sensibility are superior to mine.
As far as the Booker Prize is concerned, it is safe to say that the choice of the ‘best’ book of the year is inevitably a matter of opinion rather than fact. And not even unanimous opinion. In almost every year there are press reports of disagreements among the judges, and in some years we hear of ‘compromise choices’ or the chairman’s casting vote. We also know that, in one particular case, the eventual winner was unusually fortunate.
In 2002 the winner of the Booker Prize was Life of Pi by Yann Martel. Many newspaper reports at the time told us that this book had been rejected by Faber, the firm which had published Martel’s earlier work; the book had also been turned down by at least five other major publishers. So if the eventual publisher, Canongate, had not taken the book, it is likely that the manuscript would have remained in the author’s filing cabinet. Furthermore, if the book had been accepted by one of the bigger firms, it would not even have been entered for the Booker Prize in the first place, because the big firms are only allowed two nominations and have to enter their most famous authors; if they don’t, the famous authors are likely to go elsewhere.
The Life of Pi saga provides a beautifully clear demonstration of the random nature of decision-making in publishing. Here we have a book which was turned down for publication by numerous ‘good judges’. It was entered for the Booker Prize by a small firm which had no stronger candidates. And it so happened that the particular set of judges who were reading in 2002 happened to like it best. Or a majority of them did.
All rational observers will agree that Life of Pi, or any other Booker winner, cannot sensibly be described as the best book of the year in any absolute sense. The Life of Pi episode shows us, undeniably, that there might have been other books that year which could, quite possibly, have found favour with the judges if they had been submitted. The most that can be said of the book which wins the Booker Prize is that it is the one which (of those presented for their judgement) the judges liked the best.
But observe, please, what happens when the winner of the Booker Prize is announced (in any year). What happens is that the media, the critics, and the public, all behave as if there is some absolute sense in which the winner is the best book of the year. They act as if the book has been held up against a ruler, a universally agreed scale, and has been found, indisputably and scientifically, to be ‘better’ than any other.
This very week, for instance, I was given a copy of the New York Review of Books, in which there is a lengthy review of the most recent Booker winner; the article runs to 108 column inches. Similar articles are no doubt published every year. And this ‘superstar treatment’ will be repeated in newspapers and magazines throughout the English-speaking world.
It is the winning novel, please note, which is treated in this way – not the runners-up; and certainly not the good books which were not submitted by their publishers; and definitely not the books which didn’t even make it into print. It is the winning author who will be interviewed on television, invited to writers’ conferences, and made the subject, in due course, of earnest PhD theses by bespectacled young people who can think of nothing better to do with their time than waste it by deconstructing a novelist’s prose. This is the winner-take-all mechanism in its most unforgiving form.
The runners-up, the non-shortlisted books, and the unpublished books, all those are the drowned worshippers and dead rats of the fiction world; they are losers who disappear from our sight, never to be heard of again. And yet we know, beyond doubt, that but for the workings of randomness, which favoured the winner and disfavoured the drowned worshippers, there might be one, ten, or a hundred other books which could, in different circumstances, have proved to be more enticing to the judges than the eventual winner.
Survivorship bias in the book world is therefore brutal, vicious, and deadly. There is no point in complaining about it: it is just the way things happen. The world in general, and the book trade in particular, is unfair, unjust, and patently absurd in its workings. But all those who work in the book trade, in particular those who write and sell novels, need to be aware of this situation. And they need to ask themselves whether a business in which randomness is so powerful a factor in the distribution of rewards is a business which sensible people should allow themselves to be involved in.
Nietzsche’s error
Nietzsche, you will recall, told us that ‘What does not kill me makes me stronger.’
What, if anything, can we make of Nietzsche’s alleged words of wisdom in relation to publishing; in particular, in relation to the slush pile?
Having your novel rejected by a slush-pile reader certainly does not kill you. So suppose we examine the view that rejection makes you a better writer.
Rejection will certainly have noticeable and possibly dramatic effects, the chief of which are usually emotions such as anger, disappointment, and disgust. (The rats, by the way, had no choice about participating in the experiment. But it is not compulsory to submit anything to a slush pile. Therefore any injury which results from slush-pile experiences is a self-inflicted wound.)
Does rejection make you more determined to succeed? Possibly. But it may also cause a writer to decide (probably quite correctly) that this writing business is a fool’s game. A writer who produces a professional piece of work, and is yet unable to persuade an agent to take him on, or a publisher to invest in his book, may well conclude that the slush-pile procedure is impossibly flawed and that sensible people would not continue to waste postage on it. I happen to believe (without empirical evidence to support my view) that the writers who come to this conclusion are probably the ones who would do best in publishing if they actually persevered; because they, at least, have proved capable of rational thought. By contrast, it is the all-time, never-going-to-be-publishable losers who will see rejection as a reason to redouble their efforts.
The rejected writers, whether they drop out altogether or not, disappear from sight; just like the dead rats. Their strengths and weaknesses, virtues and shortcomings, are lost to us. Let us consider the consequences of that.
We noted, above, that MASH was rejected by 21 publishers, and The Scarlet Pimpernel by 12. Suppose both these authors had decided not to bother with the book business any more, after 20 and 11 rejections respectively. In that case these two works would have been lost to us for ever.
And the survivors, of course – those who are selected for submission by agents, and are subsequently put under contract by publishers – they will tend to assume, human nature being what it is, that they survive and prosper because of their natural inborn talent plus a great deal of hard work. Such writers will assume, in 99 cases out of 100, that they are superior to those who were rejected.
The same assumption will confidently be made by those who have done the selecting; because they, remember, have sufficient skill to decide within fifteen seconds, or a page and a half, whether a book is worth pursuing.
Such assumptions, on the part of both selectors and selectees, are in my view wholly unjustified, and I hope we have accumulated enough examples by now for you to understand why.
The swimmer’s body
Errors in thinking of the ‘swimmer’s body’ type are widespread in the field of writing and publishing.
Essentially, swimmer’s body errors involve getting things back to front. For example, in most trades and industries, companies boast about the high sales of their product; but publishers boast about its high cost (in the form of advances to authors).
We also noted earlier that it would be foolish to take up swimming in the hope of acquiring a beautiful body. Champion swimmers are not beautiful because they swim; they swim because they have bodies that are eminently suitable for swimming.
Similarly, it would be foolish for writers to emulate the working methods of successful and famous writers on the assumption that use of the same working methods will necessarily lead to similar success.
Many years ago, when I was young and impressionable, I read the famous volumes of interviews with writers which had been published in the Paris Review. What struck me most forcibly was that many of those writers seemed to operate on the basis of inspiration (or claimed to). What happened, allegedly, was that these writers never pre-planned their novels: they simply sat down at their desks and embarked on the great adventure of writing fiction.
Two points. First, people who have become rich and famous, often after many years of struggle, do not normally reveal all their trade secrets, not even to a sycophantic, starry-eyed interviewer from a prestigious literary journal; so we should take these revelations of ‘working methods’ with a shovelful of salt. And second, common sense surely tells us that writing a novel without having a clear concept of its overall shape, at the start, is a recipe for disaster. It compares with a surgeon going into an operating theatre without knowing whether he is going to remove an appendix or perform a lobotomy; and who knows, if he gets really inspired, he may amputate the patient’s left leg.
And yet, and yet…. As you and I know full well, young and ambitious writers are trusting souls, a trifle simple-minded pretty much by definition; and if a working method is good enough for André Gide and William Faulkner (or whoever), then by golly it’s good enough for them – common sense notwithstanding. Thus is futility piled upon futility.
No. We need to look a little harder at the concept of the swimmer’s body to discern anything truly useful to the art of writing.
If it is true that champion swimmers become champions mainly because of their basic physique, what is the writer’s equivalent?
The answer, surely, is a capacity to write effective prose; a writer needs to be able to use the English language to full effect.
In England, which is the source and home of the English language, the ability to write even halfway decent prose is today a rare quality. The cause, as you doubtless realise, lies in the failure of English education, a topic too big to go into here. Suffice it for our purposes to note that hardly anyone under the age of fifty has been taught to spell and punctuate properly, much less polish their prose style.
A powerful command of the English language does not necessarily require a massively high IQ: Muhammad Ali had the former but not the latter. Neither does it require a particularly long or arduous training; but it is a training which is best given on a one-to-one basis (I used to teach English to small boys myself), and in recent years such personal instruction has seldom been available.
It is doubtless the case that many slush-pile readers use the ‘basic command of English’ test as a preliminary filter; they discard immediately those books which may be described as semi-literate. I do so myself, as a matter of fact. In my capacity as owner and operator of a small press (Kingsfield Publications), I am regularly sent manuscripts by hopeful writers. (This despite a notice on the firm’s web site telling them that such submissions are a waste of time.)
Sometimes the submissions come on paper, and sometimes as email attachments. Either way, if I look at them at all, my inclination to read more than a few lines is heavily influenced by their grasp of, say, punctuation. A recent manuscript came with quite a sensible covering letter, but the first page of the accompanying text revealed a complete inability to punctuate dialogue in the orthodox manner. It is perfectly true that some writers adopt their own method of punctuating speech: James Joyce is an example. But in this case I was convinced that the oddities were the result of incompetence. Enough. I read no more.
It is not, in my view, unreasonable to use this crude ‘basic command of English’ filter to reduce the size of the pile of manuscripts. However, it is a mistake to assume that this test is reliable.
I once found a manuscript which was more or less functionally illiterate – being spattered with hundreds of elementary errors – but which, when read aloud, revealed that the author had a perfectly clear story to tell in her own distinctive voice. Unfortunately, the economics of having such a text moulded into shape by a professional editor usually preclude any further consideration of the manuscript – unless the publisher is prepared to gamble that it’s a black swan.
In most cases, it is evidently at the later stages of the slush-pile selection process that mistakes are made, rather than at the stage where the basic command of English test is applied.
In statistics, you commit a Type 1 error if you reject the hypothesis when it is true; and a Type 2 error if you confirm the hypothesis when it is untrue. As we have already seen, the slush-pile procedure produces both these types of errors: not all the time, but frequently. Books which turn out to be excellent (either by literary or commercial criteria), are rejected; and some books are selected which later prove to be duds.
The selection of duds does not matter in and of itself. Most books are duds, in the sense that they do not become black swans. They are capable, competent, even entertaining pieces of work – but they do not generate massive enthusiasm, either in the publisher or in the book’s readers. Such duds are harmless and may even be beneficial in that they give the authors practice in the difficult art of learning to write; they invite some critical attention and slowly build a reputation; and with sensible costing they may even do rather better than break even.
Where the selection of duds does do great damage is in those cases where the wrong book is selected for the ‘full treatment’. Giving a book the works these days involves paying a big advance (which is used to generate publicity and arouse interest), and providing a large publicity budget.
If both of these big-money sums (big, at any rate, by publishing standards) are committed, and the book fails, damage has been done in several quarters. The publishing firm’s profits suffer; its reputation is damaged; the author’s confidence may be shattered; and booksellers will be disgruntled.
In the past, publishers were often able to keep the gory details of such failures more or less secret within the firm. But with the advent of computer-based point-of-sale databases, which record actual sales in the high-street shops, and which are accessible to almost everyone in the trade (on payment of substantial subscriptions), the truth will out.
In the 2002 edition of the Writer’s Handbook, Barry Turner gave a couple of examples. Author A was paid £300,000 for two novels. The first came out in 2001 and sold less than 4,000 copies. Author B was paid £250,000 for one book, by a different publisher, and this one generated sales of 1,500 copies.
Private Eye recently identified a similar case. An author with an established but modest track record was paid £500,000 for her new book. The hardback (published in 2003) sold just over 2,000 copies, and the paperback (2004) managed 10,000.
Similar errors are made on a smaller scale. With my publisher’s hat on, I once sold a book to a major UK publisher for £15,000. Five years later, when the rights reverted, there was still an unearned balance of £11,000 on the royalty statement.
Some of the deals that are announced in UK publishing are so bizarre that in any other industry one would assume that money was changing hands in brown-paper bags in various Tesco car parks. In this industry, however, we can say with absolute confidence that the participants are more likely to be financially clueless than corrupt.
Casanova: a case history
You will recall that Casanova’s memoirs reveal that he continually escaped from tight corners, where his testicles or his life itself were endangered. We also noted that we would be unwise to conclude from these events that Casanova was exceptionally talented or clever. On the contrary, it was simply the random flow of events, over which he had no influence, which ultimately determined whether he lived or died.
In publishing, it is easy to mistake the effects of chance for those of unusual talent. Most readers will have had the experience of seeing a book heavily recommended in the press. A full-page glossy advert in the Bookseller or Publishers Weekly announces ‘a very special publishing event’. The book cover features endorsements from famous names (who by a curious coincidence are published by the same firm). And yet, when you actually come to read the book, you find yourself puzzled to understand what all the fuss was about. You sit there and ask yourself what you are missing.
The answer is that you are missing nothing, because there is nothing there to miss. You are expecting the book to live up to its hype; you seek evidence that it is quite exceptional. But the truth is that what you have in your hand is simply a perfectly workmanlike novel which, through the workings of randomness, has cleared successive selection hurdles and has survived to be given the benefit of a huge publicity budget. And that’s all.
How reliable and valid are the slush-pile results?
In statistics, two important concepts are reliability and validity.
A test, such as a questionnaire about people’s attitudes to a given topic, is said to be reliable if it yields the same results when completed by the same person at two different times (always assuming that the individual’s attitudes have not changed in the meantime). And validity can be defined as the extent to which an instrument actually measures what the researcher wishes it to measure (and not something else).
The slush-pile analysis, which takes place every day in hundreds of offices of agents and publishers all over the world, is clearly not conducted on a rigorous and scientific basis. Nevertheless, it will be useful, I believe, to consider to what extent the analysis is reliable and valid.
Let us remind ourselves of what it is that the slush-pile readers wish to ‘measure’. What they wish to do is to find those books which are going to be of most benefit to the firm in question, whether it be publisher or agent: the books which are going to make money and/or bring literary prestige in their wake.
In today’s book trade, it is certainly the case that publishers and booksellers alike are increasingly dependent upon a constant flow of big sellers. These can to some extent be manufactured, but the real prize is the black swan: the book by an unknown which will turn out to transform the revenues of all those stakeholders who get so much as a sniff of it.
In theory, slush-pile readers should develop, or be instructed to apply, a series of criteria (tests) which would be reliable, in the sense that if the same book was subjected to the test some months apart (after the first reading had been forgotten), it would receive the same rating on the second reading as on the first. And secondly, the test criteria should be valid, in the sense that they would lead to the infallible detection of any and all black swans.
Now…. There are not many things in the world of books which can be said to be certain; but one thing that I feel reasonably certain of (on the basis of the evidence quoted above, plus much else in a similar vein) is that the slush-pile procedure is pitifully unreliable, and it is not remotely valid. In fact, so utterly inadequate is the procedure (as generally applied) that it is tempting to say that you can almost guarantee that a black swan will be overlooked.
A general conclusion
I submit that we can now draw up a general conclusion for Part 3 of this essay. And the general conclusion is this: The selection of books from the slush pile, provided the books reach a certain basic standard of professionalism, is essentially random.
Please note: This does not mean that every writer whose book is chosen for publication is a very average talent who just got lucky, and that every unchosen writer is simply unlucky. Chance favours the prepared, and writers can certainly help themselves: they can make sure that their spelling and punctuation are perfect; that the manuscript is crisp and clean; that the story is not hackneyed and stale; and so forth. And agents/publishers can employ mature, intelligent and well-read readers, whose judgements are certain to be superior to those of recent graduates in Eng. Lit.
What our conclusion does mean is that the book-selection process is much more random than we think; and that unless you have objective evidence as to why one book was preferred to another, then the workings of randomness are the most likely explanation.
We have seen that the experiment with rats was flawed, and it failed to identify the ‘strongest’ rats. A similar set of flaws have been shown to be present in the slush-pile procedure. The procedure is unreliable in that ten publishers may reject a book, only for the eleventh to accept it and have a success with it. And the procedure is invalid in that, time after time, it fails to identify books which prove to be strong when eventually presented to the market.
Prior to the emergence of a black swan, there is no known test which will identify it as a superior entity. The book has its own unique characteristics, of course, but we cannot pin down beforehand what it is that will make it a huge success. If Barry Cunningham, of Bloomsbury, had been on holiday when Harry Potter first came into the office, and if the book had been rejected by an ‘intern’ (whatever they), Harry would have for ever remained, as the author herself pointed out, in his cupboard under the stairs.
Furthermore, in looking back on the black swan, after the event, we would be foolish to attribute its success to factors which it clearly shares not only with other books which were not huge hits, but also with other books which never even made it out of the slush pile.
In seeking explanations for black swans in publishing, it would be wise to accept that there may be no specific reasons – not even a spectrum of possible explanations. It may be that the most we can say about a black swan is that it sold a lot of copies because a lot of people liked it: a statement which is both tautological and unenlightening.
Naturally, the publishing powers that be, and many others who have an emotional or financial stake in publishing, will resist this statement that randomness lies at the heart of the book-selection process. It offends their sense of their own importance; and it is always tempting to indulge in what psychologists refer to as as hindsight bias.
‘I always knew that book was a winner, Daphne’ is a sentence which will be pronounced in many quarters when ‘that book’ proves to be a black swan. Conversely, ‘I never had any faith in it, Daphne, but the acquisitions committee just wouldn’t listen’ will be heard loud in the land when the unearned million-pound advance has to be set against profits at the year end.
Writers who are published, rather than rejected, will likewise resist my general conclusion; this is true whether they prove to be wildly successful or not. God knows, they have laboured hard enough, often for years, in the face of massive indifference, so it is understandable that they should attribute any kind of success to the innate qualities of their work; they will assume, whenever they make even a modest breakthrough, that it is all down to their natural superiority to the rest of the herd.
The opinion of such writers is reinforced when the reviews come out. (‘Brilliant’, said The Times.) It is easy to forget that, except for the big established names, reviewers do not bother to write about a book at all unless they can say something encouraging; space is too limited. This is survivorship bias again: the reviewers who read the book and hated it are lost to us, just like the dead rats.
The axe man cometh
In the course of time, after a publisher has put out two or three books by a writer, that writer is likely to be given the axe. There is nothing personal in this; it’s just the nature of today’s business. If a writer has been given two or three chances to find readers and to enthuse the critics, and the sales figures just aren’t there, and the reviews aren’t there either, then it’s Dear Jane time; and usually via an email to the author’s agent rather than during lunch at the Groucho (which is what it was when everyone had such high hopes).
But even the writer’s ultimate catastrophe, in the shape of a promising career terminated by an inadequate bottom line, even that will be ascribed by the victim to conscious and deliberate decision rather than to the effects of randomness. ‘The bastards didn’t give me the publicity budget they promised.’ Or ‘They got taken over by some South African fascists and my face didn’t fit.’
For a writer who has been dumped, it is a particularly bitter blow to discover that all that effort was futile. And the publisher isn’t going to be particularly thrilled by a less than sparkling sales record either. Which is all the more reason for us now to try to learn some lessons, and to develop strategies which will improve the situation.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment