Thursday, July 05, 2007

Nassim Nicholas Taleb: The Black Swan

Some introductory remarks

I've been putting this off, you know -- writing a review of Nassim Nicholas Taleb's new book, The Black Swan. Why? Because it's hard work for one thing: requires concentration; and because it's a deuced risky enterprise for another.

Consider, for example, the fate of the man from the New York Times. I think we can assume that, generally speaking, the NYT selects book reviewers who are fairly well qualified, if not uniquely well qualified, in terms of the subject matter of the book in question. But in the case of The Black Swan, the NYT man had no sooner published his review than Herr Doktor Professor Taleb produced a PDF file on his own web site, spelling out in some detail the several points on which the reviewer had misrepresented or misunderstood his book.

Which immediately demonstrates one of my problems in writing this review. I am very anxious, for a variety of reasons, not to misrepresent what The Black Swan tells us, or to identify the author as being something that he is not. But already, in paragraph two, I have been forced to write something which might suggest that Taleb is one of those four-star whingers who habitually write to newspaper editors to complain about the inadequacy of the reviews of their book.

I don't believe that Taleb is that sort of person at all. On the contrary, judging by the evidence of this book and its predecessor (more re the predecessor shortly), and other sources of information, Taleb seems to me to be a very even-tempered, easy-going, relaxed sort of person. He advises us not to worry about things we can do nothing about -- advice which I for one have tended to follow for some time, all the more so now that I know about black swans -- and I imagine that he has a rather calmer approach to reviews than do most writers.

That having been said, however, Taleb does express some disappointment, from time to time, that many so-called experts turn out to be anything but. This he refers to as the 'empty suit problem'. Many professionals, he argues, 'have no differential abilities from the rest of the population, but for some reason, and against their empirical records, are believed to be "experts".'

As examples he gives us clinical psychologists, academic economists, and political analysts. Such people, he warns us, 'dress up their expertise in beautiful language, jargon, mathematics, and often wear expensive suits.'

In one extensive empirical study, it was found that there was a negative correlation between the reputation of experts and their actual ability to make sound predictions. And if we transferred that circumstance to publishing, we might well find (does it ring any bells?) that the secretaries of big-time editors would turn out to be better predictors of a book's success than their bosses. (But still not very good.)

More specifically, perhaps, if I tell you that the whole point about 'black swans', as defined and described by Taleb, is that they are unpredictable events, you can understand our author becoming a trifle testy when newspaper reporters ask him to list 'the next ten black swans' for the edification of their readers. Which does not invalidate my contention that he is, by and large, a remarkably placid individual; particularly when you reflect upon the import of what he says.

All of this being the case, I think I am going to write you two separate reviews of Taleb's new book. In fact, I don't think I'm going to call them reviews at all: I shall describe them as discussions. First, then, a short discussion; and then a longer one.

The longer one will be subdivided into various bits; and I offer no guarantees of logical order or clarity of explanation. All I can promise you is that, in the faint hope of not making a complete fool of myself, I shall make every effort to ensure that what I tell you about the book is correct in terms of what the book actually says. When stating an opinion -- any and all of which are likely to be of dubious value -- I shall make it clear that it is my opinion, and not a matter of fact.

A SHORT DISCUSSION

What is The Black Swan about?

Well, the subtitle is 'the impact of the highly improbable'. And Niall Ferguson, author of a review of the book which Taleb approves of, explains:
Before the discovery of Australia, people in the old world were convinced that all swans were white, an unassailable belief as it seemed completely confirmed by empirical evidence. The sighting of the first black swan might have been an interesting surprise for a few ornithologists (and others extremely concerned with the coloring of birds), but that is not where the significance of the story lies. It illustrates a severe limitation to our learning from observations or experience and the fragility of our knowledge. One single observation can invalidate a general statement derived from millennia of confirmatory sightings of millions of white swans. All you need is one single (and, I am told, quite ugly) black bird.
This logical problem has often been remarked upon. For example, William James, the brother of Henry, was interested in psychic research. He soon became aware, however, that no matter how many hoax mediums you exposed, you were always left in a position of doubt. You only had to find one genuine case, among 10,000 con artists, to prove that communication with the dead was possible.

Black swans, in Taleb's terms, are unexpected, and indeed unforeseeable, events which have enormous impact. Examples: the events of 9/11 (bad, negative); Harry Potter (good, positive, particularly if your name is J.K. Rowling); the rise of Christianity; the rise of Islam; the first and second world wars. But after a black swan has occurred, human nature is such that we concoct explanations for it, making it both explicable and, we feel, predictable.

In short, black swans are some of the most important and influential events in the history of the human race.

'A small number of Black Swans,' Taleb tells us, 'explain almost everything in our world, from the success of ideas and religions, to the dynamics of historical events, to elements of our own personal lives.' And their impact is increasing, given the nature of the world we now live in.

But hold on a minute, hold on, I hear you cry. Surely there were lots of people who saw, for instance, that the two world wars were inevitably going to happen. Yes indeed. Or so it appears, afterwards. Afterwards we can readily find evidence that such conflicts were 'inevitable'.

At the time, however, such black swans were far from obvious. Taleb draws our attention to the pre-WWII Berlin diaries of the historian William Shirer, which show, for instance, that the French thought that Hitler was a short-term phenomenon. And the Harvard historian Niall Ferguson has demonstrated, through an analysis of bond prices, that the financial markets certainly did not expect war in 1914.

And so on, and so forth, in considerable closely argued detail, for 400 pages.

I hesitate to call The Black Swan a work of philosophy, because experience has taught me that I am crap at philosophy, and have grave difficulty in following even the simplest philosophical argument. But I didn't have any real difficulty in following the drift of The Black Swan. On the contrary, I found it an exceptionally agreeable read. So I think I would prefer, if you don't mind, to think of this book as a work of ideas. Talk philosophy to me, and my mind glazes over pretty rapidly; but tell me some of your ideas and I am interested.

True, The Black Swan does get very technical at times. (Example: 'Sometimes a fractal can make you believe that it is Gaussian, particularly when the cutpoint starts at a high number.') But Taleb does tell us when things are going to get technical; he advises us how many pages can safely be skipped by lay readers; and he provides summaries of his ideas, in layman's terms, either before or after the technical bits. The result is that the book is, in my estimation, painless and rewarding to read.

I emphasise that, because, so often these days, reading a serious non-fiction book is not a happy experience. You may well end up feeling deeply worried, guilty, upset, horrified, appalled, angry, and so forth. Well, there is plenty to worry about in The Black Swan, if you're the worrying kind. But on the whole the author does not seem to me to be particularly anxious to convert us to the cause, or to have us take up arms.

On the contrary: as I read him, Taleb tells us that it is not comfortable to recognise the truth about black swans, or randomness; but he seems disinclined to inconvenience or distress us by emphasising the point. Here are the facts, he says, rather apologetically. I am sorry to burden you with them, but this is the way things are.

A LONGER (?) DISCUSSION

Background: the author

Taleb is himself something of a rare bird. He combines practical experience of the financial world, at a high level, with academic interests and standing.

For example: Taleb has been, at various times, managing director and proprietary trader at Union Bank of Switzerland, and Managing Director and worldwide head of financial option arbitrage at CIBC-Wood Gundy. He is currently 'taking a break' by serving as the Dean's Professor in the Sciences of Uncertainty at the University of Massachusetts in Amherst.

The dust jacket of The Black Swan describes Taleb as a man who has devoted his life to the problems of luck, uncertainty, probability, and knowledge. He is said to be part literary essayist, part empiricist, part no-nonsense mathematical trader.

As I say, an unusual combination.

Background: previous books

Taleb has written two previous books. The first, Dynamic Hedging: Managing Vanilla and Exotic Options (1997), was a specialist book published by John Wiley. His second book, Fooled by Randomness, was, however, aimed firmly at the lay reader, and was sufficiently successful to have been translated into twenty languages.

Fooled by Randomness was reviewed here on 20 August 2004. The ideas contained in that book seemed to me to have some major implications for writers and publishers. Consequently, I wrote a further series of posts about Taleb's ideas on randomness on the following dates:

23 August 2004
27 August 2004
7 September 2004
15 September 2004
23 September 2004
28 September 2004

Later in 2004, I noticed that Taleb was beginning to post some draft chapters of The Black Swan on his web site. I read these, combined some thoughts on them with the thoughts and ideas which I had already posted on my blog, and incorporated the melange into a 72-page essay entitled On the Survival of Rats in the Slush Pile. This essay attempted to provide writers, agents, and publishers, with some new insights into the nature of their business. The essay is available free online, in the form of a PDF file. In fact that's the only form it is available in, as I haven't yet got around to publishing it in book form. Though I may before long.

In other words, as you will have gathered, I am quite interested in the ideas of Nassim Nicholas Taleb, and the way in which they impact upon the lives of writers and publishers.

A declaration of interest

At this point I ought to declare an interest of another kind.

When reading The Black Swan, I noticed early on that, in his 28-page bibliography, Taleb lists two of my books. There is the Rats essay, referred to above, and also The Truth about Writing, which you can also read free of charge as a PDF if you wish, though the paperback copy may be more convenient.

Not surprisingly, human nature being what it is, a reviewer tends to look kindly upon an author who has read the reviewer's books, and you will need to bear that in mind. However, there is more. In his six pages of acknowledgements, Taleb says this:
Michael Allen wrote a monograph for writers looking to get published [Rats], based on the ideas of Chapter 8 -- I subsequently rewrote Chapter 8 through the eyes of a writer looking at his lot in life.
Now, I have to say that this little paragraph is both flattering and alarming. It is flattering because it is easily the greatest compliment that has been paid to me in an intellectual context (and this, if I may modestly say so, in a life not entirely without some academic distinction). And alarming because, as you may already have concluded, I do not belong in the same company as the others listed in Taleb's acknowledgements section. I am not, as testified earlier, any sort of philosopher, much less a mathematician. I do not belong in the same part of a book as such famous dead names as Hegel and Montaigne, nor even alongside famous living names such as (to pick a few at random) Chris Anderson, Daniel Kahneman (a Nobel laureate), and Benoit Mandelbrot.

Before leaving Taleb's acknowledgements section, it is worth noting (from the point of view of this blog) what he says about his editor.

'He protected me from the intrusions of the standardizing editors. They have an uncanny ability to inflict the maximal damage by breaking the internal rhythm of one's prose with the minimum of changes.'

Yes indeed. We've all met some of those.

Implications

Rather than attempt to summarise Taleb's ideas (a risky business, at indicated at the outset), I think I will just mention a few of the implications of them, chiefly in non-publishing contexts.

Taleb's background, as we have seen, is in the financial world, and it is here, perhaps, that the implications of black swans are most worrying. More than once, I think, Taleb reminds us that bankers normally think of themselves as the most conservative of men; but in fact, he says, they are sitting (metaphorically speaking) on top of a pile of dynamite.

'The creatures with the largest cortex have the highest intelligence,' says Taleb. 'We humans have the largest cortex, followed by bank executives, dolphins, and our cousins the apes.' Our author, has you see, a gentle touch, with which he guides us painlessly through the book.

Taleb argues that our brains are genetically predisposed to make us rationalise after an event: to convince us that, for instance, had we been smart enough, we could have seen the black swan coming; and to persuade us that, when we look at the evidence, it is obvious that the black swan was inevitable.

This inbuilt tendency makes us over-confident about predicting the future, and over-confidence can lead to disaster. By way of example, Taleb offer us the case of Long-Term Capital Management. This was a speculative trading firm which offered investors (in crude terms) a sure thing. The brains behind this enterprise were all drawn from the top ranks of academia or finance, and were considered geniuses (except by Taleb). Two of them were Nobel prize-winners in Economics.

The brainy boys had all built careers and spent lifetimes in the business of risk management, and they had built up their sophisticated theories to the point where, when applied in the real world, they couldn't possibly fail to make vast profits. Except, of course, that they did fail. Very nearly dragging down the financial system of the entire western world with them. Total losses were $4.6 billion. You can read the details elsewhere.

Taleb's theories constitute a telling critique of the whole 'science' of economics. I was going to write a 'brutal' critique, but that would give the wrong impression. Taleb does not go around kicking the shit out of people. He just states his case, mildly, and with good manners, and then responds with dignity when people lose their temper with him.

If you read Chapter 17 you will discover that Taleb tends to regard economists, in particular, as not just the blind leading the blind, but leading them perilously close to the cliff's edge. However, you will also learn (if you don't know it already) that academics who have built a successful career in economics do not take kindly to a man who comes along and points out (politely) that they are either fools or charlatans. Don't underestimate the viciousness with which academic counter-attacks can be conducted, and on an ad hominem basis, at that. And don't underestimate the importance of Taleb's underlying ideas, either.

If you begin, at this point, to worry about your pension, then there is a strategy available to you: Taleb calls it the barbell strategy. It involves placing 85-90% of your assets in US Treasury bills, or your local equivalent (unless, of course, you live in Zimbabwe), and investing the remaining 10-15% in plenty of small, speculative bets.

Taleb goes on to point out that the barbell strategy can also be applied to other aspects of life. In fact, parts 3 and 4 of the book provide a number of practical steps that you can take to protect yourself against the negative black swans, and to enable you to benefit from the positive ones.

(For further discussion of the mathematical/statistical and economic/financial aspects of Taleb's ideas, you might wish to read the reviews by David A. Shaywitz and Roger Lowenstein. Thanks to Dave Lull for the links.)

By and large, says Taleb, human beings are blind to the impact of randomness. And we confidently predict the future in any number of ways. Unfortunately we are nearly always wrong, though later, when things have happened, we tend to remember only the times when we were right.

'Our cumulative prediction errors for political and economic events are so monstrous,' says Taleb, 'that every time I look at the empirical record I have to pinch myself to verify that I am not dreaming.'

If you want an example, bear in mind that, in 1970, the US government's official view was that, by 1980, the price of foreign crude oil might well decline, and would not, in any case, show a substantial increase. In fact, oil prices went up tenfold by 1980.

At the risk of misquoting and misrepresenting, perhaps I can summarise what is, for me, one of the key ideas of this book, namely that we fail to appreciate the asymmetry in our perception of events. We attribute our successes to our skills, and our failures to external events outside our control, namely to randomness. (Eight-four per cent of Frenchmen think that their lovemaking abilities put them in the top half of French lovers.)

We also forget very easily. As Taleb points out, it is possible to read a book such as his, agree entirely with it, and yet fail to take such things into account when thinking about the future.

Conclusion

I was once taken to a pub in Yorkshire which was called The Black Swan. The locals referred to it as The Mucky Duck. That may help.

This particular Black Swan (i.e. the book) is full of marvellous and amusing insights, for page after page after page. It would constitute a wearying catalogue of human folly, were it not illuminated by good humour.

The author does not hector you and bully you, and urge that you take these three simple steps to transform your life -- though for once it might be true. Instead, he seems to say to us: Go ahead, fool yourself if you must. Ignore what I tell you if you find it uncomfortable. See if I care. But when things go wrong -- and they may well do, despite everyone's best endeavours -- don't say you weren't warned.

Not every reader will find the book enjoyable, but those who stick with it, and particularly those who devour it with enthusiasm, will find it immensely rewarding.

I could go on, of course. For many thousands of words. But it would be second-best for you, when you can read the book for yourself. Actually, buy it first, and then read it. You will need to refer to it later.

And if I may venture to offer a prediction of my own (it is allowed, despite the hazards), then I would say that this is a book which will still be read in fifty years' time. (Possible comparator: The Open Society and its Enemies, by Sir Doctor Professor Karl Raimund Popper.)

By the way, if you're a publisher, watch out for that Yevgenia Nicolayevna Krasnova (see index). She's had her ups and downs, but this kid could yet be big.

21 comments:

Andy O'Hara said...

A thoughtful and well considered critique--and superbly written. Taleb's ideas have long made sense to me and leave me more than queasy at the 'sound reasoning' of today's economists.

I'm delighted to hear Taleb's well deserved recognition of you. Your 'Rats in the Slushpile' is a masterpiece for writers and, like Taleb's words, will be with us a long time, I'm sure.

elberry said...

i really enjoyed that piece, thank you.

i'm rather choosy about what i read, because i don't have regular internet access, and spend most of my life doing a rather banal job, and hence have little spare time; but i've read your whole blog & it's done me a power of good.

Peter L. Winkler said...

I'm afraid I won't be reading Mr. Taleb's book. I think that he loads the case in his favcor through his choice of Black Swan pheonomena that really were not totally unpredictable.

Take the 9/11 attacks, for instance. It's absurd to say that they were either totally unpredicatable or that no one predicted or hypothesized the possibility of crashing an airplane into a large building. And there had already been terrorist attacks in the US: the Oklahoma City bombing and the 1993 WTC bombing.

Making WW II another Black Swan is even worse. The isolationist movement in the United States certainly expected another European war, which is why they were so adamant to keep the United States out of such a war. When FRD campaigned for his third term, he promised to keep the US out of another war. At the same time, he managed to get Congress to increase funding for armaments production prior to the attack on Pearl Harbor. To sum up, there was considerable foresight and anticipation of a coming war by both interventionists and isolationists.

elberry said...

"Take the 9/11 attacks, for instance. It's absurd to say that they were either totally unpredicatable or that no one predicted or hypothesized the possibility of crashing an airplane into a large building"

Peter, the thing about 'Intelligence' is that they get so much data, now that we have such advanced means of gathering intel; hence terrorists usually 'have a MI5 file' - the thing is so do thousands of others who will never do more than read a few websites and mouth off occasionally. It is very hard to distinguish between the two.

There are usually people who have predicted X happening, it's just that A-Z has been predicted at some point by lots of other people, and it all seems pretty plausible. After the event we say "Well, MI5 had a file on him, so why didn't they stop him?".

Maxine said...

This is all why empirical scientists work by adding small pieces, and build up their syntheses that way, rather than trying to decide whether 9/11 was predictable or not (things flying into or blowing up towers having been imagined many times over in books and films).
There seems to be a large element of subjectivity in the Popperian "black swan" as defined by world events.

Jon said...

I agree totally about the book but I have to rebut your libel of the real black swan, which is a very attractive bird: see here and here.

Francis Ellen said...

I'm interested in what Taleb has to say about predicting financial events.

I once wrote an article criticizing Long Term Captial Management's underlying equations (the Nobel Prize winning Black-Scholes model, upon which their strategies were based).

But I wrote it before LTCM almost brought down the whole financial system; therefore I was a little twerp talking rubbish and I should know my place.

The article was not published. And I was told how stupid I was by the editor whom I had hoped might see my argument that failure was indeed inevitable in the absence of an infinite supply of money.

My contention was that there was no variable within the equation to contain sentiment (an idea I later found out was first put forward by the trader Victor Sperandeo).

I wrote an article around four months ago outlining my current belief that the world financial system is heading for disaster because of excess lquidity in the mortgage backed securties market. (And in it I described interest-only mortgages as options instruments.)

Three months later (and after much of what I had 'predicted' had happened) the article did get published in a highly watered-down version that actually misunderstood some of my points. And it got published on the toilet wall of the magazine (their e-mail broadcast/ web site).

I await the world-wide crash and I look forward to all the books explaining how easy it all was to see.

I did it in my other, actual name, for as Francis Ellen I will not be edited.)

dylan said...

nicely done review. for those who want to pursue his work, talebs website is the stuff of legend- for me, the 'ideal' website- pure content. it is almost unbelievable to me that a thinker i admire has his notes out for me to look through.

Steph said...

I have just finished reading The Black Swan after hearing Taleb on "Start the Week" I found it enthralling and fairly easy to understand.

Winkler has missed the point. Taleb does not state that Black Swan's are "unpredictable" he states that they are not "foreseen". Indeed the very essence of his argument is that we would be able to foresee future events if we could understand the relationship of all matter in the universe.(The nth billiard ball in chapter 11). Since it is not possible for even the most powerful computer to understand the relationship of all matter in the universe we have to rely on empirical study and empirical study produces "black swans".

As Taleb says he wont be losing any sleep if no one reads or takes notice of his book so Winkler can continue with his Gausian theory of the universe if he wishes. Taleb makes the point quite succinctly that Gausian economists and practicianers invariably produce the narrative "after the event". I have no doubt that even after the black swan in Australia was discovered an ornitholigist somewhere was saying "Told you so".

Rob Hopcott said...

Your article is beautifully balanced as are the arguments of Nassim Taleb which fill me full of excitement and hope.

Excitement, because Taleb's ideas present a new take on ways of thinking which might help me understand our complex world which I frequently find largely incomprehensible.

Hope, because his ideas have proved very popular in the best seller lists and this implies to me that I am not the only person who cries 'whence bohemia?' and decries the shallowness of popular press thinking.

Anonymous said...

This kind of ilussions about predicting events and reevaluating the evidence post event keep us mentally sane ( well, at least not with major depression). We are designed to think like that and I think for our own good, we should keep thinking like that. Taleb s insights are for a minority the vast majority cannot handle the truth and would develop distimic symptoms or even depression.

PeWe-HH said...

Taleb is really a gifted writer, with the ability to put thing very clearly and lively. But of course, 400 pages full of statements and stories about something we all already know or at least feel is a little bit too much.
Even if it's written in an entertaining way. My grandma always said that everything humans can think of will become reality at time. I experienced myself once in a while that this is true. So it is true for more then 6 billion times, since so many people live on earth now. Under this perspective how 'unpredctable' was 9/11?

Hanspeter Bornhauser said...

When I finished the book I was very disappointed. Main reason: Taleb gives more than one and even contradictory definitions of a Black Swan, e.g.: Something that´s ´purely accidental and unforeseen´ (page 272) cannot be ´Highly Improbable` (see front cover) ,the latter being already thought of and attributed a very low probability; therefore it´s not unforeseen! Furthermore Taleb makes fair amount of mistakes and doubtful remarks (starting on page XX ´What you know cannot really hurt you´ which is wrong; it may not surprise you even if it hurts), etc. etc. But I must admit: The book is nevertheless entertaining and to some extent inspiring.

Hanspeter Bornhauser said...

..makes a fair amount ...

Please add the ´a´, thanks.

AJINKYAP said...

i am afraid this book of taleb does not make that much of an impression. the irony of this book is the knowldge that taleb seems to be so certain about HIS ideas of uncertainty that it rather creates an amusing read. an engineer as reader, this book makes some statements which are not always true and are consequential. the book argument about out of control things rather question the whole process of working of human brain neglecting the very way we learn the things. i am afraid this book rather pokes the mistakes in current system which we (engineers) completely know without offering any conclusive insight. taleb seems to forget that "data fitting" or "trend in data" though not alwayes helpful is the essence of human endowments. further the concept of uncertainty has been already accepted at most basic atomic level(pop in the book of BASIC Physics. ranting that concept again with rather vague remarks may not be helpful to those informative people to whom this book is aimed at. but all in all one time good reading .

guy said...

would that be the Mucky Duck in Scarborough? If so I live opposite it:-)

Anonymous said...

коттедж
восстановление зрения
зеленый лазер
электрошокер

home for sale costa rica said...

hi people
wow i didn't notice it...

i think this spot is so interesting!

niz said...

Hello .. firstly I would like to send greetings to all readers. After this, I recognize the content so interesting about this article. For me personally I liked all the information. I would like to know of cases like this more often. In my personal experience I might mention a book called Generic Viagra in this book that I mentioned have very interesting topics, and also you have much to do with the main theme of this article.

Anonymous said...

great book and a wonderful article

Frontierville Guide

Akai.Shuu said...

http://vietlockerz.com <<<< biggest MMO site, Akai.Shuu is handsome!!!
PPI ClaimsGingerbread House