Reposted from:
http://www.spectator.co.uk/the-magazine/features/3213246/the-natural-order-of-things.thtml

Matt Ridley says that Darwinian selection explains the appearance of seemingly ‘designed’ complexity throughout the world — not just in biology but in the economy, technology and the arts

Charles Darwin, who was born 200 years ago next month, has spent the 150 years since he published The Origin of Species fighting for the idea of common descent. Though physically dead, he is still doing battle for the notion that chimps are your cousins and cauliflowers your kin. It is a sufficiently weird concept to keep Darwin relevant, revered and resented in equal measure. But in some ways it is less radical and topical than his other, more philosophical legacy: that order can generate itself, that the living world is a ‘bottom-up’ place. On the internet, Darwinian unordained order is now ubiquitous as never before.

Living beings are eddies in the stream of entropy. That is to say, while the universe gradually becomes more homogeneous and disordered, little parts of it can reverse the trend and become briefly more ordered and complex by capturing packets of energy. It happens each time a baby is conceived. Built by 20,000 genes that turn each other on and off in a symphony of great precision, and equipped with a brain of ten trillion synapses, each refined and remodelled by early and continuing experience, you are a thing of exquisite neatness, powered by glucose. Says Darwin, this came about by bottom-up emergence, not top-down dirigisme. Faithful reproduction, occasional random variation and selective survival can be a surprisingly progressive and cumulative force: it can gradually build things of immense complexity. Indeed, it can make something far more complex than a conscious, deliberate designer ever could: with apologies to William Paley and Richard Dawkins, it can make a watchmaker.

Ideas evolve by descent with modification, just as bodies do, and Darwin at least partly got this idea from economists, who got it from empirical philosophers. Locke and Newton begat Hume and Voltaire who begat Hutcheson and Smith who begat Malthus and Ricardo who begat Darwin and Wallace. Before Darwin, the supreme example of an undesigned system was Adam Smith’s economy, spontaneously self-ordered through the actions of individuals, rather than ordained by a monarch or a parliament. Where Darwin defenestrated God, Smith had defenestrated government. Neatly, this year also sees a Smith anniversary, the 250th birthday of his first book, The Theory of Moral Sentiments, a book that is very Darwinian in its insistence that sympathy is what we would today call innate, that people are naturally nice as well as naturally nasty.

Darwin’s debt to the political economists is considerable. In his last year at Cambridge in 1829, he reported in a letter, ‘My studies consist in Adam Smith and Locke’. At Maer, his uncle Josiah Wedgwood’s house in Staffordshire, he often met the lawyer and laissez-faire politician Sir James Mackintosh (whose daughter married Darwin’s brother-in-law and had an affair with his brother). On the Beagle, he read the naturalist Henri Milne-Edwards, who took Adam Smith’s notion of the division of labour and applied it to the organs of the body. Darwin promptly re-applied it to the division of labour among specialised species in an ecosystem: ‘The advantage of diversification in the inhabitants of the same region is, in fact, the same as that of the physiological division of labour in the organs of the same individual body — a subject so well elucidated by Milne-Edwards.’

Today, generally, Adam Smith is claimed by the Right, Darwin by the Left. In the American South and Midwest, where Smith’s individualist, libertarian, small-government philosophy is all the rage, Darwin is reviled for his contradiction of creation. Yet if the market needs no central planner, why should life need an intelligent designer? Conversely, in the average European biol- ogy laboratory you will find fervent believers in the individualist, emergent, decentralised properties of genomes who prefer dirigiste determinism to bring order to the economy.

So long is the shadow cast by the determinism of Karl Marx that it is often forgotten how radical the economic liberalism of the political economists seemed in the 1830s, the decade when Darwin’s thinking crystallised. This is well illustrated by the case of Harriet Martineau, who had a small but seminal influence on Darwin. The daughter of a Norwich cotton manufacturer ruined by a bank crash in the 1820s, Martineau lived by her pen. She was a radical, outspoken feminist, who toured America bravely inveighing against slavery and became so notorious that there were plans to lynch her in South Carolina. Yet before that, she had shot to fame with a series of short fictional books called Illustrations of Political Economy, which were intended to educate people in the free-trade, free-market ideas of Adam Smith (‘whose excellence is marvellous’, she said), David Ricardo and Robert Malthus, and in particular to persuade the working classes that their interests were congruent with those of their employers.

Martineau’s Illustrations were written while Darwin was on HMS Beagle. After she returned from America she became a very close friend of his elder brother Erasmus, who saw her almost daily in the late 1830s. Erasmus introduced Harriet to Charles, who was soon hanging on her every word. Had he not been ‘astonished to find how ugly she is’, Charles might have justified his father’s worry that one of his sons would marry her (as it was, cautious Charles preferred his god-fearing mouse of a cousin Emma Wedgwood to this free-thinking literary lioness). Undoubtedly they discussed slavery, which had horrified Darwin in Brazil. But since Martineau had been a close confidante of Malthus — despite his speech impediment and her deafness — there is little doubt that they also talked political economy. Was it a coincidence that Darwin read Malthus, probably not for the first time, in October 1838, just as he was looking for a mechanism to explain evolution?

Malthus taught Darwin the bleak lesson that overbreeding must end in pestilence, famine or violence — and hence gave him the insight that in a struggle for existence, survival could be selective. But the notion that, with random variation, this selective survival could then generate complexity and sophistication where there had been none before, that it is a cumulative and creative force, is entirely his. It is also one that applies to more than the bodies of living beings.

Technology is a case in point. Although engineers are under the fond illusion that they design things, nearly all of what they do consists of nudging forward descent with modification. Every technology has traceable ancestry; ‘to create is to recombine’ said the geneticist François Jacob. The first motor car was once described by the historian L.T.C. Rolt as ‘sired by the bicycle out of the horse carriage’. Just like living systems, technologies experience mutation (such as the invention of the spinning jenny), reproduction (the rapid mechanisation of the cotton industry as manufacturers copied each others’ machines), sex (Samuel Crompton’s combination of water frame and jenny to make a ‘mule’), competition (different designs competing in the early cotton mills), extinction (the spinning jenny was obsolete by 1800), and increasing complexity (modern cotton mills are electrified and computerised).

Technology also experiences progress and ‘arms races’ between competitors. Just as a modern horse could outrun a Mesohippus three-toed horse from 30 million years ago, so a car can outrun a horse-drawn carriage. Yet horses can only just go fast enough to escape today’s lions, and Land Rovers can only just perform well enough to maintain market share against Toyotas. Such running to stay in the same (improving) place is known to biologists as a Red Queen process after the character in Lewis Carroll’s Through the Looking Glass.

Software inventors have learnt to recognise the power of trial and error rather than deliberate design. Beginning with ‘genetic algorithms’ in the 1980s, they designed programmes that would experiment with changes in their sequence till they solved the problem set for them. Then gradually the open-source software movement emerged by which users themselves altered programmes and shared their improvements with each other. Linux and Apache are operating systems designed by such democratic methods, but the practice has long spread beyond programmers. Wikipedia is a bottom-up knowledge repository and, though far from flawless, is proving easily capable, even in its first flush of youth, of matching expert-written encyclopaedias for accuracy and reach. It grows by natural selection among edits.

The internet is an increasingly Darwinian place, where decentralised, self-organising sophistication holds sway: swarm intelligence is the fashionable term. Trey Ratcliff, founder of a computer games company in Texas, tells me he feels more like a victim than a designer of technology’s evolution: ‘saying Edison invented the phonograph is like saying a spider invented silk’.

The supreme example of bottom-up, rather than top-down, complexity is the market itself. As the economist Paul Seabright has written, the almost miraculous system by which he can go out and buy a cotton shirt on a whim — and expect the cotton grower, the weaver, the shirtmaker, the shipper and the retailer to have got it ready for him just when he enters the shop — is not planned or designed, it evolves. The top-down alternative does not have a great track record. Can you doubt that if the shirt industry was run by a National Shirt Service, there would now be queues, quotas and shortages?

Dirigisme has a place, of course, in the regulation and operation if not the design of institutions. A school cannot work without a teacher, a firm without a manager, or an army without a general — just as a body is directed by a brain in its everyday operations. But hubristic human beings tend to exaggerate the degree to which they are in charge of, rather than at the mercy of, organisations.

The dark side of bottom-up Darwinism is that cumulative complexity can come about only through selective death or selective celibacy. Wonderful life may result, but it is born red in tooth and claw. The social Darwinists of the 19th century and the eugenicists of the 20th were of the view that the strong should therefore be encouraged to succeed, the better to keep natural selection going. But this is to misread human society. The human body may have come about through three billion years of natural selection among genes, but civilisation and prosperity came from 50,000 years of much more rapid natural selection among ideas. It is easily possible to blunt genetic selection in the name of kindness, while allowing cultural selection to continue: the death of an idea need not be cruel.

There is, however, one more disturbing and topical parallel between biological and cultural evolution. Just as natural selection’s constructive capacity did not prevent mass extinctions, one of which, 251 million years ago, eradicated over 96 per cent of marine species, so the market’s ability to build order cannot prevent crashes. Even sophisticated, entropy-defying complex systems are subject to the weather-like vagaries of mathematical chaos — and there Darwin cannot help.