Skip to Main Content (access key 1)
Skip to Search (access key 2)
Skip to Search GO (access key 3)
Skip to comments (access key 4)
Skip to navigation (access key 5)
Skip to top of page (access key 6)
Sunday, March 25, 2007 | Science : Math and Tech | print version Print | Comments

Document Artificial Intelligence, With Help From the Humans

by Jason Pontin

Reposted from the NYTimes:
http://www.nytimes.com/2007/03/25/business/yourmoney/25Stream.html?_r=1&th&emc=th&oref=slogin


Computers still do some things very poorly. Even when they pool their memory and processors in powerful networks, they remain unevenly intelligent. Things that humans do with little conscious thought, such as recognizing patterns or meanings in images, language or concepts, only baffle the machines.

These lacunae in computers' abilities would be of interest only to computer scientists, except that many individuals and companies are finding it harder to locate and organize the swelling mass of information that our digital civilization creates.

The problem has prompted a spooky, but elegant, business idea: why not use the Web to create marketplaces of willing human beings who will perform the tasks that computers cannot? Jeff Bezos, the chief executive of Amazon.com, has created Amazon Mechanical Turk, an online service involving human workers, and he has also personally invested in a human-assisted search company called ChaCha. Mr. Bezos describes the phenomenon very prettily, calling it "artificial artificial intelligence."

"Normally, a human makes a request of a computer, and the computer does the computation of the task," he said. "But artificial artificial intelligences like Mechanical Turk invert all that. The computer has a task that is easy for a human but extraordinarily hard for the computer. So instead of calling a computer service to perform the function, it calls a human."

Mechanical Turk began life as a service that Amazon itself needed. (The name recalls a famous 18th-century hoax, where what seemed to be a chess-playing automaton really concealed a human chess master.) Amazon had millions of Web pages that described individual products, but it wanted to weed out the duplicate pages. Software could help, but algorithmically eliminating all the duplicates was impossible, according to Mr. Bezos. So the company began to develop a Web site where people would look at product pages and be paid a few cents for every duplicate page they correctly identified.

Mr. Bezos figured that what had been useful to Amazon would be valuable to other businesses, too. The company opened Mechanical Turk as a public site in November 2005. Today, there are more than 100,000 "Turk Workers" in more than 100 countries who earn micropayments in exchange for completing a wide range of quick tasks called HITs, for human intelligence tasks, for various companies.

PriceGrabber.com, a comparison shopping site, uses Mechanical Turk to match images to the product pages. "Harnessing the power of this enormous, decentralized work force allows us to obtain images for a wide variety of items in a fraction of the time it would have taken to do it ourselves," said Sagar M. Jethani, PriceGrabber's director of content development and community.

Mechanical Turk's customers are corporations. By contrast, ChaCha.com, a start-up in Carmel, Ind., uses artificial artificial intelligence — sometimes also called crowdsourcing — to help individual computer users find better results when they search the Web. ChaCha, which began last year, pays 30,000 flesh-and-blood "guides" working from home or the local coffee shop as much as $10 an hour to direct Web surfers to the most relevant resources.

Amazon makes money from Mechanical Turk by charging companies 10 percent of the price of a successfully completed HIT. For simple HITs that cost less than 1 cent, Amazon charges half a cent. ChaCha intends to make money the way most other search companies do: by charging advertisers for contextually relevant links and advertisements.

Harnessing the collective wisdom of crowds isn't new. It is employed by many of the "Web 2.0" social networks like Digg and Del.icio.us, which rely on human readers to select the most worthwhile items on the Web to read. But creating marketplaces of mercenary intelligences is genuinely novel.

What is it like to be an individual component of these digital, collective minds?

To find out, I experimented. After registering at www.mturk.com, I was confronted with a table of HITs that I could perform, together with the price that I would be paid. I first accepted a job from ContentSpooling.net that asked me to write three titles for an article about annuities and their use in retirement planning. Then I viewed a series of images apparently captured from a vehicle moving through the gray suburbs of North London, and, at the request of Geospatial Vision, a division of the British technology company Oxford Metrics Group, identified objects like road signs and markings.

For all this, my Amazon account was credited the lordly sum of 12 cents. The entire experience lasted no more than 15 minutes, and from my point of view, as an occluded part of the hive-mind, it made no sense at all.

I was also interested in learning what it was like to be a consumer of crowdsourcing. So at 2:40 p.m. on March 14, I asked ChaCha, "Who was Evelyn Waugh's commanding officer in the Commandos during World War II?" In an instant-messaging window, CandieSue22087 immediately welcomed me to ChaCha and asked me to be patient.

At 2:44, CandieSue threw up her virtual hands and transferred me to another guide, Tressie57635, who referred me to an academic paper on "suffixal sound symbolism in the novels of Evelyn Waugh." When I protested, Tressie complained that it was a hard search, and at 2:49 she gave up, typing that I might do better with yet another guide. When I agreed, Tressie accidentally ended the search altogether — but not before serving me a page of 12 search results, not one of which was relevant.

A quick search on Google quickly provided the right answer.

THERE have been two common objections to artificial artificial intelligence. The first, confirmed by my own experiences searching on ChaCha, is that the networks are no more intelligent than their smartest members. Katharine Mieszkowski, writing last year on Salon.com, raised the second, more serious criticism. She saw Mechanical Turk as a kind of virtual sweatshop. "There is something a little disturbing about a billionaire like Bezos dreaming up new ways to get ordinary folk to do work for him for pennies," she wrote.

The ever-genial Mr. Bezos dismisses the criticism. "MTurk is a marketplace where folks who have work meet up with folks who want to do work," he said.

Why do people become Turk Workers and ChaCha Guides? In poor countries, the money earned could offer a significant contribution to a family's wealth. But even Mr. Bezos concedes that Turk Workers from rich countries probably can't live on the small sums involved. "The people I've seen commenting on blogs seem mostly to be using MTurk as a supplemental form of income," he said.

Mitch Fernandez, 38, a disabled former United States Army linguist, said by e-mail that he became a Turk Worker for various reasons: "At first, I was just curious about the idea of crowdsourcing." But he said he soon found that by working about two hours a day, he could often earn more than $100 a week. In the last nine months he made around $4,000, which he used to buy a high-definition television, a DVD player and a new subwoofer — all from Amazon.com.

"I do this primarily for the money, but I also view it as a form of therapy to get me used to working again." he explained. "The experience has gotten me thinking about pursuing a library science degree."

We probably have at least another 25 years before computers are more powerful than human brains, according to the most optimistic artificial intelligence experts. Until then, people will be able to sell their idle brains to the companies and people who need the special processing power that they alone possess through marketplaces like Mechanical Turk and ChaCha.

Jason Pontin is the editor in chief and publisher of Technology Review, a magazine and Web site owned by M.I.T. E-mail:
pontin@nytimes.com.

Comments 1 - 11 of 11 |

Reload Comments | Back to Top | Page Numbers

1. Comment #27504 by denoir on March 25, 2007 at 3:59 am

 avatarWhat a horrible article.

First of all, crowdsourcing is generally used to collect training data for computer based adaptive systems to use. Typically artificial neural networks (ANNs) learn by example - you give them an input pattern and tell them what the desired response is. From this ANNs learn to generalize and extrapolate on new data. You need however training data and some use humans for that.

So it is really humans teaching the software, but it is a one-time thing. Once the adaptive system learns how to perform the task, no human intervention is needed.

We probably have at least another 25 years before computers are more powerful than human brains, according to the most optimistic artificial intelligence experts.


No, actually not 25 years but 6 years. Moore's law tells us that 2013 we should have super-computers that are roughly equivalent in processing power to one human brain. In 25 years a supercomputer will have the equivalent computational power of all human brains. (See Kurtzweil et al for an elaborate discussion of this).

We still need to have better software - raw computing power can't solve many things directly. The author of the article however seems to think that there is no development on the software front, which is nonsense. There are tons of systems such as artificial neural networks and genetic algorithms that can outperform humans in areas that used to be exclusively human (i.e pattern recognition).

Raw computing power however can be enough. There is an interesting development of meta-optimization algorithms today. Instead of a human programmer designing a clever algorithm, a dumb optimization algorithm is used to generate it. In the same way that natural selection trades design for time, there approaches trade design for computing costs. It is not efficient, but the results can be astonishing.

In the next step of course you use the generated algorithms to evolve new and better algorithms.

Although 'intelligent design' has been very popular in software development a wall of complexity has emerged - software systems have become so complex that our human brains can't handle them and all the emergent phenomena that go with that complexity. It would seem that we've hit our intelligence limit as designers. So we can expect a boost in the post-Darwinian approach with software writing software.


Other Comments by denoir

2. Comment #27512 by steve_kap on March 25, 2007 at 5:09 am

Not quite true about the ANN (ref above). There is such a thing as unsupervised learned in an ANN. The network itself creates the catagories in which things are mapped, based on the history of previous input. It is thought that the mapping between indivdual eye nerons (cones, rods) are mapped to the brain in this way. Clearly such a mapping it too complicated to be hard wired in the DNA, but rather a procedure is so hard wired, and it is thought by some that this prodecure is similair to unsupervised learning in ANNs.

Other Comments by steve_kap

3. Comment #27514 by denoir on March 25, 2007 at 5:29 am

 avatar
Not quite true about the ANN (ref above). There is such a thing as unsupervised learned in an ANN. The network itself creates the catagories in which things are mapped, based on the history of previous input. It is thought that the mapping between indivdual eye nerons (cones, rods) are mapped to the brain in this way. Clearly such a mapping it too complicated to be hard wired in the DNA, but rather a procedure is so hard wired, and it is thought by some that this prodecure is similair to unsupervised learning in ANNs.


What isn't exactly true?

Self-Organizng Maps and other simialar unsupervised approaches do clustering, not classification. The tasks mentioned in the article are typical classification tasks. Perhaps the 'finding duplicates' is more of a clustering task but if that was their approach they wouldn't be needing all the data.

While most supervised varieties of ANNs have extremely little to do with actual biological models, unsupervised varieties are not much better in that respect. Yes, we do have some form of hebbian adaptation in our brains and yes, there is evidence of self-organization, but generally the artificial varieties have little in common with the biological counterparts.

Not that it is a deficiency in any way. The point is making useful systems and not in imitating nature as closely as possible.

Evolution through natural selection is far from the best optimizing method imaginable. One major problem with it is that it is a so called "greedy" algorithm – it does not have any look ahead or planning capabilities. Every improvement, every payoff needs to be immediate.
This creates systems that carry a lot of historical baggage – an improvement isn't made as a stand-alone feature but as a continuation of the previous state.

It is not a coincidence that a brain cell is a cell like any other – nucleus and all. Nor is it a cell because it is the optimal structure for information processing. It was what could be done by modifying the existing wetware. It is not hard to imagine how that structure could be improved upon if not limited by the biological building blocks that were available to the genetic machinery.

Another point worth making is that our brains are optimized not for the modern type of information processing that humans engage in – such as writing software for instance. Humans have changed little in the last 50,000 years in terms of intellectual capacity but our societies have changed greatly. Our technological progress is a side effect of the capabilities we evolved that increased survivability when we roamed the plains of Africa in small family hunter-gatherer groups.

To assume the resulting information processing system (the brain) would the ultimately optimal solution for anything else is not justifiable.

Other Comments by denoir

4. Comment #27528 by NormanDoering on March 25, 2007 at 7:12 am

Both of you guys seem to know your way around neural nets, so let me ask you about Stuart Hameroff and Penrose and their theory:

http://andrewsullivan.theatlantic.com/the_daily_dish/2007/03/debating_the_bi.html

First, by soul I mean that consciousness (and/or unconscious processes) may be accompanied by: 1) nonlocal interconnectedness among living beings, 2) interaction with a Platonic wisdom, or cosmic intelligence inherent in the universe, and 3) existence outside the body.

I am not claiming proof of the soul, but of a scientifically plausible explanation for it based on these three factors.


Is that total Woo-woo? It sounds like it.

http://normdoering.blogspot.com

Other Comments by NormanDoering

5. Comment #27537 by denoir on March 25, 2007 at 7:49 am

 avatar
Is that total Woo-woo? It sounds like it.


Total woo-woo. Penrose is a respected physicist, but he is consider a joke in the field of neuroscience.

His and Hameroff's theories are for the lack of a better word ludicrous. Essentially, they are trying to save the soul by theorizing about quantum effects in the brain. Not only don't they have any data to back them up but there is very solid scientific works that show that what they are suggesting is impossible. The brain works at a molecular level, not at a level of elementary particles where the quantum effects would be of any significance. The brain being a wet and warm place is the worst imaginable place for quantum coherence etc..

The quantity of desperation can be seen in attempts to explain away the delay between conscious response and the activations of other parts of the brain. It has been known for a while that our conscious response is delayed relative other functional parts of the brain. Essentially that when you consciously think you are making a decision, the decision has already been made by parts of your brain that you are not actively aware of. Consciousness it seems is a post-processing function that puts decisions already made in a structured context - giving the illusion of making a conscious decision.

Penrose for instance argues that yes, there is an average 500 ms delay (as demonstrated by data), but that is compensated by quantum effects that are time symmetric – that parts of the brain actually sees into the future, which then is delayed to create a real-time decision process.

While this of course is rejected as absurd by a majority of neuroscientists and physicists, it is a good example of how passionately some people feel about the role of consciousness.

Other Comments by denoir

6. Comment #27546 by Logicel on March 25, 2007 at 8:42 am

 avatarclear and easy-to-understand explanation, denoir, thanks.

If you are aware of The Secret, I would appreciate if you would give your opinion concerning the 'science' of the law of attraction supposedly underpinning that approach.

http://www.thesecret.tv/

Other Comments by Logicel

7. Comment #27571 by denoir on March 25, 2007 at 10:38 am

 avatar

If you are aware of The Secret


No, fortunately I was not, alas that has changed.

if you would give your opinion concerning the 'science' of the law of attraction supposedly underpinning that approach.


No, not really, life is too short for wasting time on such nonsense.

I will say that I fully support that people use this method to cure themselves of cancers, brain tumors and other fatal diseases. Natural selection has its uses.

Other Comments by denoir

8. Comment #27631 by kurzweilfreak on March 25, 2007 at 7:47 pm

Very interesting topic of discussion here. Has anyone read Jeff Hawkins' book "On Intelligence"? If so, what do you think of his HTM theories?

Other Comments by kurzweilfreak

9. Comment #27647 by -TheCodeCrack- on March 26, 2007 at 1:17 am

 avatarI disliked this topic. There was nothing new, nothing interesting.

Just because a program says "cant do it - help" dosn't mean to much to me.


Computers in the future I'm sure will manage the tasks, maybe altered humans in the future will get in first and do it for a while.

A couple of cents for saw eyes and frustration, no thanks.

Other Comments by -TheCodeCrack-

10. Comment #27722 by nickthelight on March 26, 2007 at 10:32 am

 avatarThis sounds like the digital age version of "EARN £££ IN YOUR SPARE TIME"! Work from home! *Must have own computer.

It used to be packing envelopes, now its searching through incredibly boring documents of information most users will have no knowledge of. Who guarantees accuracy and integrity?

Other Comments by nickthelight

11. Comment #29142 by NotWithoutMyMonkey on April 1, 2007 at 9:27 pm

I'm not an expert on neuroscience but you know all for this talk about the limitations of the naturally evolved human brain when stacked up against the inevitability of Moore's Law (raw computing power outstripping the capabilities of the human brain), the fact is we haven't even begun to tap the brain's inherent potential (and incidentally for all the almost religious certitude expressed by some here I remain skeptical that an exponential growth in raw computing power will ever be enough to reach some theorized mininal requirement for artificial intelligence without some fundamental paradigm shift - who is to say that these future computing systems however powerful in raw number crunching capability will necessarily be intelligent in any meaningful sense?

Considering the untapped potential of the human brain I stumbled across this pertinent documentary. Some of you may well know the subject:

http://video.google.com/videoplay?docid=-3372301236664593143

As alluded to in this documentary it's theorised that in cases such as this increased cross-talk between regions of the brain dedicated to certain functions occurs - far more than within in a 'normal' brain. So this capacity is dormant within each and every one of us imagine what would be possible if we could replicate this in a controlled way within ourselves upon understanding what exactly is happening here?

--------

The likes of Kurzweil forget one important thing. The motivation behind change and improvement in our society should always be human-centric (it should serve humanity foremost). To advocate that humans must radically alter or adapt to their creations in order to compete or even survive is absurd and abhorent.

Other Comments by NotWithoutMyMonkey
Reload Comments | Back to Top

Comment Entry: Please Login

Register a new account

Username:

Password:

Send a letter to the editor of the original media outlet.
letters@nytimes.com