An unusual Distributed Work Meetup was held last night in four different cities simultaneously, arranged through many hours of hard work by Lukas Biewald and his colleagues at distributed work provider CrowdFlower.
With all the sharing of experiences and the on-the-spot analyses taking place, I didn't find an occasion to ask my most pressing question, so I'll put it here and ask my readers for comments:
How can you set up crowdsourcing where most people work for free but some are paid, and present it to participants in a way that makes it seem fair?
This situation arises all the time, with paid participants such as application developers and community managers, but there's a lot of scary literature about "crowding out" and other dangers. One basic challenge is choosing what work to reward monetarily. I can think of several dividing lines, each with potential problems:
* Pay for professional skills and ask for amateur contributions on a volunteer basis.
The problem with that approach is that so-called amateurs are invading the turf of professionals all the time, and their deft ability to do so has been proven over and over at crowdsourcing sites such as InnoCentive for inventors and SpringLeap or 99 Designs for designers. Still, most people can understand the need to pay credentialed professionals such as lawyers and accountants.
*
Pay for extraordinary skill and accept more modest contributions on a volunteer basis.
This principle usually reduces to the previous one, because there's no bright line dividing the extraordinary from the ordinary. Companies adopting this strategy could be embarrassed when a volunteer turns in work whose quality matches the professional hires, and MySQL AB in particular was known for hiring such volunteers. But if it turns out that a large number of volunteers have professional skills, the whole principle comes into doubt.
*
Pay for tasks that aren't fun.
The problem is that it's amazing what some people consider fun. On the other hand, at any particular moment when you need some input, you might be unable to find people who find it fun enough to do it for you. This principle still holds some water; for instance, I heard Linus Torvalds say that proprietary software was a reasonable solution for programming tasks that nobody would want to do for personal satisfaction.
*
Pay for critical tasks that need attention on an ongoing basis.
This can justify paying people to monitor sites for spam and obscenity, keep computer servers from going down, etc. The problem with this is that no human being can be on call constantly. If you're going to divide a task among multiple people, you'll find that a healthy community tends to be more vigilant and responsive than designated individuals.
I think there are guidelines for mixing pay with volunteer work, and I'd like to hear (without payment) ideas from the crowd.
Now I'll talk a bit about the meetup.
Venue and setup
I just have to start with the Boston-area venue. I had come to many events at the MIT Media Lab and had always entered Building E14 on the southwest side. The Lab struck me as a musty, slightly undermaintained littered with odd jetsam and parts of unfinished projects; a place you could hardly find your way around but that almost dragged creativity from you into the open. The Lab took up a new building in 2009 but to my memory the impact is still similar--it's inherent to the mission and style of the researchers.
For the first time last night, I came to the building's northeast entrance, maintained by the MIT School of Architecture. It is Ariel to the Media Lab's Caliban: an airy set of spacious white-walled forums sparsely occupied by chairs and occasional art displays. In a very different way, this space also promotes open thoughts and broad initiatives.
The ambitious agenda called for the four host cities (Boston, New York, San Francisco, and Seattle) to share speakers over videoconferencing equipment. Despite extensive preparation, we all had audio, video, and connectivity problems at the last minute (in fact, the Boston organizers crowdsourced the appeal for a laptop and I surrendered mine for the video feed). Finally in Boston we disconnected and had an old-fashioned presentation/discusser with an expert speaker.
In regard to the MIT Media Lab and Architecture School, I think it's amusing to report that Foursquare didn't recognize either one when I asked for my current location. Instead, Foursquare offered a variety of sites across the river, plus the nearby subway, the bike path, and a few other oddities.
We were lucky to have Jeff Howe, the WIRED contributor who invented the term Crowdsourcing and wrote a popular book on it. He is currently a Nieman Fellow at Harvard. His talk was wildly informal (he took an urgent call from a baby sitter in the middle) but full of interesting observations and audience interactions.
He asked us to promote his current big project with WIRED, One Book, One Twitter. His goal is to reproduce globally the literacy projects carried out in many cities (one happens every year in my town, Arlington, Mass.) where a push to get everyone to read a book is accompanied by community activities and meetups. Through a popular vote on WIRED, the book American Gods by Neil Gaiman was chosen, and people are tweeting away at #1b1t and related tags.
Discussion
With the sponsorship by CrowdFlower, our evening focused on crowdsourcing for money. We had a few interesting observations about the differences between free Wikipedia-style participation and work-for-pay, but what was most interesting is that basic human processes like community-building go in both places.
Among Howe's personal revelations was his encounter with the fear of crowdsourcing. Everyone panics when they first see what crowdsourcing is doing to his or her personal profession. For instance, when Howe talked about the graphic design sites mentioned earlier, professional designers descended on him in a frenzy. He played the sage, lecturing them that the current system for outsourcing design excludes lots of creative young talent, etc.
But even Howe, when approached by an outfit that is trying to outsource professional writing, felt the sting of competition and refused to help them. But he offered respect for Helium, which encourages self-chosen authors to sign up and compete for freelance assignments.
Howe is covering citizen journalism, though, a subject that Dan Gillmor wrote about in a book that O'Reilly published We the Media, and that he continues to pursue at his Mediactive site and a new book.
Job protection can also play a role in opposition to crowdsourcing, because it makes it easier for people around the world to work on local projects. (Over half the workers on Mechanical Turk now live in India. Biewald said one can't trust what workers say on their profiles; IP addresses reveal the truth.) But this doesn't seem to have attracted the attention of the xenophobes who oppose any support for job creation in other countries, perhaps because it's hard to get riled up about "jobs" that have the granularity of a couple seconds.
Crowdsourcing is known to occur, as Howe put it, in "situations of high social capital," simply meaning that people care about each other and want to earn each other's favor. It's often reinforced by explicit rating systems, but even more powerful is the sense of sharing and knowing that someone else is working alongside you. In a blog I wrote a couple years ago, I noted that competition site TopCoder maintained a thriving community among programmers who ultimately were competing with each other.
Similarly, the successful call center LiveOps provides forums for operators to talk about their experiences and share tips. This has become not just a source of crowdsourced help, and not even a way to boost morale by building community, but an impetus for quality. Operators actually discipline each other and urge each other to greater heights of productivity. LiveOps pays its workers more per hour than outsourcing calls to India normally costs to clients, yet LiveOps is successful because of its reputation for high quality.
We asked why communities of paid workers tended to reinforce quality rather than go in the other direction and band together to cheat the system. I think the answer is obvious: workers know that if they are successful at cheating, clients will stop using the system and it will go away, along with their source of income.
Biewald also explained that CrowdFlower has found it fairly easy to catch and chase away cheaters. It seeds its jobs with simple questions to which it knows the right answers, and warns the worker right away if the questions are answered incorrectly. After a couple alerts, the bad worker usually drops out.
We had a brief discussion afterward about the potential dark side of crowdsourcing, which law professor Jonathan Zittrain covered in a talk called Minds for Sale. One of Zittrain's complaints is that malicious actors can disguise their evil goals behind seemingly innocuous tasks farmed out to hundreds of unknowing volunteers. But someone who used to work for Amazon.com's Mechanical Turk said people are both smarter and more ethical than they get credit for, and that participants on that service quickly noted any task that looked unsavory and warned each other away.
As the name Mechanical Turk (which of course had a completely unrelated historical origin) suggests, many tasks parceled out by crowdsourcing firms are fairly mechanical ones that we just haven't figured out how to fully mechanize yet: transcribing spoken words, recognizing photos, etc. Biewald said that his firm still has a big job persuading potential clients that they can trust key parts of the company supply chain to anonymous, self-chosen workers. I think it may be easier when the company realizes that a task is truly mechanical and that they keep full control over the design of the project. But crowdsourcing is moving up in the world fast; not only production but control and choice are moving into the crowd.
Howe highlighted Fox News, which runs a UReport site for volunteers. The stories on Fox News' web site, according to Howe, are not only written by volunteers but chosen through volunteer ratings, somewhat in Slashdot style.
Musing on the sociological and economic implications of crowdsourcing, as we did last night, can be exciting. Even though Mechanical Turk doesn't seem to be profitable, its clients capture many cost savings, and other crowdsourcing firms have made headway in particular fields. Howe hails crowdsourcing as the first form of production that really reflects the strengths of the Internet, instead of trying to "force old industrial-age crap" into an online format. But beyond the philosophical rhetoric, crowdsourcing is an area where a lot of companies are making money.
To Learn More Click Here
