Note: (Distributed) Collective Intelligence, the Internet and the Crowd

The need to search and find non-structured information on the Web was perhaps one of the most pressing drivers of innovation during the 1990s. Shear scale meant manual indexing was impossible, and computers are not ‘smart enough’ to understand freeform queries and match this to. To do this, information scientists drew on social science and biology (e.g. the collective work of insects), and the concept of collective intelligence which had been developed in the field of complex adaptive systems.  Collective intelligence approaches note the emergence of intelligence' from agents, such as ants, following simple rules. (Bollen and Heylighen 1996).[1]

The internet provides a platform for large scale distributed collective intelligence processes, tapping into the many small bits of work that users do for themselves (Bollen and Heylighen 1996). This was applied to solutions on how to organise the work necessary to classify and catalogue this huge among of data created by the Web, which entrepreneurs developed a whole range of 'Web2.0' and social computing system, from Google PageRank search algorithm (Brin  and  Page, 1998), to systems that use user tagging, user recommendations and user linking.

Taking this idea further, a collective intelligence approach suggests that a 'crowd' of people, if large enough and working autonomously, can produce a 'better' solution to a problem than a small group of experts, just because to the shear diversity. Drawing on empirical examples on the web, and the science of collective intelligence enabled a number of popular writers to ignite broad excitement around ideas of 'the wisdom of the crowd' and 'crowdsourcing' (Surowiecki 2004; Howe 2008).  This seemed to provide a theoretical explanation for the success of projects such as Wikipedia. These ideas were also incorporated into the design of the open innovation mentioned above.

In fact, the peer production model does not accord with the 'crowd' model as there are many forms of social organisation, often with extensive interactions between people (agents) that is necessary for a collective production, rather than merely responding autonomously to a problem set by an anonymous other.

A variety of analytical frameworks have been proposed to capture the ways the internet has create and shaped the way that work can be done, and tasks distributed among a large crowd. These have generally sought to identify some key qualities of distributing work in this way that makes it unlike any other sort of organisation or management approach, and to explore the economics of the approach. Two of the approaches that have achieved most attention, and stimulated considerable entrepreneurship and research are 1) what is variously called 'distributed or networked collective intelligence' and 2) crowdsourcing.

Both identify a common idea – that bringing many otherwise disconnected people together to work on a problem can be more effective, accurate or creative in finding a solution than an individual, computer or team of people can be.

Of the two approaches, the ' collective intelligence' is by far the older (Levy P, 2010 From social computing to reflexive collective intelligence: The IEML research program Pierre Lévy Information Sciences Volume 180, Issue 1, 2 January 2010, Pages 71–94, special issue on collective intelligence)[2] , predating the internet, and  maybe considered a discourse more situated in a scientific domain, The 'crowdsource' approach (2006) can be seen as a popular science version of collective intelligence, but is actually more empirically driven, based on a decade of successful internet-based that emerged from the 'Web2.0' The crowdsource approach,  which draws on another popular science  idea of 'The wisdom of the crowds' (Surowiecki 2004) has probably captured the imagination of a far broader range of people, and stimulated innovation outside the world of computer science.

The “Crowd” and crowdsourcing

Surowieski (2004) claims that under right circumstances groups are remarkably intelligent, often smarter than smartest individual.  Best outcomes comes from aggregation (Page 2007)  even a crowd of non experts can provide cognitive diversity to solve problem” (Brabham 2012)

Work on Crowd approaches tried to Identify different ways in which a crowd without necessarily specialised knowledge can nonetheless produce a 'better' outcome than consulting experts.

Surowiecki 2004 in Wisdom of the crowds (204)  suggests that asking a 'crowd' can produce better results when: 1) cognitive diversity (people have private info); 2) independence; 3) decentalization; 4 aggregation.

However, contemporary crowdsourcing systems break all these ‘rules’