Wisdom of crowds - reality or virtuality Saturday, March 24, 2007

On Friday I met with COO of one of Europe's largest brokers, who is an old friend. During the "hellos" we reminisced about how in the 1990's it had been possible to analyse retail broking transactions and identify clusters of investors that might have appetite to participate in IPOs.

I mentioned how similar crowd "knowledge" was being generated amongst communities online now and gave him a few examples. In each case people are acting for personal gratification but their actions/behaviour is providing useful indicators when aggregated with other individuals.

PicksPal which is a sporting event prediction site. Its' users vote on amateur and professional sporting events but do so for fun rather than money. You to spend points to try and beat the odds makers - you win or lose points depending on the accuracy of your picks and your risk appetite. PicksPal does award prizes to the top performers and says they've over 100,000 registered users, albeit not that many playing regularly.

Like Betfair and Sporting Index, you can bet on a lot more than just the outcome of the game. such as who will be winning at various points in a game, or which player will score the most runs.

The value add though is that PicksPal sells "expert" picks, based on the selections of their top performing members, on which people can bet real money. They claim to have a 52% win rate for their last 25 picks, with a 63% win rate overall.

In similar fashion, Marketocracy looks to monitor the best investors and aggregate their collective knowledge to create a virtual fund. Users are allocated an initial $1 million when they sign-up and 60,000 users have joined to date.

However, Marketocracy went a stage further in November 2001 in launching a real mutual fund based on the virtual investments of its 100 most successful members (as determined by a computer ranking). The Masters 100 Index fund, has $44 million in assets and has outperformed the S&P 500 Index with an average annual return of 11.4% since its inception.

The only issue with these two system is that the users don't face any moral hazard or financial impact of participating. Consequently, users can "afford" to take greater risks than they otherwise would with their own money - it's not clear they would follow their own forecasts. In the case of Marketocracy, the output assumes that all investors have the same risk profile, when clearly that is not the case. In the "game" the experts are deemed to be those generate the highest returns. Yet the real world doesn't work like that. Certainly everyone wants to see their wealth grow but people temper that by the amount they are prepared to lose in an investment strategy.

It is for this reason that one has look a little deeper when looking at the results of "crowd knowledge" - are people really behaving as they would in the real world and will they suffer the consequences. If not, it's likely to be unreliable as a guide on which to base real world actions.

Labels: , ,

posted by John Wilson @ 7:58 PM Permanent Link ,

newsvine reddit



Crowdswarm - circular popularity syndrome Monday, January 15, 2007

A fascinating study by Matthew Salganik of Columbia University in New York re-enforces a point I was making to Nic Brisbourne of Esprit Capital Partners. Nic had posted about the wisdom of crowds (Last.fm) v machine intelligence (e.g. Pandora).

I feel that there is an important sub-division in the “wisdom of crowds” category, between trust networks like LinkedIn and crowd sites which give you data on which you may or may not choose to rely like Toptable reviews. Clearly the former is much stronger since your reliance on the information is based on your assessment of the source. In the latter, popularity is your main guide, which doesn’t necessarily equate to wisdom. Sam Sethi (Vecosys and formerly techcrunchuk) and I also recently chatted about this and its impact on the long-term success of “social networks”.

As regards machine recommendations, the ability of computers to spot correlations in behaviour to help make “insightful” suggestions I believe is a very powerful force but only to the extent that the dataset and population on which it draws is sufficiently sizeable to allow meaningful conclusions to be drawn. Too often social networks have/use insufficient data points in making comparisons to allow them to identify genuine correlations e.g. male & works in capital markets as our shared points isn’t sufficient to guess our interest/tastes will overlap. Add on music, hobbies, education….. At which point we start to use something akin to “dating service” algorithms to identify people we “correspond” with and hence have increased likelihood that the machine recommendations will resonate.

The Columbia study created a "music market" and identified that when songs were ranked by how many times they'd been downloaded, people followed the crowd. When the songs were not ordered by rank, but displayed the number of downloads, there was a less significant "social influence". It wasn't wisdom, just popularity that drove behaviour - "Swarming" as they termed it.

As separate study by the Florida Institute of Technology is testing out "swarming" in supermarkets, by providing shoppers with a device that indicates to them how many people currently in the shop have bought the item (or from the shelf) they are walking past. The idea is supermarkets benefit from increased sales without giving discounts and consumer get the satisfaction of buying a "popular" product. This is obviously a vicious/virtuous circle depending on your view, with visibly unpopular products being further shunned.

Oddly enough, whilst reading about these studies last night, the TV was showing a BBC Nature programme on the migration of the wildebeest across Africa, with an annual migration of one million of them on a round trip of 3,000km. A large chunk of the programme was devoted to the "watering hole" inhabited by 300 or so crocodiles. Despite the crocs repeatedly grabbing victims as they drank, the crowd kept returning to drink every few minutes. Admittedly there was nowhere else to drink for 30km, but maybe it shows that sometimes the crowd never learns.

Labels: ,

posted by John Wilson @ 9:10 AM Permanent Link ,

newsvine reddit