With the rise of Web 2.0 and the push to make the users of the internet producers and not just consumers, there has been a rise in “crowdsourcing” — if you need an answer to question, ask a large number of people and someone will give you the right answer. Websites like Digg and Reddit (and to some extent, Wikipedia) have risen to power on the basis of this concept and a good argument could be made that crowdsourcing actually does work. A few days ago, another crowdsourcing project went into public beta — StackOverflow. This is the brainchild of two software-engineers/ bloggers whom I have a fair amount of respect for: Joel Spolsky and Jeff Atwood. The basic idea is that there is a large amount of very good and useful out there on the web, but most of it is in the form of random blog posts, long forum posts, IRC transcripts, so on and so forth and so it’s not really accessible to people looking for a straight answer. The goal of StackOverflow is to unlock that knowledge by being a hybrid of forum, wiki, blog and digg/reddit style rating systems.
While I certainly understand the goal of the project and understand it’s value, I’m not quite convinced if it will work the way it’s supposed to. As a programmer, a lot of the time I really don’t want to read through detailed documentation or long mailing list discussions. I just want a simple “What does this button do?” style answer. And as any programmer who has gone digging through documentation knows, that’s not an easy thing to come by. StackOverflow hopes to promote a community of developers in the hopes that someone has already solved the problem you’re facing, and is ready to come out and help. There has been some criticism that StackOverflow will only attract mediocre developers looking for quick fixes and superficial knowledge thereby leading to the lowering of the competence of the people who are there. I think that this argument holds for crowdsourcing in general. There has been a pretty good answer made to this criticism and I suggest you go read it for a fair evaluation of the problem.
But let’s explore the alternative. If you don’t want to be pulled down to the lowest common denominator of competence, you’ll have to learn from someone better than you, not someone worse. Of course, the best way to do that is probably to find an expert in the problem you’re looking at. While this looks like a good solutions, there are as always, a number of problems. The one you’re likely to run into first is that the so-called experts are often far too busy to answer your questions. After all, they didn’t become experts without spending a lot of time actually working on stuff. Secondly, I have personally found that experts are often prone to give you a packaged answer without telling you all the inner details. This isn’t because they are arrogant or trying to confuse you, but rather because the underlying reasons are so clear to them that they don’t bother mentioning them. They are telling you their approach to the problem, which may not be the same as your own. Finally, just because they are considered to be experts doesn’t necessarily mean that they are the fountain of all knowledge. They can be wrong, imprecise or may just not have the answers.
So can crowdsourcing actually provide the easy-to-get, mostly correct information that so many of us are looking for so much of the time? I don’t think there is a real consensus and I can’t quite tell if there can ever be a definitive answer to this question. In a perfect world we would each have our own personal AI that plugged into various search mechanisms and databases and gave us the information we wanted, tuned to our needs. Of course this AI would keep evolving to better anticipate what we want to know. Think of it as a digital butler to serve our informational needs. I have personally not had to rely on crowdsourcing very much. Living in a small liberal arts college environment, I’ve had professors at arms reach, most of whom are very willing to help out and explain things to me. However, I understand that I am probably a special case in the programming world. Probably as a direct result of that, I have come to favor direct communication between myself asking a question and someone who has the answer (or can at least point me in the right direction). At the same time, I find the ideas of digital communities very interesting and I would love to see StackOverflow grow into something genuinely good and useful. I will be keeping my eyes on it and maybe in a few months it might be time to revisit my opinion of crowdsourcing.