Clay Shirky - Filter Failures

Clay Shirky is an American writer, consultant and teacher on the social and economic effects of Internet technologies. He teaches New Media as an adjunct professor at New York University's (NYU) graduate Interactive Telecommunications Program (ITP).

How many of you have seen this chart about information? (Chart of exponential data growth.) For 15 years we've been reading the same story about information overload. You can find stories in 93 that are the same as those that showed up in your RSS feed 3 minutes ago. Why is this?

Here's why I think this is: Guttenberg back when he invented the printing press, the cost of printing and the volume of what was being printed in those times was greater than a human could read in their whole life. Since you had to print the books in advance you had to take on all the risk of printing the books. The economic decision was pretty simple: make the editorial team at the publisher figure out what is worth printing.

The internet introduced post-guttenberg economics. The cost of producing anything by anyone has fallen through the floor. There's no economic logic requiring that you filter what's good from bad is now way down stream. What we're dealing with now isn't information overload, we're always dealing with that. Thinking about filter failure is the problem. Everyone has a morning ritual of removing spam from their e-mail. Some combination of a mechanical filter and manual filter.

All of the filter solutions are temporary because you have to assume that the ammount of information coming in is only going to increase. This is a general system design problem. Let me tell you what happened to a friend of mine: a former student, a colleague and a good friend. Last December decided to break off her engagement. She also had to engage in the 21st century practice of changing the status of her relationship. Might as well buy a billboard. She has a lot of friends on Facebook, but she also has a lot of friends on Facebook. She doesn't want all these folks, especially fiance's friends, to find out about this. She goes on to Facebook and thinks she's going to fix this problem. She finds their privacy policy and the interface for managing her privacy. She checks the appropriate check boxes and she's able to go from engaged to single. Two seconds later every single friend in her network get the message. E-mails, IMs, phone is ringing off the hook. Total disasterous privacy meltdown. She did her gradutae thesis on the differences of Friendster, Facebook, and MySpace. She's not an average user. Hard to blame Facebook when they've made so much of an effort. The problem is that managing privacy preferences is an unnatural act. Not something we're used to.

Privacy is a way of managing information flow. What she wanted to do is tell four or five of her friends and allow the information to slowly sweep through the network. That's how it used to work. The big problem we're facing with privacy now is how we're moving from a social system to an engineered, binary system. We live most of our lives not in the bubble of privacy, we have what we called back in the day our personal life. We don't have so much personal life anymore. The inneficiency of information flow wasn't a bug, it was a feature. There was something nice about it being difficult to say things in public. This is a question of filtering.  That's a story of outbound information flow. Spam is a story of inbound.

There are some stories that you can't even tell the source. Here' s another story. By the time Chris Avenir was 15 MySpace existed. When he gets to college he enrolls in chemistry class. Because he's 18 he starts the study group on Facebook. He gets 146 of his classmates into the group. Suddenly he's facing 147 charges, one for setting up the group, one for every student he's working with. They claim he's cheating. What has he done? He's crashed two information flows together. To the inside world they say "come here, have conversations" to the outside world they have quality control "students are top notch." What keeps these apart are that the real world is outside of the walls. What he did on Facebook is break down that wall which gives the outside world the ability to see 146 students in a 'study group'. Facebook is not like a fax machine, not like a studygroup meeting. Facebook turns out to be a lot like Facebook. It's different from what's gone before. Only worth spending time on it because of that difference. There is no simple solution to this problem.

This isn't a design problem, it's a mental shift. It's understanding we'll have information overload just the way fish have water. A quote: "when you've faced a problem long enough, maybe it's not a problem, it's a fact." Some of this can be solved with programs: look at Digg, Google, etc. But a lot of it can't. When you're getting deluged with information the question you need to ask yourself is "what filter broke?"