As I mentioned yesterday, I’m a big fan of ‘Rebooting the News’. That goes for both meanings: I love the podcast series by Jay Rosen and Dave Winer; and I’m also totally intrigued by the phenomenal transition of our system of news which is happening right under our noses.
In the 9-minute passage of RBTN 82 that I transcribed, our hosts talk about an idea that Dave put forward in a recent blog post, ‘Find me stuff that I’m interested in‘. It’s a discussion about the concepts of a personal recommendation system for news, on Dave’s part inspired by collaborative filtering technology which underpins Amazon’s personal product recommendations.
Not only do I agree with all the conceptual choices that Jay and Dave favor, – such as avoiding categories, using gestures, using feeds, looking at other users’ previous behavior, including information about authoring as well as consumption, including serendipity… – ; I have actually been thinking about these exact concepts for years.
Now, I’m not going to say, “It’s all been done already”, because Dave would think I’m trying to pitch a product 
Truth is, had it been done, we would all be using it. A personal system of highly relevant information is pretty much the Holy Grail of the Internet.
One potential complication with applying collaborative filtering to news content is that, when news breaks, there is no critical mass of gestures from previous users. This may cause some delay in the build-up of a recommendation. Instead of immediate, mass-scale amplification of the breaking news event, the news item might be a more slowly developing “trending topic” as per Twitter.
Also, when the news is very fresh, and its relevance is very personal (i.e. highly relevant to a small number of people), it may take too much time for a collaborative filtering system á la Amazon to collect sufficient gestures from other users in order to deliver the recommendation to the right people.
Therefore, rather than waiting for a new news item to pick up the critical mass which can enable collaborative filtering the Amazon way, we could instead look at the *history* of users’ gestures. If the stuff I have “gestured” in the past is very similar to the stuff you have “gestured” in the past, there is a likelihood that what you “gesture” next will be of interest to me.
So what I propose, instead of collecting many gestures from different users in order to generate a recommendation to one specific user, is to identify pairs of users whose gesture behavior is most similar, and let their behavior inform their mutual recommendations.
One could calculate a “similarity-percentage” for each combination of two users based on their gestures. With a view to serendipity, the ideal similarity is not necessarily approaching 100 percent. The system could offer users a feature to mix their own doses of serendipity. Want more off-beat news today? Turn the potmeter down to 70 percent signal and get 30 percent noise!
BTW, one headache which this idea would take care of is the eternal question: “What is news?” Whatever news means to you is defined by what you “gesture”. Hence the more accurate question to ask would be: “What is relevant?” or, indeed: “What is interesting?”
Like said, I’ve been pondering over this stuff for a while and I’d just love the opportunity to help make it happen.
The post Booting up a personal recommendation system for news appeared first on josschuurmans.com.