Quant Macro Investing

Risk Taking Disciplined

How the Netflix Prize Was Won

Vicktor Capital is looking to build up algorithms to enhance our successful rate in investing.  At this stage, we do not expect 100% quant model investing.  Instead, quant models would flash alerts to the investment manager when the price is right.   The investment manager would know the economic rationale behind the quant model (no black box).  It would be the manager’s sole decision to take investment action or not.

The idea is to build hundreds of quant models.  We bet that the signals from these hundreds of models in aggregate would improve the chance of successful investing.

To build up hundreds of models, how about setting a prize like Netflix to gather the brain powers worldwide?  We need to develop some rules to safeguard the intellectual property rights of people submitting works for the contest.

***********************************************

http://www.wired.com/epicenter/2009/09/how-the-netflix-prize-was-won/

Like BellKor’s Pragmatic Chaos, the winner of the Netflix Prize, second-place The Ensemble was an amalgam of teams which had been competing individually for the million-dollar prize. But it wasn’t until leaders joined forces with also-rans that real progress was made in the contest’s goal to improve the Netflix movie recommendation algorithm by 10 percent.

Besides the sheer thrill of seeing the winner place first by mere minutes after years of work, the Netflix Prize competition has proffered hard proof of a basic crowdsourcing concept: Better solutions come from unorganized people who are allowed to organize organically. But something else happened that wasn’t entirely expected: Teams that had it basically wrong — but for a few good ideas — made the difference when combined with teams which had it basically right, but couldn’t close the deal on their own.

The secret sauce for both BellKor’s Pragmatic Chaos and The Ensemble was collaboration between diverse ideas, and not in some touchy-feely, unquantifiable, “when people work together things are better” sort of way. The top two teams beat the challenge by combining teams and their algorithms into more complex algorithms incorporating everybody’s work. The more people joined, the more the resulting team’s score would increase.

“It’s been quite a drama,” said Netflix chief product officer Neil Hunt at Monday’s awards ceremony. “At first, a whole lot of teams got in — and they got 6-percent improvement, 7-percent improvement, 8-percent improvement, and then it started slowing down, and we got into year two. There was this long period where they were barely making progress, and we were thinking, ‘maybe this will never be won.’

“Then there was a great insight among some of the teams — that if they combined their approaches, they actually got better. It was fairly unintuitive to many people [because you generally take the smartest two people and say ‘come up with a solution’]… when you get this combining of these algorithms in certain ways, it started out this ’second frenzy.’ In combination, the teams could get better and better and better.”

Ironically, the most outlying approaches — the ones farthest away from the mainstream way to solve a given problem — proved most helpful towards the end of the contest, as the teams neared the summit.

For instance, BellKor’s Pragmatic Chaos (methodology here) credits some of its success to slicing the data by what they called “frequency.” As it turns out, people who rate a whole slew of movies at one time tend to be rating movies they saw a long time ago. The data showed that people employ different criteria to rate movies they saw a long time ago, as opposed to ones they saw recently — and that in addition, some movies age better than others, skewing either up or down over time. (Finally, someone has explained whySnakes On A Plane seemed more fun at the time than it does now.)

By tracking the number of movies rated on a given day as an indicator of how long it had been since a given viewer had seen a movie, and by tracking how memory affected particular movie ratings, Pragmatic Theory (later part of the winning team) was able to gain a slight edge, even though this particular algorithm isn’t particularly good at predicting which movies people will like when run on its own.

Another example: According to Joe Sill of The Ensemble, Big Chaos (the Austrians who also became part of the winning team) discovered that viewers in general tend to rate movies differently on Fridays versus Mondays, and certain users are in good moods on Sundays, and so on. The team essentially devised a three-dimensional model that incorporated time into the relationship between people and movies.

Taken on its own, the fact that a viewer rated a given movie on a Monday is a horrible indicator of what other movies they’ll want to rent — a crucial part of Netflix’ business (it says its recommendations are better indicators of what people will rent than their “most popular” lists). But combined with hundreds of other algorithms from other minds, each weighted with precision, and combined and recombined, that otherwise inconsequential fact takes on huge importance.

“One of the big lessons was developing diverse models that captured distinct effects,” said Sill, “even if they’re very small effects.”

**************************************************

http://www.netflixprize.com/

The Netflix Prize sought to substantially improve the accuracy of predictions about how much someone is going to enjoy a movie based on their movie preferences.

On September 21, 2009 we awarded the $1M Grand Prize to team “BellKor’s Pragmatic Chaos”. Read about their algorithm, checkout team scores on theLeaderboard, and join the discussions on the Forum.

We applaud all the contributors to this quest, which improves our ability to connect people to the movies they love.

Stay tuned for details of the next contest,Netflix Prize 2.

October 15, 2009 - Posted by | Uncategorized

2 Comments »

  1. As I know, many large companies such as IBM, google may organize this kind of competition. For the company, they can collect good ideas, for the students, they can have a good chance to get close to their interested companies. These companies have many experiences in organizing these activities.

    But for a new firm, collaborating with universities is a good way, many organizing works can be done by the universities. On the other hand, quant is a dream work for many students majoring in science such as mathematics and physics. This kind of competition will be welcome.

    Comment by Bruno | October 16, 2009 | Reply

    • Agreed. Think collaborating with universities the way to go.

      Maybe some hedge funds are inviting students to contribute ideas already, through projects or competitions.

      Comment by vicktorquant | October 16, 2009 | Reply


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: