Practical Tips from #1 on the Leaderboard with Scott Leibrand

1. Practical tips on making better
predictions
Or, how to get to #1 on DAGGRE

2. “Who are you? And why are you
presenting?”
• Scott Leibrand, age 32
• B.S. in Cell & Molecular Biology
• Advanced Services Architect for
Limelight Networks
• Nope, no professional qualifications
(for forecasting world events,
anyway)
• But, I worked my way up to #1 on
the DAGGRE leaderboard, so here I
am.

3. Overview
• Be judicious
• Be less wrong
– than the next guy
– than you were yesterday
• Gather and synthesize information
– Act like a fox, not a hedgehog
• Share your reasoning
• Prioritize
• Trade well
– Understand the scoring system, and trade accordingly
• Keep track of your positions (it’s easy)

4. Be Judicious
• The biggest mistake people make in forecasting is
being overconfident.
• On DAGGRE, this manifests itself as large edits
way outside the range of every else’s estimates,
in the absence of new information.
• If you make a large edit like that, you’re throwing
points away. If you’re confident the consensus is
wrong, make a number of small edits. If
someone partially reverses you, you’ll be able to
accumulate a much larger position much more
cheaply.

5. Be Less Wrong
• You’re going to be wrong. Just be less wrong.
• Be less wrong than the next guy: If someone
makes an overconfident edit, (partially) reverse it.
• Be less wrong than you were yesterday: don’t
unwaveringly stick to your previous position. If
you come across new information, update your
estimate.
• Be Bayesian: use new info to update your
probabilities appropriately.
• There’s no penalty for changing your mind and
reversing your own estimates.

6. Gather and synthesize information
• You can’t make a good estimate without
considering all the relevant, available facts.
• Read the comments and links others have
posted in the comment section.
• Google makes searching easy. Google News
Alerts are also essential for efficiently keeping
up with a topic you’re invested in.
• Be a fox, not a hedgehog. Make estimates
considering everything you know, not just the
one reason that seems most important to you.

7. Share your reasoning
• DAGGRE is designed to help the group make
better predictions. The more information
everyone has, the better.
• If you have information that others don’t seem
to be considering, post it. That helps the market
to make better decisions, and helps you: if people
agree with your assessment you can take profits.
• DAGGRE now requires comments on >10%
changes. Make your comments helpful.

8. Prioritize
• You only start with 1000 points. You can’t take
a large position on every question.
• Your time is limited.
• You won’t be an expert on everything.
• Even if you are, some questions are just plain
uncertain.
• Calibrate the size of your position to your
confidence.

9. Trade well
• If you want to maximize your points, you have to
understand how the scoring system works, and
trade accordingly.
• Good trading is not a substitute for making good
predictions: doing both is the only way to climb
the leaderboard very fast.
• There are good trading strategies for different
kinds of questions, but first you have to identify
them. That means making good predictions.

10. Understanding Scoring
• You move one estimate from 5% to 10%, one
from 50% to 55%, and another from 90% to
95%. Which costs more? Which gets you
more points if you’re right?
• You’re confident that both question A and B
will resolve false. A is at 10% and B is at 20%.
Which one(s) should you move? How much?
• When should you reverse an edit to take
profits? (Hint: focus on liquidity.)

11. Trading Strategy 1: Patient Accumulation
• If question legitimately has a low (or high) probability, you
want to profit off the occasional overconfident misprediction.
• Build up a cheap contrarian position when the estimate is
really low (high), so you have points to reverse any large
contrarian moves.
• Repeatedly move the estimate from 1% to 2%. If someone
moves it up to 50%, move it back down to 10%.
• You’re not betting the consensus is wrong, just that someone
else will come along and make an unwise edit. If not, you are
only out a few points.

12. Trading Strategy 2: Genuine Uncertainty
• If a question is legitimately uncertain, you want to profit off
edits that move the estimate too far in either direction.
• This strategy is simple: when the current estimate moves
away from your last estimate, make a new estimate that
splits the difference. If you estimated 50%, and someone
puts it at 70%, move it back to 60%.
• The advantage of this strategy is that any position you take
naturally gets liquidated over time, as edits move in both
directions, and you partially reverse them.

13. Trading Strategy 3: Use Real Odds
• Sometimes there’ll be questions (especially sports) where you can
find a quantitative probability of the event.
• For example, http://www.playoffstatus.com/ and
http://www.oddschecker.com/
• Move the estimates closer to the actual probabilities.
• Diversify: you’ll win some and lose some, but if you spread your
estimates across all the questions, you’ll likely gain points overall.
• Update your estimates often as games are played and the odds
change.

14. Trading Strategy 4: Be First
• New questions come out at 50%, but usually don’t stay there.
Edit early (in the correct direction) and gain points fast.
• Consider related questions, and make conditional edits.
• Invest according to your confidence level. Treat others’
estimates as new information.
• Once the probability is accurate, start gradually liquidating
your position. Always move the estimate back down to at
least 98% (or up to 2%) if it gets all the way to 99% (1%).
• Consider switching to Patient Accumulation when you
manage to fully liquidate.

15. Track your positions
• The strategies above require knowing when
estimates change.
• Periodically check “My Positions” to see which
ones have moved the most.
• Check the comments and your Google News
Alerts for new information.
• Make a few edits each login. Check in
frequently.

16. Follow-up
• Play the workshop market: it’s the fastest way
to learn.
• Play the practice questions. I’ve offered free
points to anyone demonstrating a good
trading strategy there.
• Read Nate Silver’s “The Signal and the Noise”.
• Come talk at breaks/lunch.
• Connect with me on LinkedIn, Facebook, or
scottleibrand@gmail.com

17. Questions?