Measuring Prediction Accuracy

We all love to make predictions, but how good are we really at making them? The Brier score is an easy way to quantify the accuracy of your decisions. It's quite simple to calculate:

BS = 1/N * SUM((FORECAST(instance) - OUTCOME(instance))^2)

Where
  BS = Brier Score
  N = number of instances that predictions where made for
  OUTCOME(instance) = binary outcome (1 or 0)
  FORECAST(instance) = prediction probability that OUTCOME(instance) = 1