In my last post, I compared the Steamer projections to the NFBC ADP list and noted where the two agreed and diverged. I want to take a step back, though, and compare the two approaches at a more fundamental, philosophical level.
On the Steamer side you have a machine, programmed to account for the factors that are generally most predictive in player performance and to exclude those that are usually noise. We do this ourselves while making forecasts, but not as efficiently, comprehensively or rigorously as a machine can. We also are prone to overweighting some factors, underweighting others and falling for narratives that might not have any predictive value.
On the NFBC side, you have the wisdom of an informed crowd constituting the ADP, those with "skin" (their entry fees) in the game. They might be subject to groupthink - players do seem to rise and fall on momentum sometimes - but these are some of the top players in the world, many of whom work our their valuations and strategies independently. NFBC drafters are nimble - they can bump up a player slated for more playing time or having a hot spring immediately. (They also might overweight short-term developments in exhibition games.) But in my opinion the biggest advantage the human drafters have is the capacity to identify which players are not like the majority and therefore not subject to its general rules.
For example, Tom Glavine should not have been a 300-win Hall of Fame pitcher based on stuff or peripherals. But his career ERA was 3.54 and career FIP 3.95 over 4,413 innings, i.e., it was not luck. Glavine never struck out 200 batters in a season and walked 60 or more 20 times. Mariano Rivera, the Jerry Rice of closers, threw only one pitch, somehow had a BABIP of .263 pitching in front of mediocre defenses and a HR/9 of 0.5 in a homer-friendly park over 1284 career innings. A machine might have regressed both as if they were normal players, subject entirely to good and bad bounces and the way the wind blows.
The question at stake is whether players are all more or less the same - that is, roughly subject to the same forces when it comes to their future output. At one level, the answer is obviously yes, the laws of physics and biology (aging) and the rules of the game (four balls is a walk, three strikes is an out) apply equally. But not all players age at the same rate, and not all pitchers are equal with respect to balls in play. Perhaps a pitcher like Glavine walks people not due to lack of command but strategically, to avoid giving in when behind in the count. While the walks for an ordinary pitcher might signal command problems and portend disaster, Glavine's might not mean anything except an extra runner on base in a situation from which he can usually extricate himself.
Identifying these outliers is something with which the machine is likely to struggle because its accuracy and reliability is premised on what usually happens, not the exceptions. It's premised on what's happened in the past, not only to that player but to the player pool generally, not what will happen to a particular player in this particular instance. The machine's success depends on things usually going how things usually go. And they usually do.
But the NFBC - or even your individual home league draft - is not an overall projections accuracy contest. It's more a contest of who can make the fewest devastating mistakes (early-round busts) and find the biggest breakouts (late-round steals). In other words, it's very much about identifying the exceptions to what usually happens. Some humans - and one could argue, the collective wisdom of an informed crowd - will be better than average at identifying those exceptions.
The counterargument is that even if this is true - that some have a skill at identifying the outliers (and it may not be) - most people will see outliers where there are none, creating false narratives to justify their conclusions and overpay. That the machine, not being subject to this bias, will simply generate incremental value pick after pick, proportional to the humans' hubristic mistakes. Sometimes the Steamer picks will get unlucky due to injury, and sometimes they'll break out surprisingly, even where no breakout was predicted, but overall, the return will be better.
I see both sides of this argument, and my conclusion is both are simultaneously true. Players are both more or less the same in many relevant ways and also entirely unique in others. It's worth running the machine numbers to see what usually happens and using those valuations as baselines. In general, that's the right play. But one must also trust one's instincts when something anomalous presents itself. How well one interprets the hard-to-handicap players, the situations that jump out as unique, small-sample departures from prior performance, etc. - disproportionately affects ultimate outcomes.