Don't read too much into college stats
by Jim Callis
CHICAGO--Michael Lewis' bestseller "Moneyball" continues to resonate two years after its publication. The book made Billy Beane the rock star of baseball general managers, offended most scouts and drove more teams to pursue statistical analysis.
The term "Moneyball player" has become part of the scouting lexicon, a tag applied to any college hitter or pitcher whose performance is significantly better than his tools, especially if it includes an impressive strikeout-walk ratio.
These players are associated with the idea, believed by some clubs and perpetuated by "Moneyball," that college statistics can be crunched to make valid comparisons between players. In the book, Lewis describes a radical thought by Beane:
"Billy had his own idea about where to find future major league baseball players: inside (assistant GM) Paul (DePodesta's) computer. He'd flirted with the idea of firing all the scouts and just drafting the kids straight from Paul's laptop. The Internet now served up just about every statistic you could want from about every college player in the country, and Paul knew them all."
Jeff Lunhow, who came to the Cardinals from a software and technology company in 2003 and will direct their draft this year, says that St. Louis isn't satisfied with mining just NCAA Division I stats.
"We're starting to crack the code for Division II and junior college and some of the other levels," Lunhow told SportsBlurb.com.
When Bill James developed his Minor League Equivalencies in the mid-1980s, translating minor league stats into big league numbers, he didn't venture below Double-A or mess with pitchers because the future performance records didn't hold up. Yet now some teams are putting faith into statistics from a level of baseball light years removed from Double-A.
The upper tiers of Division I college baseball are roughly equivalent to a short-season or advanced Rookie leagues. Within Division I, the quality of play varies so widely that comparisons between the best conferences (Southeastern, Big 12) and the worst (Southwestern Athletic) are problematic. Still, the SWAC produced 2003's second overall pick in Rickie Weeks, who set an NCAA record with a career .473 average while facing almost no pro-caliber pitching.
"The different levels of play in Division I are as wide-ranging as going from the World Series champion to the worst team in professional baseball," says an area scout for a statistics-oriented team who has college coaching experience. "Even in the best conferences, there's a big range from your best and worst teams."
Merely adjusting for the quality of teams faced isn't enough. Playing a team on a weekend usually means facing its top starting pitchers, while matching up with it midweek means batting against a lesser arm.
Restricting analysis to conference games only, so teams are facing the same opposition, reduces the sample size to 20-30 games. Don't forget to make adjustments for the widely different ballparks teams play in.
If you have to run the numbers through more plastic surgery than Joan Rivers, how much faith can you really put in them? Not too much, warns an official from another club that relies heavily on stats.
"You can't just draft off the stats," he says. "They're only a guide. You can't 'crack the code' because there are too many variables."
Good college stats, translated or otherwise, aren't a guarantee of future major league (or even minor league) success. "Moneyball" revealed the A's list of their top 20 targets for the 2002 draft, eight of whom were stat darlings but not consensus first-five-round picks: pitcher Steve Obenchain and position players Jeremy Brown, Steve Stanley, John Baker, Mark Kiger, Brian Stavisky, Shaun Larkin and Brant Colamarino. Oakland drafted all but Larkin and may not get more than a couple of reserve first base/DH types to show for it.
Jered Weaver put up glitzy numbers last year at Long Beach State that matched what Mark Prior did as a junior at Southern California, but that doesn't mean Weaver is the next Prior, no matter what Scott Boras says.
The best use of college statistics is to raise red flags. If a player is struggling in college, it's unlikely he'll have a dramatic turnaround after turning pro. Georgia Tech's Jason Neighborgall may have the most electric arm in the draft, but he rarely has been anywhere close to the strike zone for the last two seasons. He has 71 strikeouts in 53 innings, but also 50 walks and a 6.66 ERA.
Statistics do have value. To think otherwise would be as misguided as the belief that scouts care only about tools and "the good face." GM Terry Ryan's Twins are seen as the antithesis of "Moneyball," yet they too look at performance and have drafted as successfully as any club in recent years.
"In the draft, stats are part of the evaluation process but they're not all of it," Minnesota scouting director Mike Radcliff says. "Most of us have been using stats as part of the process forever. They can help you in a supporting fashion, but you can't use them as your base."
You can reach Alan Schwarz by sending e-mail to firstname.lastname@example.org.