Kiley McDaniel on Scouts vs Stats Doug Gray January 10, 2018 14 Comments The debate of scouts vs stats has long been dead for those who are actually in the game. While you will have those on the airwaves, or in the newspaper columns who want to tell you that the stat nerds are wrong, or the new to sabermetrics guys who tell you the scouts aren’t good enough, both of those groups are out of their element. Every front office in baseball is using both aspects and relying on input to inform their decisions. The scouts learn the statistical stuff. The stats guys learn the scouting stuff. The debate is over. There’s no “winner”. The answer is both and anything to the contrary is just wrong. Why anyone would ignore data on either side is foolish. Fangraphs writer, and former Major League Baseball scout with the Yankees, Orioles, and Pirates, as well as most recently the Assistant Director of Baseball Operations with the Braves, typed up an article on The Status of the Scouts vs Stats Debate for Tuesday. There’s a lot of very interesting things within the article. Let’s start with the idea of grading scouts. At least one team he worked for has tried this. They concluded something that I’ve often spoke about when referring to trying to grade a draft: Too much time is needed to get a good idea of whether it was good or bad. One of the clubs with which I worked tried to “scout the scouts” and see if it was possible to identify empirically who the best scouts were, or even their relative strengths. With most scouts working on one-year deals, you’ll likely promote/demote them at least once and give them four or five new contracts before you even know if they were good in year one. On the amateur side, you’d need something like a half-dozen years of a scouting director’s entire draft board before you had a statistically significant idea of how good he was. In some organizations, you’ve already fired his replacement by the time you find out how good the first guy was. It’s essentially impossible to prove how good a scout is in a timely enough fashion that it would constitute actionable information. Simply making pro, amateur, and international staffs entirely composed of above-average scouts (that’s upwards of 50 evaluators in total) is easier said than done. You’d only have this scouting-the-scouts data (that is of limited use) on the scouts who had already worked for you for years. You don’t know who made the final call on a high-profile decision or what their whole list was (i.e. how lucky they were to sign the right player) if you’re poaching a scout from another club. It’s an interesting idea, but it’s certainly not easy to do. It’s something that seems impossible to accomplish given the landscape of how things work in the industry. What is interesting, though, is that while the scouts vs stats debate is indeed over, there are still battles in baseball among them. Both sides appreciate what the other does, but also can – at times – feel that their information is better. And both sides are probably correct in their belief. I’ve spent years playing scout on the internet. If you’ve read my work, you’ve also seen that I am a proponent of stats. Stats can tell you things about the current and the projection. Scouting can do that as well. Stats also miss things about the projection. Scouting, likewise, can also miss things in the projection. The scout can tell you why someone may be underperforming based on what they see. A guys timing is off – and it could be corrected by changing “this” mechanically. The analytics may tell you that a guy is underperforming what he could be, based on say his exit velocity. Fixing his launch angle could change that. Instead of hitting hard ground balls, the new launch angle is leading to hard hit fly balls. Which leads to more extra-base hits. These days, analytics are almost just a different kind of scouting. Front offices aren’t looking at stats and saying “oh, this guy hits 22 home runs”. They are looking at his stats and saying “this guy has an exit velocity that’s 93 MPH on average, and he’s most often in this range of exit velocity and this launch angle”. And that information helps give them a good idea of projection on the player moving forward based on what that type of batted ball tends to go for. They all have models built, using far more advanced information than they used to, to project and predict the future performance of players. What’s been interesting to see with the “analytics” explosion in the public sphere over the last 10 years since we started tracking pitchers with Pitch F/X and gotten massive amounts of data, and since then added in the Statcast tracking of batted balls and defense, is that by-and-large, most of the data is simply confirming a lot of what scouts were saying. The difference is, you can now attach a specific number to it. “This guy has tons of spin on his breaking ball” can now be refined to the actual rate of spin. It’s just a little more accurate. It’s been quite a while, probably nine years or 10 years ago, but a Major League Baseball employee contacted me when I first started writing about the Pitch F/X stuff. They worked in player development at the time. They now work for a Major League team at the big league level. Their organization wasn’t doing much with the analytics at the time. But this person was curious as to what the information could be used for, to help him with his job. We kicked around a bunch of ideas of things that it could possibly be used for. This was a guy who, while he wasn’t a scout by job description, was a scout based person in terms of how he had to go about his job. But, he understood that the data being spit out by this new system could possibly help him do his job better. Over the last few years the Trackman system has showed up in Minor League ballparks around the country. I’m not sure if there’s a ballpark that doesn’t have one these days. It’s been interesting to talk with scouts in the past few seasons about it. Even they are talking about things like exit velocity, spin rates and such. At the same time, some also express concerns about what comes along with some of that. Call it the radar gun generation. With the tracking of everything, starting in middle school for a lot of kids, there’s an increased emphasis on throwing harder and less on the art of pitching. I can understand the problem with that. I also understand that throwing harder is better than not throwing harder. Ideally you want guys to both throw harder and know how to pitch. Wrapping up my thoughts on the subject This has been a fairly long post. And I could go on for days, probably, on this topic. But rather than write about 10,000 more words on the topic, I’ll leave it with one more thought. The best organizations are constantly asking questions and seeking answers. They aren’t ignoring data, but actively seeking out more data and information to help them. Statistical analysis does that. Scouting does that. So do other aspects that front offices are implementing that weren’t talked about here. Every team has scouts. Every team has analysts. The best teams are going to be the ones that can figure out how to best weigh each side. How you weigh which side depends on the players involved. Being able to understand which situations that each will bring more value to is more important than ever. The majority of teams are working with similar input from scouts and analysts. Things are won these days on the margins with these decisions. There’s isn’t likely a next “no one understands the importance of on-base percentage” coming to baseball that will give teams that understand that a huge advantage over the competition. Finding the right decision makers to run the franchise is key, and it always has been. But these days it may be more about things that aren’t directly related to baseball itself than it’s ever been. 14 Responses Cinvenfan January 10, 2018 Good post and conclusion. Both are equally important. Hate to see some armchair wanna-be GM in other site attempting to underestimate the scouting side of baseball and pretending that their laptops and moneyball formulas are the answer. He (you know who I’m talking about) makes fun of posters who talk about “intangibles” or “make-up” as if the players were robots. Stats (always been used by the way) are necessary and the evolution in sports (like in every other field) tends to incorporate more data. But the human side of the game cannot be counted out. Neither the flaws in many of those stats (i.e defense). Colorado Red January 10, 2018 well said, could not agree more victor vollhardt January 10, 2018 Mr. Gray’s article does a very good job and CINVENFAN has a great line “both are EQUALLY important”. Mr. Rickey hired the first full time stat guy (Allan Roth) in 1947! The Brooklyn Dodgers were always looking for an “edge”–how to find-how to teach-how to train baseball players. A good organization is always looking at how to get better—- inside baseball or in any other kind of business. I think many of the so called “new stats” are good and indeed have already proved their value. And I think some are overkill and may even have had the effect of passing over good MLB players in the draft and in figuring out roster formation. But not to worry that kink will be fixed by market (W/L) results. I sell no stat guy short, but what is irksome is that many of the new breed have no patience for any thoughts from the “old school” even if over time those methods have been proven to work Michael B. Green January 10, 2018 In many instances, stats support scouts. They should never replace scouts though. The great part of this game is that there are still aspects that are perhaps not statistically measurable. If a scout watches a player and he sees some qualities that remind him of Alan Trammell, his observation is perhaps based on his experience and his understanding of athletes. Later on, statistics may in fact show some similarities to Trammell. I would say, however, that statistics are an extremely important aspect of the game. A hitter is foolish not to listen to a team’s analytics staff to inform them that Clayton Kershaw throws a change up 83.2% on a 1-0 count (made up stat to illustrate my point). I think both are integral and I think neither should replace the other. Some folks could do both but that is quite possibly the exception to the norm. I seriously doubt a team would want Yogi Berra handling their analytics division but they absolutely would have wanted him from a scouting perspective. sultanofswaff January 10, 2018 Good article Doug! I’m excited to see how the game will evolve with the increased use of Trackman data. Yes, it’s near impossible to scout the scouts. So in the absence of that how do teams acquire their scouting talent? Sadly, it’s who you know………..the old boys network. It’s at the very core of the game, and I would argue this culture is even more concentrated in baseball relative to other businesses. Not that this is a screed against Bryan Price, but for anyone who thought there was this fresh, progressive voice taking over, they failed to understand this culture……..a culture that says there’s an established pecking order and that pecking order informs most every decision, from veteran privilege to the way young players are used. Price has adhered to that standard to the detriment of the team more times than I can count. The Duke January 10, 2018 I think an argument can also be made for traditional stats vs advanced metrics. Sure, some of them are very situationally dependent, but it does show how you performed in those situations. High RBI guys are dependent on people being on base, but it shows that you came through in that pressure situation when someone was on base. Traditional and advanced stats should be used together and not as much one over the other like I’ve seen argued ad nauseum over the years. Michael B. Green January 10, 2018 That’s an intriguing point, the Duke. You have my analytical brain thinking about RBI success rates and opportunity rates, high leverage situations, etc. Perhaps the RBI generic stat factors all of that! Billy January 10, 2018 How are scouts signed and trained? Is there, for instance, a consistent training program that the Reds’ scouts go through to ensure that they’re all providing the same kind of information and/or looking for the same characteristics? Do they have any annual refreshers to make sure everyone is on the same page – maybe including some video evaluations that could be used to ensure some consistency? It seems a little surprising to me that a team can’t control the variance in scout quality more than what is suggested. MK January 10, 2018 There is an MLB Scouting School, plus I am sure each team does their own thing as well. Doug Gray January 10, 2018 Major League Baseball, as MK noted, has “scout school”. It takes place during the Arizona Fall League each year. Teams send people out there to learn the ins-and-outs of how to grade tools, and some other aspects of the game. With that said, most scouts have a background in playing or coaching. They’ve been in the game, at high levels. Many scouts are former professional players, either minor league or Major League guys. There isn’t exactly a “refresher” course. But, each team does tend to have their own “system” in terms of how they grade things, what they expect, and so on. That’s communicated to them. The biggest problem with controlling the variance in scout quality is that guys switch jobs too frequently. As noted, the best scouts move up and become directors and front office members, usually. And when you are running an organization that has 35+ scouts, some of which are a year or two into their careers or even just in your organization, it’s tougher to peg the quality of them based on their evaluations. First, you can’t use any data to say this guy is right or wrong, because you just need time in order to prove that, and time isn’t on your side here if you are talking about amateur or even minor league scouting. Let’s take amateur scouting, for example. It’s going to start with your local guy, who isn’t a full-time employee. He’s out watching the local high schools and colleges. He sees a dude and sends in his report. If the report is good enough on a guy, then his boss comes out to double check/get another look. That guy is probably a full-time scout. Still a local/area guy, but someone who has been in the game a bit longer and has more experience. If his report matches up, that’s probably a good sign. But, if his report doesn’t match up, we don’t know if that says anything at all about the first report of the guy who put it in. This is especially true when dealing with non-pros. There could be a lot of reasons as to why a guy looks different from one day to the next. While that can apply to pros, it’s more likely with non-pros. It just makes it tough to “scout the scouts”. Maybe there is a way to do it. I’m just having a very tough time trying to figure that out. Kap January 10, 2018 With cole on is way to houston, the goal this year for a reds should be to not finish in last place. Im partially joking around but am serious. That can be the realistic expectation for them to gain positive momentum for the future The Duke January 10, 2018 I think .500 is a reasonable goal for this year. If we can finish with 81+ wins, that’s a successful season. 75-89 might be a step in the right direction, but I still wouldn’t call that successful. Shawn January 10, 2018 The goal this year should be (1) can Peraza be our SS of the future (2) which young pitchers will be in our rotation in the future and who will be in the BP Jon Ryker January 11, 2018 I generally agree with the gist of what you wrote. However, I don’t agree that there are not systematic outages to be exploited. There are, and always will be. For example, now, everybody throws hard and tries to hit home runs. People are scouted and paid accordingly. There are distinct, exploitable downsides to both these approaches, however. First, throwing hard leads to more injuries and also leaves out a ton of talent which doesn’t throw so hard, but can still get people out. Somebody will get tired of paying pitchers to be injured and start drafting pitchers who are durable because they know how to pitch. This means, you’ll have to invest in that sort of instruction throughout the minor leagues, but when you do, there is a huge payoff: your pitchers are systematically injured less often and, because they don’t throw hard and strike out a ton of guys, they cost less. This is a distinct advantage……at least, until everybody imitates it. Second, the obsession with power-hitting….power hitting has a built-in inefficiency to it, although having some of it is quite helpful. The inefficiency is that it happens disproportionately against bad pitchers…..meaning, in low-leverage situations when good pitchers aren’t pitching. Fact is, a 3-run homer when you’re up 6 in the eight is completely without value……but teams are paying for that now. In the mean time, guys who swing from the heals all the time are substantially less capable of making productive outs…..which is the essence of how you score in high leverage situations, where you face good pitchers and the game is close. So, the team that systematically decides to teach contact, place hitting, and situational approaches will have a distinct edge in close games and against good pitchers…..somebody will do that….this will take more time, though, because cutting down swings and place hitting are nearly lost arts. Going after such players also will lead to a different talent pool than what is normally scouted, and such a team will have a talent edge, a pricing edge, and a strategic edge, if their pitching is good enough to keep games close. Facts are, what wins baseball games is timeless…….balance….speed, power, pitching, defense, and situational competence……all applied relentlessly……that’s always worked…..it will always work….somebody will figure that out…..then everybody will imitate, and we’ll all be watching a much better game.