So, a question about scouting:
Hickory's scout is 60-year-old Stephen Conway. Relevant to this conversation, Conway is rated 17 in terms of his ability to rate minor leaguers, and 17 in terms of his ability to rate amateur talent. In terms of the former (minor leaguers), that makes his rating fourth in the game (behind Seattle's Cash [19], River Cities' Collins [18], and Walla Walla's Shirai [18]). In terms of rating amateurs, he is tied with three others for the second-best rating in the game (behind only Cash [20]).
My understanding is that this rating should, hypothetically, mean that Conway will rate players more closely to their "actual" value than a league-average scout, or be accurate a higher percentage of the time.
So now we come to second baseman Oscar Costillo, currently on the major league bench. When I took over the franchise a few weeks into the 2013 season, Costillo was at AA Jersey City. Conway had him as a five-star prospect. This included a potential rating of 10 for contact and gap power, a potential rating of 9 for batting eye, and ratings of 6 for home runs and strikeouts.
Having no other way to judge Costillo, and again given that Conway is a highly rated scout, I assumed he would be a useful player at worst. His age-22 season in AA was mediocre (OPS = .693), but he wasn't completely overmatched in 120+ major league at bats during a desperation call-up in September, so he started 2014 at AAA Beulaville. His potential ratings still showed him as a 5-star prospect; his batting eye potential rating increased to 10, and his power potential rating increased to 7.
2014, his age 23 season, was a breakout for Costillo statistically. At AAA, he hit .315/.356/.499, with 11 home runs. His 82 RBI led all of AAA. He produced about 30 runs over replacement, didn't strike out a lot. In 25 major league at bats in September, he hit .360. At the end of the season, and indeed up until last night's sim, he continued to be rated a 5-star prospect with no change to his various potential ratings and steady increases to his actual rating. He sits on the major league roster right now; my plan was to teach him first base during spring training and use him as a caddy for Federico Soriano during the season.
Following last night's sim, all of his ratings have fallen 20 points or more. His potential has dropped from 5-stars to *one*; he is no longer a prospect, at least according to my scouting team. Again, this is following an age-23 season during which he hit extremely well in the high minors.
Hey, nothing's a sure thing, right? Perhaps so, but in this sim OF Dave Quinones (.302/.374/.479 with 24 steals for single A Avon as a 20-year-old), 3B Carlos Pais (admittedly a terrible .727 OPS season at AA Jersey City), and perhaps most frustratingly, OF Mario Lopez (my first round draft pick last year, hit .299 with 11 steals in his first season and at 20 years of age), ALL dropped from five stars to one. All also experienced huge drop-offs in current ability.
Am I missing something? The potential ratings provided by my scout, theoretically in the 75th percentile of all scouts, apparently have literally nothing to do with the player's actual potential. Neither, for that matter, does actual performance. A good counterexample is Bill Spurrier, who is now my starting second baseman. In 2013, Spurrier was rated with half-star potential. He OPSed .705 at AAA at age 26, which is notably worse than Costillo did in his age-23 season. Last year, I promoted Spurrier and he hit very well in the majors; after the most recent sim his star ratings have all been upgraded.
So I guess I don't get the structure of this game. Neither my highly-rated scout nor their comparative statistics would have suggested that Spurrier would be a better prospect than Costillo. My highly-rated scout was the only evidence I could use to pick Mario Lopez, and that after one decent season his rating has dropped from 5-stars to 1 suggests that the scouting reports are essentially useless, doesn't it?
I have noticed that when the other owners approach me regarding trades, they are often interested in players that my own scout considers marginal players, at best. When I made the Bill Dewall deal with New York last year, it was for Pais and Quinones, two five-star talents according to my scout. So clearly, our scouts vary tremendously in their assessments of players.
All of this strongly suggests to me that the various scouting reports are virtually unrelated to a player's actual chances of progressing well, that player development is literally random and thus completely unpredictable. Is there some aspect of this that I am not grasping?
There are several things that go into the ratings.
ReplyDelete1. A scouts assessment of a player
2. The player's recent statistical performance
3. The players performance relative to all other player's at his position in the league
All of these should, if I have things right, also be filtered through these factors:
A. Scouts can had tendencies. They can favor tools or talent. Depending on a kid's performance and abilities different scouts with the same Scout Minors rating can have very different results.
B. Majors or minors. Conway's opinion is going to vary depending on the location of a player. He's a 17 to scout minors but a 13 to scout majors. So when the kid is in the majors Conway's reliability on evaluation drops quite a bit.
C. Injuries. Even a minor injury can seriously impact a players numbers. After, I think, breaking his leg and being out for a few months Marzano didn't recover his numbers for a year.
D. Player personality. Each player has ratings on work ethics and so forth. A player with low numbers in the personality fields is going to have a tendency to lose ground.
E. Coaching. How good their minor league coaches are at each level has a STRONG impact on player development and growth. At the major league level I note that your hitting coach is a 16 at teaching hitting but a 12 at handling rookies. Combine that with the teaching ability of your AAA coaches (which I can't see here) and there could be an issue with development.
F. Lastly, and potentially most disastrously, is the random factor. Marcus is on the record as saying there is such for player development. Sometimes a player will 'have the light bulb come on' and suddenly advance. The converse is true as well.
I'm not saying these are exhaustive reasons for why evaluations vary so wildly over time but they are real reasons why they COULD.
From my experimenting with the game, I think the massive variation is caused by two significant changes from version 6.5 In 6.5, scouting was more or less uniform and player development was more or less predictable. Scouts generally agreed who the good prospects were and the good prospects generally became good major league players.
ReplyDeleteIn version 9, it's much more true to life and also much more variable. Scouts differ often and wildly on many prospects' talent levels. And, prospects' development paths are anything but smooth. Good players come bursting out of what appeared to be no-talent hacks and great prospects completely fall apart.
As a result, we all need to be prepared for many if not most of our beloved prospects to fall apart.
The lessons I've learned from this?
1. The value of prospects and draft picks compared to major league players is much lower in version 9 than it was in version 6.5.
2. The best strategy for player development is not to accumulate a few great prospects, but simply as many decent prospects as possible so that you hopefully have enough ones that blossom to offset those that fall apart.
I'm not claiming this is a universal truth, it's just how I'm thinking about these things.
By the way, Frank, if it makes you feel better, the new scouting reports I got devastated my feelings about my minor league system. I thought I had a number of minor league starting pitchers with potential.
ReplyDeleteTurns out, I was very, very wrong.
Further info I dug up from the manual:
ReplyDelete1. For coaching issues 'Hangle Rookies' denotes how well a coach can impact player development for players under 25 years old.
2. One of the key factors for player development is playing time. A great prospect riding the pine isn't going to develop.
There are others, of course. I show it on page 109 of the manual without pictures.
I know I've always looked for coaches who are good at hangling rookies.
ReplyDelete