Thursday, February 12, 2009

How many games does it take to accurately determine a player's UZR/150 defensive rating?

Question: How many games does it take to accurately determine a player's UZR/150 defensive rating? (Link to explanation of of UZR and UZR/150)?


Why I asked the question: In The Hardball Times 2009 Baseball Annual, Coco Crisp was given a defensive grade of F-. It seems that he was not the worst possible center fielder and his ranking was unfair. I need to find out how close a single season's value corresponds to a player's lifetime value.

Analysis: For the study, I needed to find at what levels are seasonal UZR/150 reflective of lifetime skills. I chose to use the data for the 43 Center Fielders who had a combined 1000 innings played in center field since 2004 (year that UZR/150 goes back to). This data was obtained from wwwl.FanGraphs.com.

Note on Data: I wished to increase the sample size, but being able only being able to copy and paste each player's stats, so I limited the scope on this initial look. Also the data only goes back to 2004 for UZR and it is only divided into single season data. I understand that there is some definite issue about lack and quality of the data, but I feel I used the best that is available.

For these 43 players, I wanted to compare the player's available lifetime UZR/150 to the UZR/150 the player had for each year. I used standard deviation to compare the difference of the two values. I also divided the data for the number of games played in a season. From the research I got the following results:


Minimum Games Played Minimum Innings Played Standard Deviation
0 0 19.98
25 225 6.56
50 450 5.95
75 675 5.81
100 900 5.24
125 1125 5.36
150 1350 6.45

So generally, a 66% (1 standard deviation measure that value that 66% of all available values fall in) of all players will have have their UZR/150 vary from 0 to 6 runs while 95% of all players will have their UZR/150 vary from ~ 0 to 12 runs.

Also I looked at how a player's lifetime UZR1/150 compares to the UZR/150 at each time in the player's career. Again I used standard deviation to measure the variation:


Minimum Games Played Minimum Innings Played Standard Deviation
0 0 22.92
50 450 3.45
100 900 3.07
150 1350 2.65
200 1800 2.78
250 2250 2.74
300 2700 2.14
350 3150 2.09
400 3600 1.88
450 4050 1.67
500 4500 1.39
550 4950 1.18
600 5400 1.05
650 5850 0.80
700 6300 0.60
750 6750 0.76

As you can see, over a player's career they get within 2 points of their final UZR/150 within 400 games or ~3 years of time. The reason these numbers have less variation vice the yearly data is that with so few of years worth of data, and with some players only having a couple years in majors, the final value me be closer to the current value.

To make sure the previous data isn't too far off, I took the 24 players that had 5 or more years in center field and compared their UZR average from the first 3 years to the their final. They had a standard deviation of 2.62 runs over 1028 innings which compares close to the preceding graph.

Next, I need to find how UZR/150 changed as the players aged. I grouped the players depending on their age and got the following information:


Age UZR/150
22 0.02
23 -0.06
24 0.48
25 0.97
26 0.92
27 0.01
28 1.08
29 0.64
30 0.54
31 -0.06
32 0.00
33 -0.62
34 -0.22
35+ -0.88

It seems that defensive the center fielders peak between the ages of 25 and 28 (not sure what they are doing in their 27th year) and seem to decline to the league average at the age 31. Not great information here, there does seem to be an ~2 point swing in UZR/150 over a player's career.

Finally, did Coco deserve better than his F- ranking. To begin with her is a look at Coco's available defensive stats over the years he has played center field:

Season Team Position Innings Lifetime Games UZR UZR/150 Lifetime total UZR Lifetime UZR/150
2002 Indians CF 269 30 -0.2 -1 -0.2 -1
2003 Indians CF 462 81 -2.2 -6.43 -2.4 -4.43
2004 Indians CF 807 171 6.6 11.04 4.2 3.69
2005 Indians CF 79 180 -1.7 -28.98 2.5 2.09
2006 Red Sox CF 900 280 0 0 2.5 1.34
2007 Red Sox CF 1216 415 22 24.42 24.5 8.86
2008 Red Sox CF 886 513 -8.9 -13.56 15.6 4.56
Totals

4620
15.6


Since the data in the annual was base on just the 2008 season, his score of -13.56 corresponds to their grade of F- . The problem is that 1 year's worth of data is not enough to determine how well a player plays defense over his career. Coco's defensive numbers have been all over the place over his career (much more then the average center fielder), but his lifetime value of 4.56 for UZR/150 pretty close to his ability (average center fielder – Grade: C). In 2008 he was about 18 runs off his lifetime average, which is about 3 standard deviations (18/3) off. Only 5% of all players will vary is much in a season, but it is not totally out of the norm. In my opinion a respectable publication like the Hardball Times should included, besides the seasonal defensive grade, is a lifetime or 3-year grade.


No comments: