Skill, Luck and Hot Hands on the PGA Tour
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Skill, Luck and Hot Hands on the PGA Tour
June 21, 2005
Robert A. Connolly Richard J. Rendleman, Jr.
Kenan-Flagler Business School Kenan-Flagler Business School
CB3490, McColl Building CB3490, McColl Building
UNC - Chapel Hill UNC - Chapel Hill
Chapel Hill, NC 27599-3490 Chapel Hill, NC 27599-3490
(919) 962-0053 (phone) (919) 962-3188 (phone)
(919) 962-5539 (fax) (919) 962-2068 (fax)
connollr@bschool.unc.edu richard_rendleman@unc.edu
We thank Tom Gresik and seminar participants at the University of Notre Dame for helpful comments. We provide
special thanks to Carl Ackermann and David Ravenscraft who provided significant input and assistance during the early
stages of this study. We also thank Mustafa Gultekin for computational assistance and Yuedong Wang and Douglas
Bates for assistance in formulating, programming and testing portions of our estimation procedures. Please direct all
correspondence to Richard Rendleman.Skill, Luck and Hot Hands on the PGA Tour
1. INTRODUCTION
Like all sports, outcomes in golf involve elements of both skill and luck. Perhaps the highest
level of skill in golf is displayed on the PGA Tour. Even among these highly skilled players,
however, a small portion of each 18-hole score can be attributed to luck, or what players and
commentators often refer to as “good and bad breaks.”
The purpose of our work is to determine the extent to which skill and luck combine to
determine 18-hole scores in PGA Tour events. We are also interested in the question of whether
PGA players experience “hot or cold hands,” or runs of exceptionally good or bad scores, in relation
to those predicted by their statistically-estimated skill levels.
From a psychological standpoint, understanding the extent to which luck plays a role in
determining 18-hole golf scores is important. Clearly, a player would not want to make swing
changes to correct abnormally high scores that were due, primarily, to bad luck. Similarly, a player
who shoots a low round should not get discouraged if he cannot sustain that level of play, especially
if good luck was the primary reason for his score. At the same time, it is important for a player to
know whether his general (mean) skill level is improving or deteriorating over time and to understand
that deviations from past scoring patterns may not be due to luck alone.
From a policy standpoint, it seems reasonable that the PGA Tour and other professional golf
tournament organizations should attempt to minimize the role of luck in determining tournament
outcomes and qualification for play in Tour events. Ideally, the PGA Tour should be comprised of
the most highly-skilled players. Also, the Tour should strive to conduct tournaments that reward the
most highly skilled players rather than those who experience the greatest luck.
In some cases, luck in a round of golf can easily be identified. In the final round of the 2002
Bay Hill Invitational, David Duval’s approach shot to the 16th hole hit the pin and bounced back into
a water hazard fronting the green. Duval took a nine on the hole. Few would argue that Duval’s
score of nine was due to bad judgment or a sudden change in his skill level, and it is highly unlikely
that Duval made swing changes or changes to his general approach to the game to correct the type of
problem he incurred on the 16th hole.
In contrast, consider the good fortune experienced by Craig Perks in the final round of the
2002 Players Championship, when he chipped in on holes 16 and 18 and sunk a 28-foot putt for2
birdie on the 17th hole en route to victory. Was Perks’ victory, his first on the PGA Tour, a reflection
of exceptional ability or four consecutive days of good luck? The fact that his best season since 2002
placed him 146th on the PGA Tour’s Official Money List suggests that luck may have been the
overriding factor. Similarly, as of June, 2005, Ben Curtis has had only has had only one top-10 finish
since winning the 2003 British Open. At this point, few would argue that Curtis’s victory was
anything other than a fluke.
In many situations, specific occurrences of good and bad luck may not be as obvious as in the
examples above. Luck simply occurs in small quantities as part of the game. Even players as highly
skilled as Tiger Woods and Vijay Singh cannot produce perfect swings on every shot. As a result, a
certain human element of luck is introduced to a shot even before contact with the ball is made. (An
excellent article on the role of luck in golf, with a focus on how the ultimate outcome of the dual
between Tiger Woods and Chris DiMarco in the 2005 Masters was determined as much by luck as by
skill, is provided in Jenkins (2005).)
Our work draws on the rich literature in sports statistics. Klaassen and Magnus (2001) model
the probability of winning a tennis point as a function of player quality, context (situational)
variables, and a random component. They note that failing to account for quality differences will
create pseudo-dependence in scoring outcomes, because winning reflects, in part, player quality, and
player quality is generally persistent. Parameter estimates from their dynamic, panel random effects
model using four years of data from Wimbledon matches suggest that the iid hypothesis for tennis
points is a good approximation, provided there are controls for player quality. The most important
advantage of their approach from our perspective is that it supports a decomposition of actual
performance into two parts: an expected score based on skill and an unexpected residual portion.
In this paper, we decompose individual golfer’s scores into skill-based and unexpected
components by using Wang’s (1998) smoothing spline model to estimate each player’s mean skill
level as a function of time while simultaneously estimating the correlation in the random error
structure of fitted player scores and the relative difficulty of each round. The fitted spline values of
this model provide estimates of expected player scores. We define luck as deviations from these
estimated scores, whether positive or negative, and explore its statistical properties.
Our tests show that after adjusting a player’s score for his general skill level and the relative
difficulty of the course on the day a round is played, the average (and median) standard deviation of
residual golf scores on the PGA Tour is approximately 2.7 strokes per 18-hole round, ranging3
between 2.1 and 3.5 strokes per round per player. We find clear evidence of positive first-order
autocorrelation in the error structure for over 13 percent of the golfers and, therefore, conclude that a
significant number of PGA Tour players experience hot and cold hands. We also apply some
traditional hot hands tests to the autocorrelated residuals and come to similar conclusions. However,
after removing the effects of first-order autocorrelation from the residual scores, we find little
additional evidence of hot hands. This suggests that any statistical study of sporting performance
should estimate skill dynamics and deviations from normal performance while simultaneously
accounting for the relative difficulty of the task and the potential autocorrelation in unexpected
performance outcomes.
The remainder of the paper is organized as follows. In Section 2 we describe our data and
criteria for including players in our statistical samples. In Section 3 we present the results of a
number of fixed-effects model specifications to identify the variables that are important in predicting
player performance. In Section 4 we formulate and test the cubic spline-based model for estimating
player skill. This model becomes the basis for our analysis of player skill, luck and hot hands
summarized in Section 5. A final section provides a summary and concluding comments.
2. DATA
We have collected individual 18-hole scores for every player in every stroke-play PGA Tour
event for years 1998-2001 for a total of 76,456 scores distributed among 1,405 players. Our data
include all stroke play events for which participants receive credit for earning official PGA Tour
money, even though some of the events, including all four “majors,” are not actually run by the PGA
Tour. The data were collected, primarily, from www.pgatour.com, www.golfweek.com,
www.golfonline.com, www.golfnews.augustachronicle.com, www.insidetheropes.com, and
www.golftoday.com. When we were unable to obtain all necessary data from these sources, we
checked national and local newspapers, and in some instances, contacted tournament headquarters
directly.
Our data cover scores of players who made and missed cuts. (Although there are a few
exceptions, after the second round of a typical PGA Tour event, the field is reduced to the 70 players,
including ties, with the lowest total scores after the first two rounds.) The data also include scores for
players who withdrew from tournaments and who were disqualified; as long as we have a score, we
use it. We also gathered data on where each round was played. This is especially important for4
tournaments such as the Bob Hope Chrysler Classic and ATT Pebble Beach National Pro-Am played
on more than one course.
The great majority of the players represented in the sample are not regular members of the
PGA Tour. Nine players in the sample recorded only one 18-hole score over the 1998-2001 period,
and 565 players recorded only two scores. Most of these 565 players qualified to play in a single US
Open, British Open or PGA Championship, missed the cut after the first two rounds and subsequently
played in no other PGA Tour events.
As illustrated in the top panel of Figure 1, 1,069 players, 76.1 percent of the players in the
sample, recorded 50 or fewer 18-hole scores. As illustrated in the bottom panel, this same group
recorded 5,895 scores, which amounts to only 7.7 percent of the total number of scores in the sample.
1,162 players, representing 82.7 percent of the players in the sample, recorded 100 or fewer scores.
The total number of scores recorded by this group was 12,849, or 16.8 percent of the sample. The
greatest number of scores recorded by a single player was 457 recorded by Fred Funk.
The players who recorded 50 or fewer scores represent a mix of established senior players
such as Hale Irwin (19 scores), Jim Thorpe (18) and Larry Nelson (17), relatively inactive middle-
aged players including Bobby Clampett (16), Jerry Pate (16) and Seve Ballesteros (22), “old-timers”
such as Arnold Palmer (32), Raymond Floyd (24) and Gary Player (18), up-and-coming stars
including Adam Scott (32) and Chad Campbell (20), established European Tour players such as
Andrew Coltart (49), Niclas Fasth (48) and Robert Karlsson (46), and a large number of players that
most readers of this paper, including those who follow the PGA Tour, have probably never heard of.
Clearly, this group, which accounts for 76.1 percent of the players in the sample, but only 7.7 percent
of the scores, is not representative of typical PGA Tour participants. Therefore, including them in the
sample could cause distortions in estimating the statistical properties of golf scores on the Tour.
In estimating the skill levels of individual players, we employ the smoothing spline model of
Wang (1998) which adjusts for correlation in random errors. Simulations by Wang indicate that 50
sample observations is probably too small, and that approximately 100 or more observations are
required to obtain dependable statistical estimates of cubic-spline-based mean estimates. After
examining player names and the number of rounds recorded by each player, we have concluded that a
sample of players who recorded more than 90 scores is reasonably homogeneous and likely to meet
the minimum sample size requirements of the cubic spline methodology.5
3. MODEL IDENTIFICATION
The model used in our tests, based on Wang’s smoothing spline, is computationally intensive
and requires approximately a full day to run on a 3.19 GHz PC operating under Windows XP.
Because of its computational requirements, we employ two simpler models to identify the relevant
variables and appropriate model form for our tests. In model 1, we employ a fixed-effects multiple
regression to predict a player’s 18-hole score for a given round as a function of the level of skill
displayed by the player throughout the entire 1998-2001 period and the relative difficulty of each
round. In model 2 each player’s skill level is allowed to change (approximately) by calendar year,
but otherwise, the model is identical to model 1. We show that including approximate calendar year-
based time dependency significantly improves the predictive power of the model. We also estimate
various versions of model 2 to determine whether alternative model specifications might be
warranted.
Using model 2 we show that the best way to estimate the relative difficulty of a given golf
round is through a round-course interaction dummy variable. Although most tournaments on the
PGA Tour are played on a single course, several are played on more than one course. Using round-
course interactions to predict the relative difficulty of each round is significantly more powerful than
employing a dummy for the round alone (without interacting with the course), the course alone
(without interacting with the round), or the tournament alone (without interacting with either the
course or the round). We also show that individual player performance is not affected by the courses
on which he plays.
3.1. Model 1
Let si , j denote the 18-hole score of player i (with i = 1,....n , alphabetically) in PGA Tour
round-course interaction j (with j = 1,....m ). A round-course interaction is defined as the interaction
between a regular 18-hole round of play in a specific tournament and the course on which the round
is played. For most tournaments, only one course is used and, therefore, there is only one such
interaction per round. However, over the sample period, 25 of 182 tournaments were played on more
than one course. For example, in the Bob Hope Chrysler Classic, the first four rounds are played on
four different courses using a rotation that assigns each tournament participant to each of the four
courses over the first four days. A cut is made after the fourth round, and a final round is played the6
fifth day on a single course. Thus, the Bob Hope tournament consists of 17 round-course interactions
– four for each of the first four days of play and one additional interaction for the fifth and final day.
According to model 1, the predicted score for player i played in connection with round-course
interaction j is determined by the following fixed-effects multiple regression model:
n m
si , j = α + ∑ β k pk + ∑ γ c rc + ε i , j (1)
k =2 c=2
In (1), pk is a dummy variable that takes on a value of 1 if player i = k and zero otherwise,
rc is a dummy that takes on a value of 1 if round-course interaction j = c and zero otherwise, and
ε i , j is an error term with E ( ε i , j ) = 0 . In the model, the first round of the 1998 Mercedes
Championships, played on a single course, is round-course interaction j = 1 . Therefore, the
regression intercept, α , represents the expected score of the first player ( k = 1 ) in the first round of
the 1998 Mercedes Championships, β k represents the differential amount that player k > 1 would be
expected to shoot in the first round of the 1998 Mercedes, and γ c denotes the additional amount that
all players, including the first player, would be expected to shoot in connection with any other round-
course combination. The only restriction placed on the data is that a player must have recorded at
least four 18-hole scores, the minimum number required for the regression to be of full rank, to be
included in the regression estimate. With this restriction, the data include 75,054 scores for 810
players recorded over a possible 848 round-course interactions, although no single player participated
in more than 457 rounds.
Berry (2001) employs a similarly-constructed random effects model that takes into account
the skill level of each player and the intrinsic difficulty of each round to measure the performance of
Tiger Woods relative to others on the PGA Tour. However, Berry’s model does not take account of
the potential for a given round to be played on more than one course.
It should be noted that when estimating round-course interaction coefficients, no specific
information about course conditions, adverse weather, etc. is taken into account. Nevertheless, if
such conditions combine to produce abnormal scores in a given 18-hole round, the effects of these
conditions should be reflected in the estimated coefficients. Under model 1, the highest round-course
interaction coefficient is associated with the portion of the third round of the 1999 ATT Pebble Beach
National Pro-Am played on the Pebble Beach course. For this particular round-course interaction,
wind and rain combined to make the famed course extremely difficult and forced tournament officials7
to cancel the event during the fourth round due to unplayable conditions. (Our final analysis shows
that the Pebble Beach course played 5.8 strokes more difficult on the third day than on the first two
days.)
3.2. Model 2
In the previous model, a player’s skill coefficient is assumed to be constant throughout the
entire 1998-2001 period. In model 2, a player’s skill coefficient is allowed to vary through time on
an approximate calendar year basis.
In principle, we wish to estimate a single regression equation with separate skill coefficients
estimated for each player in each calendar year. However, experiments with a model of this form
reveal that a number of players whose scores are included in the estimation of model 1 would have to
be removed from the original sample for the regression to be of full rank.
To overcome the potential problem of singularity, while maintaining the same sample
employed in the estimation of model 1, we employ the following method for allowing an individual
player’s skill coefficient to vary through time. If a full calendar year has passed, at least 25 scores
were used in the estimation of the previous skill coefficient for the same player, and at least 25
additional scores were recorded for the same player, then a new incremental skill coefficient for that
player is estimated. For players who participated actively in all four years of the sample, this
procedure results in the estimation of different mean skill levels in each of the four years. Although
the 25-score criterion is somewhat arbitrary, it is not critical, since the model we will actually use for
estimating player scores is based on Wang’s smoothing spline methodology and does not use the 25-
score criterion.
As before, let si , j denote the 18-hole score of player i in connection with PGA Tour round-
course interaction j. Define pk ,t ( k ) as a dummy variable that takes on a value of 1 if player i = k and
the player-specific (approximate) calendar year-based time period equals or exceeds t ( k ) = 1,...T ( k )
and zero otherwise, where T ( k ) denotes the total number of player-specific (approximate) calendar
year-based time periods attributable to player k. Thus, if i = k , pk ,1 = 1 in all time periods, pk ,2 = 1
in periods 2 through T ( k ) but zero otherwise, pk ,3 = 1 in periods 3 through T ( k ) but zero otherwise,8
and pk ,4 = 1 in period 4 only. Finally, let β k ,t ( k ) denote the value of incremental skill coefficient
t ( k ) for player k. Then, the basic version of model 2, denoted as 2.1, is specified as follows:
T (k ) n T (k ) m
si , j = α + ∑ β k =1,t ( k ) pk =1,t ( k ) + ∑ ∑ β k ,t ( k ) pk ,t ( k ) + ∑ γ c rc + ε i , j (2.1)
t ( k )=2 k = 2 t ( k ) =1 c=2
As in model 1, the first player in the sample is player k = 1 , and the first round of the 1998
Mercedes Championships is round-course combination j = 1 . Therefore, the regression intercept, α ,
represents the first player’s expected score in the first round of the 1998 Mercedes Championships.
T (k )
The term ∑β
t ( k )=2
k =1, t ( k )
pk =1,t ( k ) picks up potential incremental changes in skill for the first player starting
in his second estimation period, typically 1999. In all forms of model 2, the skill coefficient β k ,1 is
estimated in each of the T ( k ) estimation periods for all players k > 1 . Starting in the second period,
the coefficient β k ,2 is estimated for all players, k = 1,....n for whom T ( k ) > 1 , and so on.
To determine the best way to estimate the relative difficulty of each round, we estimate three
additional forms of model 2, denoted as models 2.2 through 2.4, respectively. In model 2.2 we
substitute 728 round dummies for the 848 round-course interaction dummies in model 2.1. We
define a “round” as a round of play in a specific tournament, regardless of whether the round is
played on more than one course. Therefore, this specification ignores differences in scores for the
same round that are played on different courses. In model 2.3 we substitute 182 tournament dummies
for the round-course interaction dummies model 2.1. Each tournament dummy denotes a particular
tournament played in a given year. For example, there are four tournament dummies associated with
the Masters, played in each of the four years 1998-2001. According to this specification, the relative
difficulty of each round is determined by the tournament and does not vary from day to day within
the tournament or over multiple courses that might be used for the tournament. Finally, in model 2.4
we substitute 77 course dummies, which indicate the courses on which the rounds are played, for the
round-course interaction dummies in model 2.1. As such, this specification assumes that the relative
difficulty of a round is determined strictly by the course on which the round is played and that the
difficulty of the course does not change from day to day within a tournament or even from year to
year. Table 1 summarizes the performance of these models using both a full sample of players
(75,054 observations) and a sample restricted to the players who recorded more than 90 scores
(64,364 observations).9
Using the full sample, the adjusted R 2 is 0.2996 for model 1 and 0.3077 for model 2.1. The
F-statistic for testing the difference between model 2.1 and model 1 is 2.44, which, given the large
sample size, is significant at any normal testing level. This indicates that there is added value to
including (approximate) calendar year-based time variation in mean player skill levels. F-statistics
for testing model 2.1 against alternative specifications using dummies for rounds, tournaments and
courses, rather than for round-course interactions, are 5.66, 9.66 and 10.67, respectively, indicating
that model 2.1, which uses round-course interactions, is superior. Very similar results are obtained
for the same tests using the restricted sample. It should be noted that Berry’s (2001) model for
predicting player scores is equivalent to our model 1 using random effects for actual rounds rather
than fixed effects for round-course interactions. The results in Table 1 indicate that including
calendar-year-based time variation in player skill levels and adjusting scores for the relative difficulty
of round-course interactions is a superior model specification.
We also attempted to estimate a fifth version of model 2 to test whether there is added value
to including interactions among players and courses. With the full sample of players, the number of
coefficients to be estimated increases from 2,224 using model 2.1 to 17,201 when player-course
interactions are included. With the sample restricted to only those players that competed in more
than 90 rounds, the number of coefficients increases from 848 to 13,383. Unfortunately, the
regressions are of such large scale that we were unable to obtain sufficient resources to estimate them
on the university’s statistical mainframe computer. Therefore, we address the issue of player-course
interactions by including course dummies in random effects analyses of individual player residual
scores from our final spline-based estimation model. These tests, described in Section 4.1, show no
evidence of significant player-course interaction effects. Also, tests of player performance in
adjacent rounds of the same tournament, summarized in Table 5 and discussed in Section 5, provides
additional evidence of no significant effects.
4. THE SPLINE-BASED ESTIMATION MODEL
Based on the previous analysis, we conclude that the best way to adjust player scores for the
relative difficulty of each round is to use round-course interactions rather than rounds (ignoring
courses), courses (ignoring rounds) or tournaments (ignoring both rounds and courses). We also
conclude that our estimation model should reflect time-dependent estimates of mean player skill.
However, we recognize that allowing player skill to change only by calendar year is somewhat10
arbitrary. Therefore, in model 3 we employ a more general specification of time dependency in mean
player skill that does not require skill changes to occur only at the turn of the year or at any other pre-
specified points in time.
In model 3 we estimate each player’s mean skill level as a function of “golf time” using the
restricted maximum likelihood (REML) version Wang’s (1998) smoothing spline model which
adjusts for correlated random errors. Player k’s “golf time” counts successive competitive PGA Tour
rounds of golf played by player k regardless of how these rounds are sequenced in actual calendar
time. Inasmuch as there are likely to be gaps in actual time between some adjacent points in golf
time, it is unlikely that random errors around individual player spline fits follow higher-order
autoregressive processes. At the same time, however, we do not want to rule out the possibility that
the random errors may be correlated.
Model 3, which uses Wang’s REML model for estimating mean individual player skill as a
function of golf time, is formulated as follows. Define player k’s golf time as the sequence
g ( k ) = 1,....G ( k ) , and let g k ( j ) denote the mapping of the sequence j = 1,....m to the sequence
g ( k ) = 1,....G ( k ) such that the sequence g k ( j ) represents the “golf time” of all round-course
interactions, j, for which player k recorded an 18-hole score. For example, assume that player k’s
first three scores were recorded in connection with round-course interactions 6, 7, and 16. Then,
g k ( 6 ) = 1, g k ( 7 ) = 2, and g k (16 ) = 3 . With this mapping, model 3 becomes:
( )
n m
si , j = ∑ f k ⎣⎡ g k ( j ) ⎦⎤ + θ k ⎡⎣ g k ( j ) ⎤⎦ pk + ∑ ( γ c + ξc ) rc + κ i , j
k =1 c=2
( )
n m
= ∑ f k ⎣⎡ g k ( j ) ⎦⎤ + θ k ⎡⎣ g k ( j ) ⎤⎦ pk + ∑ ωc rc + κ i , j ,
k =1 c=2
with ωc = γ c + ξ c and E (κ i , j ) = 0 . In model 3, f k ( g k [ j ]) is the smoothing spline function applied
to player k’s golf scores over golf time g ( k ) = 1,....G ( k ) . θ k ( g k [ j ]) is the random error associated
with the spline fit for player k evaluated with respect to round-course interaction j as mapped into
′
( )
golf time g k ( j ) , with θ κ = θ k [1] ,....θ k ⎡⎣G ( k ) ⎤⎦ ∼ N ( 0, σ k2 Wk‐1 ) and σ k2 unknown. Note that the
intercept is absorbed by the f k ( g k [ j ]) terms.11
It is convenient to re-express θ k ( g k [ j ]) as θ k ( g k [ j ]) = ϕk ( g k [ j ]) + ηk ( g k [ j ]) , where
ϕk ( g k [ j ]) represents the predicted error associated with player k’s spline fit as of golf time g k ( j )
as a function of past errors, θ k (1) ,....θ k ( g k [ j ] − 1) , and ηk ( g k [ j ]) denotes the remaining residual
error. Substituting θ k ( g k [ j ]) = ϕk ( g k [ j ]) + ηk ( g k [ j ]) , model 3 becomes:
( )
n m
si , j = ∑ f k ⎡⎣ g k ( j ) ⎤⎦ + ϕk ⎡⎣ g k ( j ) ⎤⎦ + ηk ⎡⎣ g k ( j ) ⎤⎦ pk + ∑ ωc rc + κ i , j (3)
k =1 c=2
It is important to recognize that in estimating model 3, the autocorrelation in random errors
about each spline fit is not removed. Instead, the spline fits and the autocorrelation parameters
associated with the random error around the spline fits are estimated simultaneously.
Model 3 represents a generalized additive model with random effects η = (η1 ,....η n )′ and
ξ = (ξ1 ,....ξ m )′ associated with players i = 1,....n and round-course interactions j = 1,....m ,
respectively. Although the smoothing spline functions allow for general autocorrelation structures in
θi , we assume that the θi follow player-specific AR(1) processes. The methodology for estimating
model 3 is described in the appendix.
4.1. General Properties of Model 3
Throughout the remainder of the paper, frequent reference is made to two different types of
residual errors from the spline fits of model 3. To avoid confusion, we hereby define the two residual
error types. In model 3, θi ( gi [ j ]) is the total residual error associated with spline fit i as of golf
time gi ( j ) . ϕi ( gi [ j ]) represents the predicted error associated with player i’s spline fit as of golf
time gi ( j ) as a function of past errors, θi (1) ,....θi ( gi [ j ] − 1) , and ηi ( gi [ j ]) represents the error
component of the spline fit that is not predictable from past residual errors, θi ( gi [ j ]) . We assume
that the error θi ( gi [ j ]) follows an AR(1) process with first-order autocorrelation coefficient φi . The
spline fit fi and correlation φi are estimated simultaneously using Wang’s (1998) smoothing spline
methodology. For some of our tests we focus on residual errors ηi ( gi [ j ]) , which we refer to as η
errors. If the assumed AR(1) process properly captures the correlation structures of residual player12
scores, η errors should represent white noise. For other tests, we focus on autocorrelated residual
errors θi ( gi [ j ]) , which we refer to as θ errors.
In formulating model 3, we assume that an AR(1) process describes the residual error
structure about the cubic spline fit for each player. If this is a valid assumption, the η errors for each
player should be serially uncorrelated. However, if this is not a valid assumption, there may be
additional autocorrelation in the η errors. To check for possible model misspecification in
connection with the AR(1) assumption, we estimated autocorrelation coefficients of order 1-5 on the
η errors of each player. In no instance were any estimated coefficients statistically significant at the
5 percent level. In addition, we computed the Ljung-Box (1978) Q statistic associated with the η
errors for each player for lags equal to the minimum of 10 or 5 percent of the number of rounds
played. Ljung (1986) suggests that no more than 10 lags should be used for the test and Burns (2002)
suggests that the number of lags should not exceed 5 percent of the length of the series. Only 6 of
253 Q statistics were significant at the 5 percent level. As a final diagnostic, we ran random effects
models relating θ and η errors to course dummy variables for each of the 253 players, assigning a
dummy to a course if a player played the course at least twice, the minimum required for a model to
be of full rank. None of the F-tests associated with the 253 random effects tests were significant at
the 5 percent level for either type of error. Therefore, we conclude that the autocorrelation properties
of the spline function have been properly specified and that it is unnecessary to include player-course
interactions in the estimation model.
Figure 2 shows plots of spline fits for four selected players. All figures in the upper panel
show 18-hole scores reduced by the effects of round-course interactions (connected by jagged lines)
along with corresponding spline fits (smooth lines). The same spline fits (smooth lines) are shown in
the lower panel along with predicted scores adjusted for round-course interactions (connected by
jagged lines), computed as a function of prior residual errors, θi ( gi [ j ]) , and the first-order
autocorrelation coefficient, φi , estimated in connection with the spline fit. The scale of each plot is
different. Therefore, the spline fits in the lower panel appear as stretched versions of the
corresponding fits in the upper panel and visually, plots for the four players are not directly
comparable.13
The two plots for Chris Smith reflect 10.7067 degrees of freedom used in connection with the
estimate of his spline function, the largest degrees of freedom value among the 253 players. In
contrast, the spline fit for Tiger Woods, estimated with 2.0011 degrees of freedom, is more typical;
almost 71 percent of the 253 spline fits use 2.25 or fewer degrees of freedom and have the same
general appearance as that estimated for Woods. The fit for Ian Leggatt reflects φ = −0.2791 , the
most negative first-order autocorrelation coefficient estimated in connection with the 253 splines.
Also, the standard deviation of the θ residual errors around the spline fit for Leggatt is 2.14 strokes
per round, the lowest among the 253 players. Interestingly, after adjusting for prior θ errors and
first-order autocorrelation, φ , the predicted score for Bob Estes, shown as the jagged line in the lower
panel, is the lowest among all 253 players at the end of the 1998-2001 sample period.
Figure 3 shows six histograms that help to summarize the 253 cubic spline-based mean player
skill functions. The first histogram shows the degrees of freedom for the various spline fits. Three of
the fits have exactly two degrees of freedom, while the degrees of freedom for 179, or 71 percent of
the splines, is less than 2.25. This implies that a large majority of the spline fits are essentially linear
functions of time, such as that illustrated in Figure 2 for Tiger Woods. (It should be noted that for
each spline, an additional degree of freedom, not accounted for in the histograms, is used up in
estimating the AR(1) correlation structure of the residuals.) Fifty-three of the spline fits have three or
more degrees of freedom, implying a departure from linearity. Twelve of the splines have five or
more degrees of freedom, and as noted earlier, the largest number of degrees of freedom is 10.71 for
Chris Smith.
The histogram in the lower left-hand panel of Figure 3 shows the distribution of first-order
autocorrelation coefficients estimated in connection with the 253 splines. As illustrated in the lower
left-hand panel, the residual θ errors of 158, or 62 percent of the spline fits, are positively correlated.
We employ the bootstrap method to test the significance of individual player spline fits
against alternative specifications of player skill. To maintain consistency in our testing methods, we
apply the same set of bootstrap samples to test the significance levels of the first-order
autocorrelation coefficients estimated in connection with each fit. All bootstrap tests are based on
balanced sampling of 40 samples per player. Wang and Wahba (1995) describe how the bootstrap
method can be used in connection with smoothing splines that are estimated without taking into
account the autocorrelation in residual errors. We modify the method outlined in Wang and Wahba
so that the bootstrap samples are based on η residuals which adjust predicted scores for14
autocorrelation in prior θ residuals. Forty bootstrap samples is the minimum necessary to estimate
two-sided 95 percent confidence intervals. Although 40 samples is well below the number required
to estimate precise confidence intervals for each individual player, a total of 40 × 253 = 10,120
bootstrap samples are taken over all 253 players, requiring over three days of computation time using
a 3.19 GHz computer running under Windows XP. The 10,120 total bootstrap samples should be
more than sufficient to draw general inferences about statistical significance within the overall
population of PGA Tour players.
After sorting the correlation coefficients computed via the 40 bootstrap samples from lowest
to highest, the number significantly negative is the number of players for which the 39th correlation
coefficient is negative, and the number significantly positive is the number of players for which the
second correlation coefficient is positive. We find that six of 95 negative autocorrelation coefficients
are significant at the 5 percent level while 32 of 158 positive coefficients are significant. Although
the implications of positive autocorrelation are developed in more detail later, based on the number of
players displaying significant positive autocorrelation, it is clear that a substantial number of players
exhibit hot and cold hands in their golfing performance over the sample period.
The four remaining histograms in Figure 3 summarize bootstrap tests of the 253 cublic spline
fits against the following alternative methods of estimating a player’s 18-hole score (after subtracting
the effects of round-course interactions):
1. The player’s mean score.
2. The player’s mean score in each approximate calendar year period as defined by
model 2.
3. The player’s score estimated as a linear function of time.
4. The player’s score estimated as a quadratic function of time.
RSE ( alt ) RSE ( spline )
For each of the four tests we form a test statistic, ζˆ = − , where G is
G − df alt G − df spline − 1
the number of 18-hole scores for a given player, df alt and df spline are the number of degrees of
freedom associated with the estimation of the alternative and spline models, respectively, and
RSE ( alt ) and RSE ( spline ) are total residual squared errors from the alternative and spline models.
We subtract 1 in the denominator of the second term to account for the additional degree of freedom
associated with the estimation of the first-order autocorrelation coefficient. For the purposes of this
test, RSE ( spline ) is based on η errors, since these errors reflect the complete information set15
derived from each spline fit. The test statistic ζˆ is suggested by Efron and Tibshirani (1998, pp.
190-192 and p. 200 [problem 14.12]) for testing the predictive power of an estimation model
formulated with two different sets of independent variables. The test statistic is computed for each of
40 bootstrap samples per player. In a one-sided test of whether the spline fit is a superior
specification to the alternative model, ζˆ should be positive in 38 or more of the 40 bootstrap
samples.
The first line of Table 2 shows that the spline model is significantly superior (at the 5 percent
level) to the player’s mean score for approximately 30 percent of the 253 players in the sample. It is
significantly superior to a linear time trend for 8 percent of the players, to the mean score computed
for each (approximate) calendar year for 5 percent, and to a quadratic time trend for 5 percent of the
players. The second line of the table shows that the percentage of players for which the alternative
model is generally superior (but not statistically superior) is 30 percent and 17 percent for the linear
and quadratic time trends, respectively, and only 8 percent and 4 percent for the mean score and mean
score per (approximate) calendar year. (For a given player, the alternative model is generally
superior if ζˆ is positive for less than 20 of 40 bootstrap samples.)
Although the spline model is superior to the alternative models at the 5 percent level for a
relatively small percentage of players, it “beats” the alternative models a much larger proportion of
the time than could be predicted by chance. Therefore, we conclude that the spline model is superior
to any of the four alternative specifications.
5. PLAYER PERFORMANCE
5.1 Skill
Based on bootstrap sampling, the preceding analysis shows that approximately 30 percent of
cubic spline-based player score estimates are significantly superior at the 5 percent level to estimates
based solely on mean scores. Moreover, the mean squared residual error for each of the original
RSE ( spline )
cubic spline fits, measured as , is less than the mean squared residual about the mean,
G − df spline − 1
RSE ( mean )
measured as , for 70 percent of the players in the sample. These results provide strong
G −1
evidence that the skill levels of PGA Tour players change through time. For many players such as16
Tiger Woods, the relationship between the player’s average skill level and time is well approximated
by a linear time trend, after adjusting for autocorrelation in residual errors. For others, such as Chris
Smith, Ian Leggatt and Bob Estes, the relationship between mean player skill and time is more
complex and cannot easily be modeled by a simple parametric-based time relationship.
The player-specific cubic spline functions can provide point estimates of expected scores,
adjusted for round-course interactions, at the end of the 1998-2001 sample period, and, therefore, can
be used to rank the players at the end of 2001. Table 3 provides a summary of the best players
among the sample of 253 as of the end of the 2001 based on the cubic spline point estimates shown in
column 1 of the table. Column 2 shows estimates of player scores after adjusting for autocorrelation
in θ residual errors around each spline fit. The values in column 1 can be thought of as estimates of
mean player skill at the end of the sample period. In contrast, the values in column 2 can be thought
of as estimates of each player’s last score as a function of his ending mean skill level and the
correlation in random errors about prior mean skill estimates. The Official World Golf Ranking is
also shown for each player as of 11/04/01, the ranking date that corresponds to the end of the official
2001 PGA Tour season. The World Golf Ranking is based on a player’s most recent rolling two
years of performance, with points awarded based on position of finish in qualifying worldwide golf
events on nine different tours. The ranking does not reflect actual player scores. Details of the
ranking methodology can be found in the “about” section of the Official World Golf Ranking web
site, www.officialworldgolfranking.com.
Despite being based on two entirely different criteria, the two ranking methods produce
similar lists of players. Mike Weir is the only player in the top 10 of the Official World Golf
Ranking (number 10) as of 11/04/01 whose name does not appear in Table 3 (he is actually 21st
according to our ranking), and only five players listed in Table 3 were not ranked among the Official
World Golf Ranking’s top 20.
Among the 20 players listed in Table 3, the predicted score for Bob Estes, adjusted for
autocorrelation in residual errors (column 2), is the lowest. Although it may come as a surprise that a
player with as little name recognition as Estes would be predicted to shoot the lowest score at the end
of 2001, the plots of Estes’ spline function in Figure 2 show why. Estes exhibited marked
improvement over the last quarter of the sample period, and the improvement was sufficiently
pronounced that his estimated spline function picked it up.17
Table 4 provides lists of the 10 players showing the improvement and deterioration in skill
over the 1998-2001 sample period based on differences between beginning and ending spline-based
estimates of player scores adjusted for round-course interactions. Cameron Beckman was the most
improved player, improving by 3.36 strokes from the beginning of 1998 to the end of 2001. Chris
DiMarco and Mike Weir, relatively unknown in 1998 but now recognized among the world’s elite
players, were the fifth and sixth most improved players over the sample period. Among the 10
golfers whose predicted scores went up the most were Lanny Wadkins (8.18 strokes, born 1949),
Fuzzy Zoeller (2.99 strokes, 1951), Feith Fergus (2.85 strokes, 1954), Bobby Wadkins (2.74 strokes,
1951), Craig Stadler (2.65 strokes, 1953), Tom Watson (2.64 strokes, 1949), and David Edwards
(2.32 strokes, 1956). Clearly, deterioration in player skill appears to be a function of age, with a
substantial amount of deterioration occurring during a golfer’s mid to late 40’s. But despite the
natural deterioration that occurs with age, 133 of the 253 players in the sample actually improved
from the beginning of the sample period to the end.
5.2 Luck
To what extent does luck play a role in determining success or failure on the PGA Tour?
Based on the premise that our model for predicting 18-hole scores is correct, luck represents
deviations between actual and predicted 18-hole scores, either positive or negative, that are not
sustainable from one round to the next. Even if a golfer plays at an unusually high skill level in a
given round and cannot point to any specific instances of good luck to explain his performance, we
would consider him to have been lucky if his high skill level cannot be sustained. In this view, luck
is a random variable with temporal independence.
5.2.1 Average Residual Scores in Adjacent Rounds. In the analysis that follows, we test
whether deviations between actual and predicted 18-hole scores are sustainable from one round to the
next by examining whether players who record exceptionally good or bad scores in one round of a
tournament can be expected to continue their exceptional performance in subsequent rounds of the
same tournament. For this test we examine the performance of players in all adjacent tournament
rounds that are not divided by a cut. A typical PGA Tour tournament involves four rounds of play
with a cut being made after the second round. Therefore, for a typical tournament, we compare
performance between rounds 1 and 2 and also between rounds 3 and 4. We do not compare
performance between rounds 2 and 3, however, because approximately half of the players who18
participate in round 2 are cut and do not continue for a third round. For the Bob Hope Chrysler
Classic, which involves five rounds of play with a cut being made after the fourth round, we compare
performance between rounds 1 and 2, rounds 2 and 3 and rounds 3 and 4 but do not compare
performance between rounds 4 and 5. A similar procedure is employed for other tournaments that do
not employ a cut after the second round.
For the purposes of this test, we sort all θ and η residual scores in the first of each pair of
qualifying rounds and place each player into one of 20 categories of approximately equal size based
on the ranking of residuals in the first of each qualifying two-round pair. Within each sort category
we compute the average residual score in both the first and second of the two adjacent rounds. Table
5 summarizes the results of this test for both θ and η residuals.
The section of the table that summarizes the analysis for θ residuals shows a very slight
tendency for scores in the 20 sort categories to carry over from one round to the next. In sort
categories 1 – 10, the average first-round θ residual score is negative, and the average second-round
θ residual is also negative in each of these categories. For example, on average players in sort
category 1 have a first-round θ residual score of -5.184. This means that on average, players in the
first group scored 5.184 strokes better (lower) than predicted by model 3. If a portion of the 5.184
strokes represents a change in skill for these same players, or an advantage due to some players
having a competitive advantage on the course they played in the first round, rather than random good
luck, scores in the next round should continue to be lower than predicted. Otherwise, scores of
players in the first sort group should revert back to normal. In the next adjacent round (not divided
by a cut), the average θ residual for these same players is –0.097. This same pattern, negative
average residual first-round scores followed by negative average residual second-round scores, is
present throughout the first ten sort categories. However, all of the average second-round residuals
are very close to zero, with the largest, in absolute value, being 0.135 strokes for sort category 2.
To shed further light on the relationship between residual scores in adjacent rounds, we run a
simple least squares regression within each sort group in which the second residual in each pair is
regressed against the first. If players within a group tend to continue with similar performance, this
regression coefficient should be positive and significant. However, as shown in Table 5, there is no
discernable pattern among the regression coefficients within the first ten sort groups, and none of the
regressions is significant.19
Among sort groups 11 – 20, all average first-round θ residuals are positive, and seven of ten
average second-round residuals are positive. But except in sort categories 19 and 20, with average
second-round residuals of 0.219 and 0.328, respectively, the average second-round residuals would
be of little if any significance in golf.
One possible explanation for the different pattern of average residuals in categories 19 and 20,
and the significant positive regression coefficient in sort group 20, is that players who have
performed poorly prior to a cut may take more chances in an attempt to make the cut. If riskier play
tends to be accompanied by higher scores, this pattern should emerge. An alternative explanation is
that many players in the last two sort groups may consider it a forgone conclusion that they will miss
the cut and, perhaps, do not give their best efforts in the second of the two adjacent rounds. Although
both explanations are quite different, they both involve players changing their normal method of play
when the second of the two rounds is followed by a cut.
We can test for this tendency by separating the average residual scores into those for which
the second of the two adjacent rounds is immediately followed by a cut and those for which a cut
does not occur after the second of the two rounds. Although not shown in the table, for all sort
groups, except groups 19 and 20, there is little difference in performance when a cut occurs after the
second of two qualifying adjacent rounds (panel 2) and when it does not (panel 3). However, for sort
categories 19 and 20, the average of the θ residuals in the second of the two adjacent rounds are
0.381 and 0.406, respectively when a cut occurs immediately afterward the second of the two rounds,
and –0.021 and 0.212, respectively, when a cut does not follow the second round. Moreover, the
regression of second-round residuals against first-round residuals is only significant for sort category
20 when the second of the two rounds is followed by a cut. Taken together, this evidence suggests
that tournament participants who perform exceptionally poorly in the earliest round(s) may change
their method of play in the round before a cut to reflect the high probability of being cut. Otherwise,
the threat of being cut does not appear to affect player performance.
It should be noted that if a player’s exceptionally good (or bad) score as measured by his θ
residual in the first of two adjacent rounds is due to an advantage (or disadvantage) in playing a
particular course, the advantage should carry over into the next round, provided the next round is
played on the same course. Since the large majority of adjacent rounds summarized in Table 5
involve play on the same course, the fact that the average second round residual in each sort category20
is essentially zero provides additional evidence that player-course interactions do not play a
significant role in determining 18-hole scores on the PGA Tour.
Table 5 also summarizes the results of identical tests of η residuals. Unless the AR(1) model
does not adequately capture the correlation structure of player scores around their respective spline
fits and/or there are significant player-course interaction effects, η residuals should represent white
noise uncorrelated from one round to the next. Table 5 shows that regardless of the first-round sort
category, average residual scores in the second of two adjacent rounds are very close to zero, and that
there is no discernable relationship between the signs of average first-round scores and those of
adjacent second-round scores. As with θ residuals, we run a least squares regression within each
sort group in which the second residual in each pair is regressed against the first. Only one
coefficient is significant at the 5 percent level – that for sort group 4. Moreover, the fact that the
average second-round residual in group 4 is so close to zero (-0.029) indicates that any tendency for
the direction of abnormal performance to persist in this sort group is not accompanied by a level of
performance that would make much difference in determining a player’s 18-hole score. Therefore, we
conclude that significant differences between actual and predicted scores are due, primarily, to luck,
and that these differences cannot be expected to persist.
5.2.2 How Much Luck Does it Take to Win a PGA Tour Event? It is interesting to consider
how much luck it takes to win a PGA Tour event or to at least finish among the top players. Table 6
summarizes the actual scores and θ residual scores on a round-by-round basis for the top 40 finishers
in the 2001 Players Championship. We focus on θ residual scores rather than η residuals so that we
can determine the extent to which each player’s scores deviate from their time-dependent mean
values. The pattern of residual scores exhibited in this table is typical of those for all 182
tournaments. Among the top 40 finishers, the total residual score for almost all players is negative;
only Phil Mickelson, Colin Montgomerie and Steve Flesch have positive total residual scores. Thus,
to have won this tournament, and almost all others in the sample, not only must one have played
better than normal, but one must have also played sufficiently well (or with sufficient luck) to
overcome the collective good luck of many other participants in the same event.
Over all 182 tournaments, the average total θ residual score for winners and first-place ties
was –9.92 strokes, with the total residual ranging from –2.14 strokes for Tiger Woods in the 2000
ATT Pebble Beach Pro-Am to –23.48 strokes for Mark Calcavecchia in the 2001 Pheonix Open.
(Similarly, the average η residual was –9.80 strokes.) Table 7 summarizes the highest 20 total θ21
residual scores per tournament for winners and first place ties. It is noteworthy that no player won a
tournament after recording a positive total θ residual score. Although not shown in the table, Tiger
Woods’ total η residual score for the 2000 ATT Pebble Beach event was +0.32 strokes, but this is
the only positive total residual using either the θ or η measure. It is also noteworthy that Woods’
name appears 11 times in Table 7 and that all the other players on the list (Phil Mickelson, Sergio
Garcia, David Duval, and Davis Love III) are among the world’s most recognizable players. Thus,
over the 1998-2001 period, only Tiger, and perhaps a handful of other top players, were able to win
tournaments without experiencing exceptionally good luck.
5.2.3 Standard Deviation of Residual Scores. Figure 4 summarizes the standard deviation of
θ residual errors among all 253 players in the sample. The range of standard deviations is 2.14 to
3.45 strokes per round, with a median of 2.69 strokes. John Daly and Phil Mickelson, both well-
known for their aggressive play and propensities to take risks, have the third and 15th highest standard
deviations, respectively. Ian Leggatt has the lowest standard deviation. Chris Riley and Jeff Sluman,
both known as very conservative players, have the second and fifth lowest deviations, respectively.
It is interesting to consider whether average scores and standard deviations of θ residual
errors are correlated. A least squares regression of standard deviations against the mean of each
player’s spline-based estimate of skill over the entire sample period yields:
expected score = 68.22 + 1.14 × standard deviation , with adjusted R2 = 0.067, F = 19.02 and p-value
= 0.00002. Thus, there is a tendency for greater variation in player scores to lead to slightly higher
average scores.
5.2.4 Effect of Round-Course Interactions. Figure 5 summarizes the distribution of 848
random round-course interaction coefficients estimated in connection with model 3. The coefficients
range in value from –3.924 to 6.946, implying almost an 11-stroke difference between the relative
difficulty of the most difficult and easiest rounds played on the Tour during the 1998-2001 period.
Over this period, 25 tournaments were played on more than one course. In multiple course
tournaments, players are (more or less) randomly assigned to a group that rotates among all courses
used for the tournament. By the time each rotation is completed, all players will have played the
same courses. At that time a cut is made, and participants who survive the cut finish the tournament
on the same course. Although every attempt is made set up the courses so that they play with
approximately the same level of difficulty in each round, tournament officials cannot control the22
weather and, therefore, there is no guarantee that the rotation assignments will all play with the same
approximate levels of difficulty.
Figure 6 shows the distribution of the difference in the sum of round-course interaction
coefficients for the easiest and most difficult rotations for the 25 tournaments that were played on
more than one course. For seven of 25 tournaments, the difference was less than 0.50 strokes. Thus,
on average the difference for these tournaments was sufficiently small that a player’s total score
should have been the same regardless of the course rotation to which he was assigned. At the
extreme, there was a 5.45 stroke differential between the relative difficulties of the easiest and most
difficult rotation assignments in the 1999 ATT Pebble Beach Pro-Am. Within this tournament, two
of six possible rotations played with round-course interaction coefficients that totaled 10.41 and
10.23 strokes, while the total for the remaining four rotations fell between 4.96 and 5.72 strokes. The
two difficult rotations involved playing the famed Pebble Beach course on the third day of the
tournament. Described as one of the nastiest days since the tournament was started in 1947
(www.golftoday.co.uk/tours/tours99/pebblebeach/round3report.html), the adverse weather conditions
had a much greater effect on scores recorded on the Pebble Beach course than on the other two
courses, Spyglass and Poppy Hills. According to David Duval (www.golftoday.co.uk), "This is the
stuff we stopped playing in last year. … It's the type of day you don't want, for the sole reason that
luck becomes a big factor.''
It is interesting that the top nine finishers in this tournament all played one of the four easiest
rotations, as did four of the five players who tied for 10th place. Among the top 20 finishers, only two
played one of the two difficult rotations. Clearly, for this particular tournament Duval was right. The
luck of the draw had more to do with determining the top money winners than the actual skill
exhibited by the players.
5.3 Hot and Cold Hands
Hot and cold hands represent the tendency for abnormally good and poor performance to
persist over time and has been the focus of a number of statistical studies of sports. While some had
argued for the presence of a hot hand in basketball, Gilovich, Vallnoe and Tversky (1985) argued that
this belief might exist even though actual shooting was consistent with a random process of
hits/misses (wins/losses). In other words, people may find systematic patterns in what is actually
random data. Larkey, Smith, and Kadane (1989) disputed this finding. Wardrop (1995) notes thatYou can also read
