How Do We Know When a Player Is Fully Recovered? A Systematic Review About Return to Play - Bachelor Degree Project in Cognitive Neuroscience ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
How Do We Know When a Player Is Fully Recovered? A Systematic Review About Return to Play Bachelor Degree Project in Cognitive Neuroscience First Cycle 22,5 credits Spring term 2021 Student: Sarah Eriksson Supervisor: Joel Gerafi Examiner: Oskar MacGregor
2
Abstract
Athletes participating in ice hockey are at a high risk of experiencing a concussion which can
lead to impairments in cognitive functions. Deciding when a player can be returned to playing
ice hockey is part of the concussion management called return to play (RTP). This can be
based on subjective or objective measures. Returning a concussed hockey player too soon can
result in devastating consequences. This review aimed to investigate what objective measure
could be used to assess a concussed player. It specifically focused on Immediate Post
Concussion Assessment and Cognitive Test (ImPACT) and a cognitive motor integration
(CMI) performance task. It was hypothesized that by combining the two could contribute to
an optimal objective assessment and be used as accurate indicators in the RTP decision. This
review involved searches from PubMed, Scopus and Web of Science which resulted in a total
of four articles. The amount of articles was a big limitation. Main findings are that concussed
participants showed decreased cognitive performance relative to baseline despite subjectively
reporting being symptom free. Findings in this review suggest that adding a CMI performance
task to the ImPACT test battery could be a way to objectively catch several composites in
cognitive recovery including tasks that require higher levels of cognitive functioning.
Administering an assessment based on two objective measures could add accuracy and
contribute to a safe RTP not risking a second impact syndrome or prolonged cognitive
impairments.
Keywords: return to play, concussion, ice hockey, impact, cmi3
How Do We Know When a Player Is Fully Recovered? A Systematic Review About
Return to Play
Athletes participating in ice hockey, which is a high speed contact sport, played with a
puck and stick on a hard surface, are at a high risk of experiencing a concussion (Donaldson
et al., 2013). A definition of concussion is “a traumatic brain injury induced by
biomechanical forces” (McCrory et al., 2017, p. 839). This is a type of injury where a sudden
force is transferred through a direct blow or collision to the head, face, or neck, and can cause
significant movement in which the brain is affected (McCrory et al., 2017; Parizek & Ferraro,
2016). Concussion may involve psychomotor slowing, impairments in cognitive functions
such as memory and attention, and neurobehavioral changes, e.g., anxiety, fatigue and
depression (Gosselin et al., 2010; Ruhe et al., 2014).
Ice hockey has been shown to have a high incidence of concussion compared to other
contact sports (Ruhe et al., 2014). For decades this injury has been underreported in the way
that unidentified or unreported cases have not been included in the statistics, thus theoretical
prevalence rates have increased in recent years (Parizek & Ferraro, 2016). An example of this
is Swedish elite ice hockey that in the 1980 season reported approximately 20 concussions per
1000 games compared to recent years reporting an estimation of approximately 150
concussions per 1000 games (Gard et al., 2020; Pauelsen et al., 2017).
Sustaining a concussion can lead to negative and devastating effects for the athlete
progressing and continuing over a long time into old age, or when retired from the sport
(Pedersen et al., 2014). Although protective gear prevents the occurrence of head and neck
injuries, it does not necessarily reduce the risk or specifically protect against a concussion
(Benson et al., 2009; Donaldson et al., 2013).
The National Hockey League (NHL) is an ice hockey league in North America
consisting of 31 professional teams and the Swedish Hockey League is the highest division in
the Swedish ice hockey system consisting of 14 professional teams. Studies on male players
in both leagues show similar results, namely, in addition to contact with a stick or collisions,
the greatest cause of concussion is body checking. This is when the player uses his body to
stop, slow or separate an opponent from the puck (Donaldson et al., 2013; Pauelsen et al.,
2017; Ruhe et al., 2014).
The Swedish Ice Hockey Federation is an association in Sweden for ice hockey clubs
at all levels. To reduce and prevent concussions among players, the federation has an ongoing
project since late 2018 called “Vision Zero’'. Information and education to players and
leaders, combined with investigation of risk factors and protection aspects with manufacturers4
and medical experts, have yielded a reduction of concussions among elite players (Svenska
Ishockeyförbundet, 2021).
A critical decision in concussion management is the return to play (RTP) clearance
that allows an athlete to safely return to activity. Every concussion is unique (Doolan et al.,
2012) with an individual and non fixed recovery timeline (Johnston et al., 2004; Patel et al.,
2005). Historically, RTP decisions have been based on guidelines for which empirical
evidence to support underlying assumptions has been meager. Examples of these are
limitations regarding timeframes for RTP, the relationship between RTP and number of
concussions within a season, and most importantly guidelines that do not differentiate among
individuals (Echemendia et al., 2016; King et al., 2014). The first reported RTP rule named
“the three strike rule” was proposed in 1945 by Quigely and adapted a couple of years later by
Thorndike. If an athlete experienced three concussions with loss of consciousness during a
season, the athlete was to be removed from play for the remainder of the season (Thorndike,
1952).
At present the RTP decision is still complicated as many factors need to be taken into
consideration. For example, athletes can intentionally de-emphasize or conceal nonspecific
symptoms that are subjectively reported (Cantu & Register-Mihalik, 2011; Sicard et al.,
2020). Athletes may also not recognize that their symptoms are a result of the concussion
(Delaney et al., 2018). Premature RTP has been shown to increase the risk of second impact
syndrome, which is a condition where a second head injury can occur if symptoms from the
initial concussion have not fully cleared (Cantu, 1998; Sicard et al., 2020). In addition to
second impact syndrome, two other major categories of concern should be taken into
consideration in the decision making for RTP, namely post concussion syndrome which is a
prolonged recovery from sequential concussions, and chronic traumatic encephalopathy
which is a progressive neurodegenerative syndrome caused by trauma and force impacts to
the brain (Doolan et al., 2012; Omalu et al., 2011).
Subjective Assessment
In concussion management the RTP decision can be based on subjective or objective
tools to assess the athlete. A widely accepted tool in the RTP management is the stepwise
general protocol which is based on subjective measures. This protocol is developed and based
on a consensus statement brought forward at international symposiums on concussion in sport
(Aubry et al., 2002). From onset of injury to return to playing the game the athlete progresses
through six steps: (1) symptom-limited activity, (2) light aerobic exercise, (3) sport-specific
exercise, (4) non-contact training drill, (5) full-contact practice, (6) return to sport without5
restrictions. This stepwise protocol requires asymptomaticity at the current step in order to
proceed to the subsequent and more demanding step. Should the athlete feel symptoms within
24 hours performing activity at a step, the athlete is required to drop back to the previous step
(Johnston et al., 2004).
Objective Assessment
There are also a wide variety of objective assessment techniques. Monitoring
blood-based biomarkers is an objective measure that has shown empirical evidence for
prognosis of recovery and decision making in the RTP process (Anto-Ocrah et al., 2017;
Shahim et al., 2014, 2018). Electroencephalography (EEG) measures electrical signals
generated by the brain, and an event related potential (ERP) is a signal embedded in the
current EEG being measured (Gazzaniga et al., 2014). Examining ERPs by measuring a brain
network activation can also be used as a biomarker in an objective analysis. This could be the
next step in using EEG as an accepted tool for recovery assessment in concussed ice hockey
players (Kiefer et al., 2015).
Neuropsychological testing, which was introduced in the 1980s (King et al., 2014) is
another way of assessing athletes in the RTP decision making process (Doolan et al., 2012).
This is done by measuring different domains of cognitive function and the tests can be
administered using pencil-and-paper or be computer based. Neuropsychological tests are seen
as an essential part in the concussion management in sports and baseline tests are mandatory
in the NHL (Reinsberger & Frick, 2017). Computerized neuropsychological tests, such as the
Immediate Post Concussion Assessment and Cognitive Test (ImPACT) test battery (ImPACT
Solutions Inc., Pittsburgh PA), objectively measures aspects of cognitive functioning and
takes approximately 20 minutes to administer. ImPACT is one of the first widely used, and
scientifically validated computerized concussion evaluation systems (Covassin et al., 2009)
which measures cognitive functions, such as learning, memory, attention, concentration,
reaction time, visual processing speed and visual motor speed (Pedersen et al., 2014). In the
RTP decision it is suggested to use the ImPACT assessment in conjunction with other tools
such as the subjective stepwise assessment mentioned earlier known as the general RTP
protocol (Alsalaheen et al., 2016).
Cognitive motor integration (CMI) is when the brain concurrently uses cognition and
motor function to perform complex motor actions. Fronto-parietal networks together with
other brain areas, such as the cerebellum are involved in this integration (Dalecki et al., 2016).
CMI is used during play in ice hockey, e.g., when skating with a hockey puck, the player pays
attention to their teammate on their right and sends a pass to this teammate while skating to6
the left. When a player returns to ice hockey post concussion tasks that require simple motor
functions may have recovered but higher level tasks that require CMI performance have been
shown to not always be fully recovered (Dalecki et al., 2019).
Post-concussion management in sport lacks a standardized protocol for decision
making in RTP for athletes (Piedade et al., 2021). RTP decisions remain difficult and
controversial and guidelines can be unclear and contradictory (Doolan et al., 2012). There is
an increased awareness about the consequences of returning a concussed hockey player too
soon. Knowledge points in the direction that guidelines need to be based on scientific
evidence rather than clinical experience (Canadian Academy of Sport Medicine Concussion
Committee, 2000). In addition, these RTP guidelines ought to be practical and applicable in
the athlete’s environment to return a player safely to play. The critical and complicated RTP
process, the risk of a premature RTP, and the decision when an ice hockey player is safe to
return to play post concussion irrefutably lead to the question: “What measures can be used to
accurately assess whether a player has fully recovered from a concussion”?
Because post concussion management lacks a standardized protocol (Piedade et al.,
2021) there are different approaches in the RTP decision. Commonly used consensus
statements (McCrory et al., 2017) advise subjective measures based on self-reports, such as
the stepwise general protocol. In many cases these subjective measures are used together with
objective computerized neuropsychological tests (Echlin et al., 2010) in the RTP decision.
Although subjective and objective measures are used concurrently, the increased awareness
that players return to ice hockey before cognitive functions are fully recovered, it would be
interesting to know in what way objective measurements can add accuracy to the RTP
decision.
The aim of this review is to specifically focus on two objective measures, the ImPACT
test battery and a CMI performance task, and to investigate how accurately these objective
measures can assess whether a player's cognitive functions have recovered, and to contribute
to optimal RTP decisions with regard to deciding whether a player is fully recovered from a
concussion. The ImPACT test battery is a validated tool to measure cognitive functions and
commonly used together with the subjective stepwise protocol in concussion management,
however used solely it may not be enough to measure accuracy of recovery in a post
concussed athlete. Motor functions that require CMI have shown to not always be fully
recovered when a concussed athlete returns to play. The hypothesis is that combining the
ImPACT test battery and a CMI performance task could therefore be the most optimal7
assessment tool and used as objective, accurate indicators in the RTP decision when assessing
whether a concussed athlete is ready to return to playing ice hockey.
Methods
Search Strategy
This systematic review followed the Preferred Reporting Items for Systematic
Reviews (PRISMA) guidelines (Moher et al., 2009, see Figure 1). It involved searches from
the databases PubMed, Scopus, and Web of Science. The timespan was set to the earliest
publication to present date in each database (2021, April 12). Utilized search string in all
databases was: ((“return to play” OR RTP) AND (concussion OR mTBI) AND ( ImPACT
OR (“test battery”) OR CMI OR (“cognitive motor integration”) OR guidelines OR
assessment OR “decision making”) AND (“ice hockey” OR hockey)). No filters were applied.
PubMed provided 53 results, Scopus 8, and Web of Science 57, adding up to a total of 118
articles.
All results were imported into an online version of EndNote (Clarivate, 2021) and
duplicates were removed with the duplicate function, then double checked manually. The
remaining articles were screened for eligibility. Titles, abstracts and keywords for these
articles were scanned and selected to a “quick list” in EndNote for further examination.
Articles in the “quick list” were excluded if they failed to meet inclusion criteria.
Inclusion and Exclusion Criteria
Articles and studies were included if they were related to neuroscience,
neuropsychology or sports medicine. The preset criteria for inclusion was according to the
PICO framework (Hesser & Andersson, 2015): (a) participants were ice hockey players in the
age groups: youth (from 14 years old), junior (16-21 years old) or adults; (b) diagnosed with
mild or severe, first, second or consecutive concussion; (c) control group was non concussed
athletes or individual baseline measurements; (d) outcome were measurements of cognitive
functions from the ImPACT test battery or kinetic variables measured from a CMI
performance task. Reasons for exclusion were if they were review articles, non-english
language, not related to ice hockey or did not measure cognitive functions.
Data Extraction
From each article included in this review the following data was extracted: author,
year and study design, participant sample size and how many of these were concussed, non
concussed or healthy controls, gender and age of participants. Further, the author extracted
what tool was used to measure cognitive performance and functions, ImPACT test battery or
CMI measures, and what those measures were compared with, baseline testing or non8 concussed participants (healthy controls). Measured cognitive functions and kinetic variables were extracted and results were synthesized across all included studies. Figure 1 PRISMA 2009 Flow Diagram: Literature Search Process Note. The literature search process, illustrated in a PRISMA 2009 Flow Diagram. Adapted from “Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement,” by D. Moher, A. Liberati, J. Tetzlaff, D. G. Altman, The PRISMA Group, 2009, PLoS Med, 6(7), p. 8 (doi:10.1136/bmj.b2535). CC BY-NC.
9
Results
A total of 118 results were imported into an online version of EndNote (Clarivate,
2021) and 29 duplicates were removed with the duplicate function and 11 duplicates were
removed manually. This resulted in 78 articles to screen for eligibility. Titles, abstracts and
keywords for these 78 articles were scanned and a total of 65 articles were excluded. Reasons
for exclusion were if they were review articles, non-english language, not related to ice
hockey or did not measure cognitive functions. Remaining 13 articles were scrutinized and
assessed for eligibility then selected to a “quick list” in EndNote for further examination.
Eight of these articles were excluded as they measured the amount of days not playing ice
hockey in NHL (timeloss), amount of days for recovery post concussion, were related to
test-retest of the ImPACT test battery, mechanisms of injury, and chiropractic therapy. One
study (Brooks, 2007) was extracted late in the process as figures with ImPACT scores and
sample size of participants were not readable, visible or clearly stated. This added up to nine
articles being excluded from the “quick list” as they failed to meet inclusion criteria.
The search process resulted in a total of four studies for this review. Complete search
process according to the PRISMA flow diagram can be viewed as in Figure 1. All four studies
were based on empirical research articles and a summary of relevant information and
extracted data according to the PICO criteria from each study can be seen in Table 1.
In this review a total number of 151 participants with a concussion history were
examined where 4 of these participants (Pedersen et al., 2014) suffered a second concussion.
This added up to a total of 155 represented in this review. Three of the studies used the
ImPACT test battery to measure cognitive performance and compared scores relative to
baseline testing (Kriz et al., 2017; McGrath et al., 2013; Pedersen et al., 2014) and one study
used a CMI performance task to measure recovery compared with healthy controls. These
healthy controls were participants without a concussion history (Hurtubise et al., 2016).
Three studies were conducted in the United States (Kriz et al., 2017; McGrath et al.,
2013; Pedersen et al., 2014) and one study conducted in Canada (Hurtubise et al., 2016).
Participants across all studies were between 13-23 years. Mean age was 15.5 ∓1.3, age
range 13-17.5 years (Kriz et al., 2017), age range 16-17 years (McGrath et al., 2013); mean
age 23 (Pedersen et al., 2014); and mean age 17 ∓ 1 years (Hurtubise et al., 2016).
Two studies (Kriz et al., 2017; McGrath et al., 2013) had male and female participants
whereas two studies (Hurtubise et al., 2016; Pedersen et al., 2014) had male participants only.
There were no findings or differences between the female and male groups in both studies
(Kriz et al., 2017; McGrath et al., 2013).10
Table 1
Articles Included in Systematic Literature Review
Intervention: ImPACT
Author, Year Study Design Participants Comparison Outcome
Kriz et al., 2017 Cross sectional N=32 concussion Baseline testing. Cognitive functions measured:
study
(n=18 male, n=14 ° Verbal memory
female) ° Visual memory
° Visual motor speed
° Reaction time
Junior 16-21 years ° Total symptom score
McGrath et al., Retrospective N= 54 concussion Baseline testing. Cognitive functions measured:
2013 records review (n=43 male, n=11
study female) Participants divided ° Verbal memory
into two groups* ° Visual memory
Junior 16-21 years ° Visual motor speed
-PE fail group ° Reaction time
-PE pass group
° Impulse control
Pedersen et al., Archival data N=74 males Baseline testing. Cognitive functions measured:
2014 collected over 6 (n=14 concussion,
years n=4 second ° Verbal memory
concussion) ° Visual memory
° Visual motor speed
Adults ° Reaction time
° Impulse control
Intervention: CMI
Author, year Study Design Participants Comparison Outcome
Hurtubise et al., Case control N=102 males Asymptomatic with Kinematic variables examined:
2016 study (n=51 concussion concussion history
history, n=51 compared with ° Reaction time
healthy controls no asymptomatic no ° Ballistic movement time
concussion) concussion history ° Total movement time
(healthy controls). ° Ballistic path length
° Full path length
Both groups ° Peak velocity
Junior 16-21 years performed two ° Constant Error
computer based ° Variable Error
visuomotor tasks:
-standard condition
-nonstandard
condition
Note. *Cognitive functions measured after physical exercise according to a post exertion (PE)
protocol.
All three ImPACT articles (Kriz et al., 2017; McGrath et al., 2013; Pedersen et al.,
2014) assumed baseline testing where one of these studies (McGrath et al., 2013) completed
this on four occasions and specifically measured cognitive functions after physical movement
according to a post exertion protocol. This protocol involved 15-20 minutes of moderate
cardiovascular exercise, e.g., running on a treadmill, riding a stationary bike, or elliptical
training and ice hockey specific activities such as dribbling a puck or noncontact skating (no11
contact with other players). Following this exercise the players then rested 5-10 minutes
before completing a post exertion ImPACT test. The participants' baseline testing was then
divided into two groups depending on post exertion neurocognitive results (McGrath et al.,
2013). These independent variables, a PE fail group and a PE pass group, were used to
separate participants who showed one or more reliable change index (RCI) scores with those
participants that did not show any RCI on ImPACT scores post exertion.
Collective composite scores in all studies assessed with the ImPACT test battery were
verbal memory, visual memory, visual motor speed, and reaction time (see Table 1). Two
studies also displayed composite results of impulse control from the ImPACT test battery
(McGrath et al., 2013; Pedersen et al., 2014). The specific tests included in the ImPACT test
battery were Three Letters, Xs and Os, Word Memory, Color Match, Symbol Match, and
Design Memory (Pedersen et al., 2014).
One of the ImPACT studies (Kriz et al., 2017) also showed total symptom scores as an
additional measurement to the composite scores. Participants self-reported their concussion
related symptoms up until resolution, this was defined as the day they no longer experienced
symptoms. In this study (Kriz et al., 2017) the participants self-reported being asymptomatic
with a mean of 23.8∓16.8 days after being diagnosed with a concussion.
All three studies (Kriz et al., 2017; McGrath et al., 2013; Pedersen et al., 2014) using
the ImPACT test battery showed impairment in cognitive functions post concussion relative to
baseline testing. One study (Kriz et al., 2017) resulted in 9 participants reporting not having
concussion related symptoms albeit showing a reduction on at least one composite score in
cognitive performance, namely reaction time relative baseline testing. In the same study 2
participants showed continued impairment in cognitive functions on at least 2 composite
scores (Kriz et al., 2017). A study (McGrath et al., 2013) with a total of 54 concussed
participants showed that 15 participants reported no symptoms and returned to relative
baseline scores when at rest. After a period of moderate exertion the participants showed
declining cognitive changes on tests of memory ability. This study (McGrath et al., 2013)
specifically mentioned how cognitive functions after physical exercise not only showed
general performance effects, but how memory ability significantly changed despite intact
processing speed functions.
One study (Pedersen et al., 2014) contained 4 participants suffering a second
concussion showing significant decreases in cognitive domains including reductions in
visual-motor speed, tasks of visual processing discrimination, and errors on immediate recall
of designs. The same study (Pedersen et al., 2014) showed that 9 out of 14 concussed12
participants showed a significant decrease in cognitive performance compared to baseline
measures for immediate and delayed memory.
One study (Pedersen et al., 2014) mentioned the negative effects a concussion can
have on an ice hockey player's cognitive functions throughout the lifespan. An example is
speeding up natural late-life cognitive decline. This referred to cognitive functions such as
delayed memory, immediate memory, visual processing and reaction time. Namely
participants suffering a second concussion were at a higher risk of a more rapid decline and
late-life consequences.
CMI performance in one study (Hurtubise et al., 2016) was measured with a computer
based eye-hand coordination task. In this study participants with a concussion history were
compared to healthy controls and both groups performed two computer- based visuomotor
tasks in a standard and nonstandard condition. Kinetic variables examined in the CMI task
were movement timing, and movement execution. Movement timing was measured with the
following outcome variables: reaction time, ballistic movement time, total movement time
and peak velocity. The reaction time was related to motor planning, and the ballistic and total
movement timing related to motor execution. Movement execution was measured with the
following outcome variables: ballistic path length, full path length, constant error, and
variable error (Hurtubise et al., 2016). All 51 participants in this study (Hurtubise et al., 2016)
with a concussion history showed significantly slower reaction times in the CMI task. The
outcome variable constant error, which shows accuracy, decreased both in the standard and
non-standard condition relative to the control group, i.e., participants without a concussion
history.
All four studies (Hurtubise et al., 2016; Kriz et al., 2017; McGrath et al., 2013;
Pedersen et al., 2014;) mention underreporting of symptoms related to a concussion injury to
be a complication in the general concussion management.
Discussion
The aim of this review was to specifically focus on two objective measures, the
ImPACT test battery and a CMI performance task, and to investigate how accurately these
objective measures could assess whether a player's cognitive functions have recovered, and to
contribute to optimal RTP decisions with regard to deciding whether a player is fully
recovered from a concussion. It was hypothesized that combining the ImPACT test battery
and a CMI task could be the most optimal assessment tool and used as objective, accurate
indicators in the RTP decision when assessing whether a concussed athlete is ready to return
to playing ice hockey.13
The ImPACT test battery and a CMI performance task are both objective measures.
They measure the same cognitive functions, but in different ways. The ImPACT test battery
measures cognition and motor skills separately whereas a CMI performance task measures
cognition and motor skills concurrently.
Based on the included studies, one main finding emerges. Post concussed participants
show cognitive impairments despite reporting being symptom free. Although players
subjectively reported being symptom free or denied the presence of concussion related
symptoms, objective cognitive testing showed impairments in cognitive performance or
declines in cognitive recovery compared to baseline testing or to participants without a
concussion history.
In one study (Kriz et al., 2017) nine out of 32 participants, which is 28.1% of the
sample size, still showed signs of cognitive impairments despite reporting being subjectively
asymptomatic. The authors compared this finding with a study presented at the World
Conference on Prevention of Injury and Illness in Sports in Monaco by Taylor et al. (2014)
where 25% of the concussed participants showed impaired ImPACT performance on one
composite score by the time the players were cleared for RTP. Another study (McGrath et al.,
2013) showed that 27.7% of participants were categorized into a PE-fail group, which meant
that they were showing decline on composite scores in cognitive recovery post exertion.
Based on these findings, this could be an indication that approximately 25% of concussed ice
hockey players are assessed a premature RTP.
How many premature RTPs should be tolerated? The Swedish Ice Hockey Federation
with their "Vision Zero" project is about reducing and preventing concussions among players
by informing and educating involved parties, combined with investigating risk factors and
protection aspects with medical experts and manufacturers (Svenska Ishockeyförbundet,
2021). This is done across all levels and age groups, which since 2018 has yielded a reduction
in reported concussions among elite players. As hypothesized, using a CMI performance task
in combination with the ImPACT test battery could lead to a decrease in premature RTPs
which would be in line with the aim of "Vision Zero" for Swedish ice hockey. This suggestion
would lead to a minimized risk of a second concussion and ice hockey players not suffering
from potentially lifelong post concussion symptoms. A decrease of post concussion
symptoms, especially for the retired player, contributes to an increased quality of life. A study
assessing 76 ice hockey players with an average of five years since their most recent
concussion showed the players were encumbered with post concussion symptoms and reduced
quality of life (Gard et al., 2020).14
In this review one of the articles enrolled 145 athletes to participate in a study (Kriz et
al., 2017). Thirty-two participants were eligible and took part as they had baseline scores for
the ImPACT test. This could be a good representation of how it looks in the NHL and the
Swedish Hockey League. Players suffering a concussion but without having a baseline score
available are being returned to play too early, either by basing the RTP decision on subjective
measures or comparing the individual’s scores to normative values. Would it be relevant to
base the RTP decision on acceptable threshold levels of cognitive performance post
concussion rather than individual baseline scores or normative values? A suggestion would be
to include these threshold levels in a standard protocol for general guidelines in the RTP
management. The question however still remains, who is to decide these acceptable threshold
measures?
One study (McGrath et al., 2013) did not include participants with learning disorders,
ADHD, or if they were undergoing treatment for depressive or anxiety disorders. These
individuals are represented in ice hockey and it would therefore be of importance to study
cognitive recovery post concussion for players with these criteria, so they too can be offered
objective, accurate and a safe RTP clearance.
Studies in this review were conducted in the United States (Kriz et al., 2017; McGrath
et al., 2013; Pedersen et al., 2014) and Canada (Hurtubise et al., 2016). No studies with regard
to the aim of this review were based on ice hockey players in Sweden. Taking into
consideration the Swedish Ice Hockey Federation working with the ‘Vision Zero’ project it
would be of interest and recommended that future studies are conducted on Swedish elite ice
hockey players. There is one study conducted on Swedish professional ice hockey players in
regard to blood based biomarkers in predicting RTP (Shahim et al., 2014), but no studies
using a CMI task or ImPACT test battery. The ImPACT test battery is used by some ice
hockey teams but it is not a mandatory part of concussion management. Measuring CMI
performance using a CMI task in combination with the ImPACT test battery could be a future
research area for Swedish ice hockey.
Findings in this review support that RTP decisions can not be based on cognitive
functions solely measured by subjective self-reports or the commonly used stepwise protocol.
Although subjective assessments are used together with objective neuropsychological testing
(Echlin et al., 2010) none of the studies in this review particularly mentioned using the
subjective stepwise general protocol collectively with objective measurements in the RTP
decision. Repeated concussions can lead to changes in neural connectivity in the brain which
can be a reason for chronic cognitive problems (Gazzaniga et al., 2014). Granting the ice15
hockey player an accurate assessment based on two objective measures could contribute to a
safe return to play not risking a second impact syndrome or prolonged cognitive impairments
lasting into later in life.
Another suggestion that evolves from findings in this review is that making objective
measures mandatory in concussion management would be a way for players and teams to put
the RTP decision in the hands of objective scores, not in the hands of self-reports or norms.
Wennberg and Tator (2008) mention that increased adherence to guidelines is a way to
prevent premature RTP. Objective measures like the CMI task and ImPACT test battery that
show accurate results would be a way in the concussion management to support those
guidelines based on empirical evidence rather than subjective experience. Who is responsible
for the final RTP decision, the club or the player? Guidelines could be a way of clarifying
areas of responsibility in concern to moral and ethical decisions.
The level of task difficulty in the CMI performance task is high due to cognitive and
motor systems being involved at the same time. This requires that the connections between
the frontal (primary motor, premotor, prefrontal), parietal (superior parietal lobule,
intraparietal sulcus) and subcortical areas (cerebellum) in the brain are not damaged
(Hurtubise et al., 2016). Motor planning and movement timing, both related to reaction time,
were significantly slower in the CMI task for all 51 participants (Hurtubise et al., 2016) with a
concussion history. Also, accuracy decreased in both the standard and the non-standard task
for these participants. The authors suggest that the decrease in accuracy is because a
concussion transmits general turbulence to the brain's motor system (Hurtubise et al., 2016).
Using the objective ImPACT test battery together with measuring CMI skills could therefore
be a way to measure this general turbulence in cognitive recovery.
Findings from all studies in this review demonstrate cognitive impairments in returned
players post concussion. These impairments and declines were mainly in composite scores
from the ImPACT test battery regarding memory and reaction time, but also cognitive
functions requiring a higher level of task complexity, namely when integrating cognitive and
motor functions. Literature in concussion management regarding RTP decisions points
towards standardized protocols, including assessments of cognitive functions indicating
accurate scores on cognitive functions post concussion (Doolan et al., 2012; King et al., 2014;
Piedade et al., 2021).
In one of the studies (McGrath et al., 2013) results showed declining cognitive
changes, especially in the memory composite scores in presence of intact speed functions.
Further this study (McGrath et al., 2013) specifically measured cognitive recovery16
post-exertion using the ImPACT test battery adding physical activity as an independent
variable. This showed the consequence of moderate exercise to a post concussed player
subjectively reporting to be symptom free.
In future RTP clearance and concussion management the authors recommend to
include a post-exertion protocol when testing neurocognitive recovery. This approach is in
line with the stepwise general protocol, based on subjective measures (Johnston et al., 2004)
where the athlete progresses through subsequent and more demanding steps as the athlete
self-reports being symptom free. The difference here is adding accuracy by objectively
measuring cognitive recovery after the player has performed moderate exercise. It would
therefore be interesting to study CMI performance skills and create a CMI task including a
post-exertion protocol.
Another finding across all four studies in this review was the underreporting of
concussions (Hurtubise et al., 2016; Kriz et al., 2017; McGrath et al., 2013; Pedersen et al.,
2014). There can be several explanations for this, for example, players subjectively not
feeling any symptoms or not knowing that their symptoms are related to the injury. Within the
team's norms it may be seen as a weakness to not ‘tolerate’ a tackle, a body checking, or the
consequences from it. Players de-emphasize or conceal nonspecific symptoms that are
subjectively reported (Cantu & Register-Mihalik, 2011; Sicard et al., 2020). Another aspect
could be a financial one. Elite ice hockey involves large amounts of money. Not being able to
play could mean a loss of time, position and income for the team, and for the individual. The
financial aspect could also be an explanation why female players report higher concussion
incidence rates than male hockey players (Hannah et al., 2021) despite bodychecking being
prohibited. In Swedish elite ice hockey females require additional income for complete
financial support. A hypothesis could be that because female elite players do not rely on ice
hockey as financial support, they can afford to say no to playing and choose recovery over
return to play.
In one of the articles (Pedersen et al., 2014) where participants suffered a second
concussion, some cognitive performance measures indicated a decrease in performance whilst
other measures showed an improvement. Authors explain this as an unexpected finding, but
link it to the small sample size (n=4) rather than the ImPACT test battery as a tool. Test-retest
reliability of ImPACT has been studied and is considered good (Elbin et al., 2011; Pedersen et
al., 2014). However, another study (Resch et al., 2018) suggests being aware of the factors
that may influence test performance and understanding the properties of measurement when
making evidence-based decisions in concussion management. In the RTP decision this means17
knowing about a measurement's strengths but also being aware of its limitations. An example
of this is when participants with a second concussion improved their performance on ImPACT
composite scores, it was discussed whether the improvement was due to properties in the
measurement rather than cognitive recovery of the participant (Pedersen et al., 2014). A
strength with the CMI task was its novelty for the participants and that the order of conditions
was randomized (Hurtubise et al., 2016).
An interesting discussion mentioned in Hurtubise et al. (2016) was a potentially
increased CMI reserve in elite ice hockey players. Studies using EEG and functional magnetic
resonance imaging (fMRI) on athletes performing at elite level show neural efficiency
(Nakata et al., 2010; Yarrow et al., 2009). This means that elite athletes show lower brain
activation when performing the same cognitive task as non elite athletes. This efficiency can
be explained with a minimal energy consumption and a more specific neural circuitry in the
brain of an elite athlete. A beginner athlete requires more cognitive processes in order to
perform a movement or skill, meanwhile an elite athlete does this automatically, therefore
requiring less effort (Hurtubise et al., 2016). Depending on the level of skill expertise in elite
players, CMI deficits are reduced (Sergio et al., 2020) suggesting that experience and level of
skill increases the CMI reserve. This is a reason to include individual baseline testing in a
standard protocol not only when using the ImPACT test battery but also when using a CMI
performance task to measure cognitive performance.
NHL players at the top of their performance have during their years probably received
higher levels of medical attention which has been beneficial to them, and which means they
also probably have had a more looked after recovery when sustaining a concussion. The
authors (Hurtubise et al., 2016) discuss whether a potential increased CMI reserve is an innate
characteristic or a product of NHL players training conditions and their environment.
One study (Kriz et al., 2017) showed an average of 24 days for a junior player to
report being asymptomatic post concussion compared to adults reporting 7 to 14 days. The
authors argued that adolescent athletes require a longer recovery time compared to older
players and that younger players are more vulnerable (Kriz et al., 2017). This could add
support to arguments regarding the importance of individual concussion management and
RTP decisions. It also adds to the importance of educating coaches, parents and players that a
younger brain requires a longer recovery time.
Finally it is worth mentioning one article (Brooks, 2007) that was excluded from this
review late in the systematic search process. At initial look it contained eligible criteria
according to PICO but at closer investigation this article did not provide clarity in the data to18
be extracted. The study design seemed to be a case control but could also be interpreted as a
review. The study (Brooks, 2007) did not clarify how two case studies were extracted from a
larger sample, nor how big the sample size was. It described the case extraction in the
following way: “From the large numbers of concussions two case studies are reviewed”
(Brooks, 2007, p. 173). This can be questioned regarding how these cases were chosen and
how large the total sample size of concussed participants was. Further it showed figures with
results from the ImPACT test battery composite scores, but these were not visible to the
reader. Future research in RTP and concussion management would benefit from high quality
articles in which it is possible to follow PRISMA guidelines and extract data according to a
PICO framework.
Ethical and Societal Aspects
All studies had ethics approved. Participants completed informed consent prior to
testing (Hurtubise et al., 2016; Kriz et al., 2017), one study received letters of permission
from a junior hockey programme and 15 schools (McGrath et al., 2013), and one study
contained archival data (Pedersen et al., 2014).
Both the ImPACT test battery and CMI performance task are computer based and
require a hardware computer together with a software program. This means that in order to
complete these measurements an investment needs to be made and updated licences
administered. Also individuals knowing how to administer these tests are required.
Organisations who choose not to invest, or do not have the means to make the ImPACT test
battery or a CMI task part of their general concussion management are likely to have
concussed ice hockey players risking a premature RTP. One of the articles’ (McGrath et al.,
2013) authors were co-developers and co-owners of ImPACT Application, Inc., and a
credentialed ImPACT consultant. This adds bias and a conflict of interest to the study.
Future Research
Future research should aim to implement a protocol for concussion management based
on empirical evidence that uses objective measurements to indicate accurate cognitive
recovery.
It is of importance to separate CMI as a concept, defining the integration between
thought and action, with the CMI task measuring cognitive performance involving thoughts
and action. Hurtubise et al. (2016) used a computer based CMI task to measure cognitive
performance. For future studies this task would need to be validated in the same way
test-retest reliability has been tested for the ImPACT test battery.19
Conducting more studies using a CMI performance task would help understand
cognitive recovery when thoughts and action are integrated. CMI is a skill used when playing
ice hockey and therefore it is of importance that this cognitive function is totally recovered
post concussion. It might be worth studying CMI skills including a post exertion protocol. For
example adding 15-20 minutes of moderate cardiovascular followed by 5 minutes of rest pre
CMI testing. This would be in line with McGrath et al. (2013) who added a post exertion
protocol to ImPACT testing.
More studies regarding optimal decisions in the RTP management based on objective
measures would add value to this research area. Additional research on how post concussion
recovery affects the adolescent versus the adult brain would add to current studies. Finding
measures that are practical and applicable in the ice hockey players environment would also
be of interest. Future research should also investigate whether implementing mandatory
baseline testing or stating acceptable threshold levels will make a difference in RTP decisions.
Limitations
The biggest limitation in this review is the amount of studies included. A total of four
studies resulted from the search in line with PRISMA guidelines. Three studies were related
to the ImPACT test battery and only one study related to CMI performance. The amount of
participants across the four studies is also a limitation. The search was restricted to exclusion
and inclusion criteria according to the PICO framework. Studies related to concussion and
RTP management from other sports were therefore excluded, which limited the amount of
articles in the search process. Another limitation is with regard to outcome. The ImPACT test
battery and CMI performance task both require a baseline score to compare results with.
There are normative values that can be used (Echlin et al., 2010) but as mentioned in the
introduction every concussion is unique (Doolan et al., 2012) with an individual and non fixed
recovery timeline (Johnston et al., 2004; Patel et al., 2005) albeit each RTP decision should be
based on individual baseline scores.
Limitations specified in the four studies were small sample sizes (Kriz et al., 2017;
McGrath et al., 2013), limited representation of age groups, and that a direct comparison
between females and males could not be made due to small sample size (McGrath et al.,
2013). Another limitation mentioned in the studies was participants being excluded as they
were missing ImPACT baseline scores (Kriz et al., 2017). Two studies mentioned that tasks in
the ImPACT test battery may have been influenced by practice which could have increased
performance. They suggested additional studies would be needed with regard to some of the
test modules (Kriz et al., 2017; Pedersen et al., 2014). Further limitations to take into20
consideration were that effects of a concussion on CMI performance only were tested on male
participants and that because concussions in the control group were self-reported it was not
certain that participants in this group had not sustained an unreported concussion (Hurtubise
et al., 2016).
An overall limitation in research regarding CMI tasks on athletes in general, and ice
hockey players in particular, is the same names of researchers involved in all studies.
Conclusion
Main findings from this review show that post concussed ice hockey players
demonstrate cognitive impairments despite reporting being symptom free. It was hypothesized
that combining two objective assessments, ImPACT testing and a CMI task, could contribute
to an optimal objective assessment and be used as an accurate indicator in the RTP decision
when assessing whether a concussed athlete is ready to return to playing ice hockey. A big
limitation in this review was the scarce amount of studies included. Results overall are
limited, especially regarding CMI performance. No studies have been conducted combining
the two cognitive recovery measures. However, findings in this review suggest that adding a
CMI task to the ImPACT test battery could be a way to objectively measure several
composites in cognitive recovery including tasks that require higher levels of cognitive
functioning.21
References
Alsalaheen, B., Stockdale, K., Pechumer, D., & Broglio, S. P. (2016). Validity of the
immediate post concussion assessment and cognitive testing (ImPACT). Sports
Medicine, 46(10), 1487-1501. https://doi.org/10.1007/s40279-016-0532-y
Anto-Ocrah, M., Jones, C., Diacovo, D., & Bazarian, J. J. (2017). Blood-based biomarkers for
the identification of sports-related concussion. Neurologic Clinics, 35(3), 473–485.
https://doi.org/10.1016/j.ncl.2017.03.008
Aubry, M., Cantu, R., Dvorak, J., Graf-Baumann, T., Johnston, K., Kelly, J., Lovell, M.,
McCrory, P., Meeuwisse, W., & Schamasch, P. (2002). Summary and agreement
statement of the first international conference on concussion in sport, Vienna 2001.
The Physician and Sportsmedicine, 30(2), 57–63.
https://doi.org/10.3810/psm.2002.02.176
Benson, B. W., Hamilton, G. M., Meeuwisse, W. H., McCrory, P., & Dvorak, J. (2009). Is
protective equipment useful in preventing concussion? A systematic review of the
literature. British Journal of Sports Medicine, 43 (1), 56–67.
https://doi.org/10.1136/bjsm.2009.058271
Brooks, D. A. (2007). Use of computer based testing of youth hockey players with
concussions. Neuro Rehabilitation, 22(3), 169-179.
https://doi.org/10.3233/NRE-2007-22303
Canadian Academy of Sport Medicine Concussion Committee. (2000). Guidelines for
assessment and management of sport-related concussion. Clinical Journal of Sport
Medicine: Official Journal of the Canadian Academy of Sport Medicine, 10(3),
209–211. https://doi.org/10.1097/00042752-200007000-00012
Cantu, R. C. (1998). Second-impact syndrome. Clinics in Sports Medicine, 17(1), 37–44.
https://doi.org/10.1016/s0278-5919(05)70059-4
Cantu, R. C., & Register-Mihalik, J. K. (2011). Considerations for return-to-play and
retirement decisions after concussion. PM & R, 3(10), 440–444.
https://doi.org/10.1016/j.pmrj.2011.07.013
Clarivate. (2021, March 15). EndNote online login. https://endnote.com
Covassin, T., Elbin, R. J., 3rd, Stiller-Ostrowski, J. L., & Kontos, A. P. (2009). Immediate
post-concussion assessment and cognitive testing (ImPACT) practices of sports
medicine professionals. Journal of Athletic Training, 44(6), 639–644.
https://doi.org/10.4085/1062-6050-44.6.63922
Dalecki, M., Albines, D., Macpherson, A., & Sergio, L. E. (2016). Prolonged cognitive–motor
impairments in children and adolescents with a history of concussion. Concussion,
1(3), CNC14. https://doi.org/10.2217/cnc-2016-0001
Dalecki, M., Gorbet, D. J., Macpherson, A., & Sergio, L. E. (2019). Sport experience is
correlated with complex motor skill recovery in youth following concussion.
European Journal of Sport Science, 19(9), 1257–1266.
https://doi.org/10.1080/17461391.2019.1584249
Delaney, J. S., Caron, J. G., Correa, J. A., & Bloom, G. A. (2018). Why professional football
players chose not to reveal their concussion symptoms during a practice or game.
Clinical Journal of Sport Medicine, 28(1), 1-12.
Donaldson, L., Asbridge, M., & Cusimano, M. D. (2013). Bodychecking rules and concussion
in elite hockey. PloS one, 8(7), e69122. https://doi.org/10.1371/journal.pone.0069122
Doolan, A. W., Day, D. D., Maerlender, A. C., Goforth, M., & Gunnar Brolinson, P. (2012). A
review of return to play issues and sports-related concussion. Annals of Biomedical
Engineering, 40(1), 106–113. https://doi.org/10.1007/s10439-011-0413-3
Echemendia, R. J., Bruce, J. M., Meeuwisse, W., Comper, P., Aubry, M., & Hutchison, M.
(2016). Long-term reliability of ImPACT in professional ice hockey. The Clinical
Neuropsychologist, 30(2), 328–337. https://doi.org/10.1080/13854046.2016.1158320
Echlin, P. S., Tator, C. H., Cusimano, M. D., Cantu, R. C., Taunton, J. E., Hall, C. R., Johnson,
A. M., Forwell, L. A., Upshur, R. E., & Skopelja, E. N. (2010). A prospective study of
physician-observed concussions during junior ice hockey: Implications for incidence
rates. Neurosurgical Focus, 29(5), E4.
Elbin, R. J., Schatz, P., & Covassin, T. (2011). One-year test-retest reliability of the online
version of ImPACT in high school athletes. The American Journal of Sports Medicine,
39(11), 2319-2324. https://doi.org/10.1177/0363546511417173
Gard, A., Lehto, N., Engström, Å., Shahim, P., Zetterberg, H., Blennow, K., Marklund, N., &
Tegner, Y. (2020). Quality of life of ice hockey players after retirement due to
concussions. Concussion, [CNC78]. https://doi.org/10.2217/cnc-2020-0007
Gazzaniga, M. S., Ivry, R., & Mangun, G. (2014). Cognitive neuroscience: The biology of the
mind (4th ed.). W.W. Norton & Company.
Gosselin, N., Saluja, R. S., Chen, J. K., Bottari, C., Johnston, K., & Ptito, A. (2010). Brain
functions after sports-related concussion: Insights from event-related potentials and
functional MRI. The Physician and Sportsmedicine, 38(3), 27-37.
https://doi.org/10.3810/psm.2010.10.180523
Hannah, T. C., Li, A. Y., Spiera, Z., Kuohn, L., Dai, J., McAuley, F., Ali, M., Durbin, J. R.,
Dreher, N., Marayati, N. F., Gometz, A., Lovell, M., & Choudhri, T. (2021).
Sex-related differences in the incidence, severity, and recovery of concussion in
adolescent student-athletes between 2009 and 2019. The American Journal of Sports
Medicine, 036354652110085. https://doi.org/10.1177/03635465211008596
Hesser, H., & Andersson, G. (2015). Introduktion till metaanalys och systematiska översikter.
Studentlitteratur AB.
Hurtubise, J., Gorbet, D., Hamandi, Y., Macpherson, A., & Sergio, L. (2016). The effect of
concussion history on cognitive-motor integration in elite hockey players. Concussion,
1(3), CNC17. https://doi.org/10.2217/cnc-2016-0006
Johnston, K. M., Bloom, G. A., Ramsay, J., Kissick, J., Montgomery, D., Foley, D., Chen, J.,
& Ptito, A. (2004). Current concepts in concussion rehabilitation. Current Sports
Medicine Reports, 3(6), 316–323. https://doi.org/10.1007/s11932-996-0006-3
Kiefer, A. W., Barber Foss, K., Reches, A., Gadd, B., Gordon, M., Rushford, K., Laufer, I.,
Weiss, M., & Myer, G. D. (2015). Brain network activation as a novel biomarker for
the return-to-play pathway following sport-related brain injury. Frontiers in
Neurology, 6, 243. https://doi.org/10.3389/fneur.2015.00243
King, D., Brughelli, M., Hume, P., & Gissane, C. (2014). Assessment, management and
knowledge of sport-related concussion: systematic review. Sports Medicine, 44(4),
449–471. https://doi.org/10.1007/s40279-013-0134-x
Kriz, P. K., Mannix, R., Taylor, A. M., Ruggieri, D., & Meehan, W. P. (2017). Neurocognitive
deficits of concussed adolescent athletes at self-reported symptom resolution in the
Zurich Guidelines Era. Orthopaedic Journal of Sports Medicine, 5(11),
2325967117737307. https://doi.org/10.1177/2325967117737307
McCrory, P., Meeuwisse, W., Dvořák, J., Aubry, M., Bailes, J., Broglio, S., Cantu, R. C.,
Cassidy, D., Echemendia, R. J., Castellani, R. J., Davis, G. A., Ellenbogen, R., Emery,
C., Engebretsen, L., Feddermann-Demont, N., Giza, C. C., Guskiewicz, K. M.,
Herring, S., Iverson, G. L., Johnston, K. M., … Vos, P. E. (2017). Consensus statement
on concussion in sport-the 5th international conference on concussion in sport held in
Berlin, October 2016. British Journal of Sports Medicine, 51(11), 838–847.
https://doi.org/10.1136/bjsports-2017-097699
McGrath, N., Dinn, W. M., Collins, M. W., Lovell, M. R., Elbin, R. J., & Kontos, A. P.
(2013). Post-exertion neurocognitive test failure among student-athletes following24
concussion. Brain Injury, 27(1), 103-113.
https://doi.org/10.3109/02699052.2012.729282
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group (2009). Preferred
reporting items for systematic reviews and meta-analyses: The PRISMA statement.
PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
Nakata, H., Yoshie, M., Miura, A., & Kudo, K. (2010). Characteristics of the athletes' brain:
evidence from neurophysiology and neuroimaging. Brain Research Reviews, 62(2),
197-211. https://doi.org/10.1016/j.brainresrev.2009.11.006
Omalu, B., Bailes, J., Hamilton, R. L., Kamboh, M. I., Hammers, J., Case, M., &
Fitzsimmons, R. (2011). Emerging histomorphologic phenotypes of chronic traumatic
encephalopathy in American athletes. Neurosurgery, 69(1), 173–183.
https://doi.org/10.1227/NEU.0b013e318212bc7b
Parizek, A., & Ferraro, F. R. (2016). Concussions in ice hockey. Current Sports Medicine
Reports, 15(1), 23–26. https://doi.org/10.1249/JSR.0000000000000209
Patel, D. R., Shivdasani, V., & Baker, R. J. (2005). Management of sport-related concussion in
young athletes. Sports Medicine, 35(8), 671–684.
https://doi.org/10.2165/00007256-200535080-00002
Pauelsen, M., Nyberg, G., Tegner, C., & Tegner, Y. (2017). Concussion in Ice Hockey—A
Cohort Study Across 29 Seasons. Clinical Journal of Sport Medicine, 27(3), 283–287.
https://doi.org/10.1097/jsm.0000000000000347
Pedersen, H. A., Ferraro, F. R., Himle, M., Schultz, C., & Poolman, M. (2014).
Neuropsychological factors related to college ice hockey concussions. American
Journal of Alzheimer's Disease and Other Dementias, 29(3), 201–204.
https://doi.org/10.1177/1533317513517036
Piedade, S. R., Hutchinson, M. R., Ferreira, D. M., Cristante, A. F., & Maffulli, N. (2021).
The management of concussion in sport is not standardized. A systematic review.
Journal of Safety Research, 76, 262–268. https://doi.org/10.1016/j.jsr.2020.12.013
Reinsberger, C., & Frick, B. (2017). The impact of mandatory helmet use on career length in
the national hockey league (nhl). British Journal of Sports Medicine, 51(11),
A71-A71. http://dx.doi.org/10.1136/bjsports-2016-097270.184
Resch, J. E., Schneider, M. W., & Cullum, C. M. (2018). The test-retest reliability of three
computerized neurocognitive tests used in the assessment of sport concussion.
International Journal of Psychophysiology, 132, 31-38.
https://doi.org/10.1016/j.ijpsycho.2017.09.011You can also read
