Home » ImPact » Computerized Neurocognitive Testing: Important Role in Concussion Evaluation, Return To Play Decision

Computerized Neurocognitive Testing: Important Role in Concussion Evaluation, Return To Play Decision

 

Computerized neuropsycognitive testing for concussions has become increasingly popular in recent years and have been shown to have value in making the all-important return to play decision.

Because they can detect subtle signs of cognitive impairment indicating that an athlete's brain has not fully healed, even where the athlete claims his symptoms have cleared, the tests are designed to help to protect young athletes against the risk of suffering a second concussion by returning too soon, which can lead to short- and long-term cognitive problems, and catastrophic injury or even death from second impact syndrome.

The current international consensus of experts (Zurich consensus statement),[1] views computerized neuropsychological or neurocognitive (NP) testing as having clincal value in evaluation for concussion and as an aid in determining when it is safe for an athlete to return to play after a concussion, and recommends formal baseline NP screening of athletes in all organized sports in which there is a high risk of concussion (e.g. football, hockey, lacrosse, soccer, basketball), regardless of the age or level of performance.  The National Athletic Trainers' Association 2014 position statement on the management of sport-related concussion[14] says all athletes should "ideally" undergo a preseason baseline assessment, but, that, at a minimum, athletes who are at a high risk of concussion based on their sport should be included in any baseline testing program, with athletes with a significant concussion history, or other relevant pre-existing condition, such as attention-deficit hyperactivity disorder, tested on an individual basis.Hand on computer mouse

Baseline pre-injury and post-injury computerized neurocognitive testing is now commonplace at the professional and collegiate level, and is increasingly used at the high school level as well.

A recent study[2] found that computerized neuropsychological testing was used to assess fully four in ten concussions suffered by high school athletes during the 2009-2010 school year (albeit at high schools with an athletic trainer on staff), up from one-fourth (25.7%) in the short space of just one year.[3]

In the absence of NP and other testing (e.g. formal balance assessment), the Zurich statement says a more conservative approach to return to play approach may be appropriate, especially for children and adolescents.

Computerized tests 

For years the only way to test cognitive function was with so-called "paper and pencil tests".

More recently, computer-generated neuropsychological test programs have been developed and are currently being validated in the sports setting. They include:

  1. Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) developed at the University of Pittsburgh Medical Center (by far the most widely used and studied neurocognitive test);
  2. XLNTbrain Sport,TM which provides neuro-cognitive baseline testing that includes emotional measures and is integrated with compliant education, advanced concussion detection and recovery protocols with an annual subscription.
  3. Computerized Cognitive Assessment Tool (CCAT) marketed in North America by Axon Sports and developed by CogState of Victoria, Australia;
  4. Concussion Resolution Index (CRI) developed by HeadMinder, Inc. of New York, NY; 
  5. Automated Neuropsychological Assessment Metrics (ANAM) system: a PC Windows-based test protocol developed by the National Rehabilitation Hospital Assistive Technology and Neuroscience Center in Washington, DC (Editor's note: a joint investigation by National Public Radio and Pro Publica, which aired on NPR's "All Things Considered" on November 28, 2011, severely questioned the reliability of this program, which was developed and is primarily used by the military);[8] and
  6. Concussion Vital Signs, a web-based neurocognitive assessment platform developed by CNS Vital Signs and marketed by Pearson, which provides a brief, web-based neurocognitive baseline and post-injury testing solution, a self-reported history, mobile-deployed sideline assessment, and a post-concussion symptom checklist.[13]   

Advantages

Computerized tests have four significant advantages over traditional pencil and paper neuropsychological tests:

  • Rapid scoring: tests take less time (more traditional pencil and paper neuropsychological tests usually take an hour or more to administer);
  • Ease of administration: the tests do not need to be administered by a neuropsychologist.
  • Increased test-retest reliability.  Some tests allow for infinite variety in the test questions that reduce the practice or learning effects seen with more traditional neuropsychological test batteries.
  • Greater accessibility: The computerized test batteries are accessible to a wide range of clinicians, including athletic trainers and, even to parents of athletes (although experts strongly caution against use by untrained or undertrained personnel in "cookbook fashion."
One recent study suggests that the tests may also help clinicians predict, when combined with an athlete's scores on the Post-Concussion Symptom Scale in the first 2 to 3 days post-injury, whether a concussion will require protracted recovery (more than 14 days),[5] although the study have been criticized  for using only post-injury test results without comparison to pre-injury baseline test results, and as lacking in practical value in the care of individual athletes which, current concussion management guidelines say, requires individualized assessment and treatment.[4]

Important role 

A 2011 study [7] shows for the first time the important role computerized neuropsychological testing is playing in concussion assessment and return to play decisions. Athletes who had undergone pre-season, baseline computerized neuropsychological testing, and were then re-tested after suspected concussion, were less likely to return to play on the same day, and less likely to return to play within a week of their injury, than the three out of four injured athletes who did not undergo such testing.

The authors suggested three possible reasons:

  1. computerized tests were more reliable in gauging whether an athlete's cognitive functioning (ability to think, concentrate, learn, and reason) had returned to baseline than self-reporting by athletes of signs and symptoms (which, in the interest of a quick return to play, an athlete may downplay or fail to report altogether);
  2. the use of such tests by those providing concussion management leads them to be more conservative in return-to-play decisions; and/or
  3. neurocognitive testing is used more often in cases of severe concussions that require extended recovery times before return to play.

Limitations

Despite their increasingly widespread use, computerized tests faces many of the same challenges as with use of  traditional paper-and-pencil neuropsychological tests, and, because they do not test all the cognitive domains that some specific pencil-and-paper tests measure, have other limitations, including:

  • Questions regarding test reliability: As noted in a recent clinical report by the American Academy of Pediatrics, "one critique of the computerized tests is that the vast majority of studies have been conducted by the developers of the tests themselves, which raises some concern for bias, because some independent study results have suggested slightly less reliable results." [4] The ANAM test has recently came under severe criticism,[8] and the test-retest reliability of ImPACT has also been questioned in a number of recent studies. [10,11]
  • Validity, sensitivity, and specificity in peer-reviewed literature.[6,14] While neurocognitive testing is widely viewed as the cornerstone of the concussion-assement process, when used in isolation, it should never be used by itself, but rather in conjunction with symptom and other (e.g. balance, visual) assessments in diagnosing concussion and the readiness of an athlete for return to play after completing a symptom-limited graduated exercise protocol;
  • Required user training and qualifications:  The Zurich consensus statement says "[n]europsychologists are in the best position to interpret NP tests by virtue of their background and training," while the Centers for Disease Control's FAQs about Baseline Testing [9] states that, ideally, and where possible, a neuropsychologist should interpret the computerized or paper-and-pencil neuropsychological test components of a baseline exam. As for who should administer baseline tests, the CDC was even more emphatic, stating flatly that "baseline tests should only be conducted by a trained health care professional."  While the NATA position statement says an AT can conduct the baseline testing, but is silent on who should interpret the results to determine whether they are valid.  As for interpretation of post-injury data, the NATA says a neuropsychologist or physician with specific concussion training should interpret the data.[14]
  • Hardware and software issues inherent to computerized testing; and
    •
  • User costs.
    •

Children and adolescents: different rules

The Zurich consensus statement recognizes that in the "majority of cases, NP testing will be used to assist return to play decisions and will not be done until [the] patient is symptom free."

For children and adolescents, however, different testing rules may apply:

  • Testing while symptomatic. In contrast to adults and athletes in their late teens, NP testing may be performed while the young athlete is still symptomatic in order to assist in school and home management;[5,6] young athletes need to limit exertion in day-to-day activities and scholastic and other cognitive stressors (e.g. text messaging, video-games, etc.) while symptomatic ("cognitive rest"), which in some cases may require restricting school attendance and extracurricular activities to avoid making symptoms worse.1 Clinical evaluation of such athletes for concussion may also need to include both patient and parent input, as well as teacher and school input, where appropriate. If an athlete experiences post-concussion symptoms over several months (post-concussion syndrome) or has had multiple concussions, formal assessment by a neuropsychologist may be helpful, specifically to identify areas for which the athlete may need academic accommodations.
  • Adjusted for age and maturity level. Whatever cognitive testing is performed, it must be sensitive to the fact that athletes younger than their late teens are not only growing physically, but cognitively, at a rapid rate, which may limit the value of test results when comparing them to either the athlete's own, earlier baseline performance or to what is considered "normal" among his peer population. Note: different rules will apply for children below age 10, because such children report different symptoms - thus requiring a different, age-appropriate symptom checklist [12] as an assessment component.There is currently no established, validated computerized neuropsychological test for the grade school athlete, although a computerized test for use in the athletes younger than 12 years is currently being developed.
  • Repeated annually or every two years.  The Centers for Disease Control [9] recommends that most components of baseline tests (such as balance assessment, and the presence of any concussion symptoms) be repeated annually to establish a valid test result to which post-concussion results can be compared, and that baseline computerized or paper-pencil neuropsychological tests be repeated every two years.  The CDC notes that more frequent neuropsychological testing may be needed if an athlete has a history of concussions or if the athlete has a medical condition that could effect results of the test."  The NATA recommends that athletic trainers conduct annual baseline tests on athletes, if appropriate resources are available, most crucially in adolescents, whose brains are continuing to develop, and in those who have sustained a concussion since their previous test.[14]

1.  McCrory P, et al. Concussion statement on concussion in sport: the 4th International Conference on Concussion in Sport held in Zurich, November 2012. Br J Sports Med 2013; 47:250-258.

2.  Meehan WP, d'Hemecourt P, Collins C, Comstock RD, Assessment and Management of Sport-Related Concussions in United States High Schools. Am. J. Sports Med. 2011;20(10)(published online on October 3, 2011 ahead of print) as dol:10.1177/0363546511423503 (accessed October 3, 2011).

3.  Meehan W, d'Hemecourt P, ComstockD, "High School Concussions in the 2008-2009 Academic Year: Mechanism, Symptoms, and Management" Am. J. Sports. Med. 2010; 38(12): 2405-2409 (accessed December 2, 2010 at http://ajs.sagepub.com/content/38/12/2405.abstract?etoc).

4.  Halstead, M, Walter, K. Clinical Report - Sport-Related Concussion in Children and Adolescents. Pediatrics. 2010;126(3):597-615.

5.  Lau BC, Collins MW, Lovell MR. Sensitivity and Specificity of Subacute Computerized Neurocognitive Testing and Symptoms Evaluation in Predicting Outcomes After Sports-Related Concussion. Am. J. Sports Med. 2011; 20(10).

6.  Broglio SP, Ferrara MS, Macciocchi SN, Baumgartner TA. Test-Retest Reliability of Computerized Concussion Assessment Programs. J. Athl. Tr. 2007; 42(4): 509-514.

7.  Lincoln A, Caswell S, Almquist J, Dunn R, Norris J, Hinton R. "Trends in Concussion Incidence in High School Sports: A Prospective 11-Year Study" Am. J. Sports Med. 30(10) (2011), accessed January 31, 2011 @ http://ajs.sagepub.com/content/early/2011/01/29/0363546510392326.full.pd...

8. Zwerdling D, Sapien J, Miller TC. Military's Brain-Testing Program A Debacle (http://www.npr.org/2011/11/28/142662840/militarys-brain-testing-program-...)(accessed November 30, 2011).

9. Centers for Disease Control and Prevention. FAQs about Baseline Testing among Young Athletes (http://www.cdc.gov/concussion/pdf/baseline_testing_FAQs-a.pdf)(accessed April 16, 2012)

10.  Resch J, et al. ImPact Test-Retest Reliability: Reliably Unreliable? J Athl Tr. 2013;48(3):000-000 doi: 10.4085/1062-6050-48.3.09 (ePub in advance of print)

11. Randolph C, McCrea M, Barr WB. Is neuropsychological testing useful in the management of sport-related concussion? J Athl Train.
2005;40(3):139-152.

12. Child SCAT3. Br J Sports Med 2013;47:263. 

13. Gualteri CT, Johnson LG. Reliability and validity of a computerized neurocognitive test battery, CNS Vital Signs.  Arch Clin Neuropsychol. 2006;21:623-643.

14. Broglio SP, Cantu RC, Gioia GA, Guskiewicz KM, Kutcher J, Valovich McLeod TC.  National Athletic Trainers' Association Position Statement: Management of Sport Concussion.  J Ath Tr. 2014;49:245-265. 

Most recently revised April 6, 2015


Related articles