Quest for a discard survival predictive scoring system to use on board fishing vessels

Releasing tagged Atlantic cod, John Clarke Russ

The European Union Common Fisheries Policy (CFP) ban on discarding allows for animals to be discarded if “scientific evidence demonstrates high survival rates”. Estimating discard survival for fisheries has become a priority for implementation of the CFP. Limited data on discard survival and mortality is available and methods for estimation have not been standardized. Ideally, a standardized numerical scoring system can be developed and validated, based on readily observable responses and symptoms present in animals that are candidates for discarding and survival. 

RAMP is an example of a predictive scoring system for vitality, survival, and mortality, based on animal reflex actions, barotrauma symptoms, and injury that can be observed in fishing operations where real time decisions must be made about potential discarding. See post for RAMP development and validation; also Davis (2010) and Stoner (2012) for reviews of RAMP method. Other uses for RAMP are in live fisheries, aquaculture, and pollution research and monitoring.

For inspiration and alternative perspectives, examples of validated mortality predictive scoring systems can be found in human and veterinarian intensive care unit (ICU) settings, where patients present with symptoms and disease likely to result in morbidity and mortality (Rockar et al. 1994, Bouch and Thompson 2008, Timmers et al. 2011). Measurements of blood plasma and urine variables commonly made in ICU settings are not contemplated for RAMP since they are not readily made on board fishing vessels.

Below is an example of human ICU mortality prediction using the SAPS II scoring system. Note the similarity to RAMP curves for mortality prediction.

SAPS II mortality predictive scoring system, ClinCalc

Celinski and Jonas (2004) discussed scoring systems developed for the human ICU environment:

“How are scoring systems developed? All available data types and variables can potentially be used to create a scoring system. However, to make it useful, variables have to be selected to be appropriate for the predictive properties of the scoring system. The information must be unambiguous, mutually exclusive, reliable and easy to determine and collect. Ideally, the variables should be frequently recorded or measured.

The variables can be selected using clinical judgement and recognized physiological associations or by using computerized searching of data (collected from patient databases) and relating it to outcome. The variables are then assigned a weighting in relation to their importance in the predictive power of the scoring system (again by using clinical relevance or computerized databases).

Logistic regression analysis, a multivariate statistical procedure, is then used to convert a score to a predicted probability of the outcome measured (usually morbidity or mortality) against a large database of comparable patients. Lastly, the scoring system must be validated on a population of patients independent from those used to develop the scoring system.”

For discard survival prediction, groups of animals, rather than individuals, are the appropriate unit for consideration since proportion mortality is the determined outcome during index development and validation. These groups can represent various scales of resolution in fisheries of interest, i.e., single tows or traps, sets of longline, trap, or gill-net, daily catch.

Jean-Roger Le Gall (2005) discussed the appropriate use of ICU severity scoring systems:

“A good severity system provides an accurate estimate of the number of patients predicted to die among a group of similar patients; however, it does not provide a prediction of which particular patients will in fact die. Using a well-calibrated severity model, we can reasonably expect that approx. 75% of patients with a probability of mortality of 0.75 will die, but we cannot know in advance which of those patients will be among the 25% who will live. Furthermore, these 25% will not have falsified the odds but will have confirmed the validity of the probabilities. 

       The possibility that clinical decisions can be augmented by having an objective (although not always more accurate) assessment of a patient’s severity of illness is appealing. Physicians are interested in severity systems for individual patients as an adjunct to their informed but subjective opinion. Using these tools as part of the decision-making process is reasonable and prudent. Using these tools to dictate individual patient decisions is not appropriate. Decisions will and should remain the responsibility of the individual physician and should be based on a number of criteria, one of which is severity as estimated by a well calibrated scoring system.”

Stacy et al. (2013) discussed development and appropriate use of a predictive scoring system for survival in Kemp’s ridley sea turtles:

“Three mortality prediction indices (MPI) scoring systems were developed using different combinations of blood analytes, with anticipation that at least one of the three would be more accurate in predicting mortality in sea turtles within 7 days after admission. Turtles with higher scores were categorized as physiologically deranged to a degree that could result in death, and turtles that received lower scores were categorized as physiologically stable and likely to survive. Categorization of each turtle was then compared to the known outcome for that individual.

Receiver operating characteristic (ROC) analysis was used to assess the diagnostic performance of each MPI scoring system (Greiner et al., 2000; Giguere et al., 2003). The ROC analysis produces a plot that is used to estimate the area under a ROC curve, which is a summary statistic of diagnostic accuracy. A perfect test [i.e., sensitivity (SE) = 100% and specificity (SP) = 100%] will produce an area under the curve (AUC) = 1. The AUC can be used to distinguish a non-informative test (AUC = 0.5), a less accurate (0.5 < AUC ≤ 0.7), moderately accurate (0.7 < AUC ≤ 0.9), highly accurate (0.9 < AUC < 1), and perfect test (AUC = 1)."

ROC analysis of Kemp's ridley sea turtle mortality predictive scoring system, Stacy et al. 2013

"It is clear that results of mortality prediction indices (MPI) scoring systems cannot be used indiscriminately to make euthanasia decisions, because this would result in euthanasia of some turtles with a falsely positive MPI score that would otherwise survive. As with other health scoring systems in human and veterinary medicine, the MPI scores should not prevent clinicians from providing care to an individual, and euthanasia decisions should only be made in light of numerous other clinical factors, including neurological status, vision, ability to forage, ability to swim, pain and suffering, and duration of illness. Finally, MPI scores may be useful when applied retrospectively in a stranding event for comparison of various treatment outcomes within a facility or among different facilities. Thus, the MPI could provide an objective assessment tool of treatment success and contribute to the advancement of medical care in sea turtles.”