Counting Steelhead Comes with Lots of Error: But it Took the Approach Created by an 18th Century Minister Named Thomas Bayes to Help Us Better Appreciate Those Errors
Who in our vast steelhead angler community has not been disappointed at one time or another with a steelhead run forecast when the number of fish running upriver ended up being much lower – or on rare occasions, higher – than the forecast used to set fishing regulations? Wild Steelhead Coalition Science Advisor, Guy Fleischer, has thoughts on how some level of uncertainty is unavoidable and a natural characteristic of fisheries modeling, and what is needed to better understand steelhead run forecasts. If you are ready to take the plunge with Fleischer and wade deep into what affects forecast model performance, keep this in mind: without reliable forecasts and the underlying data needed to construct and explain them, policy makers are more challenged to defend responsible fisheries management. Here Fleischer explains how moving to modern probabilistic models that better capture and display uncertainty actually leads to more clarity of what is (or is not) likely driving the trends seen in steelhead abundance. As an engaged angler, the take home is that these steelhead models now offer a tool to identify key influences of all those factors involved in the survival and productivity of steelhead that will lead to increasingly more informed management decisions.
/ / / / / /
By Guy Fleischer, WSC Science Advisor
Everyone understands the concept of error – at least in the most basic sense of making a guess and being wrong. Pessimists and those unfortunate souls who lost a bet can only see the outcome as only a mistake. Right or wrong. Black and white. Close, but no cigar.
Scientists don’t see it that way. In fact, scientists embrace error. When reading scientific literature, one will run across terms like “confidence intervals”, “bias”, “probability” or “likelihood”, all of which can be more broadly described as “uncertainty.” Not that they like being wrong, but whenever scientists need to make any kind of prediction, they actually strive to understand, to the best possible degree, the errors involved in all the parameters or observations as contributors to the overall uncertainty. These errors can affect both the precision (how reliable) and the accuracy (how close to the truth) of the prediction. Ignorance is not bliss with scientists.
Using Models to Count Fish
Measuring the size of a population over time is - in theory - as simple as tallying the fish and then subtracting those that are caught and those who died of natural causes. The surviving adults are then able to reproduce and continue the next generation’s life cycle. But most anglers or other observers know it is hardly that simple in practice. Counting fish, or even some index or proxy (such as redd counts), is rarely easy. Determining the absolute number that were caught or died from natural causes is another challenge. Predicting the number of offspring from a given number of spawners - the principal goal of fishery management - is a whole other feat.
In all instances, to derive an estimate to predict the size of any fish population, one needs some type of systematic approach to processing available data. These approaches are called models.
Theoretical ecology, which includes fisheries, is founded on the concept of the “model.” Models are a mathematical formula - or more broadly, an algorithmic construct - designed to simulate the dynamics observed in nature. For fish, it should not come as a surprise that there are almost as many models in use as there are fish in the ocean! Scientists choose the model best suited for a particular fish stock’s life history and may, in some cases, apply multiple models to find the best possible fit for a particular population.
Despite different approaches, all of these models are based on three primary categories of information: catch rates, abundance, and biology or ecology. Key point here is that in many cases this depends on data availability – the more data, the more options scientists have to better describe the true state of a fish population.
When managing steelhead populations, scientists and fishery managers rely on annual measures of spawner abundance, mainly numbers of returning adult fish (“stock”), and a measure of the offspring produced by those spawners (“recruitment''), typically the mature fish from each previous brood year that returned to spawn. This stock-recruitment relation is vital to characterizing a steelhead population’s productivity. This estimate is essential in order to establish some level of escapement goal to allow sustainable harvest including catch-and-release impact levels.
Ricker and Beverton-Holt
Two stock-recruit models are principally used for steelhead: the Ricker curve and the Beverton-Holt curve, named after the scientists who first used population theory to derive functions to describe these important reproductive relations. From both theory and observation, it is understood that the level of recruitment will increase with increasing numbers of spawners up to some point and then begin to level off due to natural population limitations such as saturation of spawning habitat or other habitat constraints. It is also possible for recruitment to decline at higher spawner numbers as represented with the two curves on the graph below. (This is shown in the Ricker Recruitment Curve.)
Basically: watersheds and populations have maximum carrying capacities based on the amount of available habitat, food, water, etc. Sorry, no trout unlimited!
(To be fair, most of our steelhead rivers are not being filled to this capacity. It is the primary reason to protect wild fish and make sure they are able to fully utilize the habitat that is available and especially good reason to restore lost habitat to the greatest degree possible).
For any given model, without going into the gory computational details, the mathematical function is ‘fit’ to the available data, and from this fitting process, the shape of the curve is derived. It is this critical shape, and the resulting model parameters, that translate to the biological reference points for a fish population. These reference points are then used as the basis for setting harvest levels intended to maintain a sustainable level of recruitment.
Below is one such example used previously for Skagit River wild steelhead.
Stock-recruitment analysis for Skagit River steelhead with Beverton-Holt function (red line) and the family of curves derived by adjusting the number of points used in the analysis (gray area). Source: Skagit River Steelhead Fishery Resource Management Plan.
What is seen in this example, and common to many other stock-recruit analyses, is the use of a deterministic model (meaning it does not include elements of uncertainty) applied where run estimates span only a portion of the full range (termed a lack of contrast), but also where the data points are highly scattered, like a shotgun pattern.
As a result, some of the results derived from the analysis (represented by the gray area of the graph), predict high recruitment with little or no spawners! These results are problematic and can make the results difficult to defend, not just from the points raised above, but more importantly because of a huge violation of the assumption of a constant relation between stock and recruitment over the time period – a rare occurrence in nature.
Every undergraduate in fisheries science is warned about these problems with such models, yet many examples of such limited analyses persist in managing important fisheries.
More Than Just Number of Spawners
It is obvious that something besides just the numbers of spawners is influencing the subsequent trends in the number of recruits. In any population, it is likely that one or more external natural or human-induced processes are involved and influencing the results. Then add errors and omissions in the imperfect measures of stock and recruitment. As we in the field like to say: this is not rocket science – it is much more complicated!
Probabilistic models that include some of these additional factors are needed to get a result that more closely tracks with the complexity of nature.
Enter the approach originally developed in 1763 by Reverend Thomas Bayes now known as Bayesian statistics. Bayesian inference interprets probability as a measure of confidence, or likelihood, that an individual may possess about the occurrence of a particular event or process. This is in contrast to conventional forms of statistical inference (known as classical or frequentist statistics) which assumes that probabilities are the frequency of particular random events occurring in a long run of repeated trials. Frequentist statistics tries to eliminate uncertainty by providing estimates. In contrast, Bayesian statistics tries to preserve and refine uncertainty by adjusting individual theories in light of new evidence, providing a solid mathematical means of incorporating initial or prior beliefs, and evidence, to produce new resulting or posterior beliefs.
Bayesian methods were, up until recently, not embraced by all mathematicians and statisticians due to philosophical, not to mention practical, considerations: Bayesian methods examine the relations probabilistically, by systematic random sampling from high-dimensional probability distributions, requiring a higher level of computations to complete. However, with new, more powerful computers and new algorithms, Bayesian methods have been increasingly used in broad statistical and modeling applications and offer powerful methods to critique a particular model including evaluations of both model assumptions and model predictions, and for the comparison of models, including model selection or model averaging.
Essentially, Bayesian models allow man-made and natural factors to be considered when estimating productivity of fish populations, and they result in a range of estimates rather than a single, concrete answer. These models let managers see uncertainty and make it a part of their decision-making.
Making Models That Look More Like the Natural World
OK, so, now what to do about the obvious issue of those nagging problems of all the other factors affecting recruitment (called covariates) combined with imperfect and noisy stock and recruitment data? Steelhead researchers and fishery managers recently have turned towards Bayesian Integrated Population Models (IPMs) as a means to grapple with the combined uncertainty in the data sources and covariates. This approach, already used widely in conservation research for birds and mammals, includes describing both the seemingly erratic and the unobservable population dynamics, as well as addressing incomplete data.
Enough of the academic geek stuff. In actual practice, researchers were able to analyze the incomplete data on steelhead abundance, age composition, and harvest with a Bayesian IPM to resolve important questions relevant to management of the threatened population of Skagit River steelhead. Scientists were able to produce an updated understanding of wild steelhead stock-recruitment in the Skagit River by including several candidate ecological factors that influence wild steelhead recruitment, namely maximum of daily peak flows during October to May (the first freshwater rearing year), the minimum of low summer flows occurring from June to September (the first summer of freshwater rearing), the average North Pacific Gyre Oscillation index (NPGO) from January to December (an index of conditions experienced by juvenile steelhead during their first year in the ocean), and numbers of hatchery fish released in the spring (April or May) in the Skagit Basin. The results are shown below.
Recent reanalysis of Skagit River steelhead stock and recruitment. Depicted (a) are the derived error bars for the individual points that include the influences of external covariates. Grey lines (a) show the ensemble of stock-recruitment relations based on model estimates over the entire time series. Also shown are the estimated posterior distributions for the intrinsic productivity (b) and carrying capacity (c), the parameters that determine the reference points derived from the stock-recruitment curve. Applying the median values derived in (b) and (c) produces the curve shown here (a) as a solid blue line. Source: Scheuerell, MD, et al.. An integrated population model for estimating the relative effects of natural and anthropogenic factors on a threatened population of steelhead trout. J Appl Ecol. 2021; 58: 114– 124. https://doi.org/10.1111/1365-2664.13789
To the layperson, this fuzzy picture above may not necessarily be seen as an improvement!
A close look, however, will reveal how the data points in the new models have reoriented compared to those shown in the earlier graphs. This is the result of taking into account those other ecological factors that are influencing recruitment instead of only depending on the number of spawners for the model.
Also, there is not a single result from the model, but an ensemble of resulting curves (similar to what is seen by meteorologists predicting the possible path of hurricanes). Ironically, these fuzzy results actually lead to more clarity of what is (or is not) likely driving the trends in recruitment and thus allows for more informed management decisions. Avoiding the full view of uncertainty did offer comfort in previous models, but to scientists, these new results offer greater insights and an improved sense of the overall suite of measurement and natural process errors involved.
In addition to a more defensible derivation of the biological reference points from the stock-recruitment relation, the IBM approach in this case was also able to document the significant influence of changing ocean conditions with declined wild steelhead productivity (see: Oceans of Change: Recent Conditions in the North Pacific Are Dramatically Affecting Steelhead Survival — Wild Steelhead Coalition ), show a strong influence of minimum low summer flows, and also implicate a negative influence on wild steelhead from introductions of hatchery fish (a result that is widely reported by several other studies).
In contrast, the new model and analysis revealed that maximum daily peak flows during October to May were not found to be influential to steelhead recruitment
The take home point for managers and anglers is that despite incomplete information about the abundance and age structure of an at-risk population of steelhead, the researchers were able to improve the understanding of steelhead density-dependent population dynamics in light of both natural and human-induced variability in the environment, which is a good step in the right direction for the management of Skagit River steelhead. In addition, these results not only describe this particular watershed but are also useful in application to assess other wild steelhead populations (through a standard modelling process called meta-analysis).
Embracing Known Unknowns
Understandably, this type of analysis takes place in an environment of agency decision making. Uncertainty can be difficult to openly describe to both managers and the angling public, both of whom are often not familiar with the science being used.
A model’s prediction that exhibits uncertainty in its full splendor is possibly deemed sloppy or squishy science, particularly by those not happy with the results. Or since these models now offer a range of results, some level of judgement needs to be involved. Political pressures by constituent groups may cloud the judgement of managers.
But, it is better to have the uncertainty included and documented by the model. It is far worse when the uncertainty is ignored when setting fish harvest levels, because that often leads to overexploitation of a stock or regulations built on incomplete understanding of a population’s ability to successfully reproduce itself.
Scientific uncertainty can be, and often is used, as a political handle. However, in this type of approach, the uncertainty should actually help managers and stakeholders identify key influences on the survival and productivity of an ecologically and economically important fish population (especially in a case such as the Skagit where the steelhead are at risk).
In addition, the application of IPMs can also be used to identify those data collection priorities that could go a long way to reduce uncertainty in future estimates if those missing data were included. This approach is invaluable as a guide for future research where the focus can be placed on the costs and benefits of different data types used in formal status evaluations of wild steelhead populations.
/ / / / / /
Further Reading
Bayesian Statistics: A Beginner's Guide | QuantStart