Alison C. Simcox, Ph.D.

Tufts University, Department of Civil and Environmental Engineering

Medford, MA 02155

Bias in the design of stream-sampling networks is a major cause of inaccurate characterization of water quality at state and national levels. This paper distinguishes the three types of bias that commonly occur in water-quality assessments, and places emphasis on ‘design bias’ associated with location of sampling in watersheds. An environmental index is described that can be used to recognize and reduce design bias, and to provide a consistent means for comparing the overall water quality of watersheds within a region. The index provides a means to differentiate the component parts of a watershed, its subsheds, in terms of two sets of features: natural landscape features and anthropogenic features (‘stressors’). Together, these features largely determine the variability of the quantity and quality of water discharged from watersheds. Index values increase with landscape complexity or anthropogenic stress or both. Spatial bias is reduced by ensuring that each subshed is sampled in proportion to its expected influence on basinwide water quality. Use of the index is demonstrated in a watershed in southern New Hampshire.

Currently, the principal means by which the U.S. Environmental Protection Agency (EPA), Congress, and the public evaluate the quality of water in each state is through information given in reports required under Section 305(b) of the Clean Water Act. These reports provide water-quality information for each of the 305(b) ‘target populations’ within each state: streams, lakes, groundwater, coastal waters, and wetlands. Data for 305(b) reports are obtained from a large number of sampling sites, the locations of which commonly are biased to meet a variety of state objectives, such as compliance monitoring. As a consequence, quantitative and qualitative statements concerning the overall water quality of a 305(b) target population for a state and, ultimately, for the nation may be misleading.

The primary goals of the study (Simcox, 1998) briefly described in this paper were (1) to identify sources of bias in stream-sampling networks in watersheds, (2) to develop methods to identify and reduce bias in network design, and (3) to develop measures to compare the overall water quality of watersheds in a state or specified region.

Sources of bias were identified by reviewing 305(b) reports from eight states: Alabama, Georgia, Maine, Michigan, New Hampshire, Oregon, Washington, and Wisconsin. Based on this review, bias was divided into three types: design, analytical, and statistical (Table 1). As shown in this table, errors in measured values of water-quality variables (e.g., chemical concentration values) are placed into one of two general categories: ‘random variance’ or ‘bias’. Random variance refers to (1) deviations due to natural variation inherent in a sampled population (also known as ‘sampling error’) and to (2) deviations that result from use of imprecise field or laboratory instruments or measurement procedures.

If true population values are over- or under-estimated in a consistent manner, error is classified as ‘bias’. ‘Design bias’ refers to bias associated with sampling design, which prescribes the location and frequency of sampling. When design bias is present, the sampled and target populations will not match; as a result, samples will not be representative of the intended target population. ‘Analytical bias’ refers to bias associated with field or laboratory equipment or protocols. When analytical bias is present, the sampled and target populations may or may not match. If these populations do not match (i.e., design bias is present), analytical bias will add further error. If sampled and target populations are designed to match, analytical bias will be the main source of bias preceding statistical data analysis.

Statistical bias is fundamentally different from design and analytical bias in that statistical analysis generally proceeds from the assumption that all samples consist of random, independent, identically distributed measurements from a common underlying population. There is no concern about whether or not that population is, in fact, the intended (target) population. With statistical bias, the concern is with the statistical estimator of a population parameter, such as sample mean, median, variance, or a quantile, rather than with the similarity between sampled and target populations or with the correctness of sampling and analytical protocols.

Sources of design, analytical, and statistical bias in water-quality assessments are discussed in Simcox (1998) and NCASI, Inc. (in preparation). This paper focuses on design bias, especially bias associated with the location of stations in watershed sampling networks.

Design bias can be attributed to three main factors: spatial design (i.e., location of sampling), temporal design (i.e., sampling frequency/times), and scale effects. Bias due to spatial design of a sampling network is common in 305(b) programs because states generally do not attempt to obtain observations representative of an entire 305(b) target population, such as all of a state’s streams or lakes. Rather, they use a ‘targeted monitoring’ approach in which sampling sites are selected according to the purpose of monitoring, such as compliance monitoring or assessment of water quality in an area of special ecological value. If only these data are used to make inferences about the entire 305(b) population, such as a stream population of a watershed, two consequences are likely: (1) an estimate of average population values (e.g., mean or median concentration values of water-quality constituents) may be biased if characteristics of the observed part of the population differ from those of the unobserved part, and (2) an estimate of variance of the population average may be incorrect.

Spatial bias may also result if data are spatially correlated. This can occur when observations are taken upstream and downstream on the same waterbody so that the upstream sample forms part of the downstream sample. Although spatial correlation of data between stations in many watershed stream-monitoring networks is minor because of the dispersed nature of sampling locations, consequences of correlation can be serious. Many statistical tests assume that data are independent so that these tests cannot be used or have less power when observations in a dataset are correlated. If correlation is present, hypothesis tests used to compare groups of data, such as data from two waterbodies or from two time periods, and trend tests may lead to the conclusion that differences and trends exist when they, in fact, do not. Moreover, the width of a confidence interval about the mean (e.g., mean constituent concentration value) will be larger for correlated than for uncorrelated data because a correlated dataset provides less information about the mean than does an independent dataset of the same size. Similarly, if variance of the mean is estimated for a correlated dataset using an estimator that assumes independent data, variance may be seriously underestimated so that the sample mean appears more precise than it actually is (e.g., Helsel and Hirsch, 1995; Haan, 1977).

State water-quality assessments may also contain bias due to the frequency or time of sampling (i.e., temporal design) at individual stations. Temporal-design bias may be caused by sampling only during specified flow conditions (e.g., low or high flow) or times of the year (e.g., once each season), or even by sampling at random times. In addition, as with spatial data, if temporal data from individual stations are correlated, use of statistical tests can result in significant bias. For estimates of mean annual water quality, sampling only during low or high flows will tend to emphasize constituents that generally occur in higher concentrations during those periods and de-emphasize those that occur in lower concentrations. The concentration of many water-quality constituents is correlated with streamflow and reflects the uneven distribution of storm events throughout the year. Because of this, distributing sampling throughout the year will not ensure that estimates of annual mean values are unbiased. As noted by Helsel (1995), random or arbitrary selection of sampling times may also result in bias by over-emphasizing commonly occurring lower streamflows. Helsel (1995) recommended that sampling be done more frequently during periods of greater expected variability in the concentration of water-quality variables of interest. Because expected variability may differ considerably among a suite of water-quality variables, some researchers suggest that those requiring more or less frequent monitoring than other variables be treated separately (e.g., Ward, R.C. et al., 1990). For example, many variables associated with biomonitoring such as pesticide residues can be monitored with less frequency or at different times than variables associated with water chemistry.

A less obvious source of design bias in water-quality assessments is bias associated with scale. Mixing data from various spatial scales can lead to invalid conclusions because environmental processes commonly operate differently at different scales. For example, the water-quality effects of a particular land use in a large watershed with a large number of land uses are unlikely to be the same as those in a small watershed with a small number of land uses. Another example is given by Helsel (1995), who noted that simple random sampling over a broad region tends to result in greater representation of smaller watersheds and, therefore, a greater emphasis on the water-quality effects of land uses in those watersheds.

A method was developed to help water-resource managers identify bias in sampling design and to prioritize watershed areas for sampling when the sampling objective is to assess overall water quality of a watershed. The method also provides a means for aggregating water-quality data from subsheds into basinwide water-quality measures; when the method is repeated for a group of similarly scaled watersheds, consistent and comparable water-quality measures are produced.

The method uses environmental indices, tools that are increasingly being used for management decision-making. The indices are developed on simple zero to one scales and provide a means to differentiate the component parts of a watershed, its subsheds, in terms of two sets of features: natural landscape features and anthropogenic features (‘stressors’). Together, these features largely determine the quantity and quality of water discharged from each subshed throughout the year and, thus, the relative influence of each subshed on overall quantity and quality of water discharged from the watershed. Subsheds with greater influence have higher index values, indicating greater natural landscape complexity or anthropogenic stress or both, than subsheds with lower index values. Spatial bias is reduced by ensuring that each subshed is sampled in proportion to its expected influence on basinwide water quality.

The intent was to develop indices that can be easily understood by water-resource managers. Assessment of water quality in streams is a multidimensional problem because water quality at any specified stream location is the product of the interaction between many environmental factors, including soil type, vegetation, biological activity, climate, precipitation, topography, land use, effluent discharge, channel size, basin size, and other factors. To produce useful indices, it was necessary to reduce the dimensionality of the problem by selecting a subset of environmental factors that commonly cause water quantity to vary and water quality to be degraded. Reducing dimensionality allows data requirements to be kept to a manageable level. It also avoids overwhelming decision-makers with dozens of indicators, many of which are likely to be correlated and, therefore, contain redundant information.

The indices also were developed to be compatible with a particular approach to water-quality assessment. This approach is a watershed-based approach advocated by the EPA (U.S. Environmental Protection Agency, 1997) for the 305(b) program in which ‘waterbodies’ are defined as subshed areas (11-digit or 14-digit Hydrologic Unit Code (HUC) watersheds) comprising a larger watershed. If subshed areas are at the 11-digit HUC scale, the larger watershed is an 8-digit HUC watershed; if subshed areas at the 14-digit HUC scale, the larger watershed is an 11-digit HUC watershed. The 8-digit HUC watershed is the more likely choice for the larger watershed area since these are the units used by most states for water-resource planning.

The Contoocook River watershed in southern New Hampshire (Figure 1), a tributary watershed of the Merrimack River watershed, is used to demonstrate development and use of environmental indices. Spatial data at a scale of 1:24,000 for this watershed were linked and analyzed using GIS software (ARCVIEW™) with the spatial analyst extension to support raster modeling.

The Contoocook River watershed is an 8-digit HUC watershed comprised of six 11-digit HUC subsheds, which are referred to by the last three numbers of their hydrologic-unit code (Figure 1). The watershed covers about 764 square miles and elevations range from over 3,000 feet in the southwest to almost 300 feet in the northeast. The geology is typical of the region, with crystalline bedrock overlain by Pleistocene-aged glacial deposits, which form productive aquifers along river valleys. Like other watersheds in the region, most land area is forested and rural, with agriculture, industry, and population concentrated along river valleys.

An environmental index, composed of a ‘landscape subindex’ and a ‘stress subindex’, was developed for each of the six subsheds of the Contoocook River watershed. As shown in Figure 2, each subindex was derived by combining several measures of watershed features and adjusting the value of the subindex so that it ranged from zero to one. Although three measures were used for each subindex in this case study, the method is flexible and the number of measures could be increased or decreased.

The main assumptions used in developing the landscape subindex are that, in the absence of anthropogenic activity, subsheds with similar landscape features behave similarly hydrologically (as indicated by streamflow hydrographs) and have similar water quality at similar stream discharge (as indicated by constituents that show a strong correlation with stream discharge). For the case study watershed, which is underlain by relatively unreactive crystalline rocks, it was also assumed that geology is not the dominant control on stream chemistry. This assumption will not be true in some watersheds; however, the procedure for developing indices is flexible and could be modified to include geology.

The first step was to simplify digital landscape data. Using a GIS, land cover was generalized into two categories, forested and open; topographic slope was divided into two slope categories, flatter and steeper; and elevation data were divided into four equal-sized quarters (elevation quartiles). The number of categories is not fixed and could be modified in other watersheds. The slope categories indicated the relative steepness of land area through the watershed. Another measure of slope, average stream-channel slope, was considered, but had insufficient power to discern physiographic differences between subsheds. The procedure for defining elevation categories was chosen because it is a statistically robust procedure that is not overly influenced by extreme elevation values.

Simple GIS functions were used to calculate the percentage of area within each land cover, topographic slope, and elevation category for each of the six subsheds. These percentages were entered in tables, such as those shown below for subshed 010:

The relationship between topographic slope and land cover was not as useful for characterizing subsheds in this watershed as it might be in other watersheds. The slope information, however, was useful for producing a measure of landscape complexity based on the assumption that complexity increases with the proportion of steeper to flatter areas. First, the standard deviation of entries in each of the six tables containing the slope variable was calculated. These values were normalized onto a scale of zero to one using the following operation: (s-s_min)/(s_max-s_min), where s is the standard deviation of all entries in a table for a given subshed, s_min is the lowest s for the set of six tables, and s_max is the highest s for the same set. To meet the complexity assumption, a slope value for dividing flatter from steeper areas was selected so that the proportion of flatter to steeper areas was less than one for each subshed. This is necessary because the procedure measures variability in regard to magnitude, but not position, of entries in a table.

For the elevation table, landscape complexity was assumed to increase as the proportion of area within each elevation quartile becomes more similar. To meet this assumption, the normalized elevation measure for each subshed was calculated using the compliment of the equation given above, (s_max-s)/(s_max-s_min), so that higher values of the measure represent an increase, rather than a decrease, in landscape complexity.

In addition to normalized measures of topographic slope and elevation, the landscape subindex also included a normalized measure of subshed size. This measure was simply derived by rescaling the measurement units used to describe subshed area (square miles), so that they fell within a range of zero to one. The three normalized measures were combined using equal weighting to yield a landscape subindex for each subshed. Results were found to be insensitive to unequal weights and equal weighting was deemed reasonable.

Three stress factors, population stress, point-source stress, and nonpoint-sources stress were developed to assess the influence of human activities within the Contoocook River watershed. Population stress has previously been used by the U.S. Geological Survey (USGS) (e.g., Meade, 1995) and is defined as the ratio of human population upstream from a stream location to mean annual streamflow at that location. The inverse of this stress factor can be thought of as ‘per capita annual streamflow’, or the amount of water in the stream that is theoretically available on an annual basis to each person in a subshed. This stress factor gives only a rough indication of potential impact of population on water quality because, although water quality tends to decline in heavily populated areas, it may also decline in areas of agricultural, silvicultural, industrial, or mining development that have sparse human populations.

Mean annual streamflow of each subshed of the Contoocook River watershed was estimated by transferring streamflow information from USGS gaging stations to the outlet of each subshed using a drainage-area-ratio method described in Hirsch (1979). Population in each subshed was estimated from 1990 U.S. Census Bureau data. Values of population stress were normalized by rescaling values to range from zero to one. Not unexpectedly, the three subsheds containing the main stem Contoocook River had the highest population stress.

Point-source stress is the ratio of mean annual discharge from industrial and municipal point sources in a subshed to the mean annual streamflow for that subshed. This measures the proportion of annual streamflow in a subshed that potentially contains pollutants from reported point sources. Mean annual streamflow for each subshed was previously calculated, and mean annual discharges from facilities with National Pollution Discharge Elimination System (NPDES) permits were obtained from the EPA’s Permit Compliance System (PCS) database. Normalized values for point-source stress were derived by rescaling values to range from zero to one. Like population stress, the highest values of point-source stress were for subsheds containing the main stem Contoocook. The lowest point-stress value, however, corresponded to the subshed containing the lowermost reaches of the main stem Contoocook.

Nonpoint-source stress is simply the percentage of subshed area that has agricultural or silvicultural land use or is heavily settled, land characteristics that are commonly identified as significant contributors to nonpoint-source pollution. Definition of a ‘settled’ area will vary depending on the overall population density of a region. For the Contoocook River watershed, a ‘settled’ area was defined as one that has a population density of 1000 or more people per square mile. Areas of silvicultural development were not available for the Contoocook River watershed so could not be included in calculations of nonpoint-source stress.

Point-source and nonpoint-source stress were combined using equal weighting to yield a stress index for each subshed. Population stress was excluded from the calculation because the nonpoint-source stress measure included information about population distribution. Calculation of population stress was still worthwhile, however, as an indicator of the potential impact of population on water quality. In addition, the GIS datalayer based on U.S. Census Bureau data was the best source of information for estimating extent of ‘settled’ areas for calculation of nonpoint-source stress.

Landscape and stress subindices were combined to produce an overall environmental index for each subshed. As sensitivity to weights was relatively low and as there was no compelling reason to weight subindices unequally, equal weighting was used.

Uses of the environmental indices are described below. It should be noted that the separate components, the landscape subindices and the stress subindices, also are useful. These components provide indicators of expected natural water-quality variability (landscape subindices) and of vulnerability to surface-water pollution (stress subindices) throughout a watershed. As described below, stress subindices were particularly useful for revealing the presence of spatial bias in an existing sampling network in the Contoocook River watershed. For some watersheds, it may only be necessary to produce landscape subindices or stress subindices. Use of the landscape subindex alone is equivalent to assigning little weight to anthropogenic activity. Such an approach might be appropriate for design of sampling in a watershed that is largely comprised of wilderness areas. Similarly, consideration of the stress subindex alone gives little weight to the impact of the natural landscape on water quality. This approach may be justified for watersheds that are largely comprised of urbanized, industrialized, silvicultural, mining, or agricultural areas.

The environmental indices were used to assess spatial design bias in a sampling network used by the New Hampshire Department of Environmental Services (New Hampshire DES) (Figure 1). The spatial pattern of stations has evolved over a number of years, with many stations being sampled only occasionally and for various objectives. In general, the focus of sampling has been along the main stem Contoocook in subshed 010 and near locations of NPDES facilities. Environmental-index values were used to indicate spatial-design bias that might occur if water-quality data from all stations shown in Figure 1 were used to obtain a basinwide measure of water quality. Table 2 compares the existing distribution of stations among subsheds to a distribution that results from assigning stations to subsheds in proportion to subshed weights (i.e., index values).

Even though subshed 010 has the highest index value and, therefore, a high sampling priority, there is clearly a higher proportion of stations in this subshed than are needed to obtain a basinwide water-quality measure. The number of stations is, in fact, close (13 versus 12 stations) to the number that results from a proportional allocation using point-source stress alone.

Sampling-design decisions include determining where to locate sampling stations and where to increase sampling frequency. As shown in the previous section, index values can be used as subshed weighting factors to prioritize subsheds for sampling. The first sampling station is assigned to the subshed with the highest index (i.e., weight); additional stations are assigned to subsheds in proportion to subshed weights. As well as indicating where to locate stations as they become available, the index also indicates where the most benefit can be gained by increasing sampling frequency. For example, sampling frequency commonly is uniform throughout the sampling network and is performed throughout the year, at least on a quarterly basis. If sampling frequency can be done more often, say monthly, at some stations in the network, these stations can be selected according to subshed weight.

An alternative to using index values to assign weights to subsheds is to use the index values to assign weights to water-quality data from each subshed. In this case, data are aggregated by multiplying water-quality results representative of each subshed by the weight for the associated subshed and summing over all subsheds. This takes the following mathematical form:

where,

is a basinwide measure of water-quality for variable x,

is the mean annual value of the water-quality variable x in subshed i,

Measure (for each subshed) Normalized Measure

Slope/Landcover Standard Deviation of Matrix |----------------|

Evaluation 1 0

of

Landscape Elevation Standard Deviation of Matrix |----------------|

Features 1 0

Subshed Size Area |----------------|

1 0

Landscape subindex

Population {|----------------| }

1 0

Evaluation of

Anthropogenic |----------------|

Features 1 0

Nonpoint % Area Agriculture, |----------------|

Sources Silviculture, Settled 1 0

Stress Subindex

|----------------|

Evaluation of 1 0

Landscape and |----------------|

Anthropogenic 1 0

Features

Random Variance (Precision) Bias

*Both spatial and temporal bias

(New Hampshire DES, 1997)