DRAFT – DO NOT CITE

 

The DQO/MQO process for comparability in monitoring: 

nitrate as an example

 

Katherine Alben, Jerry Diamond, Larry Keith, and Charlie Patton

Version 1.4,  Feb 5, 2003

 

1.      Introduction

 

Excess nutrients in surface and ground waters of the U.S. have been reported with increasing frequency by a number of organizations. State assessments of their waters (305[b] reports) indicate that elevated concentrations of nutrients, such as nitrates and ammonia, are among the top 5 causes of impairment to aquatic life and/or public health and recreation (USEPA, 1998; 2000a-d). The growing national concern with elevated nitrate concentrations in many surface waters, aquifers, and drinking water sources of the U.S.  (e.g., USGS, 1999), for example, has lead to greater interest in nutrient monitoring and development of nutrient criteria for protection of aquatic life (USEPA, 1998; 2000a-d).

 

With increased nutrient monitoring and the desire to detect trends in nutrient impairment, both spatially and temporally, data quality and method performance issues are becoming more critical.  Currently, multiple agencies use a variety of methods to monitor the same nutrient analyte. The performance of these methods, and the comparability of data generated by these methods, is not always clear, nor have these issues been dealt with by data users in general (ITFM, 1995a,b).  Nutrient data (some of them relatively low in concentration), are starting to be scrutinized by the public and others, in anticipation of numeric water quality criteria being set.  The quality of data generated by different methods, and documentation of method performance, are necessary to make informed interpretations of monitoring data. 

 

The ITFM (1995a,b), National Methods and Data Comparability Board (MDCB), and the National Water Quality Monitoring Council (NWQMC), have indicated that the reliance on methods without appropriate method performance documentation, has had significant negative consequences in water quality monitoring (NWQMC, 2001).  Consistent with the goals of the Clean Water Action Plan (USEPA/USDA, 1998), the MDCB under the NWQMC has endorsed the development and use of a performance-based system (PBS) as one of its top priorities (NWQMC, 2001) because this system promotes the documentation of known quality data.  A PBS should enhance data comparability assessments among various methods or programs, and encourage implementation of better and more cost-effective methods (ITFM, 1995b; Parr, 2000).

 

Key aspects of performance-based systems include:  a) establishing concise data qualityobjectives (DQOs) and measurement quality objectives (MQOs) for each parameter reported;  b) demonstrated methods capable of meeting these DQOs and MQOs or regulatory limits; c) adequate reference materials to assist laboratories in demonstrating the appropriateness of a given method; d) adequate documentation of method performance, and e) successful pilot studies demonstrating the advantages and viability of a performance-based system (NWQMC, 2001).  All of these aspects are relevant to nutrient methods and monitoring. 

 

Data Quality Objectives (DQOs) are qualitative and quantitative statements derived from the DQO Process that clarify study objectives, define the appropriate type of data, and specify tolerable levels of potential decision errors that will be used as the basis for establishing the quality and quantity of data needed to support decisions (USEPA 1994, 2000d,e). These are mandated by EPA Order 5360.1 A2 and the applicable Federal regulations which establish a mandatory Quality System that applies to all EPA organizations and organizations funded by EPA (USEPA 1994, 2000e).  Many other organizations routinely use the DQO process as well, in a variety of environmental programs (e.g., DOE:  Grumbly 1994; APHA, 1998).

 

Measurement Quality Objectives (MQOs) on the other hand are project-specific analytical parameters that are derived from project-specific DQOs.   MQOs define acceptance criteria for the data quality indicators that are important to the project, such as sensitivity (e.g., what detection or quantification limit is desired), selectivity (i.e, what analytes are to be targeted), analytical precision. They are derived by considering the quantity and quality of data needed to actually achieve the project goals (as expressed in the DQOs).  In formal terms, the DQOs and MQOs specify project requirements to demonstrate precision, accuracy, representativeness, completeness, and comparability (APHA, 1998).  As with DQOs, MQOs specify benchmarks for validating/verifying method performance, without prescribing the technology or procedures to be used in producing analytical data.

 

The DQO/MQO Process is a systematic, iterative, and planning process based on the scientific method (USEPA 1994, 2000e).  It produces quantitative and/or qualitative statements (DQOs) that express the project-specific decision goals.  The DQOs then are used to define MQOs and guide the design of sampling and analysis plans that will be able to cost-effectively produce the right kind of data. The DQO/MQO process identifies what the goals are and what the consequences may be if the decisions are made in error.  Environmental program managers usually determine how certain (i.e., confident) they want to be before making decisions that will either impact, or be impacted by, environmental conditions (Crumbling 2001).

 

An important part of theDQO/MQOprocess is developing an understanding of how uncertainties can impact the decision-making process.  In brief, DQOs and MQOs require that analytical results be substantiated by quality control measurements, which in turn can be used to calculate confidence limits about a reported mean value. As explained in the Appendix, classical statistical methods can then be used to formulate hypotheses and test the validity of data interpretations at a specified confidence level.

 

This paper illustrates the DQO/MQO and method selection process, using nitrate to provide a focused case study:

1)      development of historical perspective:  preliminary screening of  site specific data for nitrate, using examples from the USGS National Water Information System (NWIS; www.waterdata.usgs.gov/nwis/) and classic statistical methods of interpretation

2)      development of DQOs and MQOs:  side-by-side comparisons of criteria for regulatory and ambient scenarios for monitoring of nitrate, as suggested by the historical data

3)      method selection:  appropriate choices for the compliance and ambient monitoring scenarios, using nitrate methods from the National Environmental Monitoring Index (NEMI; www.nemi.gov), an online compendium of analytical methods for water quality monitoring (Peters et al., 2000; Brass et al., 2000)

The primary goal is to understand how the DQO/MQO-method selection process leads to comparability of data and methods - within a specific program (eg for different sites, monitored at different times), and between programs (carried out by different institutions, acting independently or in collaboration).  The case study exercise raises several issues for clarification of the DQO/MQO-method selection process, which are addressed in the final discussion.  For nitrate as the analyte of interest, several suggestions are made for future pilot comparability studies, which are relevant to a monitoring program for development of nutrient criteria.

 

2.      Sources of nitrate data, methods of analysis and data interpretation

 

This paper uses NWIS data for a subset of three surface water sites in New York State (cf Table 1).  The sites were chosen primarily for having a large quantity of data reported over a 20-year period.  NWIS contains a wealth of information for nitrate and nitrite, and other analytes of current interest in surface and ground waters.  Similar data from USGS monitoring stations in virtually any of the 50 states would serve equally well.

 

The NWIS data were obtained by the USGS National Water Quality Monitoring Laboratory and the New York State section.  USGS methods were used to determine dissolved and total (dissolved and particulate) species: Method I-2545-90 for dissolved nitrate and nitrite (USGS 1993b); Method I-2540-90 for dissolved nitrite (USGS 1993a); dissolved nitrate, by difference (USGS 1997);  Method I-4540-90 for total nitrate and nitrite(USGS 1993b); Method  I-4540-90 for total nitrite (USGS 1993a); Method ??? for total and dissolved nitrate (USGS 1997).

 

Basic statistical properties and Student’s t-test are used for interpretation, to find t-values and their probabilities in comparing:  a) a single mean to a target value;  b) two means, using unpaired data, with standard deviations that are i) unequal or ii) equal;  c) two means using paired data.  Equations are given in an Appendix, with definitions of basic statistical terms, including type I (false positives) and type II errors (false negatives) (Devore and Peck 2001; Evans et al 2000).  The examples of statistical t-tests use sets of data with more than 30 entries each, to avoid the requirement for a normal distribution of values, which, in some cases, may require use of mathematically transformed data.  Calculations have been carried out in Excel:  results of pre-programmed statistical tests were cross-checked manually. 

 

3.  Historical data perspective: nitrate case study

 

Analyses of historical data are generally recommended for planning a monitoring project, to determine representative concentrations for target analytes, and potential differences between sites, and thereby develop useful and attainable DQOs and MQOs for further monitoring and assessment (US EPA, 2000e; APHA, 1998).  Table 1 summarizes nitrate data for the three surface water sites in New York State that are of interest to compare.  Average values reveal large differences among the three sites.  Relative standard deviations indicate that variability of the data is also large: from 33 to 83 percent for nitrate; from 56 to 157 percent for nitrite.  The results pertain to samples collected at different times from different sites: technically, the historical data are unpaired; statistically significant differences are expected for parameters where differences between mean values are relatively large.

 

Using data for total nitrate and nitrite concentration as an example (parameter 630), differences in mean values among the three sites can be evaluated for statistical significance, to test the following hypotheses (cf Appendix):

Hypothesis (null):    results for total nitrate and nitrite at the three sites are equivalent, at a 95 percent level of confidence

Alternate hypothesis: results for total nitrate and nitrite at the three sites are different, at a 95 percent level of confidence

Results in Table 2 show that the difference between sites in mean concentration of total nitrate, /, ranges from 29 to 51 percent.  Therefore, it is not surprising that these differences are statistically significant, at a 95 and even 99 percent confidence level (probability P = 0.00 < level of significance a = 0.025 and 0.005, confidence level 100(1-2a); cf Appendix).

 

The historical data in Table 1 raise other questions about differences in the analyte(s) being determined, which are relevant to DQOs, as well as method selection in a future monitoring project.  For example, is there a need to specify nitrate as

a)      dissolved                OR       total (dissolved and particulate) nitrate  

b)      nitrate (alone)         OR       nitrate AND (possibly including) nitrite

As before, hypotheses can be proposed to test for significant differences in mean values:

Hypothesis a (null):results for the total (dissolved and particulate) are not significantly different from results for the dissolved species at a 95 percent level of confidence 

Hypothesis b (null):results for nitrate including nitrite are not significantly different from results for nitrate alone at a 95 percent level of confidence

Tables 3 and 4 give results of the statistical tests, arranged in order of increasing concentration difference relative to the measurement of the total, /a.   In both cases, the concentration differences being examined are generally small and, as expected, statistical significance is not found (probability P = 0.10 to 0.50 > level of significance a = 0.025, confidence level  100(1-2a); cf Appendix):

a)      dissolved vs total nitrate (Table 3): values of /a range from 4.5 to 31%, remaining below 12% for all cases where there is sufficient data (degrees of freedom df = N – 1 ³ 31);  both determinations are equivalent, with 95% level of confidence

b)      nitrate vs nitrate plus nitrite (Table 4): values of /a range from 4.8 to 23%, remaining below 13% for all cases where there is sufficient data (degrees of freedom df = N – 1 ³ 49); both determinations are equivalent, with 95% level of confidence

With regard to a) in particular, it is perhaps not surprising to find that USGS has also concluded that the distinction between ‘dissolved’ and ‘total’ forms of nitrate is not significant (USGS 1992).  Two results in Table 4 are reported, where there is paired data for nitrate (parameter 630) vs nitrate plus nitrite (parameter 620).   Statistical properties of the paired data are generally similar to those of the unpaired data (Table 5).  For the Mohawk River at Schenectady, it can be concluded that determinations of nitrate, with or without nitrite, give significantly different results, but only by 4.8%  (probability P = 0.000 < level of significance a = 0.05, confidence level  100(1-a); cf Appendix):  this difference is unlikely to be of practical significance to a monitoring program.

 

Inferences for DQO Selection and Subsequent MonitoringIt is important to note that the data quantity contributes greatly to the statistical significance of the differences in concentrations, while data quality (precision) is limited for the datasets examined in this case studyRelative standard deviations for concentrations of total nitrate at the three sites are moderate, ranging from 33 to 53 percent, but the degrees of freedom (df = N –1) in these calculations are very large, ranging from 122 to 181, deflating the variability presentFor a subsequent monitoring project, it might be of interest to propose a DQO for identifying sites that have a nitrate concentration that is 30 percentgreater than the mean for all sites.  For two parameters with relative standard deviations (RSD) of 30 percent, it can be shown that for a difference / to be significant at a 95 percent confidence level, each data set must only have more than 7 values (or more than 10 values, if one set has a 50 percent RSD).

 

With regard to  setting DQOs and method selection for subsequent monitoring in this program, the foregoing results indicate that it may not be necessary to differentiate dissolved from total nitrate, or to specify nitrate in the absence of nitrite.  Historical data for any of the measurements  were equivalent.  Therefore costs could be saved in method selection for a subsequent monitoring project, by not having to filter or centrifuge samples to isolate dissolved nitrate, and by not having to determine nitrite separately from nitrate.  Moreover, data quality for nitrate may be improved:  by avoiding potential contamination and loss in precision from increased sample preparation; by not having to estimate nitrate by difference (using imprecise data for nitrite), if using a method for nitrate which allows interference from nitrite.

 

In addition to the straightforward comparisons of historical data for different sites and analytes, there are remaining questions about the data variability, which could be of interest in design of a future monitoring project.  In Table 1, there  was high variability in concentrations of dissolved nitrate for the Hudson River at Waterford, with relative standard deviations of 47 to 54 percent.  At all three sites, there  was high variability in concentrations of nitrite:  relative standard deviations range from 56 to 102 percent for the dissolved species, or to 157 percent if suspended particulates are included.  Plots of total nitrate concentrations in Figure 1 suggest that the variability is random  over time.  However, it is somewhat surprising to note that, as the mean concentration decreases among the three sites, there is a decrease in variability (less separation between lines marking two standard deviations above and below the mean, Figure 1); also, the distribution of values peaks more sharply (and in a more normal distribution) about the mean (Figure 2).  A subsequent monitoring project might try to determine sources of variability, at specific sites and from specific methods of analysis. 

 

4.      DQOs and MQOs for a new monitoring project for nitrate:

requirements for data comparability

 

Given the historical data, how might DQOs and MQOs be developed, and a method of analysis selected, for  further monitoring of nitrate?  These issues are examined for two project scenarios: compliance monitoring and ambient monitoring of nitrate because these two types of monitoring reflect different regulatory interests and goals, and therefore, different potential monitoring objectives.  Some researchers have postulated direct and indirect effects of nitrate on aquatic communities at concentrations as low as 2 mg/L (a target concentration reported by Ohio EPA to be associated with ecological effects; OEPA, 2001).  Using this target as an example, an organization performing ambient monitoring might be interested in a range of nitrate concentrations ranging from 0.01 to  >  2.0 mg/L, whereas a compliance monitoring program may be satisfied with determining whether a site or sample is >  1.5 mg/L.  Following guidelines provided by EPA (USEPA, 2000e), hypotheses are proposed for the two scenarios:

      compliance monitoring

Hypothesis:  the nitrate concentration at a particular stream is [null:  less than or equal] OR [alternate:  greater than] 1.5 mg/L with a 95% level of certainty

      ambient monitoring

      Hypothesis a (sites):  entry of a particular stream into the river increases the nitrate concentration by [null:  less than or equal to] OR [alternate: greater than] 30 percent, with a 95% level of certainty, and

Hypothesis b (times):  nitrate concentrations in consecutive quarters at a particular river location increase by [null: less than or equal to] OR [alternate: more than] 50 percent, with a 95% level of confidence

All of the hypotheses require comparisons of data:  the bottom line of both the compliance and ambient monitoring project is that data can be compared, to make statistically valid  interpretations. 

 

For both scenarios, it is assumed that project objectives (POs) have been defined by decision makers (i.e., a regulatory agency, property owner, the organization funding the work, etc.). Table 6 gives hypothetical POs for the two scenarios, following guidelines for monitoring projects that have been suggested by AWWA, WEF and APHA (APHA 1998).  Criteria which are anticipated in the setting of DQOs and MQOs are shown in italics.  Clearly the two projects differ with respect to the decisions to be made and the required level of precision and accuracy at a particular concentration:  the compliance monitoring scenario places emphasis on at the targeted nitrate concentration (1.5 mg/L) whereas the ambient monitoring scenario requires good precision and accuracy across an entire range of nitrate concentrations (0.010 to 2.00 mg/L).

 

Table 6 also gives criteria for the hypothetical DQOs (confidence level, representativeness, completeness) which establish a statistical context for interpreting the results of monitoring.  These DQOs can be combined in a statement for each scenario:

compliance monitoring:

               DQO  Collect samples quarterly (i = 1 to 4) for one year from 50 specified stream locations, measure nitrate concentrations, and determine to a 95% degree of statistical certainty if the annual average nitrate concentration (Ni = 4) at each site exceeds 1.5 mg/L

ambient monitoring:

               DQO  Collect samples of river water monthly for one year (i = 1 to 12), below consecutive entry points for 15 first- and second-order streams:  measure nitrate concentrations and determine to a  95% degree of statistical certainty, if

a)      above and below the entry of any stream, there is a significant 30 percent increase in nitrate concentrations  (average of Ni = 12),

and if  b)   between quarterly samples for any river location being monitored, there is a significant 50 percent change in average concentration nitrate concentrations (average of Ni = 3)

Decision-making in the ambient monitoring project requires a greater number of samples per site (spatial, temporal comparisons, between and within sites, respectively) than in the regulatory monitoring project (independent comparisons of each site to regulatory standard).

 

Hypothetical MQOs for both projects are shown in Table 7.  Accuracy and precision are defined operationally, by project requirements that are set for analysis and data interpretation, and by the required demonstrations of proficiency and quality control.  As noted from hypotheses for the two monitoring scenarios, there are important differences in the required precision and accuracy at a particular concentration:  the compliance monitoring project places emphasis on at the targeted nitrate concentration (1.5 mg/L) whereas the ambient monitoring scenario requires good precision and accuracy across an entire range of nitrate concentrations (0.010 to 2.00 mg/L).

 

5.  Method selection for a new monitoring project for nitrate

 

Analytical performance is the basis for method comparisons, as indicated by method detection limit, precision, and accuracy (i.e., meta data; NELAC, 2000; USEPA, 1997a, b; 1999).  These three parameters are generally regarded as key attributes of any chemical method (APHA, 1997; ASTM, 2000) and are considered to be critical for judging comparability of different analytical methods, and the ability of a method to meet MQOs and DQOs for a specific project.  Performance information for analytical methods is documented for a variety of water methods in the National Environmental Methods Index (www.nemi.gov), maintained by the Methods and Data Comparability Board and the National Water Quality Monitoring Council, with database and web support provided by the U.S. Geological Survey.  NEMI is a web-based methods compendium that provides information to compare and contrast performance and relative cost of analytical methods for water quality monitoring (Peters et al., 2000; Brass et al., 2000).  A total of 16 methods for nitrate are available in NEMI, 11 of which have sufficient performance information documented.  The DQOs and MQOs provide a framework for method selection, which is discussed using the eleven nitrate methods listed in Table 8.    The eleven methods of interest are from fourdifferent sources (USEPA; USGS; ASTM;   and Standard Methods).  For method selection, it is important to note that the methods are differentiated by using three different types of instrumentation:  capillary ion electrophoresis with UV detection (CIE-UV); ion chromatography with conductivity detection (IC-CD); nitrate reduction with colorimetric detection (RD-Vis).

 

Detection levels differ greatly among methods ranging from 0.002 - 0.42 mg/L (Table 8).  Method sensitivity was the only factor that could be adequately addressed for all eight methods with the information available.  Precision and accuracy information, as well as the spiking level used to derive method precision and accuracy, is nonexistent in some methods.  EPA methods 300.0, 300.1,and 352.1, one ASTM method (D4327), and Standard Method 4500 all have accuracy and precision data.  ASTM method D5996 has neither precision nor accuracy information.  Standard Method 4110C (with direct conductivity detection) has accuracy but not precision information.  USGS method I-2057 has only precision and not accuracy data (USGS, 1985).

 

Given the information in Table 8, only four methods have sufficient performance information with which to be able to make an informed choice.  Standard Method 4500 NO3–E appears to satisfy all MQOs for both the compliance and the ambient monitoring DQOs (Table 8).  EPA methods (300.0, 300.1, and 352.1) also appear to satisfy most of the suggested MQOs for both types of monitoring given their high sensitivity (i.e., the low detection levels), relatively high precision ( <1 – 14% RSD), and satisfactory accuracy (95-103% recoveries).  Thus, the three  EPA methods should theoretically be able to detect a 30% difference in nitrate levels between samples, and accurately detect a 0.1 mg/L nitrate concentration, as required in the ambient monitoring DQOs.  However, note that the spiking concentration used to derive precision and accuracy for two of these EPA methods was 10 mg/L nitrate and the third was 0.5 mg/L.  These concentrations are greater than the MQO for the ambient monitoring DQOs, in which we desired high precision and accuracy at a 0.1 mg/L nitrate concentration.  Thus, available method performance information for these EPA methods indicates that they should be satisfactory for the compliance monitoring DQOs and perhaps satisfactory for the ambient monitoring DQOs, pending further laboratory evaluation.

 

Of the remaining methods in Table 8, only the two ASTM methods (D4327and D6508) have both precision and accuracy data.  Accuracy and precision of method D4327 meet the MQOs as evidenced by the low spiking concentration.  However, this method is less sensitive then either of the two EPA methods or Standard Method 4500, and the detection limit is higher than the desired MQO of 0.1 mg/L nitrate (Table 8).  Therefore, this ASTM method may not meet all of the desired MQOs for ambient monitoring but it should satisfy the compliance monitoring DQOs.  ASTM method D6508 meets the sensitivity and precision MQOs for ambient monitoring, however the reported accuracy does not meet the MQO for perhaps either monitoring program as defined by our DQOs and MQOs (140% recovery, Table 8).  Also, the spiking concentration for this method was 1.99 mg/L, which is somewhat higher than that desired for the MQO for either monitoring program.  Thus, neither of the ASTM methods appear to meet all of the ambient monitoring MQOs in this example.  The remaining methods in Table 8 have insufficient performance information with which to evaluate their appropriateness, regardless of the MQOs selected.

 

For the compliance monitoring scenario, essentially all of the methods listed could produce acceptable data that would meet the project DQOs and MQOs.  All of the methods are capable of quantitation above 0.5 mg/L nitrate, with greater than 80 percent accuracy and better than (less than) 20 percent precision .  From a project manager’s viewpoint, all of the methods  should yield comparable data for application to the compliance monitoring project.  Cost would be a major factor in method selection.  One of the methods (RD-Vis) costs less than the others, as long as nitrate is the major constituent, and it does not have to be determined by difference from nitrite, a minor constituent.  However, a decision based on cost could be modified by site-specific requirements:  a review of historical or pilot data would indicate the need for a high resolution method (CIE-UV or IC-CD) capable of distinguishing nitrate and nitrite in a single analysis.  IC-CD methods can also measure multiple analytes using the same sample.  Thus, there may be additional monitoring advantages of the ion chromatographic methods, depending on the program DQOs and what other related analytes a program needs to measure.   Analyses of historical data presented earlier in this paper, provide several examples of sites for which separation of nitrate and nitrite is unnecessary.

 

For the ambient monitoring scenario, only a few of the methods listed could produce acceptable data that would meet the project DQOs and MQOs.  Only the IC-CD methods are capable of accurately quantifying nitrate from 0.010 to 2.00 mg/L, with a detection limit of 0.005 mg/L.  A particular IC-CD method is not chosen from those listed, because the performance data cited in Table 8  were obtained under different conditions, by a mix of single and multiple laboratories [and not all methods give all performance information].  An actual monitoring project would need to evaluate the IC-CD methods in detail, and verify performance capabilities of the chosen method in the laboratory that would conduct the analyses.   From a project manager’s viewpoint, the IC-CD methods are potentially comparable for application to the ambient monitoring project.

 

6.  Discussion:  comparability revisited; development of nutrient criteria

 

 This case study  using nitrate raises a number of issues which merit further discussion, regarding comparability and the DQO/MQO process, in general, and as applied to the development of nutrient criteria, in particular.

 

Comparability   A concept of comparability emerges from the nitrate case study as the overall requirement of data quality in hypothesis-based monitoring projects.  The following definition is proposed:

comparability (of data): the data meet the criteria specified in the DQOs (representativeness and completeness) and MQOs (precision, accuracy), so that the project hypotheses can be tested (data can be compared) and statistical interpretations are valid at the desired level of confidence

This definition gives greater significance to comparability than found in previous documents, which imply comparability is only one of the factors by which to judge data quality, together with representativeness, completeness, precision, and bias (APHA, 1998) and measurability (USEPA 2000d).  The above definition assigns comparability a more useful role, as a descriptor for data that satisfy all of the DQOs and MQOs in a particular project.

 

Representativeness  Similarly, the development of DQOs argues in favor of a more specific role for representativeness and a clear link to sample design. As suggested in guidelines to the DQO process, DQOs were written for the nitrate case study to explicitly state the confidence level required for data interpretation and allowed rate of false positives and negatives.  In the example given, the DQOs were also written to include a statement of the proposed times and numbers (Ni) for samples to be collected, which recognized real-world variability in nitrate concentrations (s2), and the uncertainty (± ts/ÖN) of statistical evaluations to be made.  In effect, the sample design has been incorporated in the project DQOs, and representativeness is identified as the primary DQO for the adequacy of data based on the sample design.  The following definition is proposed:

representativeness (of data):  the results adequately represent the project sites in time and location:  the samples collected and analyzed are sufficient in number N to  i) make the required interpretations, at the level of statistical significance that is specified in the DQOs, and  ii) allow for the overall uncertainty (± ts/ÖN) and rate of false positives (a) and negatives (b)

As defined above, representativeness is assigned the key role in determining that a sample design is sufficient in yielding data with the desired level of confidence for a particular project.

 

Other DQO/MQO criteria:  Completeness of the sample design is essentially implied by representativeness, but can be given a common-sense definition:

completeness:  a sufficient percentage of valid results is obtained for the project sites to make the decisions required in the DQOs.

For project planning, it should suffice to incorporate completeness into DQOs for representativeness as the primary goal, and simplify the terminology used to track DQO/MQO development.  The case study for nitrate listed precision and accuracy as the major MQOs, and definitions were understood from project requirements:

precision: operationally defined  by requirements for analysis of replicate samples and replicates of spiked controls

accuracy: operationally defined by requirements for analysis of standards, blanks, spiked controls and field samples, and by specifications for interpretation and reporting of data

This paper proposes use of validation as the overall goal of MQOs, with the following definition:

validation:  operationally defined  by required demonstrations of precision, accuracy (bias), proficiency, and quality control

Therefore, recommended terminology is simplified to:  comparability for the overall DQO/MQO process; representativeness for the DQO process; validation for the MQO process (method selection, method performance verification).

 

Sample design and sample collection:  variability  Standard Methods (APHA, 1998) makes an interesting comment that, in practice, the impact of sample design on comparability is often not determined.  This paper suggests using representativeness to clarify the link between DQOs and the sample design.  The current debate between probabilistic versus deterministic sample designs suggests that this issue needs to be explored further.  Moreover, methods for sample collection need to be included in defining performance criteria for methods of analysis.  The sample design (time, location, frequency of sampling) is the primary tool for addressing environmental variability (senvir), whereas laboratory measurements determine the analytical variability (smeas); both contribute to the overall variability sand error  – m (assuming the same number of samples, N, for each term; otherwise each is divided by the appropriate Ni; cf Appendix):

         s2      =       s2envir+       s2meas

Only the overall variability (s2) can be deduced from historical data for a single analyte, such as nitrate.  A formal analysis of variance (ANOVA) requires additional measurements for sources of variability in both the field and laboratory, generally  implying an expanded dataset for other conditions and analytes.

 

Collaboration between monitoring programs – DQOs/MQOs, method selection and comparability 

 

The case study for nitrate described a relatively simplistic situation in which the compliance and ambient monitoring programs were independent.  Collaborations between monitoring programs provide a means of increasing data quantity and quality for reduced cost, but require consideration of DQOs and MQOs.  In the examples given, the compliance monitoring program was unrestricted in method selection, but the ambient monitoring program was restricted to a subset of acceptable nitrate methods.  Clearly, the compliance program, with the least restrictive DQOs and MQOs, could use all data from the ambient monitoring program.  However, the ambient monitoring program, with more narrowly defined DQOs and MQOs, could only accept limited results from the compliance program, if they were obtained by a method of analysis  (IC-CD) with the required range of analysis and low limit of detection.  In effect, the DQOs and MQOs for each monitoring program define the acceptance criteria for data comparability:

 

 

Monitoring program (DQOs/MQOs)

Data type

compliance

ambient

compliance

data accepted

(by definition)

compliance data comparable, only

 if acquired using methods meeting ambient DQOs/MQOs

ambient

all data comparable

data accepted (by definition)

 

Therefore, the DQO/MQO process not only defines performance-based criteria for method selection, as seen in the nitrate case study, but also provides a framework for determining data comparability across monitoring programs.  Potential collaborators can objectively compare their respective DQOs and MQOs to determine how best to match their needs and resources. 

 

Evolution of monitoring programs – DQOs/MQOs, method selection and comparability  As individual monitoring programs expand their historical databases, they can also be expected to want to preserve comparability for future assessments.  This need will also influence the development of DQOs/MQOs and method selection.

 

Development of nutrient criteria:  USEPA’s nutrient program depends on representative, unbiased data with which “reference” or minimally-impaired water quality conditions, including nutrient concentrations, are characterized and used as a baseline for developing ecoregional criteria (USEPA, 2000c).  Clearly this requires analytical methods with known performance characteristics to make correct management decisions.  The nutrient criteria program needs to define the level of certainty and acceptable rate of false positives and negatives that are required to determine if water quality is impaired or unimpaired.   This information would align the nutrient criteria program with the DQO/MQO-method selection process:  the acceptability (comparability) of data could be determined, which are used to develop nutrient criteria.  The increasing realization of the importance of data quality in 303(d) water body listings (i.e., impaired status) and in TMDLs underscores the difficulties encountered in environmental programs when method performance and DQOs/MQOs are not clearly documented (Heinz CSEE, 2002).

 

This paper has used nitrate as a case study of the DQO/MQO process, with little discussion of current method limitations.  The Methods Board maintains an active interest in new technologies for a number of important analytes (nitrogen, phosphorus, chlorophyll, turbidity, suspended particles, planktonic and periphytic algae, microorganisms, macrophytes, macroinvertebrates), in advanced and alternate technologies for sample collection (in-situ probes; remote monitoring), and in the development of reference materials to assess method performance (Frankovich and Jones, 1998; NOAA, 2000).  Pilot studies for any of these areas of interest would be welcome, to improve the quality and quantity of comparable analytical data used to develop nutrient criteria.


7.   Literature Cited

 

APHA, AWWA, WEF (American Public Health Association, American Water Works Association, Water for the Environment Foundation). 1998a. Standard Methods for the Examination of Water and Wastewater. 20th edition. Editors:  L. Clesceri, A. Greenberg, and A. Eaton. 1030D.  Data Quality Objectives.  APHA: Washington, D.C., pp 1-18 to 1-20.

 

ASTM. 2000. Water:  Inorganic Analytes. Volume 11.01. American Society for Testing and Materials, Conshohocken, PA.

 

Brass, H.J., H. Ardourel, J. M. Diamond, A. Eaton, L. H. Keith, and C. A. Peters. 2000. Activities of the Interagency Methods and Data Comparability Board. Proceedings of the American Water Works Association, Water Quality Technology Conference, Salt Lake City, Utah, November 2000.

 

Crumbling, D. M, Current Perspectives in Site Remediation Monitoring, EPA 542-R-01-014, October 2001.

 

Devore, J.; Peck, R. 2001.  Statistics:  the Exploration and Analysis of Data.  Duxbury:  Pacific Grove, CA.

 

Evans, M.; Hastings, N.; Peacock, B. 2000. Statistical Distributions.  John Wiley: NY.

 

Eaton, A. and J. Diamond. 1999. Reservoir dogs and performance-based systems. Envir. Testing and Analysis 8:  18-19

 

Grumbly, T. 1994. Institutionalizing the Data Quality Objectives Process for EM’s Environmental Data Collection Activities. Memorandum September 7, 1994, United States Department of Energy, Washington, D. C.

 

Heinz CSEE. 2002. The state of the nation’s ecosystems:  measuring the lands, waters, and living resources for the United States. H. J. Heinz III Center for Science, Economics, and the Environment. Cambridge University Press, Cambridge, UK.

 

ITFM. 1995a. The Strategy for Improving Water Quality Monitoring in the U.S. Report #OFR95-742, U.S. Geological Survey, Reston, VA.

 

ITFM. 1995b. Performance-based approach to field water quality methods. In:  Strategy for Improving Water Quality Monitoring in the U.S., Appendix N, Report #OFR95-742, U.S. Geological Survey, Reston, VA.

 

Kammerer, Phil A., Garn, Herbert S., Rasmussen, Paul W., Ball, Joseph R.. 1998. A Comparison of Water-Quality Sample Collection Methods Used by the U.S. Geological survey and the Wisconsin Department of Natural Resources. In Proceedings of the NWQMC National Monitoring Conference, July 7 – 9, 1998, Reno, NV.

 

NELAC. 2000. Chapter 5:  Quality Systems Standard. National Environmental Laboratory Confernce, http://www.epa.gov/nerlesd1/land-sci/nelac/index.html - June 28, 2000.

 

NOAA. 2000. NOAA/NRC Intercomparison for Nutrients in Seawater. NOAA Technical Memorandum NOS NCCOS CCMA 143.

 

NWQMC. 2001. “Towards a Definition of Performance-Based Laboratory Methods”, National Water Quality Monitoring Council Technical Report 01 – 02, US Geological survey, Reston, VA.

 

OEPA. 1999. Associations between nutrients, habitat, and the aquatic biota in Ohio rivers and streams. Ohio EPA Technical Bulletin MAS/1999-1-1. Ohio Environmental Protection Agency, Columbus, OH.

 

Parr, J., 2000, “Determining and Documenting the Suitability of Analytical Procedures Used for Analysis of Environmental Samples”, NWQMC National Monitoring Conference 2000, Austin, TX, April 2000.

 

Peters, C.A., H.J. Brass, J. Diamond. 2000.  United States Water Quality Methods and Data Comparability Board:  Creating a framework for collaboration and comparability, in proceedings of Monitoring Tailor-made III, September 25 - 28, 2000, Nunspeet, the Netherlands.

 

USDOE. 2000. Guide for developing data quality objective for ecological risk assessment at DOE Oak Ridge Operations Facilities. Report ES/ER/TM-185/R1. US Department of Energy, Oak Ridge National Laboratory, Oak Ridge, TN.

 

USEPA. 2001. Protecting and restoring America’s Watersheds. EPA-840-R-00-001, Office of Water, Washington, D.C.

 

USEPA. 2000a (December). Nutrient Criteria Technical Guidance Manual. Lakes and Reservoirs. EPA-822-B00-001, Office of Water, Office of Science and Technology, US EPA: Washington, D.C.

 

USEPA. 2000b (December). Nutrient Criteria Technical Guidance Manual. Rivers and Streams. EPA - 822-B-00-002 , Office of Water, Office of Science and Technology, US EPA: Washington, D.C.

 

USEPA. 2000c (December). Ecoregional Nutrient Criteria. EPA-822-F-00-007, Office of Water, Office of Science and Technology, US EPA:  Washington, D.C.

 

USEPA 2000d (December).  Estuarine and Coastal Marine Waters:  Bioassessment and Biocriteria Technical Guidance.  EPA-822-B-00-024. Office of Water.  US EPA:  Washington, DC.

 

USEPA. 2000e (August). Guidance for the Data Quality Objectives Process (EPA QA/G-4), EPA/600/R-96/055. Quality Assurance Management Staff, Office of Environmental Information.  US EPA:  Washington DC.  http://www.epa.gov/quality/qs-docs/g4-final.pdf

 

USEPA. 1999. Protocol for EPA approval of Alternate Test Procedures for Organic and Inorganic Analytes in Wastewater and Drinking water. EPA-821-B-98-002, Office of Water, US EPA:  Washington, DC.

 

USEPA. 1998. National strategy for the development of regional nutrient criteria. EPA-822-R-98-002, Office of Water, US EPA:  Washington, D.C.

 

USEPA. 1997a. Guidelines establishing test procedures for analysis of pollutants and national primary drinking water regulations; flexibility in existing test procedures and streamlined proposal of new test procedures. Federal Register 62:14975-15049, Washington, DC.

 

USEPA. 1997b. Streamlining EPA’s Test Methods Approval Program. EPA-821-F-97-001, Office of Water, US Environmental Protection Agency:  Washington, DC.

 

 

USEPA/USDA. 1998. Clean Water Action Plan:  Restoring and Protecting America’s Waters. National Center for Environmental Publications and Information, US Environmental Protection Agency:  Cincinnati, OH

 

USGS. 2001An Alternative Regionalization Scheme for Defining Nutrient Criteria for Rivers and Streams, Water-Resources Investigations Report 01-4073. USGS:  Middleton WI.

 

USGS. 1999. The quality of our Nation’s Waters Nutrients and Pesticides. US Geological Survey Circular 1225.  USGS:  Reston, VA.

 

USGS. 1985. Methods, Volume A1. Branch of Information Services. U. S. Geological Survey:  Denver, CO.

 

USGS.  1989. Method I-2058-85 (NWIS 00618).  Anions, ion-exchange chromatography, low ionic-strength water, automated.  In: Fishman, M.J., Friedman, L. C. (eds) 1989. Techniques of Water Resources Investigations of the U.S. Geological Survey.  Chapter A1. Methods for the determination of inorganic substances in water and fluvial sediment.  Book 5.  Laboratory analysis.  3rd edition. USGS:  Denver, CO.  pp. 527-530

 

USGS 1992 (Dec. 2).  Analytical Methods:  Discontinuation of the National Water Quality Laboratory determinations for “total” nitrite, “total” nitrite plus nitrate, “total” ammonia, and “total” orthophosphate (using the four-channel analyzer).  David Rickert, Chief, Office of Water Quality. Technical Memorandum 93.04.

 

USGS 1993a. Method I-2540-90 and Method I-4540-90??? Or just I-4540-85 in 1989 volume??? (NWIS 00613, 00615) Nitrogen, nitrite, colorimetry, diazotization, automated-segmented flow.  In: Methods of analysis by the USGS National Water Quality Laboratory. Methods for the determination of inorganic and organic constituents in water and fluvial sediments. M.J. Fishman (ed). USGS Open-File Report 93-125. USGS:  Denver, CO. pp 143-148

 

USGS 1993b. Method I-2545-90 and Method I-4545-90??? Or just I-4545-85 in 1985 volume??? (NWIS 00631, 00630) Nitrogen, nitrite plus nitrate, colorimetry, cadmium reduction-diazotization, automated-segmented flow. In: Methods of analysis by the USGS National Water Quality Laboratory. Methods for the determination of inorganic and organic constituents in water and fluvial sediments. M.J. Fishman (ed). USGS Open-File Report 93-125. USGS:  Denver, CO. pp 157-166.

 

USGS 1997  (NWIS 00620???)  (Harold A.: this title not confirmed yet)  Nitrogen, nitrate, dissolved, mg/L as Nitrogen.  Chapter 2. Water Quality System, Appendix E, Algorithms for Calculated Parameters User’s Manual for the National Water Information System of the U.S. Geological Survey, USGS Open File Report 97-634. USGS:  Denver, CO. pp E1-E12.

 

Fishman, M.J. (ed). 1993. Methods of analysis by the U.S. Geological Survey National Water Quality Laboratory.  Methods for the determination of inorganic and organic constituents in water and fluvial sediments: USGS Open-File Report 93-125, USGS:  Denver, CO. 217 pp. 

 

Fishman, M.J., Friedman, L. C. (eds) 1989. Techniques of Water Resources Investigations (TWRI) of the U.S. Geological Survey.  Chapter A1. Methods for the determination of inorganic substances in water and fluvial sediment.  Book 5.  Laboratory analysis.  3rd edition.  USGS:  Denver, CO.

 


Text Box: Total nitrate and nitrite  (mg NO3-N/L) 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Figure 1  Comparison of data for total nitrate and nitrite (NWIS analyte code 630) at select sites in New York State, showing significant differences in average values (dotted lines) and variability (± two standard deviations, solid lines)

 

Text Box: Percent frequency of occurrence 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

                                           Total nitrate and nitrite  (mg NO3-N/L)               

 
 

 


Figure 2  Comparison of data for total nitrate and nitrite (NWIS analyte code 630) at select sites in New York State, showing significant differences in the frequency of occurrence of values and their distribution about the mean


Table 1  Nitrate and nitrite data for a subset of surface waters in New York State (USGS NWIS)

NWIS analyte code

630

631

620

618

615

613

Analyte name

[NO3- + NO2-]total

[NO3- + NO2-]dissolved

[NO3-]total

[NO3-]dissolved

[NO2-]total

[NO2-]dissolved

Concentration units

mg NO3-N/L

mg NO3-N/L

mg NO3-N/L

mg NO3-N/L

mg NO2-N/L

mg NO2-N/L

Cattaragus R. at Gowanda          04213500

 

 

 

 

 

average

0.993

0.907

-

0.640

0.012

0.014

stdev

0.327

0.310

-

0.226

0.018

0.008

rsd (%)

33

34

-

35

144

56

N

101

95

-

2

38

59

max

1.70

1.70

 

0.80

0.080

0.040

min

0.50

0.25

 

0.48

0.000

0.010

start

7/16/75

9/19/79

-

 6/24/72

4/16/87

4/16/87

stop

11/5/92

2/26/98

-

10/29/85

11/5/92

11/5/92

Mohawk R. at Schenectady          01354490

 

 

 

 

 

average

0.695

0.911

0.734

0.701

0.038

0.035

stdev

0.234

0.297

0.244

0.215

0.060

0.036

rsd (%)

34

33

33

31

157

101

N

100

7

41

47

38

47

max

1.40

1.32

1.40

1.30

0.340

0.150

min

0.10

0.50

0.15

0.27

0.010

0.002

start

10/10/73

6/5/73

 10/10/73

 6/29/71

10/10/73

6/29/71

stop

9/2/81

9/25/73

7/28/77

9/25/73

7/28/77

9/25/73

Hudson R. at Waterford          01335770

 

 

 

 

 

average

0.491

0.431

0.515

0.487

0.012

0.021

stdev

0.146

0.234

0.157

0.229

0.011

0.021

rsd (%)

30

54

31

47

93

102

N

190

29

37

30

63

54

max

1.00

1.50

1.00

1.50

0.050

0.105

min

0.10

0.25

0.24

0.20

0.000

0.002

start

10/2/73

5/14/73

 9/12/72

 7/7/71

10/2/73

7/7/71

stop

8/28/91

5/13/96

7/27/77

9/5/73

8/28/91

5/13/96

 


Table 2 Comparison ofmean values for total (dissolved and particulate) nitrate vs dissolved nitrate, for results summarized in Table 1 (with sufficient data). The data are unpaired:  therefore the test for equivalence is two-sided, using eq. A.9 – A.11 in the Appendix, at a 95% confidence level (a = 0.025; conclusions remain the same for a = 0.005). Subscript:  t  total.

Parameter

Measurement A: total

Measurement B: dissolved

/a

teff

vs

tcrit

df

Peff

vs

a

a vs b

630 [NO3- + NO2-] t

Cattaragus R.: Gowanda

Mohawk R.: Schenectady

30.0%

0.30

7.43

>

1.97

181

0.00

<

0.025

different

630 [NO3- + NO2-] t

Cattaragus R.: Gowanda

Hudson R. at Waterford

50.6%

0.50

14.6

>

1.98

122

0.00

<

0.025

different

630 [NO3- + NO2-] t

Mohawk R.: Schenectady

Hudson R. at Waterford

29.3%

0.20

7.95

>

1.98

141

0.00

<

0.025

different

 

Table 3 Comparison ofmean values for total (dissolved and particulate) nitrate vs dissolved nitrate, for results summarized in Table 1 (with sufficient data). The data are unpaired:  therefore the test for equivalence is two-sided, using eq. A.9 – A.11 in the Appendix, at a 95% confidence level (a = 0.025). Subscripts:  t  total; d  dissolved.

Site

Measurement A: total

Measurement B: dissolved

/a

teff

vs

tcrit

df

Peff

vs

a

a vs b

Mohawk R. at Schenectady

620 [NO3-] t

618 [NO3-]d

4.5%

0.033

0.68

<

1.99

81

0.50

>

0.025

equivalent

Hudson R. at Waterford

620 [NO3-] t

618 [NO3-]d

5.4%

0.028

0.57

<

2.01

50

0.57

>

0.025

equivalent

Cattaragus R at Gowanda

630 [NO3- + NO2-] t

631 [NO3- + NO2-]d

8.6%

0.086

1.88

<

1.97

194

0.06

>

0.025

equivalent

Hudson R. at Waterford

630 [NO3- + NO2-] t

631 [NO3- + NO2-]d

12.2%

0.060

1.33

<

2.04

31

0.19

>

0.025

equivalent

Mohawk R. at Schenectady

630 [NO3- + NO2-] t

631 [NO3- + NO2-]d

31.1%

0.216

-1.89

<

2.36

7

0.10

>

0.025

equivalent

 


Table 4 Comparison ofmean values for nitrate (including nitrite) vs nitrate (without nitrite), for results summarized in Table 1 (with sufficient data): unpaired and paired data are evaluated using eq. A.9 – A.11 and A.13 – A.15, respectively, corresponding to two-sided (a = 0.025) vs one-sided (a = 0.05) tests for a significant difference at a 95% confidence level.   Subscripts:  t  total; d  dissolved

Site

Measurement A: nitrate (with nitrite)

Measurement B: nitrate only

/a

teff

vs

tcrit

df

Peff

vs

a

a vs b

Unpaired data a = 0.025 P = 95%

 

 

 

 

 

 

 

 

 

 

 

Hudson R. at Waterford

630 [NO3- + NO2-]t

620 [NO3-]t

4.8%

0.024

-0.85

<

2.010

49

0.398

>

0.025

equivalent

Hudson R. at Waterford

630 [NO3- + NO2-]t

620 [NO3-]t

4.8%

0.024

0.19

<

1.995

68

0.851

>

0.025

equivalent

Mohawk R. at Schenectady

630 [NO3- + NO2-]t

620 [NO3-]t

5.6%

0.039

-0.88

<

1.993

72

0.384

>

0.025

equivalent

Mohawk R. at Schenectady

630 [NO3- + NO2-]t

620 [NO3-]t

4.8%

0.037

0.66

<

1.991

78

0.511

>

0.025

equivalent

Hudson R. at Waterford

631 [NO3- + NO2-]d

618 [NO3-]d

13%

0.056

-0.93

<

2.002

57

0.357

>

0.025

equivalent

Mohawk R. at Schenectady

631 [NO3- + NO2-]d

618 [NO3-]d

23%

0.211

1.81

<

2.36

7

0.114

>

0.025

equivalent

Paired data a = 0.05 P 95%

 

 

 

 

 

 

 

 

 

 

 

Hudson R. at Waterford

630 [NO3- + NO2-]t

620 [NO3-]t

1.4%

0.024

1.28

<

1.691

34

0.105

>

0.05

equivalent

Mohawk R. at Schenectady

630 [NO3- + NO2-]t

620 [NO3-]t

4.8%

0.037

4.01

>

1.685

39

0.000

<

0.05

different

 

Table 5  Nitrate data for paired samples analyzed in Table 4 (subset of data analyzed for Table 4)

Site

Mohawk R. at Schenectady    01354490

Hudson R. at Waterford     01335770

NWIS analyte code

630

620

630

620

Analyte name

[NO3- + NO2-]total

[NO3-]total

[NO3- + NO2-]total

[NO3-]total

Concentration units

mg NO3-N/L

mg NO3-N/L

mg NO3-N/L

mg NO3-N/L

average

0.778

0.740

0.520

0.513

stdev

0.261

0.244

0.160

0.158

rsd (%)

34

33

31

31

N

40

40

35

35

max

1.40

1.40

1.00

1.00

min

0.10

0.15

0.20

0.24

start

10/10/73

10/10/73

10/2/73

10/2/73

stop

5/28/75

5/28/75

5/27/75

5/27/75

 

 


Table 6. Hypothetical project and data quality objectives, with potential project decisions in italics (from guidelines in APHA 1998 ).

Project objective

Scenario 1:  Compliance monitoring
Scenario 2:  Ambient monitoring

Reason for monitoring

Measure the nitrate concentration for 50 specified stream locations in one year, and determine if any exceed 1.5 mg/L

Therefore highest precision and accuracy needed at 1.5 mg NO3-N/L

Measure nitrate concentrations along a river, below consecutive entry points of 15 first- and second-order streams, and determine if

a)      there is a 30 percent increase in concentration below a particular stream entering the river, or

b)      if there is a 50 percent change in concentration over time at a particular site

Therefore good precision and accuracy needed across range of analysis, eg 0.010 to 2.00 mg NO3-N/L

Target analyte(s)

nitrate (NO3-), that may affect the health of aquatic organisms Therefore retrospective or pilot data evaluated to determine need to resolve nitrate from nitrite, or to measure dissolved, not total

nitrate (NO3-), that may affect the health of aquatic organisms Therefore retrospective or pilot data evaluated to determine need to resolve nitrate from nitrite, or to measure dissolved, not total

Location(s) of interest

50 streams, anywhere in US:  therefore surface water to be analyzed, with suspended as well as dissolved nitrate, possibly in the presence of nitrite

15 first- and second-order streams, below their consecutive points of entry on a river, anywhere in US:  therefore surface water to be analyzed, with suspended as well as dissolved nitrate, possibly in the presence of nitrite

Time period of assessment

one year:  therefore monitoring at quarterly intervals, as determined by retrospective data for sites of interest or similar sites

one year:  therefore monitoring at monthly intervals, as determined by retrospective data for sites of interest or similar sites

Decisions and actions

Short-term

Classification of 50 streams:  no action vs follow-up needed

Potential restrictions on discharges of nitrate

Potential restrictions on stream-water use

Long-term

Development of plans for follow-up studies of streams with nitrate above 1.5 mg/L, to determine if environmental conditions favor high concentrations

Assessment of management options for reducing nitrate concentrations below 1.5 mg/L (control of discharges, runoff; nitrate removal, denitrification processes)

Short-term

Classification of the impacts of 15 streams on river water quality - identification of high inputs of nitrate and significant seasonal changes:  no action vs follow-up needed

Long-term

Development of plans for follow-up studies on select streams to understand environmental conditions favoring high nitrate concentrations that degrade downstream river water quality

 

Level of certainty

Confidence  level                                        95%

False positives (Type I error)                        5%

False negatives (Type II error)                    10%

Therefore, good precision (15%) and accuracy (> 85%) to be demonstrated at 1.0 as well as 2.0 mg/L nitrate

Confidence level                                   95%

False positives (Type I error)                  5%

False negatives (Type II error)              10%

Therefore, good precision (15%) and accuracy (> 85%) to be achieved across range of analysis, eg 0.010 to 2.00 mg NO3-N/L

Representativeness

 

number of sites

50 (specified streams)

15 (first- and second-order streams entering a specified river)

frequency of sample collection

quarterly, in one year

monthly, in one year

total number of samples

200

180

sample number Ni for site i

4 per stream:  average to be compared to target concentration of 1.5 mg/L nitrate

12 per stream entering river:  averages to be compared, for differences between consecutive sites

  3 per quarter for each stream:  averages to be compared, for differences between consecutive quarters

Completeness

98 percent overall:

for 45 streams (90 percent) acceptable (comparable) results reported for all quarterly samples.

for 5 streams (10 percent), acceptable (comparable) results reported for three quarterly samples

95 percent overall:

for 12 streams (80 percent) acceptable (comparable) results reported for all monthly samples.

for 3 streams (20 percent), acceptable (comparable) results reported for nine monthly samples in four quarters

 


Table 7. Hypothetical MQOs in two scenarios for nitrate analysis (from guidelines in APHA 1998 and USEPA 1994).

 

Scenario 1:  Compliance monitoring DQOs
Scenario 2:  Ambient monitoring DQOs

Accuracy

Analyte identification

 

target analyte(s)

(specificity, interferences)

IF mean values for nitrate vs nitrate and nitrite each are less than 15% at a 95% confidence level

      then determine nitrate (possibly including nitrite),

ELSE (difference is significant)

determine nitrate (alone, without nitrite)

nitrate (possibly including nitrite)

or

nitrate (determined separately from nitrite)

Results reported according to method of analysis used and its ability to resolve nitrate from nitrite

matrix

surface water (therefore, containing suspended as well as dissolved nitrate)

surface water (therefore, containing suspended as well as dissolved nitrate)

physical state

IF mean values for dissolved nitrate vs total (dissolved + suspended nitrate) are less than 15% at a 95% confidence level

      then determine total nitrate (possibly including suspended nitrate),

ELSE (difference is significant)

determine dissolved nitrate (alone, without suspended nitrate)

dissolved

or

total (dissolved + suspended particles)

 

Results reported according to method of analysis used and its ability to resolve dissolved nitrate from total (dissolved + suspended nitrate)

Accuracy

Analyte quantitation

 

concentration range

1.0 to 2.0 mg/L

0.010 to 2.00 mg/L

detection limits

£  0.5 mg/L (maybe not needed)

£ 0.005 mg/L (IDL:LLD:MDL:LOQ = 1:2:4:10 APHA 1998)

Bias (blanks)

£  0.5 mg/L (maybe not needed)

£ 0.010 mg/L

Bias (recovery)

Spiked samples:  40 total, from 20 streams;  one spike per stream in two different quarters, half at final concentrations of 1.0 mg/L total and half at 1.8 mg/L; concentrations differ from expected values by less than 20% with 95% confidence

Spiked samples:  40 total, from 10 streams; one spike per stream each quarter; two each at final concentrations of 0.25, 0.50, 0.75, 1.0 and 1.5 mg/L nitrate; concentrations differ from expected values by less than 20% with 95% confidence

Precision

 

 

sensitivity

Replicate standards or spiked reagent water:

5 percent    (0.1 mg/L)     at 2.0 mg/L

10 percent   (0.1 mg/L)     at 1.0 mg/L

Replicate standards or spiked reagent water

2.5 percent  (0.05 mg/L)       at 2.00 mg/L

5.0 percent  (0.10 mg/L)       at 1.00 mg/L

5.0 percent  (0.01 mg/L)       at 0.200 mg/L

25 percent   (0.005 mg/L)     at 0.020 mg/L

50 percent   (0.005 mg/L)     at 0.010 mg/L

significant figures

two

three

   reproducibility (field samples)

£        20% RSD for replicate field samples:

Replicate samples (collected from the same site at the same time):  40 total, from 20 streams;  one replicate per stream in two different quarters; concentrations differ by less than 20% with 95% confidence

£        20% RSD for replicate field samples:

Replicate samples (collected from the same site at the same time):  40 total, from 10 streams;  one replicate per stream each quarter; concentrations differ by less than 20% with 95% confidence


 

Table 8.  Summary of nitrate methods evaluated: CIE-UV-capillary ion electrophoresis with indirect UV detection; IC-CD ion chromatography-conductivity detection; RD-Vis reduction-derivatization-colorimetric detection.  Performance characteristics from the National Environmental Methods Index database (www.nemi.gov):  N/A = not available in method; +/- = does/does not meet the Measurement Quality Objective (MQO). MQOs for this comparison were:  detection level = 0.1 mg/L; accuracy ³ 95% recovery; and precision = £ 20% relative standard deviation (RSD), at 1 mg nitrate-N/L. Performance data are for a single laboratory unless otherwise specified.

Method Source

Method Number

Technique

Relative cost per determination

Nitrate – Nitrite Speciation

Concentration Range

(mg Nitrate-N/L)

Detection Level

 

(mg Nitrate-N/L)

Accuracy

 

(% recovery)

Precision

 

(%RSD)

Spiking Level

 

(mg Nitrate-N/L)

 

 

 

 

 

 

method

MQO

method

MQO

method

MQO

 

ASTM

D6508

CIE-UV

Moderate

Mixtures resolved

0.1-50

    0.08

+

140

-

        8

   + *

1.99

APHA

4140B

CIE-UV

Moderate

Mixtures resolved

0.1-50

    0.08

+

94

-

      13

   +

0.36

APHA

4110C

IC-CD

Moderate

Mixtures resolved

N/A

  0.017

+

103

 +

  N/A

  -

8

ASTM

D4327

IC-CD

Moderate

Mixtures resolved

check method

0.42?

-

100

+

      10

   + *

0.42?

EPA

300.0

IC-CD

Moderate

Mixtures resolved

N/A

  0.002

+

103

+

        2

   +

10

EPA

300.1

IC-CD

Moderate

Mixtures resolved

N/A

0.008

+

95

+

    < 1

   +

10

USGS

I-2057

IC-CD

Moderate

Mixtures resolved

check method

    0.05

+

    N/A

-

        8

   +

0.12

APHA

4500

RD-Vis

Low

By difference

0.01-1.0

    0.01

+

99

+

      14

   + *

0.5

EPA

352.1

RD-Vis

Low

By difference

0.1-2

    0.1

+

102

+

      14

   + *

0.5

USGS

I-2540

RD-Vis

Low

By difference

check method

     

 

 

 

 

 

 

USGS

I-2545

RD-Vis

Low

By difference

check method

     

 

 

 

 

 

 

* ML = Multi-laboratory determination

 

‘Low’ vs ‘Moderate’ cost depends on the DQO ie the target analytes:  nitrate alone requires two analyses by the RD-Vis methods - nitrite and nitrate+nitrite must be determined separately, and nitrate calculated by difference; if the DQO is for nitrate+nitrite, only one analysis is required;  using IC or CIE, nitrate and nitrite are determined separately in the same analysis.  Std Methods comment (pp 4-12):  ‘Instrumental techniques that can determine multiple analytes [KA  eg nitrate and nitrite] in a single analysis, ie ion chromatography (4110C) and capillary ion electrophoresis [KA 4140B], offer significant time and operating cost savings over traditional single-analyte wet chemical analysis’.

 

 

 

 


Appendix: Basic statistical interpretations underlying DQO/MQOs and data comparability

 

Mean value and confidence interval  In the simplest terms, analysis of  N samples gives Xi values, for i = 1 to N, with a mean value  and standard deviation s, defined by:

                        s           =                     =                            (A.1)

The mean and standard deviation are used to estimate the ‘true value’ m, in an interval defined by confidence limits (‘uncertainty’) about :

                        m          =                  ±          t                                                                   (A.2)

As shown in Table A.1, the value of t varies according to the degrees of freedom (df = N-1, for N measurements) and the desired statistical level of confidence P (the percentage of values Xi that would be included between the limits  ± ts/ÖN about ).  A confidence level P = 95% means that 95% of measured values will be within the uncertainty limits calculated for the mean. Likewise, 5% of measured values will lie outside the uncertainty limits.  For the confidence level written as P = 100%(1-a), the coefficient a is identified as the level of significance.

 

Comparisons involving mean values and their uncertainty limits are discussed below.  In general it is assumed that values have a normal distribution of error about the mean.  Current statistical software generally includes procedures to test for normal behavior, as well as functions to transform data to obtain a normal distribution, before further interpretations are made.

 

Comparison of single mean to a target value

 

In regulatory programs, the confidence intervals obtained by statistical analysis provide an objective means of comparing a result (represented by ± ts/ÖN) to a desired target value mo, to decide if they are the same or significantly different. In formal statistical terms, the hypotheses being tested are:

Ho  null hypothesis              m£ m o              difference is significant:             m - mo  £ 0         ‘no action’

H1  alternate hypothesis       m> m o              difference is not significant:        m - mo  > 0       ‘take action’

When applied to real-world analyses, Ho is defined to be true for the majority of results, so that H1 is true for the exceptions.  These hypotheses are tested by determining which of the following relationships is true:                                

Ho TRUE if         +  t     £ mo    at confidence level P:       mean is below threshold             (A.3)

H1 TRUE if          +  t     > mo  at confidence level P        mean exceeds threshold             (A.4)

The comparisons being made are considered ‘one-sided’:  Ho is not true, and H1 is true, only if the upper limit,  +   ts/ÖN, exceeds mo; the test does not involve the lower limit X – ts/ÖN, because, by definition, it is smaller than the upper limit.  Therefore the value of t, designated tcritical, is selected for the degrees of freedom df  and a confidence level P corresponding to the (1-a) case in Table A.1.

 

Relationships A.3 and A.4 are often rearranged for testing, so that the results can be displayed and discussed by comparing values of t or P and a, rather than the concentrations  +  ts/ÖN and mo, which are specific to a particular investigation.  For a given set of data, an effective value (or ‘t statistic’), teff is calculated from:

                        teff         =                                                                                                (A.5)

For the value of teff and the appropriate degrees of freedom df = N-1, there is also an effective probability Peff which represents the fractional area of Student’s t distribution, outside of the limits ± teff:  values of Peff  determined from ‘tail areas for t curves’ (Devore and Peck 2001). Therefore, in statistical textbooks and software applications, it is common to see the relationships A.3 and A.4 tested and reported in the form:

Ho TRUE if         teff £ tcritical                        or   Peff > a                      mean is below threshold             (A.6)

H1 TRUE if          teff > tcritical                        or   Peff < a                      mean exceeds threshold             (A.7)

Mathematically, the same information is being used to test for significance, but the criteria for assessment and the final result are presented somewhat differently

 

Comparison of two mean values

 

The ‘two-sided’ case is more general and widely applicable, in determining the comparability of data between monitoring programs, using different laboratories and methods of analysis, or between results obtained at different locations and times, using the same or different methods. If two results a and b are obtained with corresponding standard deviations sa and sb, the difference D of the two means is:

            D     º          (a     -        b)                                                                                    (A.8)

Statistical decisions are made based on the absolute value , without regard for which of the means is greater than the other.  The hypotheses being tested are simply:

Honull hypothesis              = 0       means are equal; (difference not significant at level P)

H1  alternate hypothesis       ¹ 0       means are not equal (difference is significant at level P)

 

The test for significance involves both the lower and upper limits for the range of values,

a ± ts a /ÖN a and b ± ts b /ÖN b.  Therefore the value of t is selected for a confidence level P corresponding to the (1-2a) case in Table A.1. 

a) comparison of two means with standard deviations not equal

If sa and sb are not equal, the test for significance is:

               >          t                                                                                            (A.9)

where Va and Vb are defined by:

            Va        =                                       Vb        =                                                     (A.10)

and the degree of freedom df is calculated from

            df         =                                                                                      (A.11)

which is rounded to the nearest integer.

b)  comparison of two means with equal standard deviations

If the standard deviations are equal sa = sb = s (or the population is large), the test for significance can be simplified to:

               >          t s                                                                                 (A.12)

b) comparison of two means using paired data  Many studies are designed to compare results of two measurements, which involve a subtle difference in procedures or in the target analyte being quantified.  Such comparisons benefit greatly from use of split or replicate samples; in statistical terms, the results are paired.  Therefore, interpretations are made using the average of differences between paired samples (also denoted , but not the same as Ddefined in eq. A.3):

                              =                         =                                         (A.13)

 and the corresponding standard deviation:

                        sd         =                =                     (A.14)

The tests applied are essentially a one-sided comparison of a single mean value to a target value of zero (moºmdº 0), using:

                        teff         =                                                                                                   (A.15)

 

Type I and Type II errors

 

In the above discussion, the value of a also defines the probability of a Type I error:  the chance that a significant difference is determined between two values being compared, when in fact they are identical; in this case hypothesis Ho is true but rejected (false positive).   Type II errors are also encountered: the coefficient b is used to describe the probability that two values are found equivalent, when in fact they differ; in this case hypothesis Ho is false but accepted (false negative).  The statistical power of analysis is 1-b:  the probability of rejecting Ho when H1 is true. 

Shown schematically, the possibilities are: 

Actual condition

 
 


Analytical result

 

Ho TRUE

H1 TRUE

Ho TRUE

Correct result

Type II error (false negative)

H1 TRUE

Type I error (false positive)

Correct result

 

The coefficients a and b are inversely related: as a decreases, b increases (Devore and Peck 2001).  Separate quality assurance procedures are needed to cross-check their values.  Moreover, their relative importance is viewed differently from the perspective of a program manager, a decision maker, a budget administrator, aquatic organisms exposed to the substance being determined, and persons with various interests in using the water being analyzed.

 

 

References

 

Devore, J.; Peck, R. 2001.  Statistics:  the Exploration and Analysis of Data.  Duxbury:  Pacific Grove, CA.

Statistical vs practical significance  Devore/Peck 2001 p 386.

Evans, M.; Hastings, N.; Peacock, B. 2000. Statistical Distributions.  John Wiley: NY.

 

 


 

Table A.1  Values of t in Student’s t distribution for specified confidence levels (percent of included area with one or two tails excluded, P = 100%(1-a) or 100%(1-2a) respectively), adapted from Evans et al 2000.

 

t values

df = N-1

One-sided (1-a) test

Two-sided (1- 2a) test

 

90%

95%

99%

90%

95%

99%

1

3.078

6.314

31.821

6.314

12.706

63.657

2

1.886

2.920

6.965

2.920

4.303

9.925

3

1.638

2.353

4.541

2.353

3.182

5.841

4

1.533

2.132

3.747

2.132

2.776

4.604

5

1.476

2.015

3.365

2.015

2.571

4.032

6

1.440

1.943

3.143

1.943

2.447

3.707

7

1.415

1.895

2.998

1.895

2.365

3.499

8

1.397

1.860

2.896

1.860

2.306

3.355

9

1.383

1.833

2.821

1.833

2.262

3.250

10

1.372

1.812

2.764

1.812

2.228

3.169

20

1.325

1.725

2.528

1.725

2.086

2.845

30

1.310

1.697

2.457

1.697

2.042

2.750

40

1.303

1.684

2.423

1.684

2.021

2.704

50

1.299

1.676

2.403

1.676

2.009

2.678

100

1.290

1.660

2.364

1.660

1.984

2.626

120

1.289

1.657

2.351

1.657

1.980

2.618

¥

1.282

1.645

2.326

1.645

1.960

2.576