A.Y. Cantillo and G.G. Lauenstein

Quality Assurance Project and Mussel Watch Project Managers

National Oceanic and Atmospheric Administration, National Status and Trends Program

N/ORCA21, 1305 East West Highway, Silver Spring, MD 20910

The NOAA National Status and Trends (NS&T) Program determines the current status of, and changes over time in the environmental health of US estuarine and coastal waters. Concentrations of organic and inorganic contaminants are determined in bivalves, bottom-dwelling fish and sediments. The quality of the analytical data generated by the NS&T Program is overseen by the performance-based Quality Assurance Project. This Project has been in operation since 1985and is designed to document sampling protocols, analytical procedures and laboratory performance, and to reduce intralaboratory and interlaboratory variation. To document laboratory expertise, the QA Project requires that all NS&T laboratories participate in a continuing series of intercomparison exercises utilizing a variety of materials. The organic analytical intercomparison exercises are coordinated by the National Institute of Standards and Technology, and the inorganic exercises by the National Research Council of Canada. The QA Project will facilitate comparisons among different monitoring programs with similar QA activities and thus extend the temporal and spatial scale of such programs.

The NOAA National Status and Trends (NS&T) Program determines the current status of, and changes over time in the environmental health of US estuarine and coastal waters. Concentrations of organic and inorganic contaminants are determined in bivalves, bottom-dwelling fish and sediments. Two projects of the NS&T Program are the major producers of data: the National Benthic Surveillance Project and the Mussel Watch Project.

The National Benthic Surveillance Project collected and analyzed benthic fish and sediments from sites around the coastal and estuarine United States, including Alaska, during the years 1984-1992. This effort was performed primarily by NOAA’s National Marine Fisheries Service. The Mussel Watch Project collects and analyzes bivalve mollusks and associated sediments from around the United States, including the Great Lakes, Alaska, Hawaii, and Puerto Rico. The Mussel Watch Project began in 1986. This effort is administered by NOAA, with collection and analyses being primarily performed under contract. The NS&T core analytes include 24 polycyclic aromatic hydrocarbons, 20 polychlorinated biphenyl congeners, DDT and its metabolites, 9 other chlorinated pesticides, organotins, 5 major elements, and 12 trace elements. Sampling sites are described in Lauenstein et al. (1997). Results of the NS&T Mussel Watch Project can be found in Cantillo and Lauenstein (this volume).

The quality of the analytical data generated by the NS&T Program is overseen by the performance-based Quality Assurance (QA) Project. The QA Project, in operation since 1985, assures that despite differences in the analytical methodologies used, data are comparable between all participating laboratories. The QA Project is designed to document sampling protocols, analytical procedures and laboratory performance, and to reduce intralaboratory and interlaboratory variation.

Since contaminant concentrations vary as a function of where samples are collected, it is essential to occupy the same site year after year in order to determine environmental trends. With the new Global Positioning System (GPS) technology a site can be re-occupied to within a few meters. All sites are be plotted on charts and/or maps that are helpful for reoccupying sites in relation to prominent land features. Photographic documentation of each site, and directions as to how to reach the site, are also important. Since field samples must be collected in a consistent manner all individuals regardless of their level of scientific expertise are trained in field sampling protocols.

The NS&T QA Project is performance based and does not prescribe specific analytical methods but encourages the use of state-of-the-art procedures. This allows the use of new or improved analytical methodology or instrumentation without compromising the quality of the data sets. It also encourages laboratories to use the most cost-effective methodology while generating data of documented quality. The methods used by the various laboratories contributing to the NS&T monitoring effort have been documented by Lauenstein and Cantillo (1993 and 1997).

To document laboratory expertise, the QA Project requires that all NS&T laboratories participate in a continuing series of intercomparison exercises utilizing a variety of materials. The organic analytical intercomparison exercises are coordinated by the National Institute of Standards and Technology (NIST), and the inorganic exercises by the National Research Council of Canada (NRC). The QA Project facilitates comparisons among different monitoring programs with similar QA activities and thus extends the temporal and spatial scale of such programs.

The materials used for the intercomparison exercises include samples with unknown (to the participants) contaminant concentrations, and Standard Reference Materials (SRMs) and/or Certified Reference Materials (CRMs). The type and matrix of the exercise materials change yearly and have increased in complexity over time. Typical results of the intercomparison exercises are discussed below.

Two sediment and two tissue materials were used for the 1993 intercomparison exercise for trace metals. National Research Council Canada BCSS-1 and NIST SRM 1566a were the known materials, and NRC prepared Sediment T, a freeze-dried Mississippi Delta sediment, and Tissue S, a freeze-dried mussel tissue homogenate collected by the International Atomic Energy Agency (IAEA) in the Mediterranean off the coast of France, as the unknowns. Typical results of intercalibration exercises for the NS&T cooperating laboratories are shown in Figure 1.

Results of analyses of SRMs and/or CRMs are not compiled and evaluated as part of the trace organic intercomparison exercises. Rather, unknown materials are prepared by NIST for each exercise. SRMs and CRMs are analyzed as part of the analytical sample string in which the unknown materials are analyzed. These results are part of the control chart information described above. Some of the materials used for trace organic exercises, however, are cuts of the material used to prepare NIST SRMs or are candidate SRMs in the certification process and so are, in effect, unknowns.

As part of the 1993 trace organic exercise, a fish homogenate of carp collected in Saginaw Bay was prepared by NRC and provided to NIST. This material was analyzed for all NS&T analytes except polycyclic aromatic hydrocarbons, since these compounds are found in very low concentrations in fish tissue. Typical PCB results are shown in Figure 2. Most of the PCB results fall within the range defined as ±35% of the consensus value plus or minus one standard deviation. In this example a positive bias is shown in the data reported by laboratory A.

The analysis of reference materials, such as the NRC CRMs and NIST SRMs, and of control materials generated for use by NS&T contract laboratories as part of the sample stream, is required.

To identify suitable reference materials for use by the NS&T Program and following the recommendation of the Intergovernmental Oceanographic Commission/United Nations Environment Programme/IAEA Group of Experts of Standards and Reference Materials (GESREM), NOAA began in 1986 a compilation of standard and reference materials for use in marine science (Cantillo and Calder, 1990; Cantillo, 1993). In response to the needs of the NS&T Program, NOAA contributed funds to the production of eight NIST Standard Reference Materials and seven internal standard solutions (Table 1). The SRMs are based on natural matrices and are prepared at two concentration levels. The calibration solutions are for each of the three chemical classes of analytes quantified by the NS&T Program. The latter are used to facilitate the preparation of multipoint calibration curves. The internal standard solutions were prepared at the request of the NS&T contract laboratories and are currently available for purchase from NIST. These SRMs, CRMs, and control materials have been, and continue to be, used by NS&T contract laboratories to maintain analytical control.

A minimum of 8% of the organic analytical sample string consists of blanks, reference or control materials, duplicates, and spike matrix samples. The use of control materials does not entirely replace the use of duplicates and spiked matrix samples. A minimum of 2% of the standard inorganic sample string consists of calibration materials and reference or control materials. Analytical data from all control materials and all matrix reference materials are reported to the NS&T Program office and these data are stored as part of the NS&T data archive.

When the program began, data at or near the detection limit were to be reported following procedures defined by Keith et al. (1983) who defined the limit of detection (LOD) as the lowest concentration level that can be determined to be statistically different from a blank. The standard deviation (used to determine the LOD) was defined by replicate measures of the difference between the lowest concentration of analyte instrumentally detectable and a blank value. Any measured value below the LOD were considered to be not detected. The limit of quantitation (LOQ) was defined as 10x the standard deviation of the blank.

In 1990, the NS&T Program replaced LODs and LOQs with Method Detection Limits (MDLs). These values are not based on the variability of blanks but rather on the standard deviation of the signal from replicate analysis of real matrix samples containing, in principle, low levels of the analyte (CFR, 1990). The MDL is "x" times the standard deviation, where "x" is defined by the Student’s t-distribution to cover 99% of the distribution of possible values (for 7 analyses, x = 3.5). MDLs were developed from a minimum of 7 replicate analyses. Actual detection limits from the program are provided in Lauenstein and Cantillo, 1993 and 1997.

Acceptable limits of precision for organic control materials are ±30% on average for all analytes, and ±35% for individual analytes. These limits apply to those materials where the concentrations of the compounds of interest are at least 10 times greater than the MDLs. The application of these guidelines in determining the acceptability of the results of the analysis of a sample is a matter of professional judgement on the part of the analyst, especially in cases where the analyte level(s) are near the limit of detection.

The results of the routine analysis of reference materials, other control materials, and blanks are reported annually to the NS&T Program office and are used to prepare control charts. Examples of control charts can be found in Cantillo (in preparation). Some of the congeners quantified as part of the NS&T Program coelute when using commonly available gas chromatography columns for analyte separation. A discussion of this topic can be found in Lauenstein and Cantillo (1993) and Schantz et al. (1993).

Once data have been received for a given year of monitoring, data are compared to analyte concentrations for that same site from earlier years. Sample homogenates are reextracted and reanalyzed for sites where unexpected increases or decreases of contaminant concentrations were noted. These reanalyses are performed by an independent laboratory. Approximately 15 samples are reanalyzed annually for trace elements and organic contaminants.

Determinations of Ag concentrations in mussels and oyster tissue digested with nitric acid were low relative to those measured in undigested tissue analyzed by ultrasonic graphite furnace atomic adsorption spectroscopy (Daskalakis et al., 1997). Good results were obtained, however, using a mixture of hydrochloric and nitric acids for digestion. Archived NOAA Mussel watch samples collected in 1986-1993 and originally analyzed following HNO₃ or HNO₃-HClO₄ digestion were reanalyzed with HNO₃-HCl digestion in 1995. Results suggest that only 44% of the redetermined Ag concentrations were within 20% of the original values. Most of the re-analyses yielded higher concentrations, and one tenth of them were more than 100% higher. As a result the new data yield information that indicated there were decreasing Ag trends along the northeast coast of the US.

Laboratories competing for a Mussel Watch Project contract were required to submit results of their analysis of a test sample (Lauenstein and Cantillo, 1997). During the 1989 selection process, laboratories that appeared to qualify on the basis of their written proposals were provided a gravimetrically prepared solution with "unknown" quantities of an undefined number of organic chemicals. In 1994, competing laboratories were once again tested but this time using matrix materials for the quantification of both trace elements and organic chemicals. Three laboratory groups participated in the exercises. For the 1989 gravimetrically prepared solutions, all participating laboratories were able to identify the chemicals and in all but two cases were able to report concentrations to within ±25% of the known values. In 1994, all laboratories were within the acceptance criteria for the quantification of trace elements and chlorinated chemicals in homogenate samples, though two laboratories were outside of the acceptable range for one of the four PAHs used to evaluate laboratory performance.

It has been shown that the performance of laboratories improves with time, as the result of experience gained through participation in intercomparison exercises (Cantillo, 1995; Willie and Berman 1996; Willie, 1997). This improvement can be demonstrated through the continued analysis of a material, such as a CRM, SRM or a control material with known analyte concentrations. The NOAA intercomparison exercises for trace metals for 1991 through 1993 used BCSS-1 as part of the exercise materials. These exercises are open to all laboratories, not just those participating in the NS&T Program. Typical results reported by a laboratory joining the exercise program in 1991 are shown in Figure 3. The accuracy of the Cr, Zn and Se determinations improved with time, as did the precision of the Se analysis.

As part of the evaluation of results of the intercomparison exercises for major and trace elements, NRC assigns a performance evaluation criteria based on the number of times results reported by a laboratory fall within acceptable criteria. The percentage of laboratories achieving superior or good performances has increased since 1991 from 46% to 83% (Figure 4). Superior--rated laboratories submitted results for most analytes within the 95% confidence intervals; good--rated laboratories submitted many results within the accepted range with a minimum number of outliers (Willie, 1997).

No CRMs or SRMs are analyzed specifically as part of the trace organic intercomparison exercises, so an evaluation similar to the one done for the trace metal exercises using changes in CRM and SRM results over time is not possible.

A measure of improvement of laboratory performance can be made, however, by comparing the performance of a laboratory joining the exercises for the first time and that of a laboratory that has participated for several years. Laboratories newly joining the exercises usually have larger percent errors than the veteran laboratories. Within a year or two, however, the performance of the new laboratories typically improves and equals those of the veteran laboratories.

The QA Program should include the monitoring data. The contractor laboratories that perform the analyses provide a first level evaluation of the quality of the data based on the limits of acceptability in the NS&T contract. Results of analytical sample strings that fail to pass the QA guidelines are rejected and the samples re-analyzed. Rejected results are not forwarded to the NS&T Program office.

Once the data reach the NS&T Program office, QA evaluation of the data is done by the NS&T Program manager as the new data are manipulated and merged with the existing multi-year data set. This process has resulted in the detection of some discrepancies.

Filters to detect high analyte values and large changes in a given site from one year to another should be used on new data before it becomes part of the NS&T database. Abnormally high values submitted to the Program office often revealed differences in sampling location.

In one instance, one oyster sample of a set of triplicates obtained at the same site, was composited from oysters growing on a creosote-covered decking pole. This sample had unusually high values for PAHs and the results were discarded. The contractor had documented via photography the location of each station in the site as required by the contract.

In several instances, data were reported to the NS&T Program office in units different than those of previous submissions. This type of discrepancy is easily detected since the differences are often in multiples of 1000 (i.e., ng/g instead of µg/g).

A system should be in place to link the data QA parameters associated with an analytical sample string, such as blanks, duplicates, matrix spikes, and reference materials results, to the corresponding samples analyzed as part of that analytical sample string. The information is currently found in the contractor laboratory data submission reports. Efforts are underway to store the data quality parameters in electronic form so it can be related to the corresponding sample data.

The NS&T Program is a long-term monitoring effort. As time passes, there is a natural turn-over of personnel involved in the Program. As this happens, corporate memory about the Program decreases, and detailed documentation becomes very important. A coordinated effort is in place to archive the NS&T electronic data and critical documentation in more than one geographical location so that in case of loss of material from one location, duplicates can be found elsewhere. Although there is an effort to reduce the amount of paper documentation, it should be remembered that electronic media are ephemeral and are easily corrupted. Paper documentation, as well as electronic versions, should be encouraged for critical information.

Critical documentation should consist of, at minimum, details of sample location and sampling times, detailed description of analytical methods, the data, data QA parameters, results of the performance by the contract laboratories in the intercomparison exercises, and any changes/errors found and corrected.

The current status of NS&T Program critical documentation is as follows:

Quality assurance is an essential part of environmental monitoring programs, especially those that are not constrained by specified analytical procedures. The performance based QA project described in this paper allows for the introduction of new instrumentation and analytical techniques that may result in improved data quality or savings in time and resources. Analytical precision and accuracy of new laboratories joining an existing monitoring program can be quantified and improved, and the performance of veteran laboratories can be monitored and corrected if necessary. CRMs and SRMs provide the benchmark necessary to document laboratory performance. If data are biased during some portion of the monitoring program, the elucidation of real trends could be missed or alternately false trends could be indicated. Quality assurance should be considered in laboratory selection, sample collection, contaminant quantification, and review of analytical results.

The SRMs are two natural matrix materials and calibration solutions at two concentration levels of the three chemical classes of analytes.