Methods and Data Comparability Board
Meeting in St. Petersburg, Florida
Heritage Holiday Inn
March 7-10, 2005

Bioassessment Comparability -- Wadeable Streams Assessment

EPA-WSA Meeting 3/8/2005

Participants : Laura Gabanski, Herb Brass, Eric Vowinkel, Greg Pond, Franceska Wilde, Erich Emery, Ed Santoro, Cary Jackson, Merle Shockey, Rick Dunn, Ed Johnson, Larry Keith, Mike Miller, Gil Dichter, Eli Greenbaum

Jerry Diamond, EPA Contractor

Call-in: Mark Southerland, Jon Volstad, Eric Silldorf, Karen Blocksom, Sam Stribling

Agenda : The agenda is on the Board's internal meeting web site: wi.water.usgs.gov/methodsboard/ The username is “methods” and the password is “nocarp99”

Linkages with other on-going efforts

•  EPA's Forum for Environmental Methods (FEM) is actively involved with performance-based systems for methods and the issue of comparability across EPA. The WSA-comparability work is related. Herb will send Laura contact information with the FEM PBS Action Team.

•  The Design Workgroup under the National Council will be developing a design for the Monitoring Network associated with the National Ocean Network. This design will include biological methods and data. Already some discussion that methods need to be prescriptive, which is counter to the Board and Council recommendations and counter to the PBS approach to bioassessment methods. Need to be sure to link WSA comparability/performance work with the Network Design project. Chuck Spooner will serve as a liaison.

•  Large river bioassessment methods being developed by EPA-ORD and HECD. Board panel members should coordinate efforts with EPA. Ed Santoro (DRBC) and Erich Emery (ORSANCO) will contact Laura.

Methods Board Advisory Panel Responsibilities

Laura outlined the intended responsibilities of the Board Advisory Panel, which includes:

•  Provide input and guidance on all deliverables

•  Provide input on analyses of WSA comparability pilots

•  Review and comment on various drafts of the Guidance Document and the Critical Review Paper

•  Interact with EPA's Workgroup to provide guidance on the project and future directions.

Official members of the Advisory Panel so far are: Herb Brass, Eric Vowinkel, Erich Emery, Ed Santoro, Mike Miller, LeAnne Astin, Eric Silldorf, Russ Frydenborg, Lisa Huff, Roy Irwin, Dave Herbst, Katherine Alben, Steve Moulton, and Jan Stevenson. Other participants to the Advisory Panel are welcome and certain panel members may wish to provide more technical assistance and help write/prepare documents. Other USGS experts (particularly NAWQA staff) will be contacted by Laura and Eric Vowinkel for possible participation as well.

Project Background

Laura gave a brief introduction of the project and highlighted a few important issues:

•  EPA reports to the public on stream biological condition on a national and regional (Ecoregion Level 2) basis. Coastal assessments were done previously and resulted in two national coastal condition reports. EPA will assess and report out on lakes next.

•  EPA is exploring the notion of comparability of state bioassessments in order to combine state results for a national assessment. Need comparability – hence, this project.

•  On-going comparability pilots funded by EPA, as part of the WSA, need to be analyzed and evaluated. New, better pilots are needed that follow consensus procedures for determining performance and comparability for bioassessments.

•  States have multiple objectives for using bioassessments – different DQOs perhaps. Sampling design will differ depending on objectives.

•  The WSA-comparability project will attempt to validate the PBS approach for bioassessments. This project needs to evaluate how well PBS works in this case. This project has significance to EPA as a whole, which is evaluating PBS agency-wide.

Background information on the Methods Board approach to PBS and comparability in bioassessments was summarized by Jerry. A powerpoint presentation, given by Jerry at the EPA Region 4 state biologist meeting in 2004, is on the meeting site.

Example Case Studies

Brief summaries of on-going or recently completed studies were presented by Mike Miller (Wisconsin), Erick Silldorf (California), Karen Blocksom (EPA Cincinnati), Erich Emery (Ohio), Mark Southerland and Jon Volstad (Maryland, Virginia, West Virginia), and Sam Stribling (Maryland, Mississippi). Materials or information are on the meeting web site for work presented by Miller, Silldorf, Blocksom, Southerland and Volstad, and Stribling. Jerry will contact Dan Sullivan to get LeAnne Astin's presentation regarding the Potomac River state comparability evaluation on the meeting web site as well. A link will also be provided to the Region 4 SWPBA meeting (Lisa Huff) and to the California state biologist meeting from 2004.

Discussion during the Case Study presentations lead to the following ideas, suggestions:

•  Intra-site as well as inter-site precision is useful to know and important for characterizing performance.

•  Need to evaluate assessment level as well as data and method comparability.

•  Assessment level may be difficult to evaluate unless sufficient data have been collected to develop indices and an assessment framework.

•  Paired observations (upstream-downstream) may be very useful for determining method performance to specific stressors and levels.

•  Reference conditions will need to be put on the same par to combine assessments from different programs - could be a challenge.

•  Acceptance criteria for comparability (or performance) rarely done beforehand. Need to identify criteria or business rules for defining how good is good and whether data are comparable. Start with determining what performance characteristics that methods are currently achieving.

•  Need to balance cost with performance-better method performance may be unnecessary if less costly, simpler method meets DQOs. Difficulty identifying DQOs and whether these are comparable across states and for 305(b) reporting.

•  Biological Condition Gradient appears to be a useful tool for comparing and aggregating assessments.

•  Human disturbance gradient needs to be defined as it can be a useful means to test the bias of a bioassessment method.

•  The EAGLE project out of EPA-ORD Duluth should be checked - they are developing metrics, indices, and stressor gradients for Great Lakes tributaries.

•  Comparability assessments at the data level (including metrics and indices) appear to be very plausible and many have demonstrated how to do this for various methods.

Guidance Document

•  Audience is states, tribes, interstates, consultants, other agencies (USGS, Army Corps), volunteers

•  In the Intro section, explain DQOs and how a program sets them

•  The Intro also needs a discussion of different purposes of bioassessments, the DQOs and study objectives for each purpose, and the type of performance (MQOs) that may be needed (e.g., TMDL vs status and trends use of bioassessments)

•  The document will include a hands on protocol on how to document performance and how to determine comparability for:

•  Data, methods, assessment levels

•  Using TALU guidance/concepts

•  Incorporate lessons from various studies such as those presented at the meeting

•  Case studies need to be included (new ones, such as Region 4, using the protocol developed in the document), preferably for different ecotypes

•  EPA regions and states will need trainings (a Workshop) once the document is put together

Critical Review

•  Laura noted that the Review will be limited by EPA funding and time. The goal is to look internationally and across ecosystem and assemblage types.

•  EPA is looking for a contractor not closely associated with EPA or WSA.

•  The Review will need to be completed in a few months (this FY).

•  Results of the Review will provide input into the Guidance Document and related projects.

Next Steps

•  Board needs to give the Council an update on status of project at their April meeting.

•  Advisory panel input needed to review WSA comparability analyses.

•  Tetra Tech will develop a revised draft outline for the Guidance Document and distribute to the Advisory Panel for review and comment.

•  Perhaps form a separate workgroup involved with the Guidance Document to ensure there is an adequate review. More State involvement is desirable.

•  Laura and Tetra Tech getting current WSA cooperators on a conference call to review status and next steps for WSA comparability studies. Results of this call will be distributed to Advisory Panel.

•  Get first draft of the Guidance Document (minus the new case studies) by end of this year (December).

•  Get more NAWQA and other USGS experts involved in Panel. Laura will get contacts and invite appropriate people.

NEMI WORKGROUP MEETING NOTES

 4. Other issues discussed (including, but not limited to…ideas for future projects, issues to bring before full Board, linkages/connections to other workgroups, ideas for new members)

7. Plans for the next meeting

8. Any between meeting conference calls planned?

Approximately every two weeks a routine workgroup call will be initiated by Dan.

9. Other comments/notes/issues

WQDE Workgroup meeting notes 3/10/05

Review meeting agenda

Monitoring methods design network

•  Addressing such questions as: Why no comparability in uplands vs. coastal assessment methods/data?

•  Looking for recommendations from workgroup/Board

•  Involvement with Chesapeake Bay program

•  Conference call on Monday 3/14 to discuss what MDCB is being charged to do

Guidance document

•  Comments from workgroup, MDCB, and selected reviewers DUE 3/18/05

•  Send final draft to ACWI by 3/31/05

•  Toni Johnson to draft cover letter

•  ACWI conference call (by end of April?) to discuss guidance document

•  Toni Johnson to set exact timeline for comments, final approval

Water Quality Data Elements

•  “Other” DE lists (fish tissues, biomarkers)

•  Workgroup agreed that other priorities (PHDEs, Methods Design Network, et al.) take precedence.

•  Curtis Cude to send links to Environmental Data Sampling and Results Standards for review by group, consideration as to whether these could be adapted for fish tissues at some point in the future

•  Bio/Tox WQDEs

•  Presented to ACWI at September meeting

•  To be sent with guidance document for comments (see above)

•  Again, Toni will set final timeline for approval

•  Some discussion re: module-specific QA elements vs. “global” QA elements which apply to an entire dataset, and whether to create a separate module (“How Good”) for the latter (no).

•  Workgroup agreed that Definitions in module 7.0 (Sample Analysis and QC Module) for elements 7.2, 7.3, and 7.4 should be clarified for population/community-level assessments (LeAnne will work on this)

•  PHDEs

•  Re-sent to workgroup members in early February for comments

•  Group discussed whether or not module numbers should be “aligned” with number system in original chem./micro WQDEs (we decided not to mess with these)

•  Considerable discussion about whether the draft PHDE's capture necessary information about classes of attributes. Revital provided list (attached) mapping how she groups physical measurements into related categories.

•  Also discussed differing levels of organization in result reported/endpoints for habitat assessments: e.g. single qualitative parameters (ex. Embeddedness), individual quantitative measurements (ex. Flow), or numeric scores based on multimetric indices (ex. qualitative multimetric assessments like RBP, indices combining qualitative and quantitative measures like Maryland DNR's Physical Habitat Index (PHI).

•  Several workgroup members agreed to test the PHDEs for individual measures and index values by “running them through” the PHDEs (i.e. replacing the definitions with actual information). Test results will be shared with the entire workgroup by e-mail.

•  Results & suggested modifications will be discussed in a Spring conference call

Web Page and Tools

•  Excel WQDE template

•  Examples were sent to group prior to meeting

•  Template is intended as a demo to illustrate how to organize monitoring data using data elements, and that the DE lists are not “too onerous,” that most entities already collect this info

•  Workgroup agreed a tool like this might be useful

•  Some discussion of how far we want to go with this…How much time and effort are we willing to spend on development (answer: not too much). How do we intend for it to be used? Do we want site visitors to be able to download workbooks or worksheets?

•  Renewed call for workgroup to suggest links, esp. about data management, metadata, using PDAs in the field, case studies using the data elements, etc.

•  Web page design issues will need to be discussed in a future conference call (hopefully with Dan Sullivan's participation)

Two-year workplan objectives

Guidance document – published

Outstanding WQDEs (bio, tox, hab) – approved

Web site – updated

New Tools - ??

return to meetings home