DERR Data Quality Guidelines

epa.ohio.gov • 50 W. Town St., Ste. 700 • P.O. Box 1049 • Columbus, OH 43216-1049 • 614-644-3020 • 614-644-2737 (fax)

Division of Environmental Response and Revitalization

December 2023

There needs to be a sufficient quality and quantity of environmental data to support

remedial and compliance decisions. Proper data quality consideration facilitates effective

use of resources and making informed remedial and compliance decisions when protecting

public health and the environment.

Overview

Ohio EPA has a quality system that includes a quality management plan (QMP) for ensuring that environmental data

meets quality and quantity expectations to support the intended assessment or regulatory decisions and is legally

defensible when needed. This system consists of the objectives, principles, authority, responsibilities,

accountability, and QMP that ensures quality in our work and provides a framework for planning, assessing,

documenting, and reviewing data quality. It is essential that when data is collected, analyzed, used, reported and/or

compiled by, or for, Ohio EPA that the data is of acceptable quality to meet the needs of users and decision-makers.

The principal tools and practices used to manage data quality include systematic planning, data quality assessment,

and data quality audits. This data quality strategy presents guidelines and expectations to ensure data quality

practices are implemented consistently within the Division of Environmental Response and Revitalization (DERR).

These guidelines represent the Ohio EPA’s position related to data quality assessment. While this document is

consistent with data verification requirements under the Ohio EPA’s Voluntary Action Program (VAP), this

document does not have the force of law.

Systematic Planning

Systematic planning is essential to assure quality in environmental data that is generated for remedial decision-

making and compliance purposes. It utilizes an objective approach to generating acceptable results. Systematic

planning includes:

•

Defining the project goal, objectives, and issues to be addressed

•

Designating available resources (i.e., personnel, expertise, information sources, equipment, funds, etc.)

•

Identifying the type and amount of information needed, how the information will be used, and any

constraints on information collection

•

Specifying performance criteria for measuring data quality and the quality assurance/quality control

(QA/QC) activities needed to assess quality

•

Determining how, when, and where the information will be obtained

•

Describing how the acquired information will be analyzed, evaluated, and assessed

Systematic planning involves establishing data quality objectives (DQOs) and developing a conceptual site model

(CSM). It should consider quality assurance in both field sampling techniques and laboratory analysis, including a

plan for verifying and validating data quality once data is obtained. The DQOs that result from systematic data

planning should be documented in a quality assurance project plan (QAPP), sampling and analysis plans (SAPs), or

summary report (e.g., remedial investigation report, Resource Conservation and Recovery Act (RCRA) facility

investigation report, phase II property assessment report, etc.).

DERR Data Quality Guidelines

Page|2

Data Quality Objectives

Environmental data that is to be used for estimating an exposure point concentration or critical point decision-

making during and assessment or remedy should be obtained using the DQO process. The DQO process is a flexible,

common-sense approach designed to generate performance and acceptance criteria for data collection. It consists of

seven iterative steps which include:

•

State the problem

•

Identify the goal

•

Identify information inputs

•

Define the boundaries of the investigation

•

Develop an analytical approach

•

Specify performance and acceptance criteria

•

Develop a plan for obtaining data

Using the DQO process can facilitate good communication, documentation, and effective use of resources. For more

information on the DQO process, refer to U.S. EPA’s Guidance on Systematic Planning Using the Data Quality

Objectives Process.

Conceptual Site Models

A CSM is an iterative, “living” representation of a contaminated site (or property) that provides a simplified and

concise summary of contamination sources and distribution, release mechanisms, exposure pathways and

migration routes, and human and ecological receptors (U.S. EPA, 2011)

Developing a CSM is an integral step in clarifying cleanup objectives for a site and determining appropriate DQOs.

The CSM is a hypothesis with the objective of making site-specific predictions about the occurrence of

contamination at a property, and its potential to adversely affect human and ecological receptors (Sayko and

LaRegina, November 2014)

. As additional data is collected, the CSM should be updated accordingly. For more

information on developing CSMs, refer to DERR’s Conceptual Site Models guidance document.

Data Quality Assessment

Why does data quality assessment matter?

Data quality assessment enables data review for consistency, quality, and relevance before it is used for making

decisions. It serves to increase confidence and reduce uncertainty when data is used evaluate compliance, make

remedial decisions, or support enforcement actions. Data quality assessment promotes the completeness,

correctness, consistency, and compliance of data against a standard or project-specific criteria.

U.S. EPA, July 2011, Environmental Cleanup Best Management Practices: Effective Use of the Project Life Cycle Conceptual

Site Model, EPA 542-F-11-011.

Sayko, S.P. and J. LaRegina, November 5, 2014, Using Conceptual Site Models to Communicate Project Understanding,

Pennsylvania Council of Professional Geologists (PCPG) Professional Development Seminar, Monroeville, PA.

DERR Data Quality Guidelines

Page|3

How should data quality assessment be performed?

DERR recommends a tiered approach to data quality assessment. The data quality assessment tiers include the

following:

Data Verification – the process of evaluating the completeness, correctness, and compliance of a specific

data set against the method, procedural, or contractual requirements. The goal of data verification is to

ensure data are what they purport to be (i.e., the reported results reflect what was done).

Data Validation – a data quality review process that extends data evaluation to determine the analytical

quality of a specific data set based on measurement quality objectives. It is an analyte- and sample-specific

process that determines the analytical quality of a specific data set based on criteria for accuracy, precision,

bias, representativeness, completeness, comparability, and sensitivity.

Tier I data validation includes a general review of sample receipt, analysis, and the ability of the

instruments to recover the elements or compounds that were analyzed.

Tier II data validation includes a more thorough review of parameters that primarily deal with

instrument calibration and analysis sensitivity.

Further data validation may be needed if data issues are identified, but validation beyond Tier II, which may include

review of the entire analytical process including raw data, is rarely needed.

Data quality assessment should also include a preliminary data review that consists of performing basic statistical

analyses and graphically depict data to aid in data interpretation. Determining basic quantitative characteristics of

the data includes detection frequency, distribution, measures of central tendency (e.g., mean, median, mode),

measures of dispersion (e.g., range, variance, standard deviation, coefficient of variation, or interquartile range)

measures of relative standing (e.g., percentiles), and measures of association between two or more variables (e.g.,

correlation). Visual displays of data can be used to identify patterns and trends in the data. Graphical

representations include displays of individual data points, statistical quantities, temporal data, or spatial data.

Who should perform data quality assessment?

While data collectors (e.g., samplers, field personnel, contractors, etc.) and data generators (e.g., laboratory analysts,

laboratory technicians, etc.) may have relevant information needed for data quality assessment, it is the

responsibility of the data users (e.g., project managers and coordinators, technical reviewers, consultants,

volunteers, potential responsible parties, and owners/operators) to ensure that the data is of sufficient quantity and

quality to serve its intended use and meets DQOs. Data users should make sure that all accumulated data,

qualifications, and limitations are evaluated considering the project’s scope and data quality requirements or

objectives to determine its usability for its intended purpose.

When should data quality be assessed?

Primarily, there are two types of data utilized while implementing DERR’s programs: 1) generated hazardous waste

data and 2) environmental media data.

Process-generated hazardous waste is sampled as it is generated for characterization and compliance purposes. All

process-generated hazardous waste should be validated. Environmental media data, which includes soil, ground

water, surface water, sediment, soil, and indoor air data, may be used to estimate an exposure point concentration,

evaluate compliance, make remedial decision, and determine disposal needs. Other types of environmental data

may include field screening, tissue sampling, toxicity testing, etc.

DERR Data Quality Guidelines

Page|4

When using environmental media data to estimate an exposure point concentration or determine compliance, data

quality should be assessed for the initial data package for each laboratory, method, and environmental media for

each project. The initial data package is typically considered the first sampling event for the dataset that will be used

for determining an exposure point concentration or making a remedial decision.

While historical data should be empirically verified, it does not necessarily constitute an initial data package.

Following the review of the initial data package for a project, data quality should be assessed as appropriate to

demonstrate the quality of data. The rate of data quality review should be determined by guidelines or an agreed

frequency by the parties involved. For example, U.S. EPA’s Uniform Federal Policy for Quality Assurance Project

Plans and Ohio EPA’s VAP Data Verification Guidance specify a rate of at least 10 percent per laboratory, method,

and environmental media.

What tools should be used to assess data quality?

DERR has a Quality Management Plan, program and contract laboratory quality assurance plans (QAPPs), and field

standard operating procedures (SOPs) that serve as a framework for producing and obtaining quality environmental

data. DERR also has guidelines on developing DQOs, preparing QAPPs, and performing Data Verification

under the

VAP and Data Validation. Checklists that were developed to facilitate data quality assessment accompany the

guidance documents on data verification and validation. Other available tools and resources may also be

appropriate. See the Data Quality Resources section below for more information on U.S. EPA data quality resources.

Where should data quality be reported?

Data quality assessment procedures should be documented in project specific DQOs, QAPPs, and SAPs as

appropriate. The data quality assessment conclusions and corrective actions taken should be summarized in a

summary report or attachment (e.g., remedial investigation report, RCRA facility investigation report, phase II

property assessment report, RCRA supplemental annual report, etc.). Laboratory reports and any checklists, if used,

should be provided to the agency for data quality audits (as attachments to progress reports, technical memos, letter

reports, or summary reports). For data generated by DERR, data that does not meet performance and acceptance

criteria will not be used, and the use of data for its intended purpose will document that is of sufficient quality.

Data Quality Audits

Ohio EPA’s QMP includes procedures for determining the suitability and effectiveness of the implemented quality

system. Based on this, DERR will conduct data quality reviews to determine whether environmental data is verified

and validated appropriately. Data reviewers (e. g., site and project coordinators, technical reviewers, data analysts,

statisticians, etc.) will review data and appropriate documentation to ensure enough data has been verified or

validated, data quality issues are appropriately disclosed, and appropriate actions are taken to manage data quality.

The findings of data quality audits should be documented appropriately (e.g., via email, technical memos, comment

letters, etc.).

DERR Data Quality Guidelines

Page|5

Data Quality Resources

There are several data quality assessment tools and resources from U.S. EPA that may be helpful in addition to DERR

tools and resources. The table below provides links to some of these resources:

Resource

Assuring Data Quality in Federal Cleanups

Data Quality Assessment: A Reviewer’s Guide

Environmental Cleanup Best Management Practices: Effective Use of the Project Life Cycle Conceptual Site

Model (U.S. EPA, 2011)

Guidance on Assessing Quality Systems

Guidance for Data Quality Assessment: Practical Methods for Data Analysis

Guidance for Geospatial Data Quality Assurance Project Plans

Guidance for Quality Assurance Project Plans

Guidance on Systematic Planning Using the Data Quality Objectives Process

National Functional Guidelines for Inorganic Superfund Methods Data Review

National Functional Guidelines for Organic Superfund Methods Data Review

Quality Assurance Project Plan Standard

Quality Management Plan Standard

Tier I Data Validation Manual

Uniform Federal Policy for Implementing Environmental Quality Systems

Uniform Federal Policy for Quality Assurance Project Plans - Part 1: UFP-QAPP Manual

Uniform Federal Policy for Quality Assurance Project Plans - Part 2A: UFP-QAPP Workbook

Uniform Federal Policy for Quality Assurance Project Plans - Part 2B: Quality Assurance / Quality

Control Compendium (Minimum QA/QC Activities)

VAP Data Verification Guidance

Contact

For more information, contact Sarah Beal at sarah.beal@epa.ohio.gov or 614.644.2972.