[Home

[Risk Assessment Index] [Next Section]

Risk Assessment Technical Approach

Section 4 - Data Evaluation

The first major step of the process involves a review of existing site investigation reports, an evaluation of available chemistry data, and development of the data sets for each specific site. This step includes:

Data gathering and review of existing reports;

Data quality and usability evaluation (data validation, preparation of Quality Assurance Summary Reports (QASRs) and data summary tables, and evaluation of data qualifiers);

Statistical evaluation of data; and

Screening and identification of COPCs.

Data collection is ongoing at CSSA and evaluation of data is currently is progress.

All data, both historical and new, have been or will be validated following USEPA procedures (National Functional Guidelines) for Level II and III validation based on the availability of raw data.

In general, the data that have not been eliminated in the data validation process will be used in the RA. Data collected over the chronological duration of each site investigation will be evaluated. As appropriate, the resulting data used in the evaluation may include all or only a portion of the existing analytical data (e.g., older data may be replaced with more recent data or data not considered useable for a risk assessment, such as grab groundwater samples).

4.1 - Evaluation of Chemistry Data for Useability

Data quality, sample types, and analytical methods will be evaluated to retain or exclude data from the data sets as appropriate. A description of the types of data issues frequently encountered by Parsons ES during this evaluation and the approach taken to address data anomalies or other inconsistencies within data sets is presented below. These issues are discussed with respect to data validation codes; nondetects and Method Detection Limits (MDL); duplicate samples; compounds analyzed using multiple analytical methods; and aberrantly high Sample Quantitation Limits (SQL).

4.1.1   Data Validation Codes

Qualifiers (i.e., data validation codes) assigned to data during the data validation process will be reviewed with respect to data usability. All data qualified with a "M, F, J, B, U" will be considered useable for the assessments. Data qualified with an "R" (rejected) will be considered unusable for any statistical calculation or risk assessment.

The definition of data qualifiers used in the document is:

J: The analyte was positively identified, the quantitation is an estimation.

U: The analyte was analyzed for, but not detected. The associated numerical value is at or below the MDL.

F: The analyte was positively identified but the associated numerical value is below the RL

R: The data are unusable due to deficiencies in the ability to analyze the sample and meet QC criteria.

B: The analyte was found in an associated blank, as well as in the sample.

M: A matrix effect was present.

The allowable final data qualifiers listed in the order of the most severe through the least severe, are R, M, F, J, B, and U.

4.1.2   Nondetects and SQLs

Data sets generally contain some samples with positive results for a particular analyte and others with nondetected ("U") results for the same analyte. Per USEPA (1989a) guidance, analytes that were never detected (i.e., not detected in any samples from the data set, including results qualified as either "U" or "R") will be eliminated from further evaluation. Exceptions to this may include nondetected values that were not analyzed at appropriate SQLs to support the evaluation. For example, data with SQLs greater than risk-based levels of concern may be unusable. In such cases, other factors will also be considered, including sufficient supporting data for the analyte at appropriate SQLs. Such data will be qualitatively described in the uncertainty section of the reports.

Where a specific chemical in a data set was both detected and nondetected, a surrogate value will be used as the nondetected concentration for further evaluation (EPA, 1989a). An evaluation of the appropriate surrogate value is chemical- and sample-specific and will follow the TRRP guidelines.

4.1.3   Duplicate Samples

This type of field sample, also known as a field split, is one sample that is divided into two separate containers for analysis to document the precision of field sampling events and sample homogeneity. These samples will be evaluated during the data validation step.

4.1.4   Compounds Analyzed Using Multiple Analytical Methods

For some specific analytes, more than one result per sample may exist in the data set. This generally occurs when one analyte is quantified using two different analytical methods, or when analyses were for both the total and individual isomers of a compound. Because multiple sets of analytical results cannot be used to quantify risk (i.e., this would result in multiple-counting a chemical), the set of data that best represents the actual chemical concentrations will be retained. The approach for selection of the best representative data is summarized below. The basis and rationale for selecting or excluding data will be provided in the individual site reports.

Two analytical methods are often used for 1,2-, 1,3-, and 1,4-dichlorobenzene, as these compounds may be detected as either VOCs or SVOCs. Typically, the procedure for volatile organic analysis (VOA) of these compounds results in less loss from potential volatilization of the compounds due to the sample collection, storage, and other handling techniques. Other considerations for determining data to be retained for evaluation include sample size, detection frequency, and SQLs for both the VOA and SVOA procedures. The most conservative (i.e., health-protective) use of these types of data will be the goal. Larger sample size, higher detection frequencies, and lower SQLs will be given higher priority for selection.

Xylene is another example of a chemical that may have more than one analytical result per sample. This is due to the analyses of total xylenes and the individual isomers, meta-xylene (m-xylene), para-xylene (p-xylene), and ortho-xylene (o-xylene). Similarly, the total and individual forms (cis and trans) of 1,2-dichloroethene may have multiple analytical results. As above, sample size, detection frequency, and SQLs will be considered in the selection of the most appropriate data. Multiple analytical methods that may occur for other compounds than the common ones listed here will be evaluated on a case-specific basis using the same general approach.

4.1.5   Sample Quantitation Limits

USEPA (1989a; 1992d) guidance describes the SQL as the MDL adjusted to reflect sample-specific factors (e.g., dilution or use of a smaller sample aliquot for analysis due to matrix effects or high concentration of some analytes). The MDL is defined as the lowest concentration of an substance which can be measured with 99 percent confidence that the analyte concentration is greater than zero. SQLs are used in the data evaluation step, because they "take into account sample characteristics, sample preparation, and analytical adjustments" and are considered to be the most relevant quantitation limits for evaluating nondetected chemicals (EPA, 1989a).

Parsons ES� chemists will perform the following calculation to obtain the SQL for each analyte when risk assessment is required for the project:

Obtain the percentage solids value from either AFCEE Form O-2, I-2, or W-2.

Divide the MDL from either AFCEE Form O-2, I-2, or W-2 for each analyte by the % Solid for each sample. Report the result as SQL. (All MDLs and RLs in AFCEE forms have been corrected for dilution factors.)

For the media of concern at each of the selected sites, analyte-specific SQLs will be compared to the maximum detected concentrations. In many cases, sample-specific problems (resulting in dilutions) or interferences (e.g., matrix interferences) can result in an SQL for a particular chemical exceeding the maximum positive result reported for the same chemical in other samples from the data set. This typically occurs when an analyte has been detected in a control blank due to laboratory or sample storage procedures. If the SQL is aberrantly high in comparison to the maximum positive result, the sample result (for that chemical only) will be excluded from the data set.

A nondetected result will be excluded from the data set if the analyte-specific SQL is greater than four times the maximum detected concentration. This approach to excluding high SQL data provides for a conservative use of data. The USEPA (1989a) guidance also states that samples with unusually high SQLs should be excluded from the assessment if they cause the calculated 95 percent UCL to exceed the maximum detected concentration for a particular sample set. Exclusion of nondetected results which are associated with SQL values greater than four times the maximum ensures that surrogate values at one-half the SQLs for all nondetects will never be greater than two times the maximum detected (i.e., non-SQL) concentration. This conservative approach will not generally result in calculating a 95 percent UCL higher than the maximum detected concentration unless, for example, there are only a few positive results and a large number of nondetected results associated with SQLs which are above the maximum, but not greater than four times the maximum. In such cases, the maximum detected value is commonly determined to be the C-term (by default) since it would be less than the 95 percent UCL. The USEPA protocol for use of the lower of either the calculated 95 percent UCL or the maximum detected concentration is discussed below.

4.1.6   Development of Data Sets

All useable data from the above procedures will be consolidated and sorted by the appropriate environmental media. Additionally, the useable soil data will be sorted by the appropriate soil exposure intervals of concern. This step of data evaluation is performed in conjunction with the human health and ecological exposure assessments for the site.

The beginning and ending depths of the soil exposure intervals will be based on the types of exposure scenarios expected for each site, but will also be dependent on the sampling intervals and depths of sampling from the previously collected analytical data. For example, the TNRCC has defined surface soil as soil that is from 0 to 15 feet bgs for residential scenarios, or 0 to 0.5 feet bgs for ecological receptors (30 TAC �350). Therefore, these are considered the likely exposure intervals for these scenarios, whereas deeper soil samples (e.g., 30 feet) may not be appropriate for these receptors, but may be applicable to qualitatively evaluate potential leaching of soil contaminants to shallow groundwater.

4.2 - Statistical Methodology

4.2.1   Major Tasks

The following major tasks will be executed in the statistical analysis of data from CSSA:

Outlier analysis to identify influential and erroneous data points.

Computation of summary statistics for each analyte to include: proportion of NDs and identification of distributional characteristics of the data.

Performance of ANOVA tests followed by post hoc multiple comparisons when statistically significant group differences are found.

4.2.2   Statistical Analysis Methods

The following paragraphs describe the steps that will be taken for each analyte and field parameter at each sampling location. Two categories of constituents will be tested in this analysis: naturally occurring and non-naturally occurring chemicals. A different approach is required for the testing of each category. At CSSA, the naturally occurring constituents include the inorganic compounds. The proposed analytical approach for evaluating naturally occurring constituents is described below. Non-naturally occurring analytes include VOCs and SVOCs. The analytical approach for handling these parameters is presented in Section 4.2.5 below. As in any statistical analysis, the exact procedure to be used for a given parameter cannot be specified � priori as it is dependent upon the distributional characteristics of the data. However, a general approach can be proposed and is set forth in this Statistical Analysis Plan as follows.

4.2.3   Outlier Analysis

The first step in any statistical analysis is to verify the quality or validity of the data by screening the data for aberrant and/or erroneous data and identifying potentially influential data points for each analyte. Outlier analyses are performed to detect the presence of outliers and aberrant data prior to beginning statistical analyses. Outliers are statistically extreme data points that lie outside the range of the total sample of cases of a particular variable. The identification of an outlier is not based on its absolute magnitude but on its relative position to the remainder of the sample. Therefore, a particular measurement classified as an outlier may not intuitively seem very low or high but it is when compared to the rest of the sample. These data values can have significant impacts on calculation of summary statistics and hypothesis tests (comparison of background to site data). Some or all may actually be valid data; however, each should be verified as such before any statistical analyses are performed since the presence of outliers may lead to erroneous conclusions concerning contamination at the site. Possible explanations for the presence of outliers include simple transcription errors, laboratory methods outside of control ranges, contamination of a particular sample, etc. Any such errors should be either corrected or deleted from the data file. For those cases for which no explanation can be found, the suspected outlier must be treated as a valid data point and remain in the data file for analysis. All outliers should be duly noted in the final outlier analysis.

4.2.4   Tests of Background vs. Site-Specific Concentrations

For each inorganic compound, an ANOVA or its nonparametric equivalent, Kriskall-Wallis Test, will be performed. Application of either the ANOVA or KW test depends upon the distributional characteristics of the parameter and the percentage of nondetects. A parametric ANOVA will be selected when homogeneity of variance and approximate normality of the data are verified using the Levene Test and Shapiro-Wilk Test, respectively, and when the proportion of nondetects is less than 15 percent. When variances are not found to be homogeneous, data dramatically depart from normality, or the percentage of nondetects is greater than 15 percent but less than 90 percent, the nonparametric KW test will be used. All ANOVAs or KW tests will be conducted at the a =.05 level of significance and will be followed by post hoc multiple comparisons tests. When 90 percent or more of the sample consists of non-detects, neither the ANOVA nor KW test will be performed. Instead, a Poisson prediction limit will be calculated.

4.2.5   Analysis of VOCs and SVOCs

The analytical approach for monitoring VOCs and SVOCs is as follows. The detected presence of all non-natural VOCs and SVOCs will be attributed to site activities. Concentrations of the detected analytes will be compared to the laboratory sample quantitation limit.

4.3 - Screening Approaches and Identification of COPCs

To develop and refine the list of COPCs for each specific site, several screening approaches will be used. To perform this screening evaluation, the technical approach presented in the following sections will be used.

4.3.1   Approach to Evaluate Frequency of Detection

USEPA Risk Assessment Guidance for Superfund (RAGS) (1989a) directs the risk assessor to reduce data prior to risk characterization via various screening techniques, including use of a minimum detection frequency. According to RAGS, chemicals that are infrequently detected may be artifacts in the data due to sampling, analytical, or other problems, and therefore may not be related to site activities. USEPA gives an example of using a 5 percent detection frequency (based on a minimum of 20 samples) as a screening level to eliminate chemicals based on these potential problems. The TNRCC (1998) states that a contaminant that is never or infrequently (<5 percent) detected may be eliminated from further consideration if: (1) it was detected in only a single media; (2) its maximum site concentration does not exceed the screening concentration; and (3) there is no reason to believe that the contaminant is present based on historical information for the site. Thus, where data for a specified media show detection frequencies of a particular analyte of 5 percent or less, the chemical will be evaluated for exclusion from the evaluation. The following conditions will be evaluated as the criteria for using the 5 percent detection frequency as a screening tool:

The analyte is not detected frequently in other environmental media, such as local and downgradient groundwater (i.e., not found above 5 percent detection frequency);

The analyte is not detected at high concentrations (i.e., not found at a concentration greater than the critical PCL);

There is no documentation (including best professional judgement) to indicate that the chemical may be present or attributable to CSSA activities (based on chemicals or breakdown products of chemicals associated with historical processes); and

There is representative sample coverage of the site and of areas where contamination may be expected.

Results showing a low detection frequency of the compound (with high concentrations) may likely be indicative of a potential contaminant "hot spot" (i.e., a small area with highly elevated site-related concentrations). The potential presence of hot spots will be qualitatively evaluated and described in the site reports. Although potential hot spots may create acute risk to human health, acute exposures are not considered by the TNRCC when calculating the PCLs.

4.3.2   Determination of Applicability of RRS Standard 1 Closure

The first step in determining the appropriate closure for a site will be a comparison of the site concentrations to established background concentrations. If the site concentrations do not exceed background, then the site will be closed using RRS Standard 1 as outlined in the risk reduction rule. If the site exceeds background, then a determination will be made regarding the feasibility of cleaning the site to meet background concentrations. If the decision is made to clean the site to background, closure under RRS Standard 1 will be sought. However, if it is determined that the site cannot be closed to background, then closure under the TRRP will be conducted. Figure 4.1 outlines this process. The following sections describe the TRRP process.

4.3.3   Determination of Critical PCLs

In the TRRP, the TNRCC has provided generic default Tier 1 PCLs that can be used to determine the health protectiveness of a site. Site-specific concentrations are compared to the appropriate PCL, and a site that does not have any exceedances of the PCLs can be closed. If desired, the operator of a facility can choose to develop site-specific Tier 2 and Tier 3 PCLs. These tiers allow the inclusion of lateral transport considerations and additional site-specific information (e.g., soil characteristics).

In situations where the groundwater or soil background concentrations are greater than the critical PCL, the site concentrations can be compared to background, in lieu of the PCL. Soil background studies have been performed at CSSA (Parsons ES, 1996, 1997, 1999), but are currently being re-evaluated.

Many chemicals found in the environment do not occur at concentrations that present an unacceptable risk to human health. Based on this information, the TNRCC has calculated Tier 1 PCLs. PCLs are concentrations below which the contaminant is not expected to pose an unacceptable risk to human health.

For major pathways of concern in surface soil, subsurface soil, and groundwater, the TNRCC has calculated PCLs for a number of chemicals. These chemicals comprise the types of contaminants typically encountered at hazardous waste sites. The Tier 1 PCLs are based on very conservative health-protective assumptions about a site and thus are considered by the TNRCC to be sufficiently protective for the majority of site conditions across Texas.

Tier 1 PCLs have been developed assuming future residential or commercial/industrial land use assumptions and related exposure scenarios. The pathways considered in deriving the PCLs include ingestion, inhalation of volatiles and fugitive dusts, and dermal contact. When available, the groundwater PCLs are based on the USEPA�s Maximum Contaminant Level (MCL). MCLs also take into consideration the technical feasibility of achieving the MCL in a public drinking water supply. These PCLs are not necessarily protective of all known possible human exposure pathways or land uses; however, assumptions behind the PCLs provide for conservative, health-protective screening levels and should be protective of the current and future land use near the selected sites at CSSA (e.g., residential and commercial). The residential PCLs essentially provide conservative comparison criteria for unrestricted land use, when future residential land use has not been definitively ruled out.

TNRCC procedures will be followed for comparing the site contaminant concentrations to the PCLs. If there are greater than 10 carcinogenic compounds or non-carcinogenic compounds present in the exposure area, then a cumulative check will be performed to ensure that the cumulative risk does not exceed 10-4 and the cumulative hazard index does not exceed 10, as indicated in the rule.

The comparisons described above provide a very conservative screening evaluation for the CSSA sites. The TRRP allows the use of statistically-derived site exposure concentrations for the comparisons. The initial screening for the areas, however, will involve comparing the maximum detected site concentrations to the Tier 1 PCLs. Maximum concentrations will first be used to preclude unnecessary statistical evaluation of the data.

For potential migration of contaminants in soil to groundwater, the PCLs for groundwater protection (GWP) will be used for comparison to the appropriate C-term. Prior to this comparison, a determination of the groundwater class will be performed using the TRRP guidance.

[Next Section]