Bottled Water Study: Study Design

Everyday, people decide what to use for drinking water. Tap water, tap water with at-home treatment, bottled water—there are many options, but which of them is best? Because we are uniquely positioned to compare bottled water quality and tap water quality due to our database of water quality tests from thousands of drinking water samples across the U.S., we did the hard work to find out.

Table of Contents:

• Motivation
• Research Questions
• Methods Summary
• Bottled Water Samples
• Tap Water Samples
• What’s Next?
• Appendix

Motivation

It is clear that many people turn to bottled water when they’re in doubt, clearly indicated by the record growth of the bottled water industry in recent years.[1][2] There is a widespread perception that bottled water is “pure” and free of contaminants, thanks in large part to misleading marketing, which compounds pre-existing distrust in tap water quality. [3-5] However, studies have shown cases where bottled water in the U.S. contained a wide range of contaminants, including bacteria, heavy metals including lead and antimony, arsenic, volatile and semi-volatile organic contaminants, disinfection by-products, radiological elements, microplastics, and many PFAS compounds.[3][5-15] 

Despite evidence that bottled water is not, in fact, free of contaminants, we know of very few studies that compare bottled water quality directly to tap water quality in the U.S., or that include assessment of the potential toxicity of the consumption of bottled water, and none that do both.

Research Questions

We set out to answer two questions with this study:

1. What is bottled water quality in the San Francisco Bay Area?
2. How does bottled water quality compare to tap water quality in the same area?

We chose to focus this study on the San Francisco Bay Area in order to assess the true choices available to a specific population. And we centered the potential toxicity of the different water sources in our analyses, rather than just comparing concentrations of contaminants without any indication of the actual effects of exposure.

Methods Summary

We made comparisons between sample sets that were representative of the actual drinking water choices available to residents of the Bay Area. To this end, we carefully selected bottled water samples such that the full sample set is representative of the products people are likely to purchase based on market share data and availability. 

The tap water data we used for comparison was taken by the end users in the same region that bottled water samples were purchased. The large sample size, and the fact that the samples were taken at the tap rather than at treatment plants, gives us confidence that the tap water data creates an accurate picture of drinking water quality across the Bay Area.

We further categorized tap water samples based on whether or not they were treated at the household scale before comparing them with bottled water sample results. The sample set referred to simply as “tap water” is composed of samples taken directly from people’s taps with no further in-home treatment (beyond the municipal water treatment plant). The other samples, referred to collectively as “household treated tap water,” have undergone further treatment at residences, typically via activated carbon, reverse osmosis, or other multi-stage point-of-use or point-of-entry systems. By comparing bottled water to tap water and household treated tap water separately, we generate a more granular and realistic view of the options people have when choosing their drinking water sources.

In order to assess the potential toxicity of the analytes measured, we compared concentrations of contaminants known to have adverse health impacts to health-based benchmarks calculated by reputable public health and governmental agencies. 

These benchmarks are set at concentrations that are protective of human health with respect to cancer and/or non-cancer impacts. The most conservative (lowest) health-based benchmark for each contaminant was used in our data analysis. 

We did the same type of analysis for aesthetic effects—aesthetic-based benchmarks were set at the lowest concentration of a contaminant at which an aesthetic impact may be perceived, i.e., a taste, odor or color. Concentrations of contaminants with known aesthetic impacts measured in the samples were then compared with the benchmarks.

Read the full results of the Bottled Water Study. More details about the bottled water and tap water samples follow below.

Bottled Water Samples

We purchased 100 different bottled water products, representing 89 brands, from stores around the San Francisco Bay Area in July of 2022. The products were differentiated by FDA-defined categories based on the water source or further treatment employed:

• 46 products were classified as “purified” (including 6 products further classified as “distilled” and 9 products that were marketed as “alkaline” but have no official designation beyond “purified”)
• 46 products were classified as “spring”, “artesian”, “mineral” or “well”, which are grouped together as “groundwater” for the purposes of analysis
• the remaining 8 products did not meet FDA requirements for any official bottled water categories.[16]

The categories of bottled water products purchased were informed by the market share of bottled water products sold by category in the U.S. in 2020. In 2020, about 41% of products sold were labeled spring or mineral water, another 41% were purified (including distilled), and the remaining 18% are described as sparkling or other.[1] 

Sparkling water can fall into any of the regulatory categories, as well as outside of them altogether, so we kept our groundwater and purified categories approximately equal and allowed for 8% of our samples to be unregulated assuming that around half of the sparkling products sold were unregulated.

In addition, products purchased purposefully included both large and small brands, as well as private labels, so as to represent the various sizes and types of companies that produce bottled water. In order to create a sample set of products that was representative of those available across the San Francisco Bay Area, we first took an inventory of the bottled water products available at 82 stores around the nine county Bay Area region. The final sample set includes at least one product from each location inspected.

Bottled Water Inventory Locations

Each bottled water product was sent, unopened, within one week of purchase, to an accredited environmental testing laboratory for analysis. Samples were tested for 103 analytes in the following categories: 

• general chemistry
• metals & metalloids
• other inorganics
• disinfection byproducts
• other volatile organic compounds
• indicator bacteria. 

The full list of analytes is included in the Appendix below.

Tap Water Samples

The tap water data used in our study is from the analysis of 714 municipal tap water samples taken by residents in the same nine county Bay Area region shown above between 2020 and 2022. Residents purchased Tap Score water testing kits from SimpleLab and were provided with sample containers and instructions regarding appropriate sampling techniques. Samples were then analyzed at accredited environmental testing laboratories around the U.S.

As discussed above, tap water samples were grouped based on whether or not they went through additional in-home treatment (we excluded samples from the treated category where there was only a water softener or pH adjustment). Samples that received additional treatment are referred to as “household treated tap water” and those that did not receive additional treatment are referred to simply as “tap water.”

What’s Next

Click here to read our summary of the full study results, which compares bottled water quality with tap water quality in the Bay Area, focusing on the potential aesthetic issues and toxicity of the different drinking water choices available for residents.