Technical Project Lead (TPL) Review of PMTAs

lfflJ

U.S.

FOOD

DRUG

ADMIN

STRATIO

Technical Project Lead (TPL) Review

PMTAs

New

Products Subject

this Review;

Submission tracking

(6) (~) I

See

Appendix A

number

(

STNs

)

Common Attributes

Submission date September 7, 2020

Rece

date September 7, 2020

Applicant

(6J

rt1

Product manufacturer

(b)(4)

Applicati

type

Standard

Product category

ENDS

APES

)

Product subcategory

ENDS

Component

Cross-Referenced Submission

All new products None

Recommendation

Issue

market

ing denial orders

for

the

new tobacco

pro

ducts subject

thi

s review.

Technical Project Lead (TPL):

Signatory Decision:

Digitally signed by David

Portnoy-S

Date: 2021.09.17 10:42:36 -04'00'

vid

Portnoy, Ph.D., M.P.

Branch Chief,

cial Science Branch 2

Divisi

Popul

Heal

ence

Concur w

ith

TPL

recommendation and

sis

recommendation

Digitally

signed

Matthew

Holman

-5

Date: 2021.09.17 10:59:24 -04'00'

Matthe

w R. Holman,

.D.

Director

Office

Science

; Product

tails,

amendmen

ts, and

dates

provided

the

Appendix.

PMTA

means

premar

tobacco

app

cation. Sci

tific

references (if any)

are

listed

the

end

this

document

and r

efer

to wi

Arabic numerals;

general

foo

are

referred

to wi

Roman

nume

rals.

(b) (4)

TPL Review of PMTAs: Page 2 of 20

TABLE OF CONTENTS

1. EXECUTIVE SUMMARY ..................................................................................................................... 3

2. BACKGROUND.................................................................................................................................. 4

2.1. NEW PRODUCTS .............................................................................................................................. 4

2.2. REGULATORY ACTIVITY.................................................................................................................... 4

2.3. BASIS FOR REQUIRING RELIABLE, ROBUST EVIDENCE TO DEMONSTRATE BENEFIT....................... 4

The Risk to Youth of Flavored ENDS Products ................................................................. 5

Balancing Known Risks to Youth with a Potential Benefit to Adults............................... 9

2.4. SCOPE OF REVIEW ......................................................................................................................... 13

3. SCIENTIFIC REVIEW ........................................................................................................................ 14

4. ENVIRONMENTAL DECISION ........................................................................................................... 14

5. CONCLUSION AND RECOMMENDATION ......................................................................................... 14

6. APPENDIX ...................................................................................................................................... 16

7. REFERENCES ................................................................................................................................... 17

(b) (4)

TPL Review of PMTAs: Page 3 of 20

1. EXECUTIVE SUMMARY

These applications for flavored ENDS

products lack evidence to demonstrate that permitting the

marketing of these products would be appropriate for the protection of the public health (APPH).

Given the known and substantial risk of flavored ENDS with respect to youth appeal, uptake, and

use, applicants would need reliable and robust evidence of a potential benefit to adult smokers

iii

that could justify that risk. Accordingly, in order to show that a flavored ENDS is APPH, the applicant

must show that the benefit to adults switching from or reducing cigarettes outweighs the risk to

youth.

Based on existing scientific evidence and our experiences in conducting premarket review employing

the APPH standard over the last several years, FDA has determined for these applications that, to

effectively demonstrate this benefit in terms of product use behavior, only the strongest types of

evidence will be sufficiently reliable and robust —most likely product specific evidence from a

randomized controlled trial (RCT)

or longitudinal cohort study, although other types of evidence

could be adequate, and will be evaluated on a case-by-case basis.

Moreover, tobacco-flavored

ENDS may offer the same type of public health benefit as flavored ENDS, i.e., increased switching

and/or significant reduction in smoking, but do not pose the same degree of risk of youth uptake.

Therefore, to demonstrate the potential benefit to current users, FDA has reviewed these

applications for any acceptably strong evidence that the flavored products have an added benefit

relative to that of tobacco-flavored ENDS in facilitating smokers completely switching away from or

significantly reducing their smoking.

We have reviewed the subject applications to determine whether they contain sufficient evidence of

the type described above to demonstrate APPH. Our review determined that the applications do not

contain evidence from a randomized controlled trial or longitudinal cohort study regarding the

impact of the ENDS on switching or cigarette reduction that could potentially demonstrate the

benefit of their flavored ENDS over tobacco-flavored ENDS. The PMTAs do contain other evidence

regarding the potential benefit to adult users; however, for the reasons explained below, this other

evidence is not adequate.

The term flavored ENDS in this review refers to any ENDS other than tobacco-flavored and menthol-flavored ENDS.

Tobacco-flavored ENDS are discussed below. Applications for menthol-flavored ENDS will be addressed separately. When

it comes to evaluating the risks and benefits of a marketing authorization, the assessment for menthol ENDS, as compared

to other non-tobacco-flavored ENDS, raises unique considerations. The term flavored ENDS also includes unflavored

“base” e-liquids that are designed to have flavors added to them. This includes e-liquids made for use with open systems

as well as closed system ENDS (e.g., cartridges or disposable ENDS) containing e-liquids.

iii

The standard described in Section 910 requires an accounting of the risks and benefits to the population as a whole,

balancing the potential impacts to both current tobacco users and non-users. This review is focused on the risk to youth

nonusers as well as the potential benefit to adult smokers as current users, as they are the group through which the

potential benefit to public health is most substantial and could overcome the known risk to youth.

A randomized controlled trial is a clinical investigation or a clinical study in which human subject(s) are prospectively, and

randomly assigned to one or more interventions (or no intervention) to evaluate the effect(s) of the intervention(s) on

behavioral, biomedical, or health-related outcomes. Control or controlled means, with respect to a clinical trial, that data

collected on human subjects in the clinical trial will be compared to concurrently collected data or to non-concurrently

collected data (e.g., historical controls, including a human subject's own baseline data), as reflected in the pre-specified

primary or secondary outcome measures.

A longitudinal cohort study is an observational study in which human subjects from a defined population are examined

prospectively over a period of time to assess an outcome or set of outcomes among study groups defined by a common

characteristic (e.g., smoking cessation among users of flavored ENDS compared with users of tobacco-flavored ENDS).

For example, we would consider evidence from another study design if it could reliably and robustly assess behavior

change (product switching or cigarette reduction) over time, comparing users of flavored products with those of tobacco-

flavored products. In our review of PMTAs for flavored ENDS so far, we have learned that, in the absence of strong

evidence generated by directly observing the behavioral impacts of using a flavored product vs. a tobacco-flavored product

over time, we are unable to reach a conclusion that the benefit outweighs the clear risks to youth.

(b) (4)

TPL Review of PMTAs: Page 4 of 20

As a result, the applicant has failed to provide evidence to overcome the risk to youth and show a

net population health benefit necessary to determine that permitting the marketing of the new

tobacco product is APPH.

2. BACKGROUND

2.1. NEW PRODUCTS

The applicant submitted information for the new products listed on the cover page and in Appendix

2.2. REGULATORY ACTIVITY

FDA issued an Acceptance letter to the applicant on October 8, 2020. FDA issued a Filing letter to

the applicant on November 9, 2020.

2.3. BASIS FOR REQUIRING RELIABLE, ROBUST EVIDENCE TO DEMONSTRATE BENEFIT

The rationale for FDA’s decision for these flavored ENDS applications is consistent with previous

decisions for other flavored ENDS and is set forth below.

The Federal Food, Drug, and Cosmetic Act (FD&C Act or Act) requires that “new tobacco products”

receive marketing authorization from FDA under one of the pathways specified by the Act in order

to be legally marketed in the United States. Under one pathway, the applicant submits a PMTA to

FDA. Section 910 of the FD&C Act requires that, for a product to receive PMTA marketing

authorization, FDA must conclude, among other things, that the marketing of the product is APPH.

The statute specifies that, in assessing APPH, FDA consider the risks and benefits to the population

as a whole including both tobacco users and nonusers, taking into account the increased or

decreased likelihood that existing users of tobacco products will stop using such products and the

increased or decreased likelihood that those who do not use tobacco products will start using such

products.

vii

It is well recognized that ENDS, and particularly flavored ENDS, pose a significant risk to nonusers,

especially youth.

1,2

After observing a dramatic increase in the prevalence of ENDS use among U.S.

youth in 2018, FDA’s Commissioner characterized the problem as a youth vaping epidemic. FDA has

initiated a series of actions to address the risk and reduce youth use. Since August 2016, FDA has

issued more than 10,000 warning letters and more than 1,400 civil money penalty complaints to

retailers for the sale of ENDS products to minors. FDA has also issued a guidance that described a

policy of prioritizing enforcement of non-tobacco/non-menthol flavored ENDS, “Enforcement

Priorities for Electronic Nicotine Delivery Systems (ENDS) and Other Deemed Products on the

Market without Premarket Authorization” (2020 Enforcement Priorities Guidance). In this guidance,

FDA described evidence that shows flavors (other than tobacco and menthol) were a key driver of

vii

This review focuses on risk to youth nonusers and the potential benefit to adult smokers as current tobacco product

users, given that these are the subpopulations that raise the most significant public health concerns and therefore are the

most relevant in evaluating the impact on the population as a whole. FDA has also considered the APPH standard with

respect to the likelihood that an authorization will increase or decrease the number of tobacco users in the overall

population. The availability of such products has generally led to greater tobacco use among youth overall,

notwithstanding the decrease in cigarette smoking for youth, which reinforces the focus in this review on having

sufficiently reliable and robust evidence to justify authorization of these PMTAs. Cullen, K.A., B.K. Ambrose, A.S. Gentzke,

et al., “Notes from the Field: Increase in e-cigarette use and any tobacco product use among middle and high school

students – United States, 2011-2018,” Morbidity and Mortality Weekly Report, 67(45);1276-1277, 2018.

(b) (4)

TPL Review of PMTAs: Page 5 of 20

the surge in ENDS use among youth and thus prioritized enforcement against certain flavored ENDS

products, with the goal of protecting youth from these products.

viii

After FDA implemented this enforcement policy prioritizing enforcement against a subset of ENDS

products known to appeal to youth, there was a meaningful reduction in youth use prevalence.

Youth ENDS use peaked in 2019 when these products were widely available. Although several other

policy changes and interventions were occurring during this same time period,

it is reasonable to

infer that prioritizing enforcement against many flavored products resulting in their removal from

the market contributed to the decline in use in 2020. Despite this decline, ENDS remained the most

widely used tobacco product among youth, with youth use at levels comparable to what originally

led FDA to declare a youth vaping epidemic. Moreover, despite the overall reduction in ENDS youth

use observed in 2020, there was simultaneously a substantial rise in youth use of disposable ENDS,

products that were largely excluded from the enforcement policy described in the 2020

Enforcement Priorities Guidance because, at that time that policy was developed, those products

were the least commonly used device type among high school ENDS users and therefore remained

on the market as a flavored option.

3,4

Section 910(c)(2)(A) of the FD&C Act requires that FDA deny a PMTA where it finds “there is a lack of

a showing that permitting such tobacco product to be marketed would be [APPH].” Through the

PMTA review process, FDA conducts a science-based evaluation to determine whether marketing of

a new tobacco product is APPH. Section 910(c)(4) requires FDA, in making the APPH determination,

to consider the risks and benefits to the population as a whole, including users and nonusers of

tobacco, and take into account, among other things, the likelihood that those who do not use

tobacco products will start using them. FDA’s scientific review is not limited to considering only

information in a PMTA, but also extends to any other information before the Agency, including the

relevant existing scientific literature (See Section 910(c)(2)). As described in greater detail below, in

reviewing PMTAs for flavored ENDS, FDA evaluates, among other things, the potential benefit to

adult smokers who may transition away from combustible cigarettes to the ENDS product, weighed

against the known risks of flavored ENDS to youth.

The Risk to Youth of Flavored ENDS Products

As noted, the APPH determination includes an assessment of the risks and benefits to the

population as a whole, and for ENDS (as well as many other tobacco products) the application of

that standard requires assessing the potential impact of the marketing of a new product on youth

use. As a group, youth are considered a vulnerable population for various reasons, including that

the majority of tobacco use begins before adulthood

and thus youth are at particular risk of

tobacco initiation. In fact, use of tobacco products, no matter what type, is almost always started

and established during adolescence when the developing brain is most vulnerable to nicotine

addiction. Indeed, almost 90 percent of adult daily smokers started smoking by the age of 18.

Adolescent tobacco users who initiated tobacco use at earlier ages were more likely than those

initiating at older ages to report symptoms of tobacco dependence, putting them at greater risk for

maintaining tobacco product use into adulthood.

On the other hand, youth and young adults who

viii

Due to the overwhelming amount of evidence showing a substantial increase in youth use of flavored ENDS products, as

well as their demonstrated popularity among youth, in January 2020, FDA finalized a guidance prioritizing enforcement

against flavored (other than tobacco or menthol) prefilled pod or cartridge-based e-cigarettes, as well as other categories

of unauthorized products.

The change in ENDS product availability coincided with other events such as the enactment of legislation raising the

federal minimum age for sale of tobacco products from 18 to 21 years (Tobacco 21), the outbreak of e-cigarette, or vaping,

product-use associated lung injury (EVALI), and public education campaigns which also may have contributed to the

decline in ENDS use.

(b) (4)

TPL Review of PMTAs: Page 6 of 20

reach the age of 26 without ever starting to use cigarettes will most likely never become a daily

smoker.

Because of the lifelong implications of nicotine dependence that can be established in

youth, preventing tobacco use initiation in young people is a central priority for protecting

population health.

2.3.1.1. Youth use of flavored ENDS

ENDS are now the most commonly used type of tobacco product among youth. In 2020,

approximately 19.6% of U.S. high school students and 4.7% of middle school students were current

users of ENDS, corresponding to 3.6 million youth and making ENDS the most widely used tobacco

product among youth by far.

As noted above, this was a decline from 2019, when 27.5% of high

school and 10.5% of middle school students reported ENDS use,

which necessitated the FDA

enforcement policy described above.

The evidence shows that the availability of a broad range of flavors is one of the primary reasons for

the popularity of ENDS among youth. The majority of youth who use ENDS report using a flavored

ENDS product, and the use of flavored ENDS has increased over time. In the 2014 National Youth

Tobacco Survey (NYTS), 65.1% of high school and 55.1% of middle school e-cigarette

users reported

using a flavored e-cigarette.

By the 2020 NYTS, the proportion of e-cigarette users reporting using

a flavored product

increased to 84.7% of high school users and 73.9% of middle school users.

Among high school e-cigarette users, the most common flavors used in 2020 were fruit (73.1%);

mint (55.8%); menthol (37.0%); and candy, dessert, or other sweets (36.4%).

Among middle school

e-cigarette users, the most common flavors used in 2020 were fruit (75.6%); candy, desserts, or

other sweets (47.2%); mint (46.5%); and menthol (23.5%).

Youth ENDS users are also more likely to use flavored ENDS compared to adult ENDS users. In PATH

Wave 5.5 from 2020, 66.8% of youth ENDS users aged 13 to 17 reported using fruit, followed by

53.8% for mint/menthol

xii

, 23.5% for candy/dessert/other sweets, and 13.3% for tobacco flavor

(internal analysis). In the 2020 PATH Adult Telephone Survey, 51.5% of adult ENDS users 25 and

older used fruit, 30.4% used mint/menthol, 23.8% used candy/dessert/other sweets, and 22.3%

used tobacco flavor (internal analysis). Youth current ENDS users were also more likely than adult

current ENDS users to use more than one flavor and to use combinations that did not include

tobacco flavors.

Studies show that flavors influence youth initiation of ENDS use. In particular, data show that

flavors are associated with product initiation, with the majority of users reporting that their first

experience with ENDS was with a flavored product. For instance, in Wave 1 of the PATH Study from

2013-2014, over 80% of youth aged 12-17, 75% of young adults 18-24, and 58% of adults 25 and

older reported that the first e-cigarette that they used was flavored.

In another PATH study, more

youth, young adults and adults who initiated e-cigarette use between Wave 1 and Wave 2 reported

use of a flavored product than a non-flavored product.

Finally, in PATH Wave 4 from 2016-2017,

93.2% of youth and 83.7% of young adult ever ENDS users reported that their first ENDS product

was flavored compared to 52.9% among adult ever users 25 and older.

In addition, nationally representative studies find that when asked to indicate their reasons for using

ENDS, youth users consistently select flavors as a top reason.

15,16

In fact, among Wave 4 youth

current ENDS users, 71% reported using ENDS "because they come in flavors I like.”

We use “e-cigarette” here to be consistent with the survey, but we interpret it to have the same meaning as ENDS.

Flavored product use in these studies means use of flavors other than tobacco.

xii

The PATH Study Questionnaire from Wave 5.5 did not assess mint and menthol separately. However, subsequent data

collections (ATS and Wave 6) have separated the two flavors.

(b) (4)

TPL Review of PMTAs: Page 7 of 20

One explanation for this high prevalence and increase in frequency of use is that flavors can

influence the rewarding and reinforcing effects of e-liquids, thereby facilitating ENDS use and

increasing abuse liability. Research shows that flavored ENDS are rated as more satisfying than non-

flavored ENDS, and participants will work harder for and take more puffs of flavored ENDS

compared to non-flavored ENDS.

Research also shows that flavors can increase nicotine exposure

by potentially influencing the rate of nicotine absorption through pH effects and by promoting the

reward of ENDS use.

Together, this evidence suggests flavored ENDS may pose greater addiction

risk relative to tobacco-flavored ENDS, which increases concerns of addiction in youth, particularly

due to the vulnerability of the developing adolescent brain, which is discussed further below.

Finally, existing literature on flavored tobacco product use suggests that flavors not only facilitate

initiation, but also promote established regular ENDS use. In particular, the flavoring in tobacco

products (including ENDS) make them more palatable for novice youth and young adults, which can

lead to initiation, more frequent and repeated use, and eventually established regular use. For

example, regional studies have found that the use of flavored e-cigarettes was associated with a

greater frequency of e-cigarettes used per day among a sample of adolescents in Connecticut in

2014

and continuation of e-cigarette use in a sample of adolescents in California from 2014-

2017.

Use of non-traditional flavors (vs. tobacco, mint/menthol, flavorless) was associated with

increased likelihood of continued use and taking more puffs per episode.

Data from a regional

survey in Philadelphia, PA found initial use of a flavored (vs. unflavored or tobacco-flavored) ENDS

was associated with progression to current ENDS use as well as escalation in the number of days

ENDS were used across 18 months.

Finally, similar effects have been found in the nationally

representative PATH study among young adults (18-24 years), where “ever use” of flavored e-

cigarettes at Wave 1 was also associated with increased odds of current regular ENDS use a year

later at Wave 2.

In sum, flavored ENDS facilitate both experimentation and progression to regular

use, which could lead to a lifetime of nicotine dependence.

2.3.1.2. The appeal of flavors across ENDS devices

The role of flavors in increasing the appeal of tobacco products to youth — across tobacco product

categories — is well-established in the literature.

23-26

The published literature is sufficient to

demonstrate the substantial appeal to youth of flavored ENDS, because it is robust and consistent.

As described above, the preference for use of flavored ENDS among youth is consistently

demonstrated across large, national surveys and longitudinal cohort studies.

National surveillance data suggest that, within the ENDS category, there is variability in the

popularity of device types among youth, suggesting there may be differential appeal of certain

product styles. Still, across these different device types, the role of flavor is consistent. As described

above, the majority of youth ENDS use involves flavored products: in 2020, the majority of high

school and middle school current e-cigarette users reported use of non-tobacco-flavored products

(82.9%)

and flavored use was favored among both users of closed (87%) and open (76%) ENDS

(internal analysis). In particular, across device types, including prefilled pods/cartridges,

disposables, tanks, and mod systems, fruit was the most commonly used flavor type among youth,

with 66.0% for prefilled pods/cartridges, 82.7% for disposables, 81.7% for tanks, and 78.9% for mod

systems among youth reporting using a fruit flavor.

It is also worth noting that the preference for device types and popularity of certain styles is likely

fluid and affected by the marketplace, that is, the options, especially flavors, that are available for

consumers to choose from. Some evidence for this was observed in the trends both leading up to,

and coinciding with, the shifting marketplace following the 2020 Enforcement Priorities Guidance.

In particular, the enormous rise in youth ENDS use from 2017-2019 coincided with the ascendance

(b) (4)

TPL Review of PMTAs: Page 8 of 20

of JUUL (and copy-cat devices) in the marketplace, suggesting a relationship between the availability

of JUUL as an option, and the sudden popularity of pod-based devices.

xiii

Then, as noted earlier,

when FDA changed its enforcement policy to prioritize pod-based flavored ENDS, which were most

appealing to youth at the time, we subsequently observed a substantial rise in use of disposable

flavored ENDS

xiv

--a ten-fold increase (from 2.4% to 26.5%) among high school current e-cigarette

users.

This trend illustrates that the removal of one flavored product option prompted youth to

migrate to another ENDS type that offered the desired flavor options, underscoring the fundamental

role of flavor in driving appeal.

2.3.1.3. The harms of youth ENDS use: The adolescent brain and risk for addiction

In addition to the high prevalence of youth ENDS use, the data also suggest this use is leading to

increases in nicotine dependence.

Indeed, responding to concerns related to youth ENDS

dependence, at the end of 2018, FDA held a public hearing to discuss the potential role of drug

therapies to support e-cigarette cessation.

In 2019, an estimated 30.4% of middle and high school student ENDS users reported frequent use

(i.e., use on 20 of the past 30 days).

By school type, 34.2% (95% CI, 31.2%-37.3%) of high school

student ENDS users and 18.0% (95% CI, 15.2%-21.2%) of middle school student ENDS users reported

frequent use.

Among current ENDS users, 21.4% of high school users and 8.8% of middle school

users reported daily ENDS use.

Additionally, in a study that examined changes in ENDS use in

youth ages 13-18 over a 12-month period, nicotine dependence (measured using the Penn State

Electronic Cigarette Dependence Index (PS-ECDI)

28,29

and salivary cotinine concentrations increased,

indicating continued ENDS use and greater nicotine xposure over time.

Youth and young adult brains are more vulnerable to nicotine’s effects than the adult brain due to

ongoing neural development.

31,32

Adolescence is a developmental period consisting of major

neurobiological and psychosocial changes and is characterized by increased reward-seeking and risk-

taking behaviors (e.g., experimentation with drugs), coupled with heightened sensitivity to both

natural and drug rewards and an immature self-regulatory system that is less able to modulate

reward-seeking impulses (e.g., diminished harm avoidance, cognitive control, self-regulation).

33-37

Furthermore, evidence from animal studies suggests that nicotine exposure during adolescence

enhances the rewarding and reinforcing effects of nicotine in adulthood

38-41

; and can induce short

and long-term deficits in attention, learning, and memory.

42-45

2.3.1.4. Risk of progression from ENDS to other tobacco products of different health risk

Among youth who use ENDS, there is a risk of progression to other tobacco products of generally

greater health risk. A 2017 systematic review and meta-analysis that summarized nine prospective

cohort studies found significantly higher odds of smoking initiation (OR = 3.50, 95% CI: 2.38, 5.16)

and past 30-day combusted cigarette use (OR = 4.28, 95% CI: 2.52, 7.27) among youth who had used

ENDS at compared to youth who had not used ENDS.

Similar associations have been observed in

longitudinal studies that have been published since the Soneji et al. review.

42,47-56

The 2018 NASEM

report concluded that there is substantial evidence that ENDS use increases risk of ever using

combusted tobacco cigarettes among youth and young adults.

The transition from non-cigarette

xiii

This is borne out by the data from 2019 NYTS, in which 59.1% of high school ENDS users reported use of this one brand.

Cullen KA, Gentzke AS, Sawdey MD, et al. e-Cigarette Use Among Youth in the United States, 2019. Jama.

2019;322(21):2095-2103.

xiv

In July 2020, FDA issued Warning letters to three companies for illegally marketing disposable e-cigarettes and for

marketing unauthorized modified risk tobacco products.

On December 5, 2018, FDA hosted a public hearing on “Eliminating Youth Electronic Cigarette and Other Product Use:

The Role of Drug Therapies.”

≥

(b) (4)

TPL Review of PMTAs: Page 9 of 20

product use to combusted cigarette use has been observed for other non-cigarette products, such

as cigars, as well.

Although it is challenging to empirically separate causality from shared risk

factors among youth combusted cigarette and ENDS users, some studies have found an association

between ENDS and subsequent combusted cigarette use while controlling for similar risk profiles.

The precise relationship between youth ENDS use and youth smoking remains undetermined. On

the one hand, the prevalence of combusted cigarette smoking in youth has continued to

decline,

9,59,60

suggesting that youth use of ENDS has not significantly slowed or impeded that positive

public health trajectory. On the other hand, there is a growing body of evidence showing a link

between ENDS use and subsequent smoking among youth that raises significant concerns. This

evidence also increases concern that over time—and particularly if youth ENDS use were to return

to the rates seen in 2019 or worsen--the trend of declining cigarette smoking could slow or even

reverse.

2.3.1.5. Other health risks associated with ENDS use

In addition to the risk of tobacco initiation and progression among youth, there is epidemiologic

evidence from the cross-sectional

xvi

Behavioral Risk Factor Survey system (BRFSS) suggesting positive

associations between ENDS use among those who never smoked and some health outcomes. Two

studies found associations between ENDS use and self-reported history of asthma, chronic

bronchitis, emphysema, or chronic obstructive pulmonary disease with increased ENDS use (i.e.,

daily use) relating to increased odds of disease.

6 62

Another found an association between ENDS

use and respiratory symptoms in younger adults (ages 18-34) but not in older adults.

ENDS use

has also resulted in acute harm to individuals through battery explosion-related burns and e-liquid

nicotine poisoning.

64-66

Ultimately, as this is still a relatively novel product category, much remains

unknown about other potential long-term health risks.

2.3.1.6. Conclusion

The exponential growth in youth ENDS use observed from 2017 to 2019, and the enduring

prevalence of youth ENDS use in the U.S. is alarming. Despite a reduction in youth use of ENDS from

2019 to 2020, there were still 3.6 million youth ENDS users in 2020 and the majority used a flavored

ENDS product. Youth users are more likely to use flavored ENDS than adult ENDS users. Flavors are

associated with ENDS initiation and progression among youth. The full extent of the harms of ENDS

use are not yet known, but evidence to date suggests they include permanent effects of nicotine on

the developing adolescent brain and the risk of nicotine addiction. Studies indicate an additive

effect of e-liquid flavorings on the rewarding and reinforcing effects of nicotine containing e-liquids.

Studies also demonstrate that e-liquid flavors affect nicotine exposure. Among youth who use

ENDS, there is a risk of progression to other tobacco products with greater health risks including

combustible cigarettes. Finally, though long-term health risks are not fully understood, studies

suggest an association between never-smoking ENDS users and respiratory and cardiovascular

health effects. This evidence demonstrates that flavored ENDS pose a significant risk to youth.

Balancing Known Risks to Youth with a Potential Benefit to Adults

Determining whether marketing a new product is APPH includes evaluating the risks and benefits to

the population as a whole. This requires FDA to balance, among other things, the negative public

health impact for nonusers against the potential positive public health impact for current tobacco

users. Accordingly, for marketing of a new product to be found to be APPH, any risks posed by a

new product to youth would need to be overcome by a sufficient benefit to adult users, and as the

xvi

Cross-sectional surveys examine these relationships at a single point in time, and as a result, do not establish causality.

(b) (4)

TPL Review of PMTAs: Page 10 of 20

known risks increase, so too does the burden of demonstrating a substantial enough benefit. In the

case of a new flavored ENDS product, the risk of youth initiation and use is substantial, given the

clearly documented evidence described above. In order for marketing of a new flavored ENDS

product to be found APPH, an applicant would have to show that the significant risk to youth could

be overcome by likely benefits substantial enough such that the net impact to public health would

be positive, taking into account all relevant evidence and circumstances, including whether there

are effective limitations on youth access.

2.3.2.1. Potential benefit of new flavored ENDS

Current scientific literature demonstrates that ENDS are generally likely to have fewer and lower

concentrations of harmful and potentially harmful constituents (HPHCs) than combustible

cigarettes, and biomarker studies demonstrate significantly lower exposure to HPHCs among current

exclusive ENDS users than current smokers.

However, whether this is true for any particular new

ENDS product, and the implications for health risks from a particular product, are considered on a

case-by-case basis during the course of FDA’s scientific review of a PMTA.

FDA also considers the potential that current cigarette smokers may experience a reduction in

health risks if they switch completely to an ENDS, or if they use both products but substantially

reduce their cigarette smoking. For a flavored ENDS product, assuming that the evaluation of the

product shows the likelihood for lower HPHC exposure, then to demonstrate the likely individual

and population benefit, applicants must demonstrate that current smokers are likely to start using

the new ENDS product exclusively or predominantly (e.g., dual use with a significant smoking

reduction).

2.3.2.2. Behavioral evidence appropriate to demonstrate the potential benefit to smokers

FDA’s PMTA review includes an evaluation of any potential benefits of the product for the likely

users, such as a possible reduction in health risks. In general, as FDA stated in its guidance for

PMTAs for ENDS,

xvii

an assessment of how a new product may be used by current smokers can be

derived from a variety of sources. FDA may consider direct behavioral evidence on the specific

products under review or indirect evidence derived from studies of behavioral intentions;

pharmacological studies of nicotine delivery, abuse liability, and/or use topography; and bridging

from studies based on comparable products. Further, in the case of a flavored ENDS product, to

demonstrate that the marketing of the new product is APPH, the magnitude of the likely benefit

would have to be substantial enough to overcome the significant risk of youth uptake and use posed

by the flavored ENDS product.

Section 910(c)(5) of the FD&C Act provides that determining whether marketing of a new tobacco

product is APPH shall, when appropriate, be based on “well-controlled investigations, which may

include one or more clinical investigations by experts qualified by training and experience to

evaluate the tobacco product.” FDA believes well-controlled investigations are “appropriate” for

demonstrating that permitting the marketing of specific flavored ENDS would be APPH given the

significant risks to youth of flavored ENDS. One type of well-controlled investigation that could

effectively demonstrate a potential benefit of a flavored ENDS product would be an RCT. In

addition, as CTP has previously described,

xviii

another well-controlled investigation that could serve

as an alternative to conducting an RCT to demonstrate adequate benefit is a longitudinal cohort

study.

xvii

Premarket Tobacco Product Applications for Electronic Nicotine Delivery Systems: Guidance for Industry (p.47); October

2019 Public Meeting on Deemed Tobacco Product Applications

xviii

Premarket Tobacco Product Applications for Electronic Nicotine Delivery Systems: Guidance for Industry (p.47); October

2019 Public Meeting on Deemed Tobacco Product Applications

(b) (4)

TPL Review of PMTAs: Page 11 of 20

For flavored ENDS, the known and substantial risk to youth in particular is high. Therefore, to show

a net population health benefit, FDA has determined that these applications must demonstrate

potential benefits to smokers from marketing such products with robust and reliable evidence –

including both robust study design and methods and the strength of the study results. In other

words, because the potential benefit to adults is gained through its impact on smoking behavior,

FDA is reviewing these applications to determine whether they demonstrate that a benefit of a new

product is significant enough to overcome the risk to youth. In particular, FDA’s review of these

applications has considered the degree of benefit to a flavored ENDS product over a tobacco-

flavored variety in facilitating smokers completely switching or significantly reducing their smoking,

given the significant increase in risk of youth initiation associated with flavored ENDS compared to

tobacco-flavored ENDS. Note that applications with this type of information may still not be APPH:

applications containing this evidence would still be evaluated to determine that the totality of the

evidence supports a marketing authorization. As it relates to the risk to youth, for example, this

assessment includes evaluating the appropriateness of the proposed marketing plan.

xix

We have been using the APPH standard for several years in reviewing previous PMTAs for non-ENDS

products. Our substantive review of PMTAs for ENDS and our completion of numerous scientific

reviews over the last 10 months have deepened our understanding of the APPH evaluation with

respect to behavior. In these reviews, the expectations for scientific evidence related to potential

adult benefit can vary based on demonstrated risk to youth. Although indirect evidence or bridged

data from the literature may still be appropriate for many new products, including tobacco-flavored

ENDS, robust and direct evidence demonstrating potential benefit has been needed when the

known risks are high as with all flavored ENDS products. At the same time, we have learned from

experience that, in the absence of strong direct evidence, we are unable to reach a conclusion that

the benefit outweighs the clear risks to youth. For instance, applicants who do not conduct their

own behavioral studies must rely on, and bridge to, the general ENDS category literature to inform

an evaluation of the potential benefit to adult users. To date, that approach has not been sufficient

in our evaluation of flavored ENDS PMTAs because, in contrast to the evidence related to youth

initiation—which shows clear and consistent patterns of real-world use that support strong

conclusions--the evidence regarding the role of flavors in promoting switching among adult smokers

is far from conclusive.

In fact, the findings are quite mixed and as a result the literature does not

establish that flavors differentially promote switching amongst ENDS users in general. Aside from

differences in study design/methods, the heterogeneity of the existing literature is likely due, at

least in part, to differences in the products studied. Therefore, given the state of the science on

flavored ENDS, and the known risks to youth, FDA has reviewed these applications for any

acceptably strong product-specific evidence.

xix

Limiting youth access and exposure to marketing is a critical aspect of product regulation. It is theoretically possible that

significant mitigation efforts could adequately reduce youth access and appeal such that the risk for youth initiation would

be reduced. However, to date, none of the ENDS PMTAs that FDA has evaluated have proposed advertising and promotion

restrictions that would decrease appeal to youth to a degree significant enough to address and counter-balance the

substantial concerns, and supporting evidence, discussed above regarding youth use. Similarly, we are not aware of access

restrictions that, to date, have been successful in sufficiently decreasing the ability of youth to obtain and use ENDS.

Accordingly, for the sake of efficiency, the evaluation of the marketing plans in applications will not occur at this stage of

review, and we have not evaluated any marketing plans submitted with these applications.

This discrepancy between the literature for youth initiation and adult switching also likely reflects fundamental

differences in the two outcomes being assessed—youth initiation and switching among adult smokers—and their

determinants. For switching among adult smokers, the behavior change is occurring in the context of nicotine

dependence. Thus, the specific product’s ability to provide adequate reinforcement and continue to satisfy a smoker’s

cravings over time, which is a function of the design of the specific product itself, are critical factors in determining

likelihood of continued use and the product’s ability to promote switching. Whereas for youth initiation, experimentation

among naïve or novice users is not driven by these factors.

(b) (4)

TPL Review of PMTAs: Page 12 of 20

More specifically, in order to adequately assess whether such an added benefit has been

demonstrated, FDA has reviewed these applications for product-specific

xxi

evidence that would

enable a comparison between the applications’ new flavored products and an appropriate

comparator tobacco-flavored product (both ENDS) in terms of their impact on tobacco use behavior

among adult smokers. Consistent with section 910(c)(5), evidence generated using either an RCT

design or longitudinal cohort study design is mostly likely to demonstrate such a benefit, although

other types of evidence could be adequate if sufficiently reliable and robust, and will be evaluated

on a case-by-case basis.

xxii

CTP will consider other types of evidence if it is sufficiently robust and direct to demonstrate the

impact of the new ENDS on adult switching or cigarette reduction. Uptake and transition to ENDS

use is a behavioral pattern that requires assessment at more than one time point. In addition, the

transition from smoking to exclusive ENDS use typically involves a period of dual use. Therefore,

evaluating the behavioral outcomes needed to show any benefit of the product requires observing

the actual behavior of users over time. With both RCT and cohort study designs, enrolled

participants are followed over a period of time, with periodic and repeated measurement of

relevant outcomes.

In contrast, cross-sectional surveys entail a one-time assessment of self-reported outcomes:

although participants can be asked to recall their past behavior, the single data collection does not

enable reliable evaluation of behavior change over time. Consumer perception studies (surveys or

experiments) typically assess outcomes believed to be precursors to behavior, such as preferences

or intentions related to the new products, but are not designed to directly assess actual product use

behavior. Moreover, the general scientific literature, though informative for evaluation of some

types of products, is not adequate to address this assessment because it does not provide product-

specific information. This is because the effectiveness of a product in promoting switching among

smokers arises from a combination of its product features—including labeled characteristics like

flavor and nicotine concentration—as well as the sensory and subjective experience of use (taste,

throat hit, nicotine delivery), and can also be influenced by how the device itself looks and feels to

the use.

While RCTs and cohort studies both enable direct assessment of behavioral outcomes associated

with actual product use over time, there are pros and cons to each type of design. While RCTs

afford greater control and internal validity; cohort studies enable stronger generalizability because

xxi

By product-specific, we mean the data are based on studies using the specific new products that are the subject of the

application(s). If the applicant has a large number of product variants (e.g., nicotine concentration and/or flavor options),

it may be justifiable to bridge data from a study including a subset of their products to one or more of their other products

(not included in the study). In contrast, because of the need for product-specific information, bridging from a different set

of products (not the subject of the application) would not be appropriate here.

xxii

Conversely, such longitudinal or product-specific data are not necessarily required to assess experimentation and

appeal among youth. The available literature on youth initiation contains valid scientific evidence sufficient to evaluate the

risk to youth of ENDS. The literature includes longitudinal cohort studies, such as the PATH study, which have been used

to assess uptake of tobacco products, including flavored ENDS, among youth and young adults. These studies have

evaluated the impact of flavors on the promotion of established regular use. Additionally, the literature includes large,

nationally representative cross-sectional surveys, which are among the best available evidence to understand patterns of

youth ENDS use and the key characteristics associated with such use These studies enable observation of youth behavior

as it naturally occurs in representative samples of the U.S. population. These data available in the literature provide clear

and overwhelming evidence that ENDS are the most widely used products by youth, the majority of youth users use a

flavored ENDS, and that youth users are more likely to use flavored ENDS than adult ENDS users. We note that, in

assessing the risks to youth from flavored ENDS, RCTs are not possible because it would be unethical to randomize youth

never or naive users to try a particular ENDS to examine what impact it would have on initiation, experimentation, or

progression to regular use.

(b) (4)

TPL Review of PMTAs: Page 13 of 20

conditions are closer to real-world. We are aware of these as trade-offs and generally do not favor

one type over the other for addressing this question.

To be informative, a study using one of these two designs would measure the impact of use of the

new or appropriate comparator product tobacco-flavored ENDS and flavored products on adult

smokers’ tobacco use behavior over time

xxiii

; include outcomes related to ENDS use and smoking

behavior to assess switching and/or cigarette reduction; and enable comparisons of these outcomes

based on flavor type. In some cases, evidence on each individual flavor option may not be feasible;

bridging data from one of the applicant’s flavors to other flavors of the applicant’s in the same flavor

category (e.g., “fruit”) may be appropriate. Furthermore, consistent with previous FDA guidance,

we would expect the applicant to provide justification to support this bridging.

xxiv

Likewise, if a

flavor is tested with one nicotine concentration, it may be feasible for the applicant to bridge the

study results to other nicotine concentrations, under certain circumstances, and with the

appropriate justification for bridging.

Data from one of these studies could support a benefit to adult users if the findings showed that,

compared to the new tobacco-flavored product, use of (each) new flavored product is associated

with greater likelihood of either of these behavioral outcomes for adult smokers: (1) complete

switching from cigarettes to exclusive new product use or (2) significant reduction in cigarettes per

day (CPD).

2.3.2.3. Conclusion

Given the known and substantial risk to youth posed by flavored ENDS, FDA has reviewed these

applications for the presence of particularly reliable product-specific

xxv

evidence to demonstrate a

potential for benefit to adult smokers that could justify that risk. Based on our current

understanding, a demonstration with sufficiently reliable and robust evidence that the flavored

ENDS have an added benefit relative to tobacco-flavored ENDS in facilitating smokers completely

switching or reducing their smoking could demonstrate the potential benefit to current users that

would outweigh the risk to youth posed by flavored ENDS.

2.4. SCOPE OF REVIEW

The reviews evaluated whether the subject PMTAs contain evidence from a randomized controlled

trial, longitudinal cohort study, and/or other evidence regarding the impact of the new products on

switching or cigarette reduction that could potentially demonstrate the added benefit to adult users

of their flavored ENDS over an appropriate comparator tobacco-flavored ENDS. These reviews

included a search of the PMTAs to determine whether the evidence is found anywhere within the

PMTAs, and if present, if certain conditions were met (e.g., was the randomized controlled trial

conducted using the new products that are the subject of the PMTA). Our review also included a

xxiii

This could include studies that are long-term (i.e., six months or longer). In FDA’s (2019) Guidance to Industry,

“Premarket Tobacco Product Applications for Electronic Nicotine Delivery Systems”, FDA has previously stated that it did

not expect that applicants would need to conduct long-term studies to support an application for ENDS. Because the

behavior change of interest (switching or cigarette reduction) occurs over a period of time, it is possible that to observe

these outcomes, investigators designing these studies may decide to follow participants over a period of six months or

longer. However, it is also possible that studies with a shorter duration would be adequately reliable.

xxiv

Bridging is discussed in FDA’s 2019 Guidance to Industry cited above (fn xxiii).

xxv

By product-specific, we mean the data are based on studies using the specific new products that are the subject of the

application(s). If the applicant has a large number of product variants (e.g., nicotine concentration and/or flavor options),

it may be justifiable to bridge data from a study including a subset of their products to one or more of their other products

(not included in the study). In contrast, because of the need for product-specific information, bridging from a different set

of products (not the subject of the application) would not be appropriate here.

(b) (4)

TPL Review of PMTAs: Page 14 of 20

search for other studies that provided product-specific evidence related to the potential benefit to

adult users.

3. SCIENTIFIC REVIEW

Reviews were completed by Allison Hoffman and Willa Dong on September 17, 2021.

The reviews determined that, although the PMTAs includes a RCT and longitudinal cohort study, the

studies did not include the actual use of the new products or compare tobacco-flavored products to

other flavored products. In particular, the data from the RCT did not sufficiently demonstrate the

relative effect of the flavored products as compared to a tobacco-flavored product or include

outcomes assessing switching or cigarette reduction and the data from the cohort study not

sufficiently demonstrate the relative effect of the flavored products as compared to a tobacco-

flavored product. Therefore, these are insufficient to evaluate the magnitude of the potential

benefit to adult users that is needed to complete our assessment.

The PMTAs referenced studies including those that that assessed exposure biomarkers and

(b)(4)

physiological response following use, the effects of on health outcomes such as lung

function, and surveys on consumer perceptions and intentions to use , but this evidence is

not sufficiently strong to support the benefit to adult smokers of using these flavored ENDS because

it was not clear that the referenced studies included the specific products in the application(s);

evaluate product switching or cigarette reduction resulting from use of these products over time; or

evaluate these outcomes based on flavor type to enable comparisons between tobacco and other

flavors. Accordingly, this evidence is not adequate and therefore, we did not assess other aspects of

the application as part of this scientific review.

4. ENVIRONMENTAL DECISION

Under 21 CFR 25.35(b), issuance of an order under section 910(c) of the Federal Food, Drug, and

Cosmetic Act that a new product may not be introduced or delivered for introduction into interstate

commerce (i.e., a marketing denial order) falls within a class of actions that are ordinarily

categorically excluded from the preparation of an environmental assessment (EA) or environmental

impact statement (EIS). To the best of our knowledge, no extraordinary circumstances exist that

would preclude application of this categorical exclusion. FDA concludes that categorical exclusion is

warranted and no EA or EIS is required.

5. CONCLUSION AND RECOMMENDATION

FDA has reviewed these applications for evidence demonstrating that the new flavored products will

provide an added benefit to adult smokers relative to tobacco-flavored products. Based on our

review, we determined that the PMTAs for the applicant’s new products, as described in the

applications and specified in Appendix A, lack sufficient evidence to demonstrate that permitting the

marketing of the new products would be APPH. Thus, a Denial letter should be issued to the

applicant. The applicant cannot introduce or deliver for introduction these products into interstate

commerce in the United States. Doing so is a prohibited act under section 301(a) of the FD&C Act,

the violation of which could result in enforcement action by FDA.

The following deficiency should be conveyed to the applicant as the key basis for our determination

that marketing of the new products is not APPH:

1. All of your PMTAs lack sufficient evidence demonstrating that your flavored ENDS will

provide a benefit to adult users that would be adequate to outweigh the risks to youth. In

(b) (4)

TPL Review of PMTAs: Page 15 of 20

light of the known risks to youth of marketing flavored ENDS, robust and reliable evidence is

needed regarding the magnitude of the potential benefit to adult smokers. This evidence

could have been provided using a randomized controlled trial (RCT) and/or longitudinal

cohort study that demonstrated the benefit of your flavored ENDS products over an

appropriate comparator tobacco-flavored ENDS. Although your PMTA includes a RCT and

cohort study, it is unclear if they included the actual use of the new products. Additionally,

the RCT and cohort studies did not compare tobacco-flavored products to other flavored

products or include outcomes assessing switching or cigarette reduction. In particular, the

data from your RCT did not sufficiently demonstrate the relative effect of your flavored

products as compared to a tobacco-flavored product or the effects on switching or cigarette

reduction and the cohort study did not sufficiently demonstrate the relative effect of your

flavored products as compared to a tobacco-flavored product. Therefore, these are

insufficient to evaluate the magnitude of the potential benefit to adult users that is needed

to complete our assessment.

Alternatively, FDA would consider other evidence but only if it reliably and robustly

evaluated the impact of the new flavored vs. tobacco-flavored products on adult smokers’

switching or cigarette reduction over time. Although your PMTAs referenced studies

including those that that assessed exposure biomarkers and physiological response

(b)(4)

following use, the effects of on health outcomes such as lung function, and

surveys on consumer perceptions and intentions to use , this evidence is not

sufficient to show a benefit to adult smokers of using these flavored ENDS because it was

not clear that the referenced studies included the specific products in the application(s);

evaluate product switching or cigarette reduction resulting from use of these products over

time; or evaluate these outcomes based on flavor type to enable comparisons between

tobacco and other flavors. Without this information, FDA concludes that your application is

insufficient to demonstrate that these products would provide an added benefit that is

adequate to outweigh the risks to youth and, therefore, cannot find that permitting the

marketing of your new tobacco products would be appropriate for the protection of the

public health.

Review of PMTAs:[(

l:5

} (4 }7

Page 16

APPENDIX

Appendix

New

Products

Common

Attributes

Submi

date September 7, 2020

Rece

date September 7, 2020

App

cant (6) (~)

Product manufacturer

(

b)(4)

Product category

ENDS

APES

)

Product subcategory

ENDS

Component

(b) (4)

TPL Review of PMTAs: Page 17 of 20

7. REFERENCES

1. U.S. Department of Health and Human Services. E-cigarette Use Among Youth and Young Adults.

A Report of the Surgeon General. Atlanta, GA: U.S. Dept of Health and Human Services, Centers

for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health

Promotion, Office on Smoking and Health;2016.

2. U.S. Food and Drug Administration. Enforcement Priorities for Electronic Nicotine Delivery

Systems (ENDS) and Other Deemed Products on the Market Without Premarket Authorization

(Revised). Guidance for Industry. Silver Spring, MD: U.S. Department of Health and Human

Services, Food and Drug Administration, Center for Tobacco Products;April 2020.

3. Wang TW, Neff LJ, Park-Lee E, Ren C, Cullen KA, King BA. E-cigarette Use Among Middle and

High School Students - United States, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(37):1310-

1312.

4. Wang TW, Gentzke AS, Neff LJ, et al. Disposable E-Cigarette Use among U.S. Youth — An

Emerging Public Health Challenge. New England Journal of Medicine. 2021;384(16):1573-1576.

5. U.S. Department of Health and Human Services. Preventing Tobacco Use Among Youth and

Young Adults: A Report of the Surgeon General. Atlanta, GA: U.S. Dept of Health and Human

Services, Centers for Disease Control and Prevention, National Center for Chronic Disease

Prevention and Health Promotion, Office on Smoking and Health;2012.

6. U.S. Department of Health and Human Services. The Health Consequences of Smoking: 50 Years

of Progress. A Report of the Surgeon General. Atlanta, GA: U.S. Department of Health and

Human Services, Centers for Disease Control and Prevention, National Center for Chronic

Disease Prevention and Health Promotion, Office on Smoking and Health;2014.

7. Apelberg BJ, Corey CG, Hoffman AC, et al. Symptoms of tobacco dependence among middle and

high school tobacco users: results from the 2012 National Youth Tobacco Survey. Am J Prev

Med. 2014;47(2 Suppl 1):S4-14.

8. Gentzke AS, Wang TW, Jamal A, et al. Tobacco Product Use Among Middle and High School

Students - United States, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(50):1881-1888.

9. Wang TW, Gentzke AS, Creamer MR, et al. Tobacco Product Use and Associated Factors Among

Middle and High School Students - United States, 2019. MMWR Surveill Summ. 2019;68(12):1-

22.

10. Cullen KA, Liu ST, Bernat JK, et al. Flavored Tobacco Product Use Among Middle and High School

Students - United States, 2014-2018. MMWR Morb Mortal Wkly Rep. 2019;68(39):839-844.

11. Schneller LM, Bansal-Travers M, Goniewicz ML, McIntosh S, Ossip D, O'Connor RJ. Use of

Flavored E-Cigarettes and the Type of E-Cigarette Devices Used among Adults and Youth in the

US-Results from Wave 3 of the Population Assessment of Tobacco and Health Study (2015-

2016). Int J Environ Res Public Health. 2019;16(16).

12. Villanti AC, Johnson AL, Glasser AM, et al. Association of Flavored Tobacco Use With Tobacco

Initiation and Subsequent Use Among US Youth and Adults, 2013-2015. JAMA Netw Open.

2019;2(10):e1913804.

13. Rose SW, Johnson AL, Glasser AM, et al. Flavour types used by youth and adult tobacco users in

wave 2 of the Population Assessment of Tobacco and Health (PATH) Study 2014-2015. Tob

Control. 2020;29(4):432-446.

14. Rostron BL, Cheng YC, Gardner LD, Ambrose BK. Prevalence and Reasons for Use of Flavored

Cigars and ENDS among US Youth and Adults: Estimates from Wave 4 of the PATH Study, 2016-

2017. Am J Health Behav. 2020;44(1):76-81.

15. Ambrose BK, Day HR, Rostron B, et al. Flavored Tobacco Product Use Among US Youth Aged 12-

17 Years, 2013-2014. JAMA. 2015;314(17):1871-1873.

(b) (4)

TPL Review of PMTAs: Page 18 of 20

16. Tsai J, Walton K, Coleman BN, et al. Reasons for Electronic Cigarette Use Among Middle and

High School Students - National Youth Tobacco Survey, United States, 2016. MMWR Morb

Mortal Wkly Rep. 2018;67(6):196-200.

17. Audrain-McGovern J, Strasser AA, Wileyto EP. The impact of flavoring on the rewarding and

reinforcing value of e-cigarettes with nicotine among young adult smokers. Drug Alcohol

Depend. 2016;166:263-267.

18. St Helen G, Dempsey DA, Havel CM, Jacob P, 3rd, Benowitz NL. Impact of e-liquid flavors on

nicotine intake and pharmacology of e-cigarettes. Drug Alcohol Depend. 2017;178:391-398.

19. Morean ME, Butler ER, Bold KW, et al. Preferring more e-cigarette flavors is associated with e-

cigarette use frequency among adolescents but not adults. PLoS One. 2018;13(1):e0189015.

20. Leventhal AM, Goldenson NI, Cho J, Kirkpatrick MG, McConnell RS, Stone MD, Pang RD, Audrain-

McGovern J, Barrington-Trimis JL. Flavored E-cigarette Use and Progression of Vaping in

Adolescents. Pediatrics. 2019 Nov;144(5):e20190789.

21. Audrain-McGovern J, Rodriguez D, Pianin S, Alexander E. Initial e-cigarette flavoring and nicotine

exposure and e-cigarette uptake among adolescents. Drug Alcohol Depend. 2019;202:149-155.

22. Villanti AC, Johnson AL, Halenar M, et al. Menthol and mint cigarettes and cigars: Initiation and

progression in youth, young adults and adults in Waves 1 - 4 of the PATH Study, 2013 - 2017.

Nicotine Tob Res. 2020.

23. Carpenter CM, Wayne GF, Pauly JL, Koh HK, Connolly GN. New cigarette brands with flavors that

appeal to youth: tobacco marketing strategies. Health Aff (Millwood). 2005;24(6):1601-1610.

24. Pepper JK, Ribisl KM, Brewer NT. Adolescents' interest in trying flavoured e-cigarettes. Tobacco

Control. 2016;25(Suppl 2):ii62.

25. Camenga DR, Morean M, Kong G, Krishnan-Sarin S, Simon P, Bold K. Appeal and Use of

Customizable E-cigarette Product Features in Adolescents. Tob Regul Sci. 2018;4(2):51-60.

26. Harrell MB, Weaver SR, Loukas A, et al. Flavored e-cigarette use: Characterizing youth, young

adult, and adult users. Prev Med Rep. 2017;5:33-40.

27. Cullen KA, Gentzke AS, Sawdey MD, et al. e-Cigarette Use Among Youth in the United States,

2019. Jama. 2019;322(21):2095-2103.

28. Foulds J, Veldheer S, Yingst J, et al. Development of a questionnaire for assessing dependence

on electronic cigarettes among a large sample of ex-smoking E-cigarette users. Nicotine Tob Res.

2015;17(2):186-192.

29. Yingst J, Foulds J, Veldheer S, et al. Measurement of Electronic Cigarette Frequency of Use

Among Smokers Participating in a Randomized Controlled Trial. Nicotine Tob Res.

2020;22(5):699-704.

30. Vogel EA, Prochaska JJ, Ramo DE, Andres J, Rubinstein ML. Adolescents' E-Cigarette Use:

Increases in Frequency, Dependence, and Nicotine Exposure Over 12 Months. J Adolesc Health.

2019;64(6):770-775.

31. Slotkin TA. Nicotine and the adolescent brain: insights from an animal model. Neurotoxicol

Teratol. 2002;24(3):369-384.

32. Yuan M, Cross SJ, Loughlin SE, Leslie FM. Nicotine and the adolescent brain. J Physiol.

2015;593(16):3397-3412.

33. Bava S, Tapert SF. Adolescent brain development and the risk for alcohol and other drug

problems. Neuropsychol Rev. 2010;20(4):398-413.

34. Bernheim A, Halfon O, Boutrel B. Controversies about the enhanced vulnerability of the

adolescent brain to develop addiction. Front Pharmacol.

2013;4:118.

35.

Casey BJ, Jones RM. Neurobiology of the adolescent brain and behavior: implications for

substance use disorders. J Am Acad Child Adolesc Psychiatry. 2010;49(12):1189-1201; quiz 1285.

36. Doremus-Fitzwater TL, Varlinskaya EI, Spear LP. Motivational systems in adolescence: possible

implications for age differences in substance abuse and other risk-taking behaviors. Brain Cogn.

2010;72(1):114-123.

(b) (4)

TPL Review of PMTAs: Page 19 of 20

37. Shulman EP, Smith AR, Silva K, et al. The dual systems model: Review, reappraisal, and

reaffirmation. Dev Cogn Neurosci. 2016;17:103-117.

38. Kota D, Sanjakdar S, Marks MJ, Khabour O, Alzoubi K, Damaj MI. Exploring behavioral and

molecular mechanisms of nicotine reward in adolescent mice. Biochem Pharmacol.

2011;82(8):1008-1014.

39. Shram MJ, Lê AD. Adolescent male Wistar rats are more responsive than adult rats to the

conditioned rewarding effects of intravenously administered nicotine in the place conditioning

procedure. Behav Brain Res. 2010;206(2):240-244.

40. Natividad LA, Torres OV, Friedman TC, O'Dell LE. Adolescence is a period of development

characterized by short- and long-term vulnerability to the rewarding effects of nicotine and

reduced sensitivity to the anorectic effects of this drug. Behav Brain Res. 2013;257:275-285.

41. de la Peña JB, Ahsan HM, Tampus R, et al. Cigarette smoke exposure during adolescence

enhances sensitivity to the rewarding effects of nicotine in adulthood, even after a long period

of abstinence. Neuropharmacology. 2015;99:9-14.

42. Conner M, Grogan S, Simms-Ellis R, et al. Do electronic cigarettes increase cigarette smoking in

UK adolescents? Evidence from a 12-month prospective study. Tob Control. 2017;27(4):365-372.

43. Holliday ED, Gould TJ. Chronic Nicotine Treatment During Adolescence Attenuates the Effects of

Acute Nicotine in Adult Contextual Fear Learning. Nicotine Tob Res. 2017;19(1):87-93.

44. Counotte DS, Spijker S, Van de Burgwal LH, et al. Long-lasting cognitive deficits resulting from

adolescent nicotine exposure in rats. Neuropsychopharmacology. 2009;34(2):299-306.

45. Fountain SB, Rowan JD, Kelley BM, Willey AR, Nolley EP. Adolescent exposure to nicotine impairs

adult serial pattern learning in rats. Exp Brain Res. 2008;187(4):651-656.

46. Soneji S, Barrington-Trimis JL, Wills TA, et al. Association Between Initial Use of e-Cigarettes and

Subsequent Cigarette Smoking Among Adolescents and Young Adults: A Systematic Review and

Meta-analysis. JAMA Pediatr. 2017;171(8):788-797.

47. Aleyan S, Cole A, Qian W, Leatherdale ST. Risky business: a longitudinal study examining

cigarette smoking initiation among susceptible and non-susceptible e-cigarette users in Canada.

BMJ Open. 2018;8(5):e021080.

48. Berry KM, Fetterman JL, Benjamin EJ, et al. Association of Electronic Cigarette Use With

Subsequent Initiation of Tobacco Cigarettes in US Youths. JAMA Netw Open. 2019;2(2):e187794.

49. Hammond D, Reid JL, Cole AG, Leatherdale ST. Electronic cigarette use and smoking initiation

among youth: a longitudinal cohort study. Cmaj. 2017;189(43):E1328-e1336.

50. Loukas A, Marti CN, Cooper M, Pasch KE, Perry CL. Exclusive e-cigarette use predicts cigarette

initiation among college students. Addict Behav. 2018;76:343-347.

51. Treur JL, Rozema AD, Mathijssen JJP, van Oers H, Vink JM. E-cigarette and waterpipe use in two

adolescent cohorts: cross-sectional and longitudinal associations with conventional cigarette

smoking. Eur J Epidemiol. 2018;33(3):323-334.

52. Kintz N, Liu M, Chou CP, et al. Risk factors associated with subsequent initiation of cigarettes and

e-cigarettes in adolescence: A structural equation modeling approach. Drug Alcohol Depend.

2020;207:107676.

53. Lozano P, Barrientos-Gutierrez I, Arillo-Santillan E, et al. A longitudinal study of electronic

cigarette use and onset of conventional cigarette smoking and marijuana use among Mexican

adolescents. Drug Alcohol Depend. 2017;180:427-430.

54. Stanton CA, Bansal-Travers M, Johnson AL, et al. Longitudinal e-Cigarette and Cigarette Use

Among US Youth in the PATH Study (2013-2015). J N

atl Cancer Inst. 2019;111(10):1088-1096.

55. Best C, Haseen F, Currie D, et al. Relationship between trying an electronic cigarette and

subsequent

cigarette experimentation in Scottish adolescents: a cohort study. Tob Control.

2017;27(4):373-378.

56. Bold KW, Kong G, Camenga DR, et al. Trajectories of E-Cigarette and Conventional Cigarette Use

Among Youth. Pediatrics. 2018;141(1).

(b) (4)

TPL Review of PMTAs: Page 20 of 20

57. National Academies of Sciences Engineering Medicine. Public Health Consequences of E-

Cigarettes. In. Washington, DC: National Academies Press; 2018.

58. Edwards KC, Sharma E, Halenar MJ, et al. Longitudinal pathways of exclusive and polytobacco

cigar use among youth, young adults and adults in the USA: findings from the PATH Study Waves

1-3 (2013-2016). Tob Control. 2020;29(Suppl 3):s163-s169.

59. Gentzke AS, Creamer M, Cullen KA, et al. Vital Signs: Tobacco Product Use Among Middle and

High School Students - United States, 2011-2018. MMWR Morb Mortal Wkly Rep.

2019;68(6):157-164.

60. Wang TW, Gentzke A, Sharapova S, Cullen KA, Ambrose BK, Jamal A. Tobacco Product Use

Among Middle and High School Students - United States, 2011-2017. MMWR Morb Mortal Wkly

Rep. 2018;67(22):629-633.

61. Osei AD, Mirbolouk M, Orimoloye OA, et al. Association Between E-Cigarette Use and Chronic

Obstructive Pulmonary Disease by Smoking Status: Behavioral Risk Factor Surveillance System

2016 and 2017. Am J Prev Med. 2020;58(3):336-342.

62. Osei AD, Mirbolouk M, Orimoloye OA, et al. The association between e-cigarette use and

asthma among never combustible cigarette smokers: behavioral risk factor surveillance system

(BRFSS) 2016 & 2017. BMC Pulm Med. 2019;19(1):180.

63. Giovanni SP, Keller TL, Bryant AD, Weiss NS, Littman AJ. Electronic Cigarette Use and Chronic

Respiratory Symptoms among U.S. Adults. Am J Respir Crit Care Med. 2020;201(9):1157-1160.

64. Chang JT, Anic GM, Rostron BL, Tanwar M, Chang CM. Cigarette Smoking Reduction and Health

Risks: A Systematic Review and Meta-analysis. Nicotine Tob Res. 2021;23(4):635-642.

65. Vyncke T, De Wolf E, Hoeksema H, et al. Injuries associated with electronic nicotine delivery

systems: A systematic review. J Trauma Acute Care Surg. 2020;89(4):783-791.

66. Rossheim ME, Livingston MD, Soule EK, Zeraye HA, Thombs DL. Electronic cigarette explosion

and burn injuries, US Emergency Departments 2015–2017. Tobacco Control. 2019;28(4):472.