Tuesday, July 7, 2015

CASAA Research Fund Launches Fundraising for Study of Gateway Claims

UPDATE September 4, 2015: CASAA is pleased to announce that thanks to your generosity and support, we have reached our fundraising goal for this project. We expect that the researchers will have a working paper ready for comments and review within the next four to five months. Thank you to all who contributed to support this important effort. This project will be a great use of your donations, providing both good science and an invaluable tool for advocacy that would not exist otherwise.


The CASAA Research Fund has an opportunity to support some valuable new research which we believe will benefit e-cigarette and other tobacco harm reduction (THR) advocacy efforts. We are seeking crowdfunding and will be matching every dollar donated (up to $7,500) one-for-one.

As you probably know, one of the most common attacks on e-cigarettes, as well as snus and other THR products, is that they will be a “gateway” to smoking. The gateway claim is part of the standard playbook of people who want to ban or otherwise restrict something that poses minimal risk: They claim that it causes another behavior that poses great risk. That way they can pretend that they are motivated by concerns about health, even when that is clearly not the case.

There are frequent claims that e-cigarettes have been shown to be a gateway to smoking. All of the claims of scientific evidence that we are aware of are based on highly limited and flawed research, as demonstrated in a research methodology paper recently published by our Chief Scientific Officer, Carl V Phillips, PhD. Indeed, there has never been a credible case that any THR products cause smoking that we are aware of. Unfortunately, there is no definitive study we can point to that provides such an assessment. Dr. Phillips’s paper provides the framework for making the point, but it is a technical analysis that requires the reader to connect the dots for any specific claim.

A team led by Karyn Heavner, PhD, as Principal Investigator, working with Dr. Phillips and Catherine M Nissen has proposed conducting the research and publishing a paper or two that will review the claims that are being made and connect the dots. The full formal research proposal appears below. To summarize, the researchers will review all of the scientific studies that include claims, or are used in claims, that e-cigarettes and other THR products are a gateway to smoking. They will also review the recent prominent claims -- from websites, blogs, the popular press, and other media -- by influential institutions and individuals that say or imply there is a gateway effect from e-cigarettes. They will trace the sources of these claims. Their hypotheses are that all the claims of evidence fail and that all popular claims trace either to one of these sources, to something that does not even ostensibly provide such evidence, or nothing at all. They will document these in detail and summarize the results.

Dr. Heavner is an epidemiologist who wrote some of the earliest research on e-cigarettes in the late 2000s. Presumably everyone familiar with CASAA is familiar with Dr. Phillips and his work, which includes a similar review of claims about smokeless tobacco about ten years ago. Ms. Nissen worked with both of them at the University of Alberta.

The budget for the project is $15,000, which makes it the type of high bang-for-the-buck project that CASAA seeks to focus on funding. It will be immediately useful for our and others’ advocacy efforts to protect access to e-cigarettes and push back against unwarranted restrictions. Since CASAA will match all donations, we need $7,500 in earmarked donations to move forward with this project. (Any donations beyond this target sum are also most welcome, and will be kept in the Research Fund for future projects.)

The Research Proposal is included below. For those who are interested in hearing Dr. Phillips's thoughts on the project, you may find the CASAA Podcast Update for July 6, 2015 of some interest.


Research Proposal

Review of claims and evidence of a gateway effect for tobacco products

Study personnel

Principal investigator: Karyn Heavner, PhD

Co-investigators: Catherine M. Nissen, Carl V. Phillips, PhD

Background An increasingly frequent claim used by those who would restrict the availability of low-risk tobacco products (e-cigarettes and smokeless tobacco) is that they are a gateway to smoking. That is, they cause would-be nonsmokers to become smokers. Despite there being little apparent reason why this would happen, the claim has gained substantial political traction and it is widely said to be empirically supported. However, the claims of empirical support we are aware of do not stand up to scientific scrutiny.

Phillips recently published a paper on what empirical evidence would be needed to support a claim of a gateway effect for tobacco products: http://www.mdpi.com/1660-4601/12/5/5439/htm. In that paper he:

  • argues that the logic of the gateway claim is such an unlikely scenario that the claim should be seen as extraordinary, requiring extraordinary evidence;
  • notes the fatal flaws in reasoning of study authors and others who claim that particular evidence supports a gateway claim – in particular that they fail to discriminate between the gateway effect they claim to observe versus causation in the other direction (switching from smoking to a low-risk product for harm reduction) or the inevitable massive confounding (anyone inclined to use one tobacco product is more inclined than average to use another);
  • provides a checklist for what a particular study or analysis would need to do in order to support a gateway claim;
  • reports observing (though not based on any systematic review) that no existing claim meets those conditions or even provides generally supporting evidence, let alone extraordinary evidence;
  • makes other methodological observations, suggests methods for actually observing a gateway effect if one exists, and provides an extended example of a useful analysis.

A practical upshot of that paper is that the author reported that none of the research he identified genuinely supported the gateway claim. However, that paper did not include a systematic review of the evidence, and so cannot be said to show there is no such evidence. Moreover, it is written as a epidemiology methods paper, and thus is relatively difficult reading (being both long and technical) and requires the reader to connect the dots. Thus, using it in the practical policy debate requires the intermediate step of understanding the paper and parsing each gateway claim that is made against the methodological analysis to determine if it passes muster.
Proposed research
To fill the gaps and create a paper that is more accessible to non-scientists, we propose to conduct a literature review applying the methodology developed in the Phillips paper. The review would have two parts:
Part 1. A systematic comprehensive review of scientific papers that are purported to show a gateway effect for tobacco products. These would then be parsed against the methodology points from the Phillips paper.
The dataset would be generated using a keyword search of PubMed and Google Scholar, supplemented by snowballing from any papers referenced in the papers we find and in the dataset generated in Part 2. The latter step is critical because activist authors have recently written papers that were designed to be the basis for gateway claims – which they then made in other forums – while carefully avoiding making the gateway claims in the paper itself (an example is discussed in the Phillips paper). This search strategy might not find archival papers that played that role (e.g., a paper from the 2000s about smokeless tobacco that did not itself include gateway claims but was used to support such claims in activist documents that are now largely forgotten), but should find all such recent papers that are being invoked in the debate surrounding e-cigarettes.
The papers would be analyzed based on the considerations presented in the Phillips methodology paper, which are necessary conditions for making a valid scientific argument that study results suggest a gateway effect (more details appear in the Phillips paper and will be reprised in the new paper):
  1. Do the authors offer a clear statement of what it even means to claim a gateway effect and do they offer any suggestion of the magnitude and timing of effect they are claiming? (I.e., is the hypothesis/claim even defined?
  2. Do the authors offer a prima facie case for why a gateway would ever occur or any testimonial evidence that even one case has ever occurred? (I.e., does the gateway conclusion have plausibility, regardless of the statistical results?)
  3. Upon observing an association between smoking and low-risk product use, is there any attempt to discriminate between gateway causation and the inevitable causation in the other direction (tobacco harm reduction)?
  4. Is there a serious attempt to discriminate between the observed association being causal or confounding?
  5. Is the attempt to deal with the inevitable confounding based on a theory of the nature of the confounding and is it empirically tested, or does it just consist of throwing in whatever covariates happen to be available?
  6. Do the study results support any prediction that would be true under the gateway hypothesis but unlikely if the hypothesis is false?
  7. Is there any attempt to detect temporal ordering of behaviors? (As noted in the Phillips paper, this is neither necessary nor sufficient for a gateway, but the right ordering tends to better support the claim for obvious reasons.)
  8. Did the authors report enough different variations of their models to make clear that a conclusion that the data supports a particular hypothesis is apparently not an artifact of the specific models and results that are reported?

Our hypothesis is that none of these conditions but 7 is met for any of the existing claims of a smoking gateway. Our preliminary review finds that the only paper that meets 2 and some of the other plausibility arguments (but provides no actual evidence of an effect) is the one that claims a gateway to cocaine use.
Part 2. A structured review of activists’ claims that there is a gateway, tracing the basis for each claim. This would obviously not be comprehensive; it would be semi-systematic, based on structured review criteria but limited to claims by authoritative or respected actors, rather than simply looking at everything that happened to rank high in a search.
This step harkens back to Phillips’s “curious history of a ‘fact’” research on the claims about smokeless tobacco and oral cancer. We would catalog the claims and what empirical research they trace to. Our hypothesis is that the claims (even limiting the review to the authoritative actors) do not address considerations 1 and 2 from the above list, and the majority are not sourced to any empirical evidence. For those that do cite empirical evidence, that evidence will be reviewed in Part 1, where our hypothesis is that the conditions for it being valid evidence are not met.
Work product
Approximately four months after securing funding, we will publish a working paper reporting on the proposed research. The paper will include: a table parsing all the empirical papers from Part 1 against the listed considerations, and whatever additional analysis of them that seems useful, as well as an appendix with all the details from Part 2 and a summary of that (probably in the form of network analysis diagrams) with accompanying analysis.
After circulating the working paper for comments we will then produce at least one (though
it may require two) journal-style papers from it and submit them for journal publication, pursuing that process until it succeeds. We will also attempt to present the results at any relevant conferences we are able to attend.

No comments: