Sign up to receive these free articles weekly.

Reader Question: We are in the process of applying for a grant to extend promising preliminary results. But should we also include mention of some contradictory data we found? This would demonstrate we did extra hours of work, but will it undercut our chances for funding?

Answer: As to the "extra" hours of work you put in, nobody cares. Only results count.

The first purpose in presenting preliminary data is to demonstrate feasibility and that you have the necessary expertise to carry out the procedures you propose in the Research Methods section. I believe it doesn't matter if you have some contradictory results, or even that the data you are describing were collected for other purposes in a different study. The focus in this section is not on the results per se, but on the fact that you were successfully able to collect and analyze the data.

But pilot data, showing positive results, are potent evidence. They indicate that you are on the right track. The reviewers will weight this heavily. There are subtleties to reporting these, however. The possible variations involve (1) primacy of the outcomes; (2) positive/null/negative results; and (3) statistical power of your pilot study to detect effects. Here are the possibilities:

Implications of preliminary results

The main problem is the null result in your primary outcome. If your sample size was small, and power inadequate, this isn't necessarily fatal - but it will require careful wording when you write the section.

One would not necessarily mention the statistical power in pilot data; the reviewer will get a hint of it from the sample size. Remember that in a small sample, even a single outlier will kill your significance.

Next scenario: Null results. One can never prove the null hypothesis, so it doesn’t matter how big the sample size may be, there could always be other reasons for the null results. However, the reviewer is probably going to believe that result and penalize you for it.

Another strategy would be to run a new, quick-and-dirty pilot. The limitation is whether you have the time and resources (and inclination) to do so. And, do you have any reason to think your results will turn out differently?

Let me close on a controversial note. You decide what to report - and what not to. In circumstances like yours, some will bend the rules (note I do not say "break"). A common strategy is to design the study to address several outcomes, in hopes that at least a few will work out (known as “fishing”, not a compliment). Further, you may simply omit mention of null results. However, don't underestimate the intelligence of your reviewers for spotting such maneuvers.

Comments by William Gerin, Ph.D., Professor of Biobehavioral Health, Pennsylvania State University.
Author, Writing the NIH Grant Proposal: A Step-by-Step Guide, SAGE Books (2006)

This eAlert is brought to you as an informational training tool by the Principal Investigators Association, which is an independent organization. Neither the eAlert nor its contents have any connection with the National Institutes of Health (NIH), nor are they endorsed by this agency. All views expressed are those personally held by the author and are not official government policies or opinions.