Junior researchers face a choice: a high or low road to success?

November 8, 2013.

 

This is a  presentation from the International Psycho-Oncology Society Conference in Rotterdam, November 8, 2013 invited by the Early Career Professionals Special Interest Group.* I am grateful for such a relaxed opportunity to speak my mind about some issues that junior researchers in psycho-oncology, like those in many fields, are facing. Senior members of the field have failed you. We need you to undo some of the damage that is being done.

As you enter the field, recognize that you are different from cohorts of researchers who have come before you. On the one hand, you are more methodologically and statistically sophisticated. You are also more digitally savvy, although I am sometimes bewildered by how little you as yet take advantage of the resources of the Internet and social media.

On the other hand, you face new accountability and pressures in terms of the monitoring of the impact factor of journals in which you publish, as well as you having to adhere to reporting standards and preregister your clinical trials before you even run the first patient. Researches who came before you had it easier in these respects.

I’ve done this kind of talk before and I recognize there is an expected obsolescence to what I present. I recall way back when I was junior person in the field, senior faculty warned me not to start using email because it was a total waste of time and inferior to communicating by snail mail. I am sure that much of the advice being offered to you is just as valuable and soon to be obsolete. And, similarly, many of the tools and strategies you will need to acquire first seem a waste of time.

Five years ago, I would have encouraged you to get more comfortable communicating about your work and even self-promoting. I would have suggested you use the now obsolete means, listserves to do so.  I would have encouraged you to challenge the gross inadequacies of peer review by writing letters to the editor, which also have the advantage of cultivating critical skills better than journal clubs do. Both listserves and letters to the editor are now obsolete, but the ideas behind these recommendations still hold, maybe even more. You just have to pursue these goals differently and certainly with different tools.

As for myself, I’m undergone a lot of changes in the past five years. Some of my best recent papers have been written with authors gathered from the Internet, often without me first meeting all of them. I was honored that one of these papers won the Cochrane Collaboration’s Bill Silverman Prize, which I guess makes my co-authors and myself certified disruptive innovators.

I now tweet, blog, use Facebook, and champion open access publishing. Later in this talk, I will provide the exciting details of the launch of a trial of PubMed Commons. I had been afraid of having to observe an embargo on discussing this. But fortunately the shutdown of the US federal government ended, and PubMed Commons was launched just in time for me to talk about it in this presentation.

Tweeting and blogging are not distractions or alternatives to writing peer-reviewed papers, they can become the means of doing so. Tweets may grow into blog posts, then a series of blog posts, and eventually even a peer-reviewed journal article. No guarantees, but looking back, that’s how a number of my peer-reviewed papers have developed.

On the other hand, the process can work in reverse. Blogging and tweeting about recent and forthcoming papers is a very important part of how to be a scholar and how to promote yourself in the current digital moment.

Here are some examples of me self-consciously and experimentally promoting recent papers with blogging.

My first bit of advice to junior investigators is figure out where such action is occurring. The form and format it takes is constantly shifting. Observe, experiment, and get involved, consistent with your own comfort level. Remain lurking if you’d like, reading blogs, and occasionally expressing approval clicking “like” or “favorite” until you are ready for get more involved.

I invite all of you to join me in participating in disruptive innovation. On the other hand, I realize this is not for everyone, and so I will spell out alternative low road.

The state of the field being what it is, offers clear opportunities for you to conform, and play the game according to the rules that work. Many of you will do so and some of you can rise to the top of a mediocrity.

My second bit of advice is that if everyone likes your work, you can be certain that that you are not doing anything important. That sage advice I got from Andrew Oswald.

The behavioral and social sciences are a mess. We have four or five times the rate of positive findings relative to some of the hard sciences, and I don’t think is because our theories and methods are more advanced.

The field of psycho-oncology is particularly a mess, as seen in rampant confirmation bias and many of our widely acclaimed papers presenting evidence that is interpreted in ways that are exaggerated or outright false. The bulk of intervention studies in psycho-oncology are underpowered and the flaws in their designs provide a high risk of bias. Studies consistently obtain significant results at an impressive, but statistically improbable rate.

At the heart of the special problems of the field is the consistent subordination of a commitment to evidence-based science to the vested interests of those who want to promote and secure opportunities for the clinical services of their professions, regardless of what the evidence suggests. This is most notably seen in the relentless promotion of screening for distress in the absence of evidence that actually improves patient outcomes.

Ultimately, data will provide the basis for deciding whether screening is a cost-effectivedistress way of improvements and whether it represents the best use of scarce resources. I think that the evidence will be negative. But I am more worried about the lasting effects on the credibility and integrity of a field in which editing and peer review have been so distorted by the felt need to demonstrate that screening has benefit.

Many celebrated findings in the field of psycho-oncology are really null findings, if you carefully look at them.

These include

  • Spiegel, D., Kraemer, H., Bloom, J., & Gottheil, E. (1989). Effect of psychosocial treatment on survival of patients with metastatic breast cancer. The Lancet, 334(8668), 888-891.
  • Fawzy, F. I., Fawzy, N. W., Hyun, C. S., Elashoff, R., Guthrie, D., Fahey, J. L., & Morton, D. L. (1993). Malignant melanoma: effects of an early structured psychiatric intervention, coping, and affective state on recurrence and survival 6 years later. Archives of General Psychiatry, 50(9), 681.
  • Antoni, M. H., Lehman, J. M., Klibourn, K. M., Boyers, A. E., Culver, J. L., Alferi, S. M., … & Carver, C. S. (2001). Cognitive-behavioral stress management intervention decreases the prevalence of depression and enhances benefit finding among women under treatment for early-stage breast cancer. Health Psychology, 20(1), 20.
  • Andersen, B. L., Yang, H. C., Farrar, W. B., Golden‐Kreutz, D. M., Emery, C. F., Thornton, L. M., … & Carson, W. E. (2008). Psychologic intervention improves survival for breast cancer patients. Cancer, 113(12), 3450-3458.

There are important negative trials of supportive expressive therapy and expressive writing being kept hidden in file drawers. Just search clinicaltrials.gov.

Zombie ideas and tooth fairy science still hold sway in the literature and win mediatooth fairy attention. I have in mind the notion that psychological interventions can extend the lives of cancer patients and exaggerated ideas about the mind holding sway over the body and defeating cancer.

You are entering a system of publication and awards that is not working fairly. Papers appear in ostensibly peer reviewed journals without adequate review. There’s rampant cronyism in opportunities to publish and widespread sweetheart deals as to whether authors have to address concerns raised by reviewers. There is sandbagging of critics and negative findings. It is an embarrassment to the field that authors of flawed ideas are able to suppress commentary on their work and censor criticism.

The respected, high impact Journal of Clinical Oncology is particularly bad when it comes to psychosocial studies. It shows consistently flawed peer-review, the influence of sloppy editorial oversight, and serious restrictions on commenting on its miscarriage of the review process. Feeble post publication peer review is continually handicapped and silenced. I believe that journal has an ethical responsibility to identify to its readers which articles have evaded peer review and to announce that authors of published papers can exercise veto over any criticism or negative commentary.

If you want to take the low road, you have lots of opportunities to succeed.

  • Pick a trendy topic.
  • Don’t be critical of the dominant views, even if you see through the hype and hokum.
  • Use biological measures, particularly ones that can be derived from saliva, even if they have no or unknown clinical significance.
  • Report positive findings, even if you have to spin and torture and suppress data.
  • No matter what your results, in your discussion section claim they confirm the dominant view and reaffirm that view, even if it is irrelevant or contradicted by your findings.

When you design studies, have lots of endpoints that you can always ignore later. Pick the one to report that makes your study look best. A lot of the positive findings in literature cannot really be replicated, but you can always appear to do so by pushing aside the results of primary analyses, and favor unplanned secondary and subgroup analyses. If necessary, construct some post hoc new outcome measures you didn’t even envision when you originally designed your study. Prominent examples of these strategies can readily be found in the published literature.

Many of you will do all this, wittingly or unwittingly following the advice and example of your advisors, but you can become more proficient in pursuing this low road.

Alternatively, I invite at least some of you to take the high road and join me and participate in disruptive innovation. Again, it’s not for everyone.

Blog, and if you’re not ready to consistently post your own, join in a group blog. I highlymental ellf1 recommend groups like Mental Elf, where you can take turns offering critical commentary on recently published papers.

If you are not ready to blog, you can tweet. You can selectively follow those on Twitter who show they can offer you both fresh new ideas with which you would not otherwise come into contact, as well as a filtering out of much that is hype, hokum and sheer nonsense.

pubmedcommonsNow I can announce the PubMed Commons revolution is upon us. Here are some links that explain what it is and how it works.

As long as you have a paper published in PubMed, even a letter to the editor, you can secure an invitation to comment on any article that has appeared in PubMed. You can have others “like” or add a response to your comment, and you to theirs as part of a continuing process of post publication peer review. With PubMed Commons, we’re taking post publication peer review out of the hands of editors who so often have aggressively and vainly taken control of a process that should be left with readers.

I’m asking you to join with me in pursuing a larger goal of creating a literature that is an honest and reliable guide for other researchers, clinicians, patients, the media, and policymakers as to the best evidence. Let’s work together to create a system where review process is transparent and persists for the useful life of a work. For this last point, I give thanks to Michael Eisen, cofounder of PLOS one and disruptive innovator extraordinaire.

*Special thanks to

Claire Wakefield, Michelle Peate, University of NSW,Sydney Australia

Kirsten Douma and Inge Henselmans, Academic Medical Center Amsterdam, The Netherlands

Wendy Lichtenthal, Memorial Sloan-Kettering Cancer Center, New York City

(CC-BY-NC-SA)

Creative Commons License
This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution 3.0 Unported License.

This entry was posted in advice to junior researchers, cancer, research, Uncategorized and tagged , . Bookmark the permalink.

8 Responses to Junior researchers face a choice: a high or low road to success?

  1. John says:

    It is not just the field of psycho-oncology that is a ‘mess’. Patients suffering from ME/CFS are all too familiar with the issues you write about re: “The field of psycho-oncology is particularly a mess, as seen in rampant confirmation bias and many of our widely acclaimed papers presenting evidence that is interpreted in ways that are exaggerated or outright false. The bulk of intervention studies in psycho-oncology are underpowered and the flaws in their designs provide a high risk of bias. Studies consistently obtain significant results at an impressive, but statistically improbable rate”.

    I think a lot of the problems with peer-review in psychological ‘sciences’ could be due to a) these papers being peer-reviewed by other psychologists and psychiatrists and not biomedical researchers, b) the peer-reviews themselves being kept secret and not published alongside the published papers and perhaps most importantly c) there being no requirement to publish raw data along with the published papers.

    The thing is that most, if not all, of these failings are simply based on physical restrictions of a paper model of publishing, a limiting factor which does not exist in the digital age. With the advent of computers and the internet, you can publish all of the above and more just as easily as you can publish the article itself. As such, there simply is no good reason to keep these restrictions in place. Patients of all conditions, as well as society at large, deserve better.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  2. Wendy Lichtenthal says:

    Jim, this is truly thought-provoking and inspiring.

    I am grateful that you posted it since I am unable to attend IPOS this year. Your “how to succeed’s” are right-on, and as a player of the game, I sincerely appreciate you shaking the ground under me and forcing me to think, “At what cost?” I blush as I confess that I don’t know if I will abandon “the formula” quite yet, but I will certainly be more thoughtful about how I play the game because of the extremely important critiques you raise. Many thanks for putting this out there and leading by example.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • Thank you very much for your post. Stay tuned. Next week I will provide the annotated slide presentation for the screening debate on this website. My presentation is “The case against screening for distress.” Hope to see you next time.

      VN:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
      • Wendy Lichtenthal says:

        I’ll look forward to it!

        In discussing the points you raised with a colleague, she raised how the issue is really the promotion system….this is really what’s behind the temptation to take the low road. Not to punt blame and just say it’s the system’s fault; we’re each ultimately responsible for our personal decisions and actions, but there is a systemic issue that really complicates things!

        Until next time…thanks again!

        VA:F [1.9.22_1171]
        Rating: 0 (from 0 votes)
        • That is a totally fair comment and I would agree. And I would add that senior faculty feel pressures that they transmit to junior researchers, namely that the junior researchers get into high impact journals that are required by external evaluators of the senior faculty and their lab groups.

          VN:F [1.9.22_1171]
          Rating: 0 (from 0 votes)
          • Wendy Lichtenthal says:

            Amen! Though I haven’t had that experience directly, I know others who have. Hopefully, what you raise at least has junior faculty taking pause and encourages alternative paths–I know it has me thinking!

            Have a wonderful time at IPOS!

            VA:F [1.9.22_1171]
            Rating: 0 (from 0 votes)
  3. Adriana says:

    Excellent post!
    Not only for junior, but I think also for senior researches (a bit of self-criticism is not bad). Unfortunately, this time I cannot attend to the IPOS conference, but of course I will share your presentation with many of my junior colleagues. I am afraid many of them are following the low road (in my words the easy road). However, I think this kind of presentations are very inspiring and challenging for some others that we have chosen the high road (hard road).
    Some years ago, when I started my first master in Spain I attended to a presentation about Science 2.0 [http://www.science20.com], this was the first time that I realized about the two faces of Science.
    Thank you very much to spread this kind of information.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>