The JACS Challenge

What makes a scientific publication good?
As scientists, are we capable of discerning good papers from bad papers?

These questions were the focus of a recent conversation I had on twitter with Stuart Cantrill, editor at Nature Chemistry, and bloggers extraordinaire, Chemjobber and See Arr Oh. This conversation started with the observation that a recent paper, containing what might be evidence of scientific misconduct, was generating a lot of buzz on-line. Chemists were rightly upset about this alleged impropriety. But, the incident highlighted a different aspect of scientific publishing to me. Precisely:

We, as a scientific community, generate many thousands of research articles a year. In my ideal world, research publications should be an ongoing conversation to better understand our world. Most of that conversation does occur in the literature. For a healthy discipline, shouldn’t that conversation be happening outside of the literature as well? This dialogue certainly happens at conferences and within research groups and in general-interest articles. Shouldn’t these conversations also be taking place in the on-line world as well? Between blogs and social media, there are some powerful methods for enabling real-time communication with scientists all over the world! Shouldn’t some topics be so vital and interesting to our field that we want to discuss them immediately? Even if they aren’t within our chosen sub-discipline? Shouldn’t we collectively be talking about some research as much as we discuss research misconduct?

With these topics on my mind, I asked Stu, (paraphrasing) “As an editor, can you recognize a paper that should garner broader discussion, and how do you effectively share *good* papers?” We went on to have a broader conversation about how any of us (seriously, we’re all experts in chemistry here) recognize a “worthy” paper. Finally, in a fit of genius, Stu asked, “If we were all given the same 10year old copy of JACS1, would we pick out the same ‘top 5′ papers?” Stu’s hypothesis for this question, which the rest of us agreed with, was: I doubt it!

So we set forth to find out! We sent out emails to a broad range of chemists (grad students, professors, editors, journalists, professionals) asking if they’d like to be involved in our experiment. Could they (we) choose the best papers from a 10 year old issue of JACS?

Best or good or important are judgements that are entirely subjective! So we asked four very specific questions:

1) Which three papers in the issue do YOU think are the most ‘significant’ (your own definition of ‘significant’ is what is important here)?

2) Without looking up the numbers, which three papers do you think will have been cited the most to-date?

3) Which three papers would you most want to point out to other chemists?

4) Which three papers would you want to shout about from the rooftops (i.e., tell anybody about, not just chemists)?

None of these questions really get at what it means to be important. They are just ways to describe how we perceive importance. Even question number 2, which evokes an easily retrieved metric, doesn’t necessarily prove a paper’s “worth”. But, are we (scientists! chemists!) even able to figure out which articles would get cited the most?

Our esteemed panel of experts have weighed in with their thoughts. How well would you do in the JACS Challenge? We’ll be back in one week with our respondents answers along with an analysis of their answers and thoughts.

But, first, we’d love to hear from you! How would you answer these four questions?! We have set up a survey on Survey Monkey in order to compile more answers. We plan on using your anonymous responses to try to glean some generalized trends about the way chemists view a study’s “importance” for a publication that we are preparing. We’d love to hear your thoughts on how you tried to analyze these papers here in the comments of this post!

Are you chemist-enough? You’ve got the questions! You’ve got the issue of JACS! Put your expertise on the line!!

We’re looking forward to hearing from you!

Survey link: HERE

- See Arr Oh, Chemjobber, Stu, and Matt

1This could have just as easily involved an issue of Angewandte Chemie or other journal. We chose JACS because it was the first journal that popped up (in considering journals that is a standard of quality and broad applicability among all chemists) and also because “The JACS Challenge” had such a nice ring to it!

This entry was posted in Uncategorized. Bookmark the permalink.

4 Responses to The JACS Challenge

  1. This is my first time I visit here. I found some interesting stuff in your post, though I find that this entertaining and very readable piece would be greatly enhanced by a brief summary of the key issues, including some schemes. A picture is worth a 1000 words! I am sure that my old friend RJKT would approve!

  2. Nice article and sound arguments. Just shared this post with a colleague…

  3. GeHava says:






    精准医疗要做到个性、高效及预防的关键在于筛查和诊断,因此基因测序等检测诊断技术的发展是关键。成本的下降让基因测序商业化市场的打开成为可能,基因测序技术的成熟和商用经过了多年的发展,1980 年自动测序仪出现,2001 年完成了人类基因组框架图标志着这一技术的成熟,2007 年二代基因测序技术大幅降低测序成本,使得这一技术应用出现可能,以走在前列的Illumina 公司为例,该公司自2007 年起把当时每个基因组的测序成本费用从1000万美元降到了当下的1000 美元, 根据Illumina 公司数据,全球NGS(二代基因测序)的应用市场规模预计为200 亿美元,药品研发和临床应用是增速最快的领域,增速超过15%,肿瘤诊断和个性化用药是最有应用前景的领域,市场规模120亿美元。乐土投资与Illumina以及新一代的基因检测公司Genalyte, Centrillion都有着合作关系。


    全球创新论坛纽约峰会由全美华人金融协会(The Chinese Finance Association, TCFA) 主办。全美华人金融协会于一九九四年在美国成立。分布在世界各地的会员来自华尔街投行、基金、监管部门、和学术界,已成为联系中美金融界最重要的桥梁之一。协会定期举行学术年会。协会本部设在纽约,并在波士顿,华盛顿,旧金山,伦敦,香港,北京和上海等金融中心设有分会。


Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>