Gavin writes:

Dear TWIM,

In episode #157 on Moonmilk there was some discussion about its medicinal properties. Calcium Carbonate is the active ingredient in Tums! In addition, Elio may find it interesting that CaCO3 is a very popular additive in the cultivation of fungi. Amateur mushroom growers often add ground up oyster shells to improve growth.

Best,

Gavin

UCSF SOM

Class of 2021

Neil Greenspan writes:

June 26, 2017

Vincent and Arturo,

I listened with great interest to TWiM 154 with Arturo, a longtime friend. Clearly, Arturo is both one of the most thoughtful individuals in biomedical science and one of the most prolific. While I have a high regard for Arturo’s intellect and views, I persist in formulating my own perspectives on many of the topics he addresses. Frequently, I agree with Arturo’s take, but sometimes my view diverges in some degree. So, I have several comments regarding the conversation on TWiM 154.

Pathogenic Potential: Overall, Arturo’s concept of pathogenic potential is an advance because it highlights the limitations of current notions of virulence. I completely support his critique of the usual classification scheme for microbes as pathogenic, symbiotic, etc. As Arturo may recall, I have devoted a fair amount of time and effort to writing about the limitations of much biological thinking around classification and categories.

With respect to the term “pathogenic potential,” I am not certain it is the best possible choice. Alternatives I would suggest for consideration are “specific pathogenic activity” or “pathogenic potency,” with explicit recognition that the cited activity or potency is for the endpoint measured and may not be generally true for all endpoints.   In various branches of chemistry, “specific activity” or “potency” refers to the magnitude of an effect per unit concentration, and essentially Arturo’s ratio would seem to be an analogous variable for microbes. As noted above an important issue is how the ratio being determined will vary for different endpoints (mortality, weight loss, production of a cytokine or other gene product, or a type of histopathology such as granuloma). I would not be surprised to find examples of one pathogen or pathogen strain being more potent for endpoint 1 and less potent for endpoint 2 in comparison to another pathogen or pathogen strain (in a given host). Similar issues could arise for different hosts with a single pathogen.  

Measurement of pathogenic potential is worth trying, but I wonder how quantitatively similar assessments of a particular pathogen strain will be for the same inbred mouse strain in different locations. Here as well, I would not be surprised by significant differences due to either cryptic genetic differences or variation in relevant environmental factors.

Also, there are other limitations of the standard concepts of virulence and virulence factors not directly addressed by the definition of pathogen potential. For example, as I have plans to write about, there is now increasing evidence that optimal virulence is a property of a microbial population, not of individual microbes. In addition, there is some evidence consistent with significant interactions between different microbes such that, for example, a pathogen can, perhaps indirectly, transform a putatively non-pathogenic microbe into a mediator of host tissue damage. Another example of an interaction for which there is some evidence would be putatively non-pathogenic microbes influencing the ability of pathogenic microbes to infect.

Scientific Rigor: I agree with Arturo and Ferric Fang regarding the five elements of scientific rigor that they outline in their mBio article. They are all necessary, but I am not certain if they are all that is needed or at least all that needs to be brought to the attention of investigators.

In addition to being concerned about error, investigators must also be concerned about artifacts, i.e. results which may reveal some aspect of the assay system used but not reveal much about the underlying biological phenomenon purportedly being studied. For example, a technique involving the fusion of GFP to every protein in a cell to study rates of turnover reportedly resulted in incorrect measurements for some proteins due to GFP-associated perturbations associated with alterations in protein trafficking or interactions with other proteins involved in forming macromolecular assemblies (Yewdell et al. Cell Biol Int. 2011 May;35(5):457-62. doi: 10.1042/CBI20110055). Artifacts can result even in the context of careful experimental design and implementation if the measurement system is not adequately validated. One might prefer to classify artifacts as an issue of inadequate redundancy, specifically assay validation, but I suggest that the problem is of sufficient importance to be confronted more explicitly.

With respect to logic, the classical logical fallacies are not the sole problems. Failures to use appropriate conceptualizations of phenomena or recognize the limitations of any one conceptual scheme can also lead to serious errors of inference. I have published on this issue in immunology.

Falsification is useful as a rough guide to scientific practice, but it is not a panacea. Karl Popper’s claims about falsification, or at least the interpretations of those claims by others, have been heavily criticized within formal philosophy of science and are not regarded as foolproof. My understanding of the consensus of philosophers of science is that typically a single finding or piece of data cannot necessarily and definitively destroy a theory due to the potential for fashioning ad hoc explanations that arguably rescue the main hypothesis at issue.  

Arturo and Ferric are correct that logic is not generally taught to graduate students as a formal part of their coursework or other training, but they fall prey to the central limitation of inductive reasoning by claiming that, “with the exception of statistics, none of these disciplines are formally taught in the training of scientists.” Beginning in 1994, I began offering a graduate course that combined material on immunology, evolution, and logic. Readings from a book on mathematical logic were a part of the course. I emphasized the nature of deductive reasoning, how it differs from inductive reasoning, and used papers from the literature to illustrate examples of logically invalid arguments. The course has changed over the years, but logic is still part of it as of the last time it was taught in the spring of 2016.

More broadly, some of my colleagues and I have devoted energy to pointing out methodological issues in both seminar and journal club presentations by students. In addition to my own graduate course, I have addressed more philosophical issues in lectures for courses directed by other faculty and in research seminars. Nevertheless, I accept Arturo and Ferric’s basic point that the vast majority of graduate programs offer no routine training in logic and philosophical issues important for the primarily cognitive aspects of investigation.

A Funding Lottery: I agree with many of Arturo’s points. However, I want to note that others, such as my former Case colleague David Kaplan, previously pointed out the underpowered nature of peer review at study sections. Dave Kaplan also mentioned the concept of a lottery and suggested a criterion for assessing innovation based on the variance in grant scores, which may be related to Arturo’s idea that grants everyone understands and likes are probably not the most innovative. As Elio might have predicted, the powers that be at NIH and the Gates Foundation hosted meetings and conversations about Dave’s ideas, but they took no action that would ruffle the feathers of the elite birds sitting atop their golden scientific nests.

I want to make one other point about ranking grants based on a standard NIH score, i.e. a scalar quantity. Grants, like pathogens with respect to virulence, have many dimensions, i.e. they are more effectively represented as vectors in a space with two or more orthogonal axes than as points along a single line. Any time multi-dimensional vectors are reduced to scalars, information is lost. Rankings necessarily require scores that can be ordered (ordinal numbers require cardinal numbers), and scores for multi-dimensional entities necessarily require applying weights to the different variables corresponding to the multiple axes to generate a single number. These weightings are necessarily subjective or even arbitrary, rendering any absolute ranking highly dubious (see the multitude of rankings for colleges and universities that vary substantially in terms of where on each list particular institutions appear).

Moonshot Science: I am a little less enthusiastic about giant science projects than is Arturo. The Genome Project was a success in certain respects, but it was sold to the Congress and the public somewhat dishonestly as the best path forward to achieve disease cures, a manifestly overstated claim. As Joseph Goldstein, eminent geneticist and Nobel Laureate, once wrote many years ago, “a gene is not a drug.”

Also, big science projects tend to concentrate authority and power in fewer hands (and fewer brains) and by the nature of the relevant selection process, these individuals sometimes appear to be above average in the desire for self-promotion and self-aggrandizement. Furthermore, the projects not pursued or otherwise damaged due to the new focus cannot be easily measured, so an honest and fair assessment of the net worth of the “moonshot” project is extremely challenging if not impossible (and not generally desired with a vested interest in asserting its overwhelming success). Thus, big science is likely to generate BS.

Best regards,

Neil Greenspan

Wolstein Research Building, Rm. 5130

Case Western Reserve University

Arturo replies:

Dear Neil,

Thank you so much for this thoughtful critique.  I consider you one of the great minds in biomedical research and as I have told you, I have been greatly influenced by your insightful immunological essays.  Hence, I greatly appreciate that you took the time to write us and comment about your reaction to TWIM 154.  I have no essential disagreement with any of the points you raise about the four papers that were discussed in TWIM 154 and I feel that your comments bring out important issues that are worth additional discussion in an open form.  I have a suggestion for you – why not take the points below and write letters to the editor?  I don’t see any many disagreement with any of the points you raise but the back and forth discussion will be useful in teasing out the issues (I am a fan of engagement and discussion…the Marxists taught me the importance of the ‘dielectic’…which of course Marx appropriated from Hegel).

Warm regards,

Arturo

Leave a Reply

Your email address will not be published. Required fields are marked *

One comment on “TWiM 158 letters