The Journal Nature Requires Reproducibility In Submitted Papers

[Billy_Bob]

The Journal Nature Requires Reproducibility In Submitted Papers

“Replication is something scientists should be thinking about before they write the paper,” says Ritu Dhand, the editorial director at Nature.

Now this will stop a lot of AGW crap from being spewed... Requiring all data and methods be disclosed is going to stop a lot of bullshit from being published. No more hiding the data or refusing to disclose your methods.. Finally!!! Real science might get accomplished and some integrity returned to science..

 

[Billy_Bob]

Will they follow through and set the higher standard? Only time will tell..

The journal Nature is going to begin requiring reproducibility in submitted papers

 

[JakeStarkey]

Bob, you don't understand the science of science.

The article will not affect AGW research, models, or projections.

 

[IanC]

The Journal Nature had very similar rules in place when MBH98 was published.

It is not necessarily the rules that count but their enforcement.

 

[Muhammed]

Perhaps our boycott is working.

 

[IanC]

The Journal Nature had very similar rules in place when MBH98 was published.

It is not necessarily the rules that count but their enforcement.

It's very analogous to illegal immigration. It's against the law but if you get across safely and stay undetected for a period of time the your safe. Having babies makes it even harder to be deported. Only rape or murder sends you back, DUI and below is OK.

Unreproducible papers that get through are basically untouchable unless immediately shown to have huge mistakes or fraud. Once portions of them are used in knock-on papers it is impossible to retract them because then they would have to retract the others as well. Auditors pointing out even numerous mistakes are powerless get changes made, just as the police are told to stand down on immigration status.

 

[Billy_Bob]

The Journal Nature had very similar rules in place when MBH98 was published.

It is not necessarily the rules that count but their enforcement.

For a very long time the likes of Michael Mann and others publishing crap and then hiding their work has gone un-dealt with. Now just maybe the whole lot will have to come clean if they want to be published in a reputable journal. I'm just happy to see movement back towards reputable reporting and holding scientists to a high standard.

 

[Billy_Bob]

The Journal Nature had very similar rules in place when MBH98 was published.

It is not necessarily the rules that count but their enforcement.

It's very analogous to illegal immigration. It's against the law but if you get across safely and stay undetected for a period of time the your safe. Having babies makes it even harder to be deported. Only rape or murder sends you back, DUI and below is OK.

Unreproducible papers that get through are basically untouchable unless immediately shown to have huge mistakes or fraud. Once portions of them are used in knock-on papers it is impossible to retract them because then they would have to retract the others as well. Auditors pointing out even numerous mistakes are powerless get changes made, just as the police are told to stand down on immigration status.

Notating each paper as 'unrepeatable' will do a lot towards keeping others from using it as a building block for their work. It puts them on notice that the work is questionable. I don't know too many who will use work labeled as such.

 

[Billy_Bob]

Perhaps our boycott is working.

I was reading an article the other day about Nature loosing 37% of its readership over the last two years. They may be viewing this as the only way to stay alive and be looked at as reputable. Definitively a no confidence vote that got their attention.

 

[Billy_Bob]

Bob, you don't understand the science of science.

The article will not affect AGW research, models, or projections.

Actually i disagree with your assessment. If you can not reproduce your work, it has no business being in a journal. If it fails over and over, it has no business being cited as grounds for policy of any kind. This will have a chilling effect on half-baked pseudoscience getting published. And its about damn time!

 

[JakeStarkey]

Bob, you don't understand the science of science.

The article will not affect AGW research, models, or projections.

Actually i disagree with your assessment. If you can not reproduce your work, it has no business being in a journal. If it fails over and over, it has no business being cited as grounds for policy of any kind. This will have a chilling effect on half-baked pseudoscience getting published. And its about damn time!

Then a point will fail. But it is going to be published until the consensus agrees that it is a fail. The consensus, in the case of AGW, is that the deniers have failed.

 

[Billy_Bob]

Bob, you don't understand the science of science.

The article will not affect AGW research, models, or projections.

Actually i disagree with your assessment. If you can not reproduce your work, it has no business being in a journal. If it fails over and over, it has no business being cited as grounds for policy of any kind. This will have a chilling effect on half-baked pseudoscience getting published. And its about damn time!

Then a point will fail. But it is going to be published until the consensus agrees that it is a fail. The consensus, in the case of AGW, is that the deniers have failed.

Too Funny;

When scientists are questioned, in specifics, about that hypothesis and its specific causes your so called 'consensus' vanishes. Consensus is a political term and has no business being used in science. I guess you still believe the world is flat and the sun moves around the earth, those were consensus science too.

 

[Old Rocks]

Bob, you don't understand the science of science.

The article will not affect AGW research, models, or projections.

Actually i disagree with your assessment. If you can not reproduce your work, it has no business being in a journal. If it fails over and over, it has no business being cited as grounds for policy of any kind. This will have a chilling effect on half-baked pseudoscience getting published. And its about damn time!

Well now, little Silly Billy continues his silly lies. Mann's graph has been reproduced over a dozen times by independent researchers using different proxies. So his paper definately passes the reproducability test.

 

[usmbguest5318]

The Journal Nature Requires Reproducibility In Submitted Papers

“Replication is something scientists should be thinking about before they write the paper,” says Ritu Dhand, the editorial director at Nature.

Now this will stop a lot of AGW crap from being spewed... Requiring all data and methods be disclosed is going to stop a lot of bullshit from being published. No more hiding the data or refusing to disclose your methods.. Finally!!! Real science might get accomplished and some integrity returned to science..

I have to say it's not altogether clear to me why this topic is in the Environment forum rather than, say, Science/Technology, Education, or perhaps Structured Debate. The thread's presence here in such a narrowly focused forum, given that Nature publishes papers covering the entirety of science disciplines, gives me pause insofar as I cannot help but sense the purpose of the thread is not to address inadequacies in the nature of Nature's published articles, but rather it exists to push an agenda pertaining to the climate change debate.

If the thread and OP-er's raison d'etre be that of advancing/advocating what appears (to me) to be a calumny theory premised upon notions of collusion of some set of actors in the debate over the politics of climate change, I don't really have something I care to share. I have nothing substantive to contribute in that regard not because I don't understand the matter, and not because I don't concur about the importance of reproducibility, for I absolutely do so concur. I don't because I know the futility of bothering here to engage in a climate change discussion. In a less petty environment, I might actively share my thoughts, or at least pose some thoughtful questions that may help advance the discussion.

If, on the other hand, the discussion theme is intended to be one of the technical merit of papers Nature (and by extension, competing science journals) publishes, I have the thoughts below to share as goes the peer review process.

integrity returned to science..

Before getting started, I have to say I think science and scientists have a good deal of integrity, but I'm not naive about the fact that there are lapses, material ones even at times. But that, I think, is a related but different discussion topic, so I will move on.

The current system of peer review is not perfect, and, while most scientists believe it is necessary, indeed, desirable, the core assumptions inherent in the process must be evaluated and adapted to the changing environment. While research has examined modifications to the process, biases, and other flaws, perhaps the purpose of peer review and some key assumptions should be examined. For example:

• Should manuscript peer review emphasize the validity of the science or its merit? While peer review is often viewed as the gatekeeper to the realm of truth, a staff of editors and reviewers cannot make that distinction in a few months.
• In terms of interesting assumptions overlooked, does the process of peer review require a true peer or expert in the field?
• Would unfettered publishing of findings lead to more efficient or faster progress in science?
• What does the stamp of peer review mean to the readers of the literature?

I think Nature's editorial board have content and intellectual integrity constantly in mind. In the mid-aughts and in response to suggestions from readers and contributors, Nature explored a new approach to its peer review process, the so-called open peer review approach. (Click the link to learn what the tested process entailed.) It found that while "open" reviewers contributed some editorially relevant feedback, very little technical input and critique resulted from the open review approach. The journal thus opted not to implement the open review process.

In light of Nature's sincere efforts to boost the qualitative value of the peer review process, I find that the publication's editors understand the importance of integrity. That said, they are science editors, not experts on every science topic about which they may receive submissions. As such they depend on the input of subject matter experts to participate in the review process. When that participation is lacking, as it was in the noted open review trial, what are the editors to do?

Now the editors are trying to force one (several) peer reviewers to perform replication exercises. Okay. It's a different approach and a new standard that manuscript authors must meet. Like the "standard" peer review processes, it too has pros and cons. Pragmatically speaking, the replication obligation is for some manuscripts fitting and for others, well, not so much.

• Should a researcher who's made a preliminary discovery need to meet the requirement? Often scientists come by ideas they explore incompletely, but they publish papers about the idea for the sake of engendering further research into the matter. Essentially they do so to say, "Hey. I just noticed 'this' and looked into it to the following extent...What I found suggests XYZ and I think it'd be worth looking into further." What such researchers observed/examined could well be an anomaly, or it could be that it's not an anomaly but not a frequent or reproducible-at-will thing. The natural world operates under a set of rules, but it's not software either.
• Should prohibitively expensive experiments also be reproducible? Ditto re: data collection? Sometimes what a scientist publishes is the story of an experiment s/he performed for the purpose of collecting data. For instance, a physicist researching gamma ray bursts is constrained by the mere occurrence of the event. Should her paper not be published because nobody else has seen another burst, or because no burst has yet occurred?

What I'm getting at is that scholarly publication is scholars' way of debating. One researchers says, "This is what I've seen and what I conclude from it." Another takes the idea and explores it from a different angle and publishes to say, "I accept part of what you say, but I looked into it 'this' way and found ABC, which it think attenuates or invalidates your conclusion." The next guy, having read both papers performs his own research inspired by the "debate" and shows how one of them is "right" given X-weighting of the factors in play, while the other is "right" given Y-weighting of them. Yet a fourth examiner looks into it and finds that there is a new factors that none of the earlier researchers noticed or considered and show that element to be the thing that defines the cause of the varying results earlier researchers found.

In short, scholarly publication is as much about the specific technical findings as it about the process of expanding a discipline's body of knowledge. The process may not go as quickly as some might like. It may be imperfect, and Nature seems to be attempting to mitigate an aspect of the imperfection. I just hope they don't go too far with the replication "thing" that by enforcing unreasonable demands that squelch idea expressions they hinder scientists' ability to have, as they do now via the paper publication process, widely accessible discussions that inspire among others innovative thought .

 

[Billy_Bob]

The Journal Nature Requires Reproducibility In Submitted Papers

“Replication is something scientists should be thinking about before they write the paper,” says Ritu Dhand, the editorial director at Nature.

Now this will stop a lot of AGW crap from being spewed... Requiring all data and methods be disclosed is going to stop a lot of bullshit from being published. No more hiding the data or refusing to disclose your methods.. Finally!!! Real science might get accomplished and some integrity returned to science..

I have to say it's not altogether clear to me why this topic is in the Environment forum rather than, say, Science/Technology, Education, or perhaps Structured Debate. The thread's presence here in such a narrowly focused forum, given that Nature publishes papers covering the entirety of science disciplines, gives me pause insofar as I cannot help but sense the purpose of the thread is not to address inadequacies in the nature of Nature's published articles, but rather it exists to push an agenda pertaining to the climate change debate.

If the thread and OP-er's raison d'etre be that of advancing/advocating what appears (to me) to be a calumny theory premised upon notions of collusion of some set of actors in the debate over the politics of climate change, I don't really have something I care to share. I have nothing substantive to contribute in that regard not because I don't understand the matter, and not because I don't concur about the importance of reproducibility, for I absolutely do so concur. I don't because I know the futility of bothering here to engage in a climate change discussion. In a less petty environment, I might actively share my thoughts, or at least pose some thoughtful questions that may help advance the discussion.

If, on the other hand, the discussion theme is intended to be one of the technical merit of papers Nature (and by extension, competing science journals) publishes, I have the thoughts below to share as goes the peer review process.

integrity returned to science..

Before getting started, I have to say I think science and scientists have a good deal of integrity, but I'm not naive about the fact that there are lapses, material ones even at times. But that, I think, is a related but different discussion topic, so I will move on.

The current system of peer review is not perfect, and, while most scientists believe it is necessary, indeed, desirable, the core assumptions inherent in the process must be evaluated and adapted to the changing environment. While research has examined modifications to the process, biases, and other flaws, perhaps the purpose of peer review and some key assumptions should be examined. For example:

• Should manuscript peer review emphasize the validity of the science or its merit? While peer review is often viewed as the gatekeeper to the realm of truth, a staff of editors and reviewers cannot make that distinction in a few months.
• In terms of interesting assumptions overlooked, does the process of peer review require a true peer or expert in the field?
• Would unfettered publishing of findings lead to more efficient or faster progress in science?
• What does the stamp of peer review mean to the readers of the literature?

I think Nature's editorial board have content and intellectual integrity constantly in mind. In the mid-aughts and in response to suggestions from readers and contributors, Nature explored a new approach to its peer review process, the so-called open peer review approach. (Click the link to learn what the tested process entailed.) It found that while "open" reviewers contributed some editorially relevant feedback, very little technical input and critique resulted from the open review approach. The journal thus opted not to implement the open review process.

In light of Nature's sincere efforts to boost the qualitative value of the peer review process, I find that the publication's editors understand the importance of integrity. That said, they are science editors, not experts on every science topic about which they may receive submissions. As such they depend on the input of subject matter experts to participate in the review process. When that participation is lacking, as it was in the noted open review trial, what are the editors to do?

Now the editors are trying to force one (several) peer reviewers to perform replication exercises. Okay. It's a different approach and a new standard that manuscript authors must meet. Like the "standard" peer review processes, it too has pros and cons. Pragmatically speaking, the replication obligation is for some manuscripts fitting and for others, well, not so much.

• Should a researcher who's made a preliminary discovery need to meet the requirement? Often scientists come by ideas they explore incompletely, but they publish papers about the idea for the sake of engendering further research into the matter. Essentially they do so to say, "Hey. I just noticed 'this' and looked into it to the following extent...What I found suggests XYZ and I think it'd be worth looking into further." What such researchers observed/examined could well be an anomaly, or it could be that it's not an anomaly but not a frequent or reproducible-at-will thing. The natural world operates under a set of rules, but it's not software either.
• Should prohibitively expensive experiments also be reproducible? Ditto re: data collection? Sometimes what a scientist publishes is the story of an experiment s/he performed for the purpose of collecting data. For instance, a physicist researching gamma ray bursts is constrained by the mere occurrence of the event. Should her paper not be published because nobody else has seen another burst, or because no burst has yet occurred?

What I'm getting at is that scholarly publication is scholars' way of debating. One researchers says, "This is what I've seen and what I conclude from it." Another takes the idea and explores it from a different angle and publishes to say, "I accept part of what you say, but I looked into it 'this' way and found ABC, which it think attenuates or invalidates your conclusion." The next guy, having read both papers performs his own research inspired by the "debate" and shows how one of them is "right" given X-weighting of the factors in play, while the other is "right" given Y-weighting of them. Yet a fourth examiner looks into it and finds that there is a new factors that none of the earlier researchers noticed or considered and show that element to be the thing that defines the cause of the varying results earlier researchers found.

In short, scholarly publication is as much about the specific technical findings as it about the process of expanding a discipline's body of knowledge. The process may not go as quickly as some might like. It may be imperfect, and Nature seems to be attempting to mitigate an aspect of the imperfection. I just hope they don't go too far with the replication "thing" that by enforcing unreasonable demands that squelch idea expressions they hinder scientists' ability to have, as they do now via the paper publication process, widely accessible discussions that inspire among others innovative thought .

The point is far easier and has basic scientific practice in its basis.

Science is the act of presenting facts, derived from the observation of physical reactions. Thus, anyone doing science must be able to reproduce the outcome, by providing the data and methods for review by others to perform replication. Failure to provide the material should bring into question the validity of the hypothesis presented. ie: Mann et al and his refusal to produce any of the data which has been shown fraudulent. aka: The Hockey Stick

A great deal of policy has been constructed from this fraud which has destroyed many economies and stopped others from growing.

 

[Billy_Bob]

Bob, you don't understand the science of science.

The article will not affect AGW research, models, or projections.

Actually i disagree with your assessment. If you can not reproduce your work, it has no business being in a journal. If it fails over and over, it has no business being cited as grounds for policy of any kind. This will have a chilling effect on half-baked pseudoscience getting published. And its about damn time!

Well now, little Silly Billy continues his silly lies. Mann's graph has been reproduced over a dozen times by independent researchers using different proxies. So his paper definately passes the reproducability test.

Poor little libtard liar..

You have been shown over and over again the Mann parlor trick and how he adjoined proxies, that have no commonality, and threw away the historical record for a fantasy he used upside down to boot.

Even the IPCC has abandoned the lie.. Now that's funny!

 

[JakeStarkey]

Bob, you are setting a standard that has not applicability to research, models, or projections.

Honestly, are you a creationist or young earther?

 

[JakeStarkey]

"Thus, anyone doing science must be able to reproduce the outcome, by providing the data and methods for review by others to perform replication"

Tell that to Einstein.

The first sentence above is a false standard.

 

[Billy_Bob]

"Thus, anyone doing science must be able to reproduce the outcome, by providing the data and methods for review by others to perform replication"

Tell that to Einstein.

The first sentence above is a false standard.

Even Einstein understood this premise. He always told people his equation was a hypothesis based on his observations and provided his work. Alarmists ... Not so much..

 

[JakeStarkey]

"Thus, anyone doing science must be able to reproduce the outcome, by providing the data and methods for review by others to perform replication"

Tell that to Einstein.

The first sentence above is a false standard.

Even Einstein understood this premise. He always told people his equation was a hypothesis based on his observations and provided his work. Alarmists ... Not so much..

Bob, no one thinks you speak for Einstein.

You do not understand the scientific terms 'hypothesis' and 'theory.' He used the term theory, which does not mean what you think it means.

Once again, if you fail to answer the question, the reasonable poster will understand that you are a creationist, a young earther.

Come on, Bob, answer the question.