Wikipedia talk:Scientific peer review

From Wikipedia, the free encyclopedia

• Material archived in 16 March, 2006 to Wikipedia talk:Scientific peer review/Archive 1
• Material archived on 20 March (Vernal/Autumnal equinox), 2006 to Wikipedia talk:Scientific peer review/Archive 2
• Material archived on 12 February, 2007 to Wikipedia talk:Scientific peer review/Archive 3

Contents 1 Clean up 2 Review of Scientific peer review 3 Recent developments 4 Stable versions

[edit] Clean up

The Scientific peer review pages have had little attention for nearly a year. I have archived old reviews, added a section on what to do when a review is completed and archived this talk page. I have removed the recently added list of chemistry review articles, just leaving a link to their page, as it seems to me that the list would be very difficult to maintain. I have also reviewed how the process has worked over the last year. See below. --Bduke 00:06, 12 February 2007 (UTC)

[edit] Review of Scientific peer review

How has this worked?

Before I started to clean up, Category:Requests for scientific peer review contained 9 items. One of these had not been added to the list in Wikipedia:Scientific peer review/recent reviews. Of the remaining 8, 3 were fairly extensive reviews, 3 had received no comment at all, 1 had a small comment but was really a request for a review of a possible rewrite of two paragraphs and not a review of the article itself - Physics.

The list at Wikipedia:Scientific peer review/recent reviews contained 6 items that were not in the category because the tag {{scipeerreview}} had not been added to the article's talk page. One of these had not created the review discussion page - it was a redlink - so I just deleted it. Of the 5 others, there was one good review, one that only linked to Peer review, one that was a rather bizarre controversy that is really beyond our rules, one with a small comment, and one with no comment. The last one, Process Physics, was only added in November last, so I have fixed up the problems and it and the very new entry are now the only two listed. The rest have been tagged with {{oldscipeerreview}}.

So, this is a pretty mixed bag. I guess some of the other review processes attract no review comments. We have had some good reviews. I suggest we leave it for now, and I will try to keep an eye on it on a regular basis. Clearly it is important to check that a new request has been added correctly, both to the list and the category with the talk page tag. --Bduke 00:06, 12 February 2007 (UTC)

[edit] Recent developments

Since doing the clean up described above, two new articles have been put up for review. As an experiment, I followed the pattern of several Project peer reviews and transcluded the review discussion also into Wikipedia:Peer review for the new one that prompted my clean up and the two that have come in since. Does this seem a good idea? The Projects get a bot to do it. However, to fit in the discussion page needs a === page name === header, while previously we used == page name ==. I have changed the instructions for submitting this. However some people come to SPR from the page that gets transcluded into all the Science Project pages and I have had to edit the discssion page before adding it to WP:PR. Anyway, something seems to have prompted more interest. I welcome discussion on improving matters.

More significantly, the two new articles are both having some controversy and that is not what we anticipated SPR was all about. What should we do about this? Do we let it stand? Do we discourage such articles being put for review? Do we reject such articles? Again I welcome discussion on this.

I'll put something on the Science project talk pages, but it will have to wait for a while. --Bduke 22:13, 13 February 2007 (UTC)

I have now done it on what I think are the main science projects. See Wikipedia talk:WikiProject Chemistry#Wikipedia:Scientific peer review for what I wrote on all of them. --Bduke 07:29, 17 February 2007 (UTC)

Has there been consideration of abandoning the application of the traditional model of peer review to open-collaboration documents? While peer review has demonstrated value in journals (PMID 8198342) and now is being used by practice guidelines to developed external validity and also used by 'textbooks' such as PIER, UpToDate, and eMedicine, applying peer review to a wiki is a different matter. What is the use of a review when at any moment paragraphs may be changed? Graham's law and Linus's law should be studied and we learn how/whether to make them work. If this can be done, I wonder if peer review has to be done on the paragraph or smaller level. I wonder if the traditional model of an expert writing a commentary, while maybe helpful here, should be enhanced or supplanted by displaying some sort of metric of editorial traffic incurred by the article.

Has there been investigation of using CME to encourage participation? Is it possible the http://www.accme.org/ would ever license a commercial company to review content on Wikipedia at the request of an author or reviewer and credit the author or reviewer with CME? The author/reviewer would pay for the CME (just as we do for other methods of CME) which would fund the precess.Badgettrg 11:49, 16 February 2007 (UTC)

I'm afraid I'm not very clear about all this. In what way does "open-collaboration documents" differ from the normal wiki "any body can edit" process". Peer review of one kind or another is common on Wikipedia. The main one is of course WP:PR. This is generally used prior to an article reaching "good article" or "featured article" status. Clearly a lot of people think it is valuable. The Scientific peer review process was an attempt to attract science experts to look at science articles. It has not been very successfull. --Bduke 07:29, 17 February 2007 (UTC)

Does this mean that the scientific peer review effort is still alive? I think it will be difficult to maintain if the experts that are supposed to do the reviewing don't know that it exists, and haven't read their local wikiproject announcements page in the recent past. One idea for keeping this semi-running would be to maintain a list of review participants and their area of expertise, so that those looking for a review could drop a note to the relevant reviewer as well as listing here.

On the matter of controversy, my personal bias (as an observer; I've never submitted or reviewed an article in this process) would be to keep the process for well-written and fairly stable articles where further improvements would require expert commentary. On the other hand, I don't know where one would go with a science-related article that's under dispute and it's not obvious to those unfamiliar with the topic who is right or who's the crank. If the relevant wikiproject doesn't help, there's really no other process to use. So I'd be inclined to discourage but not disallow such things. Opabinia regalis 02:58, 18 February 2007 (UTC)

[edit] Stable versions

I think its more important to work on "stable versions", so that users can have some assurance that they are not looking at vandalized content. Once "stable version" infrastructure is in place, then we can talk about editorial roles. I envision something akin to scientific journal editorial boards: an editorial board, after conducting a review (and making needed corrections), places its "seal of approval" on a given "stable version".

Basically, for me, it makes no sense to go through the major effort to vet an article, only to come back later to find its been completely restructured (not atypically, by some anon unfamiliar with the subject matter). linas 21:47, 17 February 2007 (UTC)