|
Assessing quality of
peer review:
An uphill task
Dr. Jamshed Akhtar*
Peer Review is the process of
subjecting a work or idea of an investigator to the scrutiny of others
who are “Experts” in the field. Peer review is needed because it is
assumed that it improves the quality of research work. But there is a
great question mark at the end of this statement.
CPSP and JCPSP
Before going into the subject
further I would like to explain as to how we work at Journal of College
of Physicians & Surgeons Pakistan (JCPSP). We feel proud that we are
among few journals indexed in Medline and is also enrolled with many
others indexing services.
CPSP is the postgraduate diploma awarding body of Pakistan. More than
80% of the specialists in various medical fields are Fellows and Members
of CPSP. CPSP recognizes various departments of different medical
institutes for imparting training to postgraduate trainees. For
imparting training consultants and faculty members have to undergo few
mandatory workshops. CPSP offers fellowship in nearly 50 specialties
including dentistry. We therefore have a long list of faculty members
from all over Pakistan who act as a pool for reviewing articles
submitted for publication to JCPSP. This opportunity lies only with us.
The circulation of JCPSP is more than any other medical journal of the
Pakistan. It is distributed free of cost to all fellows of CPSP more
than 7000 in number. It is also sent on complimentary basis to all
libraries of medical colleges. In addition many foreign medical
institutes also receive its copies.
Attributes of a
Reviewer
To become reviewer at JCPSP an
individual must be a member of teaching faculty of an educational
institute. The person must have attended a workshop on Research
Methodology & Biostatistics at CPSP. He should preferably be a
supervisor of FCPS trainee and must have some publications to his
credit.
Process of review
It involves initial assessment
(format, newness, novelty, category, language)
Review by statistician and Assessment by bibliographer.
Following this the manuscript being processed meets the following fates:
Rejection (nearly 50%), send for redressing or proceed for review.
Process of review remains confidential. We pay remuneration to reviewers
& we do not charge anything from authors. We encourage on line
submission to facilitate the process. Following review the article can
be:
* Rejected
* Accepted as such.
* Accepted with minor changes.
* Accepted if major changes incorporated.
* Accepted if category of article is changed.
The purpose of having last category is to encourage writing habits but
as most of the authors contribute articles as these are mandatory for
their promotion, they insist on original article category only. We
discourage this attitude as we believe that a beautifully conducted
randomized control trial and a simple retrospective record of patients
can not be at par and distinction must be made otherwise RCTs will never
be found in Pakistani journals. We offer different categories like
clinical practice articles, current practices, short articles, debate,
issues etc. So in a way JCPSP has a very author friendly approach.
Response of
Reviewers
The response of reviewers is
very disappointing. It can be briefly described as:
* Mostly casual.
* Not bothered to give timely response.
* Reminders are needed often time.
* Even when responses are facilitated by attached prforma no regard is
shown to it.
* Many fail to give a consolidated view which matters most, to us.
* Language at times is rude and attitude appears hostile.
* Conflict of interest is not revealed.
* The review lacks critical analysis.
* Most are not aware of statistical tests and thus fail to analyze this
part of manuscript.
* Most have no access to references quoted by authors so can not give
any comment upon truthfulness of issues compared with various studies.
* Most do not even know what guidelines are to be followed during the
process of review as adopted by specific journal (as they vary from
journal to journal). We can classify bias reviews as Discouraging,
Dishonest and Inadequate.
This was the response of reviewer when article was sent to them after
initial in house assessment We recently conducted an experiment where
articles were sent to reviewers without initially being assessed by
assistant editor (who assessed the article independently – not subject
specialist). To our surprise reviewers failed to identify even gross
inadequacies. So we put a question: Is peer review process at all
needed considering its failure on most of the occasions even when
reviewers were carefully selected from a large pool?
This concern has been raised on many forums. So one really asks what do
readers get out of review? It is natural to assume a filtered article
which is accurate and brings clarity to practically done scientific
work. But still many articles get published in high esteemed peer review
journals that later turned out to be fraudulent.
So it is right to ask “Should Every Article Submitted Get Published?” It
is a controversial statement. While BioMed Central an electronic journal
operates as “Bias to Publish” accepting anything that meets minimal
scientific and ethical standards while rejection rate at Lancet and BMJ
is well over 90%.
Another issue that surfaced is that an open review improves the status
quo. Open review was introduced in 1996 by Journal of Interactive Media
of Education. Online journal Philicia adopted a new approach. Articles
submitted are published immediately and review takes place later, while
Naboj invites readers to write review.
We are therefore still unclear what peer review actually does and can
its omission bring any change. Assessing the quality of peer review thus
becomes a tricky job. It may be clarified here that Editor is under no
formal obligation to accept the opinion of the referee.
Our (Pakistani Editors) position in this regard at present is not very
clear. We are still in process of evolution as medical journalism is
Pakistan in not recognized as a distinct entity. Editors of various
journals can hold such meetings to define the milestones that have been
achieved and what still to be done in this regard.
Recommendations
* Both authors and reviewers
need to show maturity in their approaches.
* More dedication is needed on their part to improve the job that they
are carrying out.
* More workshops are needed to update the knowledge of authors and
reviewers.
* Former Associate Editor
Journal of College of Physicians & Surgeons Pakistan
Editor, Journal of Surgery Pakistan
Associate Professor,
Department of Pediatric Surgery
National Institute Of Child Health Karachi.
E-mail: jamjim88@yahoo.com
|
PDF
