How is it that just as I write that post about validation and how stuck it can get you, I find another relevant link (via Twitter, don’t let anyone tell you it isn’t worthwhile; Sharing links is one of the benefits).

Why Lists are a Flawed Approach to Assessing Excellence

Some excerpts to tempt you to click that link:

The assumption that the rank awarded a journal equates to the quality of each paper within it is simply erroneous. This is compounded by the fact that many systems automatically award a low rank to emergent journals which are too new to yet have an impact factor. This tells us nothing about the quality of papers in them. But it does provide an excellent incentive for scholars to avoid them, however unique their mission or exciting their potential.

and particularly where US journals dominate, with their particular methodological, theoretical, and topic focus:

Particular forms of theorising and research become privileged above others. Scholarship grows more industrialised, conformist, boring and badly written.

On rejection rates as an indicator of quality:

Elite journals also have a rejection rate, typically, of over 90 per cent. […] To achieve this, desk rejection is increasingly common. Editors have become judge, jury and, mostly, executioner.

So much for the safeguards of peer review. If you dodge the bullet of desk rejection an arduous obstacle course remains. One must overcome the hurdle of increasingly critical reviewer comments, crawl through a long process of revisions, before resubmission, only to face another barrage of wounding criticism.

Those publications that emerge from this bloodied, misshapen and usually far too long process have not always been improved by it.

Some disciplines already have this type of validation system firmly ensconced. Others are under pressure to do so. After all, a list makes the process of evaluation that much quicker and easier. And it is much easier to defend decisions as being fair, at least in the sense that the same rules and procedures are applied to everyone. (The fact that the rules and procedures are unfair is a separate question. As long as they are applied equally to all, there is a certain fairness.)

You, as a member of a scholarly association, as a member of evaluation committees, as a head of department, a dean, or whatever, have some power to influence these trends. If you are unionized, your union also has some power to influence these things. These are, after all, your working conditions, and often related to key bargaining issues of hiring, salary, promotion, and so on.

Your power may be limited but pretending you don’t have it will not change anything. The more senior you are, the more important it is to speak out.