On being systematic

Over on another planet blog Darren Saunders asks what is an Associate Faculty Member (AFM).

There was some sales training on this subject last week and I sat in, so I should know. I’ve also been re-writing the FAQs for the new f1000 website and have just realized that there isn’t an FAQ relating to AFMs there, either. (Meta: how many times does a question have to be asked before it becomes ‘frequent’?) So, let’s have a stab at explaining it.

As you may or may not know, what Faculty of 1000 does is publish short reviews of the scientific (currently biology and medicine) literature. How this works is through our eponymous Faculty of over 5000 top scientists and medics, all over the world. These people are principle investigator level or higher. When they read a paper that they consider interesting, important, or otherwise worthy of wider recognition they write a review (or ‘evaluation’), assign a score (or rating) to the original article, and submit it to our editorial team (usually via a web interface). The piece is then edited in the usual way, coded to appropriate sections (i.e. sub-disciplines), and published on the website at f1000biology.com or f1000medicine.com, depending on the specialty of the contributing Faculty Member.

This system has been pretty successful for a few years now, and we know that people really like the service (because they tell us!). It lets scientists and medics see very quickly what’s happening in their own field, and rapidly get at what’s considered important in other communities (whether simply out of interest or because they’re moving into unfamiliar territory). Identifying important papers quickly and easily gauging the opinion of a field easily are not trivial tasks: f1000 is intended to help everyone, from students through to vice-chancellors, achieve this.

Critically, choice of articles to review is left entirely to the Faculty, and may come from any journal. Any journal: even the Harvard Business Review. Naturally there are a high proportion of articles from the usual suspects–_Nature_, Cell, NEJM, etc.–although about 80% come from ‘second tier’ or less popular journals (he says, desparately avoiding the ‘I’ word).You might expect this, seeing as certain journals review editorially before a paper goes anywhere near peer review, and actually are quite successful at it.

In a sense, we don’t care about the providence of the articles reviewed at f1000. If they’re good, we want to know (and ‘good’ means 1-2% of the current eligible literature). However, there are a lot of journals publishing good stuff, and how do we know we’re scanning the right ones if we’re just leaving it to serendipitous reading by the Faculty?

Enter the Associate Faculty. Currently about a thousand Faculty Members have one or more Associates: less senior members of their lab or practice (which can mean anything from a post-doc to a PI in their own right). Once a month we send these Associates a table of contents from two journals: one general, one ‘specific’; both self-nominated. The Associate checks the table against their own reading, and selects articles that they have already read that they will review. They also let us know if there are any articles that they think should be reviewed but that they will not do themselves: these then go into a ‘pot’ which we send (a couple of weeks later) to Associates who haven’t committed to producing a review that month.

When the Associate commits to reviewing an article, it’s pretty much between them and their Faculty Member as to how it’s handled. Sometimes the Associate will do the bulk of the writing, other times the Faculty Member will. In either case, the full Faculty Member has to approve the evaluation and has final say–they are the corresponding author.

We cover, at the last count, about 660 journals in this fashion. We’ve asked the Faculty to tell us what journals they think should be scanned in this scheme, and eventually we’ll be covering over a thousand different journals. This does not mean that we won’t be evaluating articles outwith this ‘core’ of journals: Faculty Members have complete freedom to evaluate papers regardless of where they are published. Our Associate Faculty help them identify the good stuff, and we help them to choose by providing the tables of content with a selection system (somewhat arcane, but we are working on it). The buzzphrase is ‘systematic and comprehensive’: we’re certainly systematic and are working on the comprehensive.

Hope that clears some things up for Darren.

Cross-posted