RT @rpg7twit: Henry Gee, aka @cromercrox: the new @stephenfry ? @lablit

Great time at the NN drinks last night, despite there being only a few regulars there; and the magnificent Pamela Ronald. Pamela said something nice about a certain blog post last year, without realizing that was me, which is, actually, the kind of fame that appeals to me. People know what I’ve done without associating it with the mental-looking redhead tripping over his own shadow and drinking Theakstons out of plastic cups.

Apparently I'm the next @stephenfry sez @rpg7twit. Big, brain size of planet, yes. Gay, famous, no. @cromercrox

On the other hand, we all (here at NN anyway) know Henry Gee and what he looks like. But the rest of the world doesn’t. Which is a shame, especially as one of us is convinced that Henry Gee is the next Stephen Fry.

RT @rpg7twit Henry Gee, aka @cromercrox: the new @stephenfry ? @kejames

So, I urge you, retweet the next internet meme. And remember where you were on 1st July 2009, when first you realized that Henry Gee is the next Stephen Fry.

I have lots of stories about kit. This isn’t one of them.

Now that I’m working with the results of science, rather than generating it myself, I don’t get to play much with really cool equipment (although I do seem to be getting introduced to lots of new ideas, which is also cool but in a different way).

However, I do, through my network of spies, maintain access to at least looking at cool gadgetry.

Stage 11 chick embryo

It is, if you like, a kind of vicarious lab life. The forty hour embryo in the picture above for example: one of my spies is learning how to dissect them. As you might imagine, this is quite a fiddly job, and decent scissors are necessary. It’s just as fiddly when they’re a bit bigger.

Stage 14 chick embryo

Now, the thing about scissors. The thing about scissors is that, basically, they’re clumsy. It’s difficult to see what you’re cutting for one, especially when you’re looking down a microscope. But enter the Lumsden BioScissors, the coolest kids on the block for micro-dissection:

Yet another use for pipette tips

At first glance they look like forceps, but instead of gripping ends they have blades. You use them just like forceps, in that you squeeze and the very ends come together first, then they slide against each other just like scissors. But from the distal end. Which means you can actually see what you’re cutting: you can place the blades over where you want to cut without guessing and tearing.

Lumsden BioScissors. Apparently belonging to Clemens

So. Cool kit. Cool titanium kit. Which leads me onto something else:

A friend of mine was telling me about her experiments last week and said that it was a pain, having to do a certain thing, and that it’d be really neat to have a tool that would do this certain thing. How to do it actually came to me in a flash, right there and then. I haven’t made any sketches yet, but I should, because it’s amazingly simple in concept and lots of people molecular biologists would buy it. And then, one day, I might actually be able to afford to go to SciFoo.

I don’t normally have much time for videos on weblogs, but I’m making an exception for Sir Paul.

Note added in proof.

Oh for goodness’ sake. I can’t embed the video. It’s here. When are we getting MT4?

Never mind author contributor ID, or DOIs for articles, or whatever (I can’t be bothered looking up the links): I’m currently trying to find correct names for and de-duplicate entire journals.

there must be a better way

I have to match up all occurrences of a journal’s name, including misspellings and tyops, in our database and correct them to the canonical abbreviation. For further enjoyment I’d like the URL of the journal’s main page, where one exists.

PubMed, frankly, is a bit crap at finding journal names and their homepages. Anyone know of a good resource? Preferably one with an API or at least a script-friendly interface.

In the meantime, my favourite journal so far is

Meded Rijksuniv Gent Fak Landbouwkd Toegep Biol Wet

closely followed by the laconic

Pain.

What I wanted to talk about yesterday, before I got distracted, was a mix of things that came out of last Monday’s ALPSP meeting, a paper in PLoS One and this whole assessment schtick that we’ve got going on.

We know, don’t we, that the Impact Factor as a measure of an individual scientist’s productivity is deeply flawed. Some might go as far as to say it’s flawed fundamentally and should be ditched forthwith: after all it is prone to gaming, unduly skewed by review articles, does not demonstrate a link between the ‘quality’ of a journal and any given article therein, can reduce the measured effectiveness of a single article (if a seminal paper is cited in a review then the review rather than the original paper can be cited more frequently), has no meaning in fields (humanities, nursing…) where citations are not routinely used as a measure of impact, and is very, very slow. (Seriously. We’re looking at a two year gap at least between the original work being done and it being cited in any meaningful manner—there has to be the first paper published, experiments thought about and performed, results analysed, papers written, reviewed and finally another publishing round before you can measure it.) It is at least one step removed from the output it might be taken as measuring, relies on everyone else playing fair, and, just like a quantum physical experiment, perturbs in the act of measurement the very thing it is supposed to measure.

There have been attempts to massage the Impact Factor, and new formulations cooked up, such as the Hirsch Index (and its variants) or the Eigenfactor, but essentially these are all measuring not a researcher’s activity, nor even how good that activity has been, but rather how someone else has managed to exploit it. Let’s not even think about research activities that do not result in a classical ‘citable unit’: database annotations, talks, training of students, fixing kit, etc., etc. And when you start looking at this in terms of stakeholders, for example the Research Excrement Excellence Framework (&c.), you get into all sorts of difficulties because people want to be able to measure you in a pretty automated way in order to finish it before the heat death of the universe.

Clearly there is no substitute for actually reading an article to determine its importance

… which is probably why certain people last Monday were very careful to say “indicators” rather than “metrics”, and brings me onto article-level metrics and a paper on the impact of Wellcome-funded research.

PLoS ONE are trying to look at the impact (however you define that) in terms of individual articles. According to Peter Binfield, for each PLoS ONE article the aim

is to provide information relating to online usage, citation activity, blog and media coverage, commenting activity, social bookmarking, "star ratings" and "best of" picks as selected by academic experts, as well as other measures yet to be determined

In other words, you’ll be measured by what you write, not by who writes about what you wrote. And even that’s a bit tricky, because really, there is no substitute for “expert” review, as was elegantly demonstrated in PLoS One last week.

Liz Allen and colleagues at the Wellcome Trust analysed the output of a whole bunch of Wellcome-funded researchers, whether project or programme grant or fellowship holders. They appointed a panel, or college, of ‘experts’ to review papers published in 2005 and essentially got this college to answer the question “how good is this stuff, I mean, really?”—for nearly 700 papers. (Okay, they didn’t assess ‘non-citable units’, but it’s a start.)

The results are interesting. For a start, it turns out that the Impact Factor or citational analysis is a pretty poor indicator of an individually paper’s importance (or quality, or influence). We all kind of suspected this, but it’s nice to finally have some data.

The other thing, and here I declare my conflict of interest, is that the Wellcome’s expert college on the whole matched Faculty of 1000’s assessment. Yes, some papers were completely missed, but we think we know why and are working to make sure that in future we do spot more interesting things that appear in less well-known journals. We’re pretty stoked about this, actually.

You can’t beat human input. Not yet, anyway. What we’d really like to do is find some way of matching PLoS’s article-level metrics indicators with our F1000 Factor and synthesize a sensible measure that looks at the quality of the research as well as its influence and usage. Here would be a good place to draw your attention to Johan Bollen’s incredibly cool network graphs, which could provide hours of fun for all the family. And yes, I talked to Johan on Monday and yes, he does plan to slice those data at the individual author/research group level.

You may have noticed that we (Faculty of 1000) already ‘score’ articles. Seeing as we have these scores, you might imagine that we’d be able to, oh I don’t know, give scores to individual journals or institutions: perhaps even individual scientists. Now there’s an idea…

But you know, I do wonder if all these indicators and claims of unfairness, fairness, whatever, might be missing the point, somewhat. Whatever the metric you use, good science will out. I wonder if there are truly any brilliant scientists who have not got jobs, or not got funded, because the current indicators have missed them? And I’m sure you’ve all got stories you can tell me. To which I say,

the plural of anecdote is not ‘data’

Furthermore, we all assume that there is necessarily a link between past results and future performance, in opposition to what the investment adverts keep telling us. The Impact Factor—or any citation metric—is at least two years out of date. So I wonder if this holds when dishing out the cash for research? Now, I’m pretty open-minded, and a strong believer in experiment. I would really like someone to test my hypotheses. So I propose the following:

Take a pot of money. Tens, if not hundreds, of millions of pounds or dollars, doesn’t matter which. Divide it into four equal portions. Invite researchers to apply for the money in the usual way, and on receipt of the applications assign them randomly to one of four groups, A, B, C or D. Group A is judged in the standard manner of competitive grant applications. Group B is judged on the basis of Hirsch Indices or Impact Factors of the PIs involved and Group C uses something like an expert college in much the same way as the papers in the Allen study were judged (but looking at individual researchers rather than their papers). Applicants in Group D get the money randomly. We’d probably want to limit to a 20% success rate for each group—someone else can work out the details.

And then, after five years, we assess, using every best method available, the productivity of the researchers who were funded (and could probably look at the careers of those who weren’t, too).

This is probably unethical, but potentially very interesting. I wouldn’t be surprised if there were a few surprises.

I was dismayed to find that Jenny had already nicked my it’s Friday somewhere in the world joke, so I had to save what I had thought would be a super Friday afternoon post for another time, and write about something half-way serious. Sorry about that.

It’s not apple blossom time anymore

I often get asked about inspiration for blog posts, and it’s a funny thing, because I have a list as long as a very long thing of serious stuff to write about, but the posts that everyone seems to enjoy and comment on are more difficult to think of. Anyway, today was Father’s Day and the summer solstice, and I was given 2.2 lb of Cadbury’s Dairy Milk by my wonderful, perfect, lovely children. I did some gardening (my tomatoes and chillies are now in grown-up pots)

and I scrounged around, came up with some random pieces of wood, and constructed a bird table.

Oh, and the other Dad thing I did was to crack bad puns break into the garage because we couldn’t find the key. I even fixed it, later.

Now, I didn’t want to write about any of that. I wanted to write about something interesting that came up at the end of last week in a peer-reviewed journal, but actually it’s been far too nice a day for that. Instead, I’m going to report from the Front: the War on Gastropoda.

You might recall that whereas certain people of our acquaintance were dealing death and destruction in the form of little blue pellets of doom, I was reverting to more subtle methods. Not, as has been rumoured, by staking my children to the bamboo growing in the corner and not letting them inside before they’ve captured and water-boarded every slug or snail in the district, but by using coffee grounds.

‘Why?’ I hear you ask. ’You’re no tree-hugging hippy greenie, are you?‘—and no, I’m not. I was a bit worried about the effects of the poison on birds that might eat dying slugs, but my major reason for using coffee grounds is because I keep forgetting to go to B&Q and buy slug pellets.

But, anyway, I’m pleased to say it worked. The joint coffee-drinking forces of F1000 and KCL (actually, I’m not totally convinced that the Fairtrade muck they provide at SNG can rightly be called ‘coffee’ under the terms of the Geneva Conventions) provided sufficient anti-gastropod power to protect my seedlings. Indeed, one morning I went out and saw a slimy trail from a limey snail glisten up to the coffee ground event horizon, and then veer off; obviously defeated.

Unfortunately, after a couple of rainy days I looked out to find that one of my beans and two mangetouts had succumbed. Trails criss-crossed the coffee grounds. Obviously I need to reinforce the defences on a regular basis.

This is a bit of a problem though, and I’m getting very odd looks as I root through dumpsters at the back of the coffee shops in W1. I’ll have to bite the bullet and remember to cycle to B&Q (now that I’ve fixed the garage door found the garage door key, this is possible).

Especially now, seeing as my lettuce seedlings are stretching their dicots and beginning to look yummy.

H1N1—here it comes!

“I don’t have it. See,” Sophie said, “I don’t have a curly tail.”

According to the newsletter Sophie brought home,

This week H1N1 (swine ’flu) has been confirmed in four Southwark schools including a school in Rotherhithe. This particular strain of the virus seemes relatively mild and responds well to the anti-virals (Tamiflu) which are used to treat it. Its symptoms are the same as those of the ’flu except it is accompanied by diarrhoea.

Which is pretty matter-of-fact for a school newsletter. But wait, it continues,

The advice from the Health Protection Agency is that schools are not to close simply because cases are reported or confirmed. At this stage they feel that the virus is so widespread in the community that closing a school woiuld make no impact on its spread. the probability is that most, if not all Southwark schools will have confirmed cases before the end of the summer term.

which is remarkably sane, logical and progressive. See, it’s just a pandemic, which simply means everyone gets it. It doesn’t mean it’s the next sodding Black Death. Getting now while it’s hot not incredibly nasty would be a Good Plan. Swine flu party, anyone?

And then Sophie was wondering if H2N2 was ‘worse’ than H1N1, and if H5N5 would be the worst of all… I stayed out of that conversation. Is there a virologist in the house? Cath Eva Åsa?

Older people will remember ‘Hong Kong’ flu which swept across the country quite a few years ago—there is no doubt that H1N1 is a nasty does of the ‘flu…If you have any concerns around your own child’s health please see your GP.

Now, if only we’d had advice like that during that whole MMR gives you autism mess.

Expect preening.

That is all.

or

I started a blog and all I got was a new job

A few weeks ago I arranged to meet Cameron at a pub just across the road from my office. At the time I could stand up at my computer, turn my head and see into the lounge. It was that close. Since then the east side of our floor has been refurbished and I sit in a sunlit bay that overlooks a courtyard, with three empty desks around me and chairs that number between one and four, depending on any number of factors.

Just to make Matt jealous, we do have air-conditioning now. And while I find it very comfortable (although these afternoons I am forced to lower the blind because I start to cook) the comment from most of the women in the lab office has been along the lines of ‘my, but isn’t it cold?’ Except less politely-worded.

All that aside, I was talking with Cameron about something that is very interesting to publishers and information service providers (in case you missed it, I now work for one). Cameron presented these slides at the Eduserv Symposium (and you can watch the recordings of his and the other talks).

Essentially, what we have in the scientific literature—and not just the literature but also when we want to give credit to those who blog, or who deposit data, or who curate databases, or do anything that might be measurable and therefore applicable to assessment—is a huge Zhang problem.

How do we distinguish authors (or to be more precise, contributors) with the same name?

How do we make sure that contributors get credited appropriately for their work, in a day and age where automated metrics are becoming more important. How do you make sure that you get the credit in the next Research Excrement Framework and not the other person with the same name? How, in fact, if you’re this guy do you make sure you’re not the one getting sued for libel; or, using Cameron’s example, if you’re Andrew Wakefield and want to work in immunology, what are you going to do?

I’ve been fascinated by this problem for a while, ever since I started getting stopped in the street and being told that Withnail and I is the best film ever. But as anyone even vaguely familiar with F1000 might not be surprised to learn, I have more than a passing professional interest in it, too.

So it’s gratifying to see the issue coming to the front of the minds of some of the brightest people in the industry. We’re pretty much resigned to having to curate our data manually for a little while, but after hearing rumours and having wishful thoughts, I finally caught up with Geoffrey Bilder at a meeting of the ALPSP yesterday.

The people who brought you the DOI are prototyping a Contributor ID. And I am all of a sudden quite excited about this. You should be too, especially if your name is Zhang.

A good summer student, it is said, will only set you back a month.

I was reminded of this rather pessimistic piece of advice when I commented on Jenny’s post about dark arts this afternoon.

When I was at the LMB in Cambridge, we used to take on summer students if they came with a recommendation, if they could demonstrate at least that they knew one end of a Gilson from the other, and if some brainless monkey work needed doing. It turned out, one summer, that I needed a shedload of protein growing so that I could solve its structure by NMR. We got a summer student, he spent all summer in the cold room, and we got enough of the 7 kDa fragment to solve it by natural abundance ( ¹ H) methods. This work was published in the then Nature Structural Biology and we were very happy, and even happier when we realized that it was a complete work of fiction and published the correct answer to our question a little bit later.

cough

Anyway, this was a good experience with summer students, and so we tried again a a couple of years later: again when I needed a lot of protein for NMR (although for this one we’d managed to get the labelling working).

Oh boy.

Now, to be scrupulously fair, it was the fault of the lab manager, who taught the chap how to grow up bacteria for protein preps. For ages (except for a couple of aberrant years when a quitter visiting post-doc from Australia got everyone except me using IPTG and GST-tags) we’d been doing transformations, taking single colonies into a litre of 2XTY in a 2 l flask, shaking overnight at 34°C and letting the little beauties auto-induce protein expression (long before Bill Studier published his impressive analysis of auto-induction, by the way. We just didn’t think it was worth publishing, which just goes to show, innit?)

So after I trialled the method, and left the student with complete instructions under the care of the lab manager, we got one successful prep. But nothing afterwards — we weren’t getting any protein at all, and wasting the horribly expensive ¹³ C and ¹⁵ N while we were at it. One day I watched this chap set up a prep.

He went to the freezer, took a frozen bacterial pellet, scraped some gunk from the top of the pellet with a pipette tip and dropped the tip into fresh media for his grow up.

After I had peeled myself off the ceiling, I asked him why.

“Zat ees how $LAB_MANAGER showed me,” he said.

One culpable homicide later, we got back into business and made the protein and I solved the structure and I blogged about it.

This summer student was not completely harmless though. Before all this, I was showing him how to do certain things.

“Oh,” he used to say, “zat ees not how vee did eet in Germany!”

Everyone in the lab used to get this. It wore a tad thin after a while.

One fine day, I was drying a protein gel between cellophane sheets.

“Oh,” he said, “zat ees not how vee did eet in Germany!”

I am sorry to report that I very nearly lost my temper. I turned around, poked him in the chest (he was bigger than me) and said,

“Well sunshine, you’re not in Germany now; you’re in the Army, and we do it my way.”

He was quiet after that.

For the entire summer.