Peer reviewing

One of the foundations of the scientific method is publication of results so that they can be assessed and reproduced by the wider community of researchers.  Peer review is the first step in that publication process, voluntary, anonymous review of work by experts in the field, with comments returned to the author so that the paper can be improved prior to publication.

I serve as a peer reviewer for several journals and probably get requests about twice a month to comment on a paper.  Generally it takes about an hour to understand the work, to assess whether it is correct, clear, open to alternative interpretation,and properly positioned within the relevant landscape of background literature. I try to write balanced comments, praising a clear hypothesis, a well-reasoned discussion, while perhaps suggesting where figures could be more clear or an alternative method might yield better results.  The revised paper comes back the next month and I generally pass it if the author has replied to my points (whether or not those result in changes).

Occasionally I run into situations where I struggle.  I received one where the author had copied an earlier paper, word for word, yet not cited it. I talked with my editor (the lead author was fairly senior) and we recommended that he include the citation: it was enough to alert him that we’d found the earlier work.   Another was proposing pages of dense calculation for an obscure visualization technique that, in the end,m didn’t reveal anything new in the data.  I think that, in six revisions, we did everything but rewrite the paper for him.

Last month, I received a work that was analyzing structure in short-and long-term heart rate changes.  It’s a tricky analysis, since it’s statistical rather than deterministic, and the author had calculated some crude ratios to support his hypothesis.  Worse, his underlying technique destroyed the very ordering among data points that he based his hypothesis on.  I pointed out that the results weren’t supported by the method, suggested some alternative approaches fo analysis, and worked out an example to illustrate my point.  Throughout I was constructive and respectful (I though), allowing that I might have missed his point, but didn’t follow the logic.

The authors absolutely blistered me in their reply.

…there are many comments made by the Reviewers with which we disagree, comments which we consider unhelpful and comments which are just wrong…we strongly believe that the Reviewers should be aware of the fact that, even though they are anonymous, they should put time and effort in reading and reviewing the papers. We are frequent reviewers for a number of peer-reviewed journals,and we always try to helpful for the Authors and never write from the position of someone who, by definition, knows better.

Most of my comments were subsequently dismissed with this is an opinion and there is nothing we can do about it, except state that we disagree.  Fair enough: I went back through my analysis and still cannot link their conclusions with their data using their method.  I had to just recuse myself after that: while we might have solved the disagreements over a beer and a whiteboard, their reaction was so strong and their comments so dismissive, that there just weren’t actually any explanations to discuss.

I felt badly about the comments and about the outcome: I do take the job seriously and give every paper a good read.  I know how much work it takes to do research and to write it up.  In the end, if they are right, they should be able to refute the counter argument and to say why.  If they are wrong, they need to make corrections.

And, fundamentally, I disagree with them that Reviewers are there to serve the Authors. 

Reviewers are present to serve Readers.

If, as a first reader, I think that a paper is wrong, I have a responsibility to (nicely) say so, suggest why, and present alternatives.  Maybe it’s a real mistake, maybe I just don’t understand, maybe the Methods contain omissions or ambiguities.  But honest and differing perspectives should be addressed openly.

We might argue that every author is entitled to his view and the Readers will sort things out. Eventually, yes.   But errors enter indexed literature, then pop up on searches, news reports, and blogs.

The Letters to the Editor don’t have much impact, and are presented as a debate that obscures rather than resolves issues.  It can take years before the truth is sorted by Readers, years of contradictions and misinterpretations in which true facts have been undermined and genuine debates get muddied. 

The argument presented by these Authors, that Reviewers should only check for clarity of language and labels on graphs, is wrong.  The peer review process is intended to engage experts as surrogate Readers, not as supplemental Editors.

The long tail of travel

I was riding the airport escalator, lost in thought, the aspirational airport ads gliding by.  Island resorts that I’ll never visit; books I’ll never read; restaurants I’ll never frequent; shows I’ll never attend.  It’s like an airline magazine brought to life.

Then an HSBC poster caught my eye – one of the many “ways of seeing” ads that they post on jetways worldwide.  This one made me smile, though.

It captured the contradictions of expat psychology so well.  It’s a Jason Bourne / Jeremiah Johnson archetype, of being more comfortable with environments than with people.  Of being lone hunters rather than communal gatherers.

It reminded me of an October 2011 graphic in Wired Magazine, reproduced below.   They surveyed everyone’s passports in the office an counted up the countries visited.  The results produced an long-tail distribution, with some countries being visited by almost everyone and others by only a few.

As with the HSBC poster, I can recognize myself within this picture, placed about a third of the way from the left.  That places me in the top 7% or so among the most travelled Wired staffers. 

I’m torn as to whether this tells me that I travel too much or that others should travel more.

And there’s an aspirational aspect to the statistics.  I’m instinctively looking right to find my next destination: prescriptively I should be visiting Brazil, Egypt, Norway, and perhaps delaying Albania and Sudan.

And no sign of when the time is right to start exploring the ‘Stan countries – I may leave that to my son, reporting in from Turkmenistan and Afghanistan.

Piddling with the plumbing

Home ownership comes with as many jobs as joys.  I don’t mind most, from yard work to wiring, but plumbing has always held a special challenge for me.  Technically, it’s no more difficult than electricity or ventilation: follow the flows, remove the blockages, replace the fittings, clean the connections.  I’ve developed a special relationship with my local hardware stores, Ernst, Home Depot, McLendon’s, who always seem to be able to supply a diagnosis, a part, and a smile that puts me on track to getting the lights on and the heat through.

Not so with plumbing.

Innumerable times, I have replaced fittings, tightened connections, and turned on the water to get results varying from drips to geysers.  Water is relentless in finding gaps and flaws.  Then it all has to be taken apart, cleaned, reassembled, two times, three, before either finally working or triggering a call to a neighbor or plumber.  There’s always a temptation to solve problems with plumbers tape and bathtub caulk.

Toilet seals (simple instructions lead to dire consequences: seat the bowl over the wax ring and press down firmly; flush once), faucets  (filled with tiny springs and valves), and underground pipes (a favorite of moles and tree roots) hold special dangers.  Even today’s simple task, “Please replace the four leaky shutoff valves”, revealed unexpected complications.

Like compression rings.

Our fixtures are old and out of date, corroded and dripping, original to the house.  The shutoff valves are the likely culprit, so I bought four at the local Home Depot, turned off the main water (provoking howls from the daughter) and attacked the process of disassembling the pipes.

And quickly ran into the day’s challenges.

Lesson 1:  Turning off the main shutoff doesn’t disconnect internal reservoirs, such as the hot water tank.  I got caught full in the face by a geyser of (fortunately warm) water as the valve popped off the pipe.

Lesson 2:  Compression rings may turn but they don’t pull.  The ring slipped easily off the first pipe, with difficulty from the second, and refused to budge from the next two.  No amount of twisting and tugging set it free, yet the corroded wall plate and worn threads of the sealing nut meant it had to come loose.  Cutting off the pipe behind it was not an option: the Internet suggested a hacksaw (but don’t nick the underlying pipe).

Lesson 3:  Compression ring pullers.  Yes, it’s $20 for something I’ll seldom use (and probably lose), but it got the job done.  Insert the tip into the pipe, the collar around the ring, turn the handle, and it pulls the ring right off the pipe.  Lovely.

Lesson 4:  The value of a box of old plumbing parts.  The job was done, and I was reassembling the drain pipes.  The main one of the bottom of the sink wouldn’t attach, in fact, it looked like it had never attached.  I took pictures up to the hardware store (third trip of the day) and they said that the old ring had literally fallen apart. They gave me a new one, but that just slipped over the threads, refusing to pull the pipe and sink together.  Rummaging through an old box of pipe parts, Karen came up with a plastic fitting that happened to fit perfectly.  I’m a skeptic about holding onto useless boxes of miscellaneous hardware, but this one saved the day.

By late evening, double the estimated time for the job, the work was done and the water was back on.   Simple, really: just unscrew the old valves and replace them with new ones.

Except when it’s plumbing.

Christmas Day

‘wishing everyone a very warm and happy Christmas with family, friends, and traditions.