CS publishing culture considered harmful

I read this article today (http://www.theatlantic.com/culture/archive/2010/07/whats-wrong-with-the-american-university-system/60458/) on what’s wrong with the US university system in between reading a bunch of papers on HCI. I’m pleased to say that I don’t think the teaching/research dichotomy is as bad in the UK and I think it’s pretty good in my own institution at St Andrews.

However, the sentence that really resonated was this one:

“The problem is that there are just too many publications and too many people publishing”

The papers I was reading today are certainly some of the ‘too many publications’ and we’ve got into this dreadful state because of our idiotic research culture. This is partly driven by tenure (in the US) and research assessment (in the UK) but, in truth, we have brought it on ourselves by favouring quantity over quality and by pretending that papers that report provisional and interim results are high-quality publications.

Most of the papers I read today were conference papers and, with one exception, they didn’t report on finished work. They reported on prototype designs, which were never evaluated for real systems or on system studies that were never followed up with the results used to modify the systems studied or used as a basis for developing some general theories that could be more generally applicable. These were HCI papers but it’s the same right across the discipline – there’s no shame in not implementing things, not evaluating work and not caring about its relevance to society.

Now I’m pretty confident that the authors of these papers are bright, thoughtful and creative researchers – they are publishing in the ‘best’ HCI conferences so it’s reasonable to think they are the top people in their field. They are smart enough to know how to play the system and are doing it well. But, the system is forcing them to always seek novelty instead of spending time doing long-term work and documenting it properly. The consequence is, sadly, mostly useless publications because they can’t really be used as a foundation for further work.

Conferences are great fun – you get to meet colleagues and friends from far away, have interesting tech talks, usually some good dinners. Hardly anyone listens to the papers though – either they are reading their email (imagine the uproar if you arranged a conference without wifi in the lecture rooms) or talking in the lobby. So lets start having (many) fewer conferences which are not about publication but which are about interaction. We need many fewer but much better and more complete papers.

Changing culture is always hard, always long term and usually needs some kind of major shock to initiate the change. Maybe the economic crisis is the shock that we need and will trigger a process that will lead to a grown-up CS research culture.

Advertisements

1 Comment

Filed under Uncategorized

One response to “CS publishing culture considered harmful

  1. Pingback: Time for a harder line on evaluation | Thoughts on software and systems engineering

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s