Scientific Publishing, Part I: Peer Review and Journal Basics

Around a week ago I was asked to review an article for a journal. For reasons that I'll explain subsequently, this request has prompted me to do some research about certain aspects of academic publishing. Rather than try to capture all my thoughts in one entry, I'll aim to write a series of shorter pieces. So, if all goes well, this will be the first in a continuing series.

I thought I should begin with some of the basic mechanics of academic journals. To those who have submitted work to peer-reviewed journals, I don't think this will be anything new. However I suspect that some undergraduates or prospective COA students read this blog, and I've found that many undergraduates -- quite understandably -- don't have that much awareness of the nuts and bolts of academic publishing. (Actually, I really have almost no idea who, if anyone, reads this blog.)

My experience with peer-reviewed journals is almost exclusively in physics and some interdisciplinary chaos/complexity type journals. I've submitted about a dozen papers to such journals, and I've refereed about twenty papers. (This hardly qualifies me as an expert referee or as a productive author.) I think that my comments apply to other fields of science, and much of the social sciences, as well.

Publishing articles in peer-reviewed journals is the standard way academic research results are communicated. The process works as follows. Let's say I have some results that I've written up as a paper. I choose a journal to submit my paper to, and I send a copy to the journal. Usually, this submission is done electronically. Over 90% of the papers sent to the Physical Review series of journals are submitted electronically, and I suspect that this figure is similar for other physics and math journals. I've never submitted a paper copy to a journal, nor do I know anyone who has done so recently.

Anyway, the editor or an associate editor of the journal gets the paper and likely gives it a quick read. Unless the editor deems the paper to be clearly unfit for publication, he or she chooses a few people to serve as referees for the submission. Often, authors will suggest referees when submitting their paper. The editor is under no obligation to choose the suggested referees, of course. The editor may choose other experts in the particular area that the paper is in, and/or may select referees from the list of references in the submission.

The paper then goes out to the referees who are asked to write a brief report and recommend publication, rejection, or something in between. The referees are anonymous to the authors of the submission. As a matter of practice, however, it is often not too hard to figure out who your referees are. In scientific fields, almost always the authors of the submission are known to the referees. In social sciences it is more common for the authors' names to be left off the copy of the manuscript that goes to the referees.

Once the editor has received the referee reports, he or she makes a decision regarding publication and communications this to the author, along with the referee reports. The author may be asked to make changes to the manuscript, in which case the referees may be asked to re-review the revised paper. The author also has an opportunity to respond to the referees comments. As one can imagine, this process can sometimes get a little messy. But eventually this process ends and the paper is either accepted or not.

Much could be said about the biases and problems inherent in the peer review process. It can be a nasty business and there is much room for abuse of the process. There are some good arguments that the peer review process doesn't work and should be done away with or radically modified. But this isn't my main concern here.

(For more about the review process, here are the editorial policies and description of the review process for Physical Review E and Chaos, two journals that I've published in. And wikipedia has a entry about peer review that seems pretty accurate to me.)

Moving on, then, let's assume that the article is accepted for publication. The editors and their staff may do some light editing of the manuscript. In my experience, they've done very little, but they have caught a few typos and formatting glitches. They've also introduced errors, but on balance they do more good than harm. The article is then formatted for publication. I don't think this requires much work, since almost all articles are submitted electronically and, in physics at least, most of these are submitted in LaTeX, which is a typesetting program that produces publication-quality formatting. There is some labor involved in making the articles look good enough for publishing, but not much.

Then, the journal is published and sent to subscribers. Most scientists don't subscribe to journals; we read them through our libraries. Some scientists might subscribe to journals directly relevant to their field, but I'd guess that few subscribe to more than one or two, if any at all. I subscribe to the American Journal of Physics and The Physics Teacher. But I'm interested in these journals because they discuss matters of interest to my teaching. I don't subscribe to any journals that publish papers I might be interested in because they related to my research.

In any event, the result is that a journal full of articles appears in the library, and/or a the new issue is made available online. What good is this system? What is the value of the journal? What services do the journal and its publisher provide the academic community? To this last question, I can think of several answers.

First, the publisher provides a nicely formatted, bound issue of a journal. This journal is a nice physical object; most journals have a pleasing heft to them, even if they do resemble telephone books at times. This physical object can be stored in libraries, or in offices or people's homes, and can be carried around and read. As for the formatting of individual articles, I'm not sure that the publisher is adding that much value. Without too much difficulty most physicists can format articles that look as good as those in a journal.

Second, the journal provides the service of peer review. Why is this important? Peer-reviewed publications are the coin of the realm in academic science. The number of publications in peer-reviewed journals is one of the first things looked at when evaluating a scientist. This is especially so for junior scientists who are up for promotion and/or tenure at universities that emphasize research and for graduate students and post-docs who are applying for jobs. The widespread practice of using a count of peer-reviewed publications as a proxy for scientific quality is, for better or worse, a fact of life. This isn't going to change anytime soon. So, peer review is an important service to the academic community.

It could also be argued that, through the peer-review process, journals maintain quality. Because all articles in a journal have been through peer review, they will be scientifically sound and more worth reading than just some random stuff someone might, say, post on a website. There is surely some truth to this, but I suspect not much. In areas that I work in, I feel pretty compentent to judge the value of an article; peer-reviewed or not, I think I can detect
garbage pretty well. Alas, I've read lots of really bad articles in peer-reviewed journals. Good journals, too. Peer review might be more valuable for areas I don't know much about. But again, I think most scientists develop their own critical eye pretty quickly; just because it's been peer-reviewed doesn't mean it's correct or interesting.

Third, through publication in journals articles are indexed in databases. If an article is published in most any journal, it will get picked up by the science citation index, INSPEC, and other databases. Researchers then use these databases to find articles.

Fourth, journals provide electronic access to papers. Individual researchers could do this, but not everyone has a webpage. And a publisher's website is likely much more permanent than any individual scientist's web page.

Before concluding, let's note some services that journals don't provide:

1. Scientific research and writing. The journal doesn't do the research, and authors are not paid for their submissions. In fact, some journals charge authors to publish their work.
   
   
2. Peer-review. Yes, the journal facilitates peer review, but the journal doesn't actually do the work. As I've lamented previously, referees are not paid for their work.
   

So, all this leads me to the main questions: How much should a journal cost? (Remember, authors and reviewers work for free.) What is reasonable to charge a library for a paper and/or online subscription to a monthly journal that contains, say, around thirty research articles each issue? Are there significant differences in journal prices? Have journal prices changed over time? Do better journals cost more?

IRS and the Quadratic Formula

I spent a while tonight writing some text for the COA academic website. It's an important task, but I have a hard time getting psyched to do it.

Anyway, a while back I stumbled across the following: If the IRS had discovered the quadratic formula... by Daniel Velleman. Amusing.

Island Astronomy Institute

I've spent part of the morning working on a grant which, if successful, would result in an astronomy class being taught at COA this spring. We've been trying for quite some time to get an astronomy class, so I'd be very pleased if we could get one. I'm far too busy with physics and math classes to think about doing astronomy. And, er... , I don't really know any astronomy.

The grant will be submitted to the Maine Space Grant Consortium. This is a funding arm of NASA. So far as I know, every state has something like this. I think the idea is to spread NASA funding around all 50 states. MSGC has awarded some nice grants to COA the last few years. Helen Hess received a small grant to conduct biomechanics workshops with area teachers, and I received a grant to add a laboratory component to my Chaos and Fractals class. Nishi Rajakaruna got a larger grant to conduct research on plants that hyper accumulate metal.

The lead organization in the astronomy grant is the Island Astronomy Institute, (IAI) which is a relatively new organization on Mount Desert Island. They have a small observatory and have given a bunch of seminars and talks locally. Since IAI is the main organization, I don't have to do the final editing of the proposal, which is nice.

Anyway, I hope we get the grant. Writing this morning has gotten me re-energized about astronomy. We're proposing a course that will interweave the history of ideas with the history of astronomy. And, perhaps more importantly, the class will focus the observational underpinnings of astronomy: how naked-eye and simple telescope-aided observations are used to draw inferences. At every stage, the emphasis will be basic epistemological questions: How do we know the moon shines because of reflected light? Why do we believe the earth revolves around the sun and not the other way around? What evidence is there for the fact that Venus revolves around the sun and not the earth? Sounds like fun to me.

[A clarification: If the grant is successful, it won't be me who teaches the class. I'm not really qualified to do so. The teacher will be Peter Lord, the founder and director of the Island Astronomy Institute.]

Various Weather-Related Observations

1. The weather report for my zip code, courtesy of weatherunderground.com says a 100% chance of ice pellets tomorrow. Ouch. Ice pellets don't sound like fun. Looks like I'm staying inside.
   
   
2. It's cold enough in my kitchen that the peanut oil has clouded up. Fortunately, the olive oil is still liquid. About once a year it gets cold enough for this to occur. Hasn't happened yet.
   
   
3. Because it's been cold and dry, the skin on my heels is dry and cracking. And it hurts. Yes, I use lotion, and it makes things somewhat better but not completely better. I'm too young for this to be happening.
   
   
4. The moonrise over the water as seen from campus today was absolutely amazing. And right now the moonshadows on the snow are very, very cool. I generally like living here, even if it's bad for my heels.
   

House Music/Home Music

There are a handful of blogs that I read semi-regularly. Many are written by academics, the majority of whom are young-ish, new faculty. A common theme at these blogs has been heading home for the December break. And this also has been the topic of a bunch of student blogs. There seem to be similarly mixed feelings among both groups.

After visiting Olin in Boston I headed to NYC/New Jersey to spend some time with my Dad. It was a nice visit, but I realized how glad I am that I don't live in New York, and especially New Jersey. I'm happy to live adjacent to a national park (even if it seems like all of New York visits here in the summer), I'm happy I'm not a (medical) doctor, I'm happy I have a job that enjoy almost all the time, I'm thankful I don't have to drive through traffic to get to work, and I'm thankful that I escaped from the cultural/social/whatever scene that was my high school. I'm not sure quite how I ended up at this weird school in Maine with a PhD and doing strange/interesting research and living in a house full of strange but mostly friendly creatures. Hmmm .... this is starting to sound like a cross between a dedication you'd hear on the Delilah radio show and something from a Woody Allen movie. This isn't how I wanted this to go.

Ok. Let's try starting again. Mostly what I wanted to write about was driving home. The last stretch of the drive I listened to a CD I got at tower records in the village: Shapeshifters house grooves, vol 2. It's pretty unbelievable. Warm, funky, delicious, in a starburst-fruit-chews-for-grow-ups sort of way.

The track, C-Mos' "2 Million Ways" is my anthem right now. Driving onto Mount Desert Island after a day on the highways that started in Englewood, NJ, it was perfect. It was almost midnight, the moon was shining on the mountains, and it was middle-of-winter cold. One website describes it as funky/vocal/disco/club house, which I suppose is a good start. If there were tropical rainforests in Iceland, forest gnomes might listen to this while they drank pink, minty drinks and danced at night under the tree canopy. I'm not sure what that exactly means, but if you were listing to the CD I was listening to, you might agree.

Anyway, I'm very glad to be home, and there's nothing like going home---in the sense of the home, the place one is from, even though I'm not from there---to make one appreciate home. My cats are happy to see me, the pipes haven't frozen yet, and I can listen to really, really loud music because Doreen is away. This dulls the pain of having to spend most of the week writing grants, writing text for our academic website, and writing administrative memos.

Impulsive Citation

Today I learned via the science citation index, that a paper I co-authored in 1998 was recently cited by an article in the journal Dynamics of Continuous, Discrete and Impulsive Systems. I think this is an awesome name for a journal. I want to publish things there.

Olin College of Engineering

A few days ago I gave a talk at the Olin College of Engineering and visited Olin for the day. I was very impressed. Olin seems like a really, really interesting place and I greatly enjoyed my visit.

First, some background. Olin was founded around five years ago; they're set to graduate their first class this June. Olin is supported by the F.W. Olin foundation. The foundation used to be in the business of helping private colleges build science buildings. Carleton College has an Olin Hall, as do many other schools. Carleton's looks like a giant radiator. I took all my physics classes in Olin, and spent many a late night working on problem sets and fiddling with electronics there. It's nicer on the inside than the outside.

Anyway, I guess around a decade ago, the Olin foundation decided to stop building other college's buildings and build some of their own. Specifically, they decided to use their entire assets, around 400 million dollars, to start an innovative college of engineering. The college now has three buildings, a bunch of dorms, and around three hundred students. I'm not sure how much of the 400 million is for their endowment, but I think around 300 million. All students get a 4-year, full-tuition scholarship, which they say is valued at around $130,000.

Basically, the school has a lot of money. Their buildings are very nice, especially on the inside. (I must admit that I don't think they're that attractive on the outside. A little too ponderous and modern-looking for my tastes. I'd bet that their style won't age well.) So one of the things that's cool about Olin is that they've got a lot of resources, both human and physical. But there's much more that's interesting to me.

For one, it's an amazing experiment to be able to start a college. Even more amazing is to be able to do it and not start off poor. Starting from scratch means being able to re-think lots of assumptions about college. From what I can tell, Olin is doing a great job of avoiding some of the perils of engineering schools: Olin seems to emphasize cooperation and collaboration and not competition among students; they place a high value on arts, humanities and social sciences; they try to make it easy for students to study abroad; and they really emphasize project-based and hands-on learning. Also, unlike most engineering schools, Olin has been successful in maintaining something close to a gender balance; 40% of the students are women.

I'll be interested to see just how innovative Olin ends up being. It's hard to be different. There are many external pressures---accrediting agencies, donors, parents---that already have a bunch of ideas about what a college should be. If one makes a college that's too different, then no one will come or give money. But, since Olin is well funded, they won't have as many pressures as other schools. So they have an opportunity to really make a mark.

Like Olin, College of the Atlantic is a young college. But we're teenagers (35 years old), while Olin is just a baby at 5 years of age. My sense as a faculty member and administrator is that COA is always walking a fine line between being innovative and being traditional. One the one hand, we really are different in some important ways. But, on the other hand, we are a college just like lots of other colleges. I don't think this is a bad thing, there's lots that's great about the institution of the college.

Another balance that I think COA and Olin are both trying to find is that between project-based learning and more traditional survey learning. It's great to learn mathematics via case studies. But there's certainly a limit to this. At some point, one probably has to do some basic theory or background that's not directly motivated at the time by a particular application. My sense is that faculty at both institutions are experimenting to find the right balance between these two. My inclination, at least for now, is that case studies work well as initial motivating schemes in introductory settings, and are great as capstone experiences for more advanced students. But somewhere in between one has to "pay some dues." I don't see a way to avoid taking a few classes on calculus and maybe one on linear algebra or differential equations and one on some sort of programming. Of course, these classes are so much fun that they shouldn't really be viewed as "paying dues."

Another thing that struck me about Olin, and reminded me of COA, was both the students and the faculty. No one comes to Olin by default, faculty or students. Students aren't going to apply to Olin just because it's the thing to do, like applying to MIT or the State University in their home state. Students decide to go to Olin because it has something they're looking for: a small size, an innovative curriculum, hands-on learning, etc. (And the fact that it doesn't cost them anything certainly helps, I suppose.) And faculty, I'm guessing, are drawn to Olin by the excitement of creating a new college, by the opportunity to build up a curriculum from scratch at a place that values innovation and has a lot of resources. If faculty wanted to do interesting research and basically ignore undergrads, they'd stay away from Olin.

So, the faculty and students at Olin struck me as a very interesting bunch. I was very impressed with all the faculty and students I met: bright, interesting, kind, and engaged with learning and doing. In this sense Olin reminded me very much of COA.

Olin and COA recently entered into a cooperative agreement wherein students at one school can take up to two terms at the other school. I was psyched about this when we first negotiated it, and now, having visited, I'm even more psyched. I think the two schools are natural partners. We have much in common, and we have some strengths that should complement each other quite nicely. I hope that students take advantage of this opportunity.

Uses for a Headlamp

I needed to use a calculator this evening, and mine was in the office. So I borrowed my wife's calculator, which is a relatively old solar model. (It's a TI-30X Solar, to be precise.) Alas, it wasn't working: the calculator is a little old and our living room is somewhat dimly lit. After a few minutes of frustration I got my little LCD headlamp. With the lamp on and pointed at the calculator, it worked fine and I happily averaged some numbers. Headlamps are handy little things -- no home should be without one.

Blog Realizations

Having had a blog for all of a week now, I've realized two things:

My life is boring. I guess I always knew this before, but somehow it's really hitting home right now. For example, today I did a bunch of coding, worked on my overheads for my upcoming seminar, went to the grocery store, went for a slow 2.5 mile run, and dealt with lots of email. Scintillating. Perhaps the only thing less exciting than being me, is reading about being me. My wife tried to console me: You blog isn't boring, but your life is pretty boring. I think she meant it in a good way.

The other thing I realized is that some of the potentially interesting or amusing items from my life I can't really blog about since I'm not doing this anonymously. I spend a lot of time teaching and a lot of time in meetings and doing administration. Doesn't seem right to blog about most of this stuff, unless I can do so in pretty general terms. So this eliminates a lot of the day-to-day stuff that otherwise might be of interest. And hence I find myself writing about mailboxes and Calculus grading.

Seminar Preparation and Household Tasks

I'm giving a seminar in Boston this Wednesday. So I spent much of the day preparing for it. I'll be talking at the Olin College of Engineering about computational models of neighborhood segregation. I've given a few talks about this at College of the Atlantic, so I have a pretty good outline. But this is the first time I'll be talking about this material to a group off-campus, so I need to tighten up some of my arguments, and will also need to generate some slightly better data. As part of this latter task, I spent a good chunk of the day coding, which was a great change of pace from the grading and grant writing I did during the week.

I also spent a while today doing stuff around the house: two loads of laundry, filled the birdfeeders, did a bunch of dishes, took stuff to the recycling place, and picked up my co-op order from Town Hill Market. I also moved a large amount firewood from the woodpile outside into our mudroom. Nothing terribly exciting, but between the household labor and some coding, it's been a productive day. Oh -- and I also made a batch of brownies. Yay.

Tidbits: Tires, Grades, Programming, Fog Tropes

1. Finished Calculus III grading, assigned grades, and submitted them. The most satisfying part was distributing a stack of papers that was around six inches tall to student mailboxes. Although it's tedious, I like putting graded assignments in student's boxes; it gives me a satisfying sense of closure.
   
   
2. I got snow tires late yesterday afternoon -- an errand I had been meaning to do for quite some time. Since I have a front-wheel drive corolla now instead of an all-wheel-drive subaru, I wanted the extra traction that a good pair of snow tires would give me.
   
   
3. Did some programming last night. Cleaned up some existing code an implemented one small new function. Nothing too exciting, but nice to have made some progress.
   
   
4. Listened last night to the CD "John Adams: American Elegies". It's a CD of John Adams conducting elegiac twentieth century classical music. Charles Ives' "Five Songs," Ingram Marshall's "Fog Tropes" and John Adams' "Eros Piano" are the selections I like the most. "Fog Tropes" is pretty amazing -- I always think of Ft. Bragg, California when I listen to it. An interesting discussion of the piece, along with an mp3, can be found here.
   
   

Tidbits: Grades, Pens, Editing Prose, Glenn Branca

1. Today I did my (next to last?) batch of Calculus III grading. In the midst of this effort the pen I was using ran out of ink. I always view it as a bit of an accomplishment when this occurs. For one, it's an indication that I've been doing work -- a whole pen's worth, in fact. It also means that I've managed to not lose the pen and I've gotten to use all the ink I paid for. So I was pretty excited. The pen was a maroon metallic gelly roll. Last week I finished a green metallic one. Grading problem sets using metallic glittery pens makes grading slightly more exciting. A student once remarked that it looked like a Harry Potter dwarf had graded her homework. I know nothing about Harry Potter; I chose to take it as a compliment.
   
   
2. Physics grades are finished. I still have most of the narrative evaluations to write, however. Assigning final grades reminded me of how much I dislike letter grades. (But that's a topic for another post.) As usually happens when I assign grades, I spent far too much time musing about the true meaning of a B+.
   
   
3. I spent an hour or so tonight carefully reading over a grant that two colleagues are writing. The grant is for $75,000 per year for four years for the purpose of increasing our graduation rate by 3% a year over the duration of the grant. The grant is pretty good -- there are some things in it that are really good ideas. One thing that will be a challenge is to merge the two writing styles into a coherent narrative. One author seems to be in love with parentheses. The other loves long sentences with almost as many commas as words. Surely there is a happy middle ground somewhere.
   
   
4. Searching for relaxing music in my CD bookcase, I've rediscovered Glenn Branca's Symphony #3: (Gloria) Music for the first 127 intervals of the harmonic series. Good stuff. Ok, maybe it's not exactly relaxing. But I do find somewhat soothing.