Saturday, March 7, 2015

How to make $1,000,000

My mother once told me about an easy way to become a millionaire: start with $10 million. This seems to be advice that second generations are particularly good at following. And not only as regards inter-generational wealth transfer. Like families, journals also enjoy life cycles, with founders giving way to a next generation. And as in families, regression to the mean (i.e. the headlong rush to average) seems to be an inexorable force. However, in contrast to the rise and decline of wealth in families, the move from provocative to staid in journals is rarely catalogued. Rarely, but not never. Here is a paper (by Priva and Austerweil (P&A)) that charts the change in intellectual focus of Cognition, a (once?) very important cogsci journal. What does P&A show? Two things: (i) It shows that the mix of papers in the journal has substantially changed. Whereas in the beginning, there was a fair mix of theory and experimental papers (theory papers predominating), since the mid 2000s the mix has dramatically changed, with experimental papers forming the bulk of academic product. Theory papers have not entirely disappeared, but they have been substantially overtaken by their experimental kin. (ii) That papers on language and development have gone from central topics of interest to a somewhat marginal presence.[1]

How surprising is this?  Let me start by discussing (i), the decline of “theory,” a favorite obsession of mine (see here). Well, first off, from one common perspective, some decline might be expected. We all know the Kuhnian trope; “revolutionary” periods of scientific inquiry where paradigms are contested, big ideas are born and old ones die out (one old fogey at a time in Plank time) give way to periods of “normal science” where the solid majestic wall of scientific accomplishment is carefully and industriously built brick by careful empirical brick. The picture offered is one in which the basic framework ideas get hashed out and then their implications are empirically fleshed out. I never really liked this way of conceptualizing matters (there is a lot of hashing and fleshing going on all the time in serious work), but I think that this picture has some appeal descriptively. Sadly, it also seems to have normative attractions, especially to next generation editors. Here’s what I mean.

Editing a journal is a lot of work. Much of it thankless. So before I go off the deep end here in a minute, let me personally thank those that take this task on, for their work and commitment is invaluable and what we think of as science could not succeed without such effort. That said, precisely because of how hard it is to do, you need to be driven (nuts?) to start a journal. What drives you? The feeling that there is something new to say but that there is no good place to say it. Moreover, not only is that something new, it must be important and new. And it is not possible to say these new important things in the current journals because the new ideas cut things up in new ways or approach problems from premises that don’t fit into the existing journalistic matrix.[2] So, at the very least, the extant venues are not congenial places to publish, and in some cases are outright hostile.

The emergence of cogsci (something that happened when I was growing up intellectually) had this feel to it. There was a self-conscious cognitive revolution, with very self-conscious revolutionaries. Furthermore, this revolution was fought on several fronts: linguistics, psychology, computer science and philosophy being the four main ones. Indeed, for a while, it was not clear where one left off and the other began. Linguists read philosophy and psychology papers, psychologists knew about Transformational Grammar and could Locke from Descartes, philosophers debated what to make of innate ideas, representations and rule following based on work in linguistic and psychology and computer scientists (especially in AI) worried about the computational properties of mental representations (think Marr and Marcus for example).  Cogsci lived at the intersection of these many disciplines, was nurtured by their cross disciplinary discussions and, for someone like me, cogsci became identified as the investigation of the structures of minds (and one day brains) using the techniques and methods of thought that each discipline brought to the feast. Boy was this exciting. Not surprisingly, the premiere journal for the advancement of this vision was Cognition. Why not surprisingly? Because the founding editors, Jacques Mehler and Tom Bever, were two people that thoroughly embodied this new combined intellectual vision (and were and are two of its leading lights) and they built Cognition to reflect it.[3]

A nice way of seeing this is to read Mehler’s “farewell remarks” here. It is very explicit about what gap the journal was intended to fill:

Our aim was to change the publishing landscape in psychology and related disciplines that became part of “Cognitive Science.” …[P]sychology had turned almost exclusively into an experimental discipline with an overt disdain for theory…Linguistics had become a descriptive discipline often favoring normative or purely descriptive over theoretical approaches. Professional journals in line with this outlook generally obliged contributors to write their papers in standard format that privileged the shortest possible introductions and conclusions, methods and procedures used in experiments. Papers by non-experimental scientists say, philosophers of mind or theoretical linguists, were rarely even accepted…. (p. 7)

In service of this, the journal was the venue of lots of BIG debates concerning connectionism, the representational theory of mind, compositionality, AI models of mind, prototypes, domain specificity, computational complexity, core knowledge and much much more. In fact, Cognition did something almost miraculous: It became a truly inter-disciplinary journal, something that administrators and science bureaucrats (including publishers) love to talk about (but, it seems, often fail appreciate when it happens).

P&A records that this Cognition now seems to be largely gone. It is no longer the journal its editors founded. There is little philosophy and little linguistics or linguistically based psychology. Nor does it seem to any longer be the venue where big ideas are thrashed out. Three illustrations: (i) the critical discussions concerning Bayesian methods in psychology have not occurred in the pages of Cognition,[4] (ii) nor have the Gallistel-like critiques of connectionist neuro-science gotten much of an airing, (iii) nor have extensive critiques of resurgent “language” empiricism (e.g. Tomasello) made an appearance. These have gotten play elsewhere, and that is a good thing, but these dogs have not barked in Cognition, and their absence is a good indicator of how much Cognition has changed. Moreover, this change is no accident. It was policy.

How so? Well, in the same issue that Mehler penned his farewell the new incoming editor Gerry Altmann gave his inaugural editorial (here). It’s really worth reading the Mehler and Altmann pieces side by side, if nothing else as an exercise in the sociology of science. I’ve rarely read anything that so embodies (and embraces) the Kuhnian distinction between revolutionary vs normal science. Altmann’s editorial is six pages long. After some standard boilerplate thanking Mehler & Co. for its path-breaking efforts, Altmann sets out his vision of the future. It comes in two parts.

First the ideal paper:

To be published in Cognition, articles must be robust in respect of the fit between the theory, the data and the literature in which the work is grounded. They should have a breadth to them that enables the specific research they describe to make contact with more general issues in cognition; the more explicit this contact, the greater the impact of the research beyond the confines of the specialized research community. (2)

It’s worth contrasting this ideal with the more expansive one provided by Mehler above. In Altmann’s, there is already an emphasis on “data” that was missing from Mehler’s discussion. In other words, Altmann’s ideal has an up front experimental tilt. Data’s the lede. The vision thing is filler. To see this, read the two sentences in reverse order. The sense of what is important changes. In the actual quoted order what matters is data fit then idea quality. Reverse the sentences and we get first idea quality and then data fit. Moreover, unlike Mehler’s pitch, what’s clear here is that Altmann does not envision papers that might be good and worthwhile even were they bereft of data to fit. It more or less assumes that the conceptual issues that were at the foundation of the cogsci revolution have all been thoroughly investigated and understood (or were largely irrelevant to begin with (dare I say, maybe even BS?)). More charitably, it assumes that if something new does rise under the cognitive sun, it will arise from the carefully fitted data. In short, the main job of the cogscientist is see how the theories fit the facts (or vice versa). Theory alone is so your grandparent’s cognition.

The second part of the editorial reinforces this reading. The last 3 pages (i.e. half the editorial), section 3, concerns “the appropriate analyses of data” (4). It’s a long discussion of what stats to use and how to use them. There is no equally long section discussing thos hot topics/problems, what issues are worth addressing and why. This reinforces the conclusion that what Cognition will henceforth worry most about is data fit and experimental procedure. Sounds like the kind of journal that Mehler and Bever had hoped that Cognition would displace. Indeed, prior to Cognition’s founding, psychology had lots of the kinds of Journals that the Altmann editorial aspires to. That’s precisely why Mehler and Bever started their journal. Altmann appears to think that psychology needs one more.

If this read is right, then it is not surprising that Cognition’s content profile has changed over the years. It is not merely that new topics get hot and old ones get stale. Rather, it is that what was once a journal interested in bridging disciplines, critically investigating big issues and provoking thought, “grew up” and happily assumed the role of purveyor of “normal” science. A nice well behaved journal, just like most of the others.

Last two points. Given the apparent dearth of interest in theory, it is not a surprise (to me) that work on language is less represented in the new Cognition. Anything that takes linguistic theory seriously in psycho study will be suspect to those with a great respect for psychological techniques (we don’t gather data the right way, there is a distance between competence and performance, we think that minds are not all purpose learners etc.). Thus taking results in linguistic theory as starting points will go against the intellectual grain where theory is less important than data points. This need not have been so. But that it is so is not surprising.

Second, there is a weird part of Altmann’s editorial concerning the “collaborative” nature of science and how this should be reflected in the “editorial structure” of the journal. Basically, it seems to be signaling a departure from past methods. I don’t really know how the Mehler era operated “editorially.” But it would not surprise me were he (and Bever) more activist editors than is commonplace. This would go far, IMO, in explaining why the old Cognition was such a great journal. It expressed the excellent taste of its leaders. This is typically true of great journals. At one time the leading figures edited journals and imposed their tastes on the field, to its benefit. Max Planck edited the journals that published Einstein’s groundbreaking (and very unconventional) papers.[5] Keynes edited the most important economics journal of his day. Mehler and Bever were intellectual leaders and Cognition reflected their excellent taste in questions and problems. It strikes me that the Altmann editorial is a non too subtle critique of this. It’s saying that going forward editorial decisions would be more balanced and shared. In other words, more watered down, more common denominatorish, less quirky, more fashionnable. There is room for this ideal, one where the aim is to reflect the scientific “consensus.” Today, in fact, this is what most journals do. Mehler and Bever’s Cognition did not.

To end: Cognition has changed. Why? Because it wanted to. It has managed to achieve exactly what the new regime was aiming for. The old Cognition stood apart, had a broad vision and had the courage of its new ideas. The new Cognition has re-joined the fold. A good journal (no doubt). But no longer a distinctive one. It’s not where people go to see the most important ideas in cognition vigorously debated. It’s become a professional’s journal, one among many. Does it publish good papers? Sure. Is it the indispensible journal in cogsci that it once was? Not so much. IMO, that’s really too bad. However, it is educational, for now you know how to make $1,000,000 dollars. Just be sure to start off with $10,000,000.

[1] This is all premised on the assumption that the topic model methodology used in the paper accurately reflects what has been going on. This may be incorrect. However, I confess that it accurately reflects what many people I know have noted anecdotally.
[2] Is this PoMo or what? With a tinge of the Wachowskis thrown in.
[3] And you know many of the others. To name a few: Chomsky, two Fodors, Gleitman, Gallistel, Katz, Pylyshyn, Gellman, Garrett, Block, Carey, Spelke, Berwick, Marr, Marcus, a.o.
[4] E.g. Eberhardt & Danks, Brown & Love, Bowers & Davis, Marcus have all appeared in other venues. See here, here and here form some discussion and references.
[5] A friend of mine in theoretical physics once told me that he doubted that papers like Einstein’s great 1905 quartet could be published today. Even by the standards in 1905 they looked strange. Moreover, they were from a nobody working in a patent office. It’s a good thing for Einstein that Planck, one of the leading physicists of his day, was the editor of Annalen der Physik.

No comments:

Post a Comment