Last year about this time, Kris McDaniel posted some important questions about the nature of the hiring process in philosophy, particular with respect to APA interviews. I’d like to resurrect one of Kris’s questions for a new round of discussion as well as add another.
My first question is similar to Kris’s: what, if anything, is the value of conducting interviews at the Eastern APA? I strongly suspect that hiring committees go into these interview sessions with a rough ranking of their 12 or so interviewees, with a distinct and agreed-on top tier. What I’m curious about, now that most interviewing departments have extended invitations to their top 3 or 4 candidates for campus visits, is whether or not these initial rough rankings (at least in the top tier) changed much, if at all, from your pre-interview stage (where folks went from being in the top tier to being out, or being out of the top tier to being in). If so, what drove such change? Was it the interview itself, the smoker, other informal discussions, or something else? If not, do you see any genuine value to the interviewing experience, at least a value worth preserving in light of the significant costs to your department or institution in terms of flights, hotels, travel time, and the psychic toll of leaving one’s family for several days in the middle of the holiday season? Why not, instead, just bring the top tier to campus?
Second, consider the campus visit. There are several puzzling elements to this event as standardly practiced, but I’ll focus for now on the main event, the “job talk.” The typical job talk involves the candidate reading or presenting a paper for approximately an hour, followed by Q&A for an hour. A typical condition is that the paper be a work in progress (so not previously published) and new to the department, i.e., not also submitted as a writing sample. What, however, is the point of this format and these conditions? What information relevant to the needs and desires of a philosophy department looking to add a colleague will be learned in this two hour performance with such preconditions?
Yes, these are typically the conditions for a conference presentation (only of a rarified sort, though—APA colloquia papers, and many other conference papers, are only 20 minutes long, and include commentators and only 20 minutes of questions). But that’s a very different environment, and anyway the goals of someone evaluating whether or not to hire the speaker are (or ought to be?) very different from the goals of an audience member in a conference.
One thing I suspect most people want to learn is whether or not the candidate is a good philosopher, where this includes the abilities to work on interesting and philosophically valuable projects, to situate one’s projects in the relevant literature, to have a command of such literature, to construct good arguments, to write clearly, to defend one’s ideas from various objections, and to have genuine publishing potential (and surely there are others). If this is the general goal (and here I set aside the possibility of different and really puzzling goals like “it’s meant to be a trial by fire”), then why is this centerpiece of the campus visit conducted in this way? Is the ability to think quickly on one’s feet to address an objection one hasn’t heard before one of these conditions? Surely it’s not necessary to see whether or not the candidate can read aloud for an hour, so why use the first hour for that when one could instead get the paper in advance and then just use that time to discuss the paper (that the audience members ought to have read) in more detail? And what is the point of restricting the types of papers that can be presented in the ways already mentioned? If one wants to evaluate the best work the candidate has to offer, why not allow the candidate to submit a published piece? Or his/her writing sample? These latter two suggestions would make particular sense when evaluating a candidate fresh out of grad school, for he or she may simply not have many presentable papers just yet. Requiring him/her to present a new and unpublished paper will often produce a rushed and weak effort (typically drawn from a later, more sketchy portion of his/her dissertation), which is likely to present an inaccurate picture of his/her writing and publishing potential or general philosophical talents.
I’m looking, then, to see what defense is on offer for these longstanding industry practices. Feel free, however, to add into the mix other typical aspects of hiring procedures you find puzzling.
I think a lot of what you say makes sense, but I want to undercut what looks like an assumption at one point, which is that the hiring committee likely has a ranking when it forms a short list. Because committees may have members who don’t agree and because different ways of getting rankings out of different people’s preferences can produce different rankings, this is often not true. All that needs to be decided at any stage is how many to interview or bring out or whatever. So voting may constructed to settle that issue without deciding an overall ranking.
Why is this relevant? Just that committee members may use interviews, etc. to try to change their colleagues minds, even when they are thinking they aren’t going to get more evidence from the process.
I’m not saying this in defense of interviews, especially as they are done by most departments. I’m just suggesting that things are a bit more complicated than presented and thus that people may use them for purposes other than to get new information.
Interesting post. Regarding APA interviews, I suspect that if the committee more or less agrees on who the top people are, for the most part they would be better off just flying them out and foregoing the costly and time consuming interview process. One interesting exception, however, is if the committee is going to be somewhat strategic about its campus visits. For some schools, such as mine, it would be very risky simply to offer campus visits to our top three people. A more sensible strategy for us is to offer a campus vist to one or two of our top tier people and perhaps to one or two of our middle tier people. The interviews can help us decide which top tier person might “fall through the cracks” and thus worth shooting for (and which would definitely get better offers) and which middle tier people might be better than their file suggests. But if I were at a top 20 program, I think flying out the top people directly would be the way to go.
As for campus visits, let me tell a brief story. Some years back when I was on the job market I had a flyout where they asked me to do a teaching demonstration on material I had never read before (on the fourth chapter of McDowell’s “Mind and World”). Most of the faculty sat in on my teaching demonstration and bombarded me with questions throughout. In essence, they had me give a talk on an assigned topic far outside of my area. At the time I thought this was outrageous (and a bit funny). In retrospect, however, I suspect that this gave them a much better sense of my philosophical abilities than my paper presentation did (which I gave later that day).
Dave,
You ask,
If the goal is just to see whether the person is a good philosopher with publishing potential, then why have the live, in-person interviews at all? As I know, you’re in favor of having a live, in person session where the faculty ask questions about one of the candidate’s papers, which the faculty have all read in advance. But why even have this? Does this give you a better idea about how good a philosopher the candidate is than you would get by just looking at his or her CV and reading more of his or her work? Or if you think the ability to address objections is important why not email the candidate a series of questions on his or her work and let him or her respond to the questions in writing after having some time to ponder them.
So I wonder whether your worries about the cost-benefit analysis of APA interviews and job talks don’t apply to all sorts of live, in-person interviews. What is the value of such live, in-person interviews? These live interviews can give us (the interviewers) a sense of whether we find the person personable and good at thinking on his or her feet under high-stress situations. But it sounds like you don’t think that this sort of information is what’s most salient. But then why have live in-person interviews at all? Why not just look at written evidence such as the candidate’s CV, teaching evaluations, written work, letters of references. Surely, all these provide the best evidence of a candidate’s philosophical and teaching abilities, and indeed live in-person interviews are more liable than not to provide unreliable and yet more vivid evidence that gets in the way of our looking at the most reliable paper evidence.
David,
I’m not sure what people say in defense of the research-based ‘job talk’ but I imagine one thing it actually does is help the department decide if they want the person around for (potentially) three or four decades. In other words, it has its facial purposes (to evaluate the person’s research potential, their ability to think on their feet, etc.), and then there’s the ‘do I want to have conversations with this person about this kind of topic and provide them feedback useful to help them get published?’ factor. So as with much else in campus interviews, since the people who are invited are likely to be basically (and almost equally) qualified, departments are deciding who they want as a colleague and who fits best with their institutional and departmental culture.
I think that Michael is right that the real purpose of live, in-person interviews is to see whether the candidate is someone whom the interviewers might like as a colleague. But I think that this speaks in favor of keeping the APA interviews. Use the APA interviews to weed out those people that you positively don’t want as colleagues, but rely on the other more reliable types of evidence to determine each candidate’s philosophical and teaching abilities. The approach to take, then, would be to make an offer right after the APA interviews to the candidate, of all those whom you could stand to have as a colleague, that the more reliable written evidence supports as being the top teacher/scholar. Offer that person the job, and perhaps offer them a visit to your campus to interview you, for perhaps they would like to visit your department, campus, and environs before making a decision about whether to accept your offer. But then you don’t need to have any job talk or Q&A session. You just need to show the candidate what your position has going for it: let the candidate meet the faculty, grad students, the relevant administrators, and give him or her a tour of the department, the library, the campus, and the surrounding environs.
Thanks for the comments thus far.
Mark: You’re right that the assumption the “hiring committee” has a ranking may not necessarily be the case. But it’s unclear to what extent the APA interviews actually help out here. Take the example you give, in which interviews might be used to help change colleague’s minds. Do you have any evidence (and anecdotal evidence is all I expect) to suggest that this works? Might there not be another way in which a push for greater consensus might be achieved than by flying everyone out to the go through the dreary standard process?
Mike: Your anecdote gives me chills. As to the point about being realistic about to whom you extend campus visits, this is really important for non-top-30 departments, I know. Do you really get the feeling that the APA interviews help in this regard, though? Everyone’s typically on their best behavior there, and I can’t recall every discovering of a candidate at an APA interview that he or she would fall through the cracks elsewhere or not be worth risking a campus visit or offer. These are things we usually know in advance, just from a study of the CV or letters.
Doug: You raise a good point. I think it is important that a candidate be able to discuss and defend what he/she takes to be his/her best work intelligently. To that end, some sort of department-wide discussion of that work should take place, primarily so we have some sort of common touchstone for later discussions (rather than having one or just a few people saying, “Well, I talked to him about X and he was unintelligible” — sometimes folks who say that just don’t get the project or are untrustworthy or have their own agenda). But so much of what makes people good philosophers takes place either in private (painstaking contemplation) or in extended discussions that it is hard to see the ultimate relevance for assessing philosophical skills whether or not someone can think on his/her feet in this particular sort of environment. So I’m not averse to the suggestion that a list of potential questions/objections might also be provided in advance to the candidate, who could then have time to think about a reply. Much in the way that a commentary on one’s paper at a conference gives one a chance to think about it prior to the talk as well (and which typically makes for a much better discussion).
That said, my main question is about the typicaly restrictions on papers presented: if what we want is an in-depth discussion of what the candidate thinks is his/her best work, why not allow discussion of published papers or writing samples?
Michael: I agree that the search for a good hire is in many ways the search for a good colleague (whether or not that amounts to three or four decades, which these days is entirely unreasonable). And what many people mean by that is, “Can I talk philosophy with this person?” The job talk may provide a glimmer of an answer to that question, but without something at least more in-depth (along the lines I’ve suggested above), it’s hard to see how it could provide more than a glimmer.
Doug: In my comments above, I’m saying your first point had merit, but I have a real problem with your most recent suggestion. Unless you can get the entire department out to the interviews (or unless you have a smaller personnel committee everyone fully trusts to make the decision, which will be rare), I can’t see you could get a true departmental vote out on who to make offers to.
I’ve earlier said I doubt that rankings change much as a result of APA interviews. But I’ve seen it happen many times that those thought to be collegial and good philosophical discussants after the brief APA interviews were subsequently thought to be problematic during the campus visit, i.e., rankings really do change post-visit, once you get a chance to talk in greater depth with the candidate.
Dave,
I have no doubt that the ranking of candidates can change after campus visits have occurred. And I have no doubt that the ranking can change after APA interviews have been conducted. You want evidence for the latter, but it is hard to produce evidence of this, since departments and individuals don’t typically go to the trouble of ranking all the candidates to be interviewed before interviewing them. What would be the point of that?
The issue, though, is not whether they change, but whether they change for the better. What evidence do you have that the rankings change for the better rather than the worse? I’ve been told that the best empirical evidence shows that in-person interviews actually lead to poorer decisions about whom to hire than are typically reached when the hiring body makes its decision without conducting any live, in-person interviews. Hasn’t Princeton abandoned in-person interviews, and for this reason?
So if the information gathered by these live encounters leads to poorer decisions being made, why have them at all? Perhaps, though, you have some reason to think that the evidence that these sorts of encounters produce is more reliable in the case of on campus visits than in the the case of APA interviews. I doubt that’s true. The problem with these in-person encounters is that the “vividness” of the evidence that they produce exceeds its reliability, and thus they lead us to give more weight to less reliable evidence just because it is more vivid in our minds. I think this problem will hold for both the shorter APA interviews and for the longer on-campus interviews.
I suspect that the only reliable information that is to be gathered by these in-person interviews is whether or not the person is a complete jerk or social moron. This is why I suggested keeping APA interviews to weed out the complete jerks and social morons.
I also suspect that you put too much evidence on whether someone is a good philosophical discussant. I have had at various institutions plenty of colleagues that are good philosophical discussants, but who are not available to discuss philosophy with, because they tend to work at home, or because they live far way, or because they don’t socialize much. So what really matters is not whether they can discuss philosophy but whether they will do so when they get here. And I don’t know how you figure that out from an interview — on -or off-campus. The same applies to collegiality. I think that you can from an interview get a sense of whether the person is a complete jerk or not, but beyond that I don’t know how to tell from an interview how active a participant the person will be in departmental affairs — except that you’ll know that the complete jerk will either be too active or not active at all, depending on what best serves his or her interests.
By the way, I’m sorry to have derailed what you take to be the main issue:
“my main question is about the typicaly restrictions on papers presented: if what we want is an in-depth discussion of what the candidate thinks is his/her best work, why not allow discussion of published papers or writing samples”
The answer, I think, is that we should allow this.
Dave,
You’ll find Gil Harman’s thoughts about this in the discussion thread of Kris’s post. More specifically, you can find it here.
So my concern is that your position is an inconsistent one. The most salient sorts of considerations that count against APA interviews count against on-campus interviews.
Dave,
I wasn’t meaning to disagree with you about the value of interviews. I think the value is minimal with respect to information gathering on the hiring side. I was just thinking that your case oversimplified in its presentation. So I think I should decline to offer evidence that interviews are an epistemically good way to generate a ranking.
The one value an interview can have by my lights, is to show a candidate that they will have good people to talk to if the interview is done well, and that can matter in recruiting candidates.
Mark: You’re definitely right that interviews give departments an opportunity to sell themselves to candidates, letting them know there will be good people about to talk to. Further, the reputation of a good interviewing department can circulate amongst various candidates such that interest in that department can increase (amongst good candidates, hopefully!). And such factors may be quite relevant in attracting and actually hiring strong contenders. But as I think you also agree, they may not be very helpful in determining who those strong contenders are to begin with.
Doug: You make many interesting and provocative points. Note first, though, that I’m simply trying to take as given certain assumptions about what the goals of a particular sort of job talk might be. Certainly if we back up to consider the goals of hiring generally and what departments are looking for from a job candidate, it certainly makes sense to question the role of both interviews and campus visits. Here, though, I suspect that the answers will widely vary, depending on the nature of the dept., the roles of research and teaching in it, the number of faculty, and so forth. So what I was doing was focusing on the two aspects of hiring that most departments adhere to (APA interviews and job talks) to see what value, if any, they find in them, and if there is some value (in the case of job talks), what value there could be such that allowing published papers or writing samples was verboten.
I’m aware of the studies ostensibly showing that the “best empirical evidence shows that in-person interviews actually lead to poorer decisions about whom to hire than are typically reached when the hiring body makes its decision without conducting any live, in-person interviews.” I’ve only heard about them, though, and I wonder how something like this could be shown. It would be impossible, for instance, to test this by engaging in a hire with in-person interviews, and then erasing everyone’s memory and engaging in precisely the same hiring process without the in-person interviews.
At any rate, you point to another ostensible value (which you mistakenly think I was espousing), having to do with finding good philosophical discussants. I completely agree that this is, in practice, utterly unpredictable. You may find someone who would indeed be good but who’s never around, or has no time to talk when he/she is. Plus, someone could be a great commentator on other people’s work (which I find extremely valuable), but one doesn’t need to be a colleague for that, and anyway that’s not what one “tests for” with a job talk.
But again, you’re questioning values I was just supposing for the sake of argument.
Is it really true that “job talks” should be work in progress? I know that I have a weak preference for not having the writing sample be the job talk, but it’s a reasonably weak preference. I have no preference at all that the job talk be unpublished. Especially at junior level, where all publications will be pretty recent (and hence almost certainly unread!) I’d be perfectly happy hearing a presentation on a published paper. But if other people would not be happy with this, I want to tell my grad students that…
I agree that it is odd to have the person read the paper rather than pre-distributing it. I have known people who have said that this is a way to test how good the candidate is at presenting material, and hence how good they are at (one aspect of) teaching. I think these are terrible inferences, but they are defences that are given.
I have to be careful here, because we are in the middle of an ongoing search. I will say, though, that the phone interviews we conducted in lieu of interviews at the APA did indeed make a difference to our collective rankings of candidates. I should probably not comment publicly on the details at this point, but some people helped themselves and some didn’t.
We will be doing the standard research paper job talk. Maybe that isn’t the best approach. For purposes of discussion, though, let me offer some points in its defense that I don’t think have been mentioned.
1. In practice, some people won’t read the papers in advance. Especially since not everyone at the talk will be from the hiring department.
2. Putting people in a position where they have to think on their feet may not tell you much about how successful they will be at publishing. Other things equal, though, I’d rather hire someone who will be successful at conferences as well. Plus, I want to hire someone who will be able to deal with students’ questions effectively. We are doing a separate teaching demonstration, but that is aimed more at teaching lower-level students. The research talk may actually give us more insight into what an upper-level or graduate class from the candidate would be like than the teaching demo will.
3. I don’t think that presenting an already published paper or the writing sample is strictly verboten. Other things equal, though, I’ll prefer the candidate who doesn’t. My school expects quite a few publications, around 7, for tenure. That is too many, I think, but it isn’t a matter for the department to decide. A candidate who has work at different points in the pipeline is more attractive to us as someone likely to get tenure.
Part of me, though, does long to do campus interviews British-style: bring all the candidates in one day, job talks in the morning, awkward collective lunch, interviews in the afternoon, and make the offer before you go home that night. Three campus interviews are a big disruption.
Dale,
I don’t understand your (3). I understand why a “candidate who has work at different points in the pipeline is more attractive to [your department] as someone likely to get tenure.” But why does that lead you to prefer, other things being equal, that the candidate doesn’t present his or her writing sample? Aren’t there other (better) ways of determining whether the candidate has work in the pipeline: e.g., asking the candidate to submit all of his or her papers and asking what their statuses are (e.g., in progress, under review, being revised and resubmitted, etc.)? Why does the candidate need to present a work in progress to demonstrate to you that he or she has work in the pipeline?
And, as Dave asks, “If one wants to evaluate the best work the candidate has to offer, why not allow the candidate to [present] a published piece? Or his/her writing sample?”
Dave,
I don’t think that the question of what instrumental value some practice has on the assumption that certain presumed goals are valuable is nearly as interesting as the question of what instrumental value some practice has given what is in fact valuable. That’s why I was focusing on the latter. I see, though, that your post is more concerned with the former. Nevertheless, you did say: “Feel free, however, to add into the mix other typical aspects of hiring procedures you find puzzling.” So I thought it was fair of me to question values that you were just presupposing and to question whether on-campus interviews (whether or not they involve a job talk and whether or not the interview focuses on a work in progress or a published writing sample) have any more utility than APA interviews do.
Let me add that I think that on-campus visits do have some value, but I see the value as being mainly for the candidate in determining whether he or she would want to accept the job and mainly for the department in recruiting a top candidate, demonstrating to him or her that this is indeed an excellent place to live and work. But if this is the value that on-campus visits have and if on-campus visits are a bad tool for selecting candidates (for the reasons Harman cites), then this suggests that they should not be used as a selection tool (as they typically are) and should instead be used only as a tool for recruiting the candidate that has been already been selected on the basis of more reliable and less distracting evidence.
Doug,
Maybe there are other ways in which the candidate can demonstrate that s/he has work in the pipeline. My point was that the current practice does achieve some worthwhile ends. I also had Brian’s post in the back of my head, in which he was wondering what advice to give grad. students given current practice. Of course, the alternative would need to let candidates show that they had more work that was fairly far along in its development. Just listing paper titles on the CV doesn’t suffice, since those might just be ideas with nothing on paper.
Dale: Just a quick question about your (2). It seems to me that job talks and teaching an undergraduate (or even graduate) class are pretty far apart, in tone, content, format, expectations, and otherwise. Why do you think, then, that the former could give any real insight into facility with the latter?
One other point I’d add: A good many other people besides department faculty will meet with candidates during campus visits, and this is pretty significant inasmuch as deans, provosts, etc. often have a great deal of input on who gets hired. Indeed, with respect to my current position, I spent a good deal of time prepping for my ‘interview’ with the dean, which (I understand) proved key to my being hired.
David,
I may be wrong, and my opinion may change in a couple of weeks. We haven’t done a lot of searches in my time in my current position, so I haven’t heard loads of job talks. But I’ll hear three in the next nine days, and maybe the experience will change my mind. I was responding, though, to what I took to be your implied doubt about whether it was useful to watch candidates being asked to think on their feet, responding to questions that they haven’t heard in advance. (At least I think that it was in one of your comments that I first saw the suggestion that candidates might be presented with questions/objections in advance.) I agree that it is not obvious that seeing people think on their feet is useful for judging their scholarly potential, at least insofar as this means their potential for publishing. When you’re writing, you can take your time. But I do think that the skill of finding good answers to challenging questions and objections quickly is important in the classroom, at least when we are talking about teaching brighter students who aren’t just asking the most obvious questions. I’m not suggesting that teaching is exactly like giving a job talk, but I am saying that doing a job talk gives you a chance to display some skills that are called upon in upper-level teaching.
Michael: You’re definitely right about that. A visit isn’t entirely about selling the candidate, but also giving the administration a look-see so they can make something of an informed evaluation of a candidate for their own purposes. That’s certainly one reason I think the campus visit generally is still important.
Dale: Fair enough, but it’s also the case that in teaching one can use a hard question to spur deeper, interesting discussions that may shape the remainder of the semester, say. Indeed, sometimes the quick, sharp answer in the classroom will stymie such opportunities. The classroom answer, “That’s really interesting, and I’m not sure what to say about that, but let’s think it through together” can be, I think, a very good teaching answer that often may not look good in a job talk (where it looks as if the candidate’s project might be in serious trouble).