In this week’s newsletter, I’m thinking aloud about a recent Urban Institute blog post (screenshotted above) which questions “objectivity” and “rigor” as ideals in social science / think tank research. (Note: the Urban blog represents only the views of the author, and not of the institution; this is also true of my own musings here.) But before I jump, here’s some items of note.
First, I helped write a new Brookings report which examines the nature of St. Louis’ racial inequalities, with particular attention to the intersection of poverty, incarceration, and educational disparities. The report also highlight grassroots, citywide efforts to end mass incarceration and advance equitable development.
Second, I wrote a super short review of The Cult of Smart for Plough magazine.
”Whereas Vonnegut’s dystopia seeks to blur or remove all distinctions in ability and talent, meritocracy justifies those distinctions in ways that undermine any sense of solidarity. Specifically, meritocracy allows us to take credit for our luck (having a natural aptitude for academics, for example) as though it were our virtue, while blaming others for their lack of luck as though it were a vice.”
Third, my workplace is unionized now (!):
Fourth, whoever greenlighted this new animated film…thank you
Wait, Are We Really Against Objectivity and Rigor?
The Urban Institute (a renowned DC thinktank) recently published a blog post entitled “Equitable Research Requires Questioning the Status Quo.” In the post, the author takes aim at both “objectivity” and “rigor” in research as being troubling or even harmful concepts. Naturally, the piece sparked debate on Twitter, with some leading economists panning it. I think the piece was poorly crafted and communicated, but the general observations are pretty standard within self-aware social science (akin to the self-aware historiography in the discipline of professional history.)
I was recently on a panel with another thinktanker (and friend) talking about research with honors college students. And as part of my comments on that panel, I talked about different ways of understanding objectivity. So in what follows, I’ll share what I shared with the students, in what I think is a much better framing than what was published by Urban.
1): To begin, here is what the Urban researcher wrote about objectivity:
This is the distance between the “researcher” and “researched.” It is based on the belief that neutrality on a subject is the best way to determine its facts. Objectivity allows researchers, intentions aside, to define themselves as experts without learning from people with lived experience. Objectivity also gives researchers grounds to claim they have no motives or biases in their work. Racism, sexism, classism, and ableism permeate US institutions and systems, which, in turn, allows for research that reproduces or creates racist stereotypes and reinforces societal power differences between who generates information (white cisgender people) and who is a subject (Black, Indigenous, and other people of color at the margins of class and gender). At best, objectivity curbs how impactful research can be, and, at worst, it irrevocably harms a community.
Reading this charitably, I think the main point is that just because we use empirical methods, does not mean that there are no power dynamics, value judgments, subjective viewpoints, or potential bias. From the conceptualizing of a problem, to the developing of a modeling or research parameters, to the way we conduct the research, to the way we interpret and contexualize our findings - there is inescapable human subjectivity.
On a somewhat parallel note, here’s the gist of what I told the honors students:
I think we often conflate two kinds of objectivity. So on the one hand, it’s absolutely true that most (not all) thinktanks strive to be objective in the research they publish; we’re not lobbying on behalf of specific parties or politicians, and we’re not doing PR for donors or special interest groups. Fine. That’s basic research ethics.
But that doesn’t mean there are not particular subjective points of view that come across. Every scholar has personal motivations for what they’re researching, has goals for what they’d like their research to accomplish, has implicit (typically unstated) beliefs about how society should operate, which political coalitions should be in power, whose policy agenda is most in line with their own preferences.
And so the research we publish, while rigorous, while not explicitly partisan, contains all those normative assumptions and judgments. Brookings and AEI will occasionally do working studies together, for example on family leave policy, and we’ll say, “hey, here’s this really good research we did together; and here’s what we agree on in terms of defining a problem and looking at basic facts. And here’s maybe two or three solutions both sets of scholars advocate. But then also, here‘s the areas that both our perspective on the issue and the solutions diverges based on our different set of starting principles.”
And one implication of this unavoidable subjectivity is that you’re always gonna get criticism of your research. Sometimes it’s because you made a numbers error. And there you just have to own the mistake and issue a correction. But beyond that, people might question your whole approach, or suggest that you overlooked an important factor, or argue that your data work doesn’t support your claims. When that happens, you listen with an open mind: be willing to learn from feedback, but also if you think your approach holds up, defend it! That process of thinking through (maybe even writing a defense) can be great for clarifying things internally, and also potentially be a teaching moment for your audience.
There’s also gonna be people who simply disagree with your philosophical first principles, and with the moral framing or normative dimensions to your work. When that happens, just acknowledge that and move on. (But you can also work on making moral arguments more central to your work moving forward, if that’s something you want to do.)
2): And then here is what the Urban researcher says about rigor:
Rigor measures whether research is reliable, accurate, and trustworthy. It’s a standard asked for by funders and research institutions alike. However, researchers often define rigor as following an established research protocol meticulously instead of ensuring data are contextualized and grounded in community experience. Rigor in this sense does not guarantee trustworthiness or accuracy.
In the most charitable reading of this, I think the researcher is basically saying, “not everything needs to be a regression to count as evidence.”
I told the honors students that I’m a huge advocate for the importance of basic exploratory data. People often get dazzled by complex mathematical modeling, or analysis that says, “we have x percentage confidence of this relationship between variables, and this cause/effect blah blah blah” but often basic observations are just as vital as the basis for research. So for example: my team is currently finishing up a project about life expectancy decline. We saw the stat that in 2020 life expectancy declined by 1.5 years overall mostly due to covid, but that there were major racial and gender disparities. Now, without any fancy regression analysis or complex modeling, those statistics are newsworthy in and of themselves. And they immediately prompted us to ask, “what’s going on here?”
So yes, regressions can be really helpful, but they aren’t the end-all-be-all. And there’s a certain kind of social scientists (particularly in economics) who scoffs at any argument that does not rely on regression - and that’s myopic and wrong-headed especially because a regression model is only as good as the assumptions behind the formulation of the research question, and the parameters of the model. And often it’s the less “rigorous” as well as the more “qualitative” research where our intuitions get sharpened and we learn how to ask the right questions and to build the right models.
Before I end, I do want to editorialize a little about what I think is the role of thinktanks. There’s the standard answer about conducting actionable research, providing policy recommendations, cultivating a group of experts who can staff the next presidential administration, etc.. But I think the deeper answer is that thinktanks provide a space where the brightest minds can gather to discuss political ideas - without the pressures and artificial timelines of political campaign cycles and without the onslaught of daily crises that politicians are forced to deal with as part of governance.
The goal is to plumb the depths, to gain clarity, to slowly stumble onto greater insight – and then to be able to share that insight with leaders who value your work. To situate this in ancient philosophy terms, the thinktank is a place for leading the kind of philosophical life which can enrich the political life, it’s a supportive space for contemplation which then becomes a good foundation for active involvement in public affairs - whether that is our own involvement, or the involvement of people shaped by our work.
What I Am Reading Elsewhere:
This piece by Osita Nwanevu on What Is Political Writing For? is a worth reading multiple times. And linger on this insight » “"Writers ought to be given the time, space, and opportunity to say not the first, second, or even third thing to come to mind, but maybe the fourth—a chance to write at an angle or with prose that challenges or surprises."
That above pieces pairs nicely with this deeply moving column by George Will in The Post: “Here, however, is the good news: Amid the cacophony, and because of it, there is an audience for something different, for what Kempton exemplified and some of us aspire to — trenchant elegance."
And also pairs with this NYT column by Charles M. Blow, Why I Write: “What I knew was that otherness, that outsiderness, that sense of being left behind and left out, that sense of being the world’s disposable people because you had little money and wielded little power.”
Peter Berkowitz (former director of policy planning at the State Department) has a fascinating exploration of the multisided Henry Jaffa as a window into the East Coast / West Coast Straussian split, and its implications for contemporary conservative politics. “Contrary to the most famous lines that he penned and the orneriness and arrogance he frequently exhibited, Jaffa's seminal writings on Lincoln's thinking and statesmanship provide good reason to conclude that the vice of extremism imperils liberty and that the virtue of moderation is essential to the pursuit of justice.”
In The Atlantic, Will Leitch has a sobering article on how the rise of micro-betting could destroy sports.
There are so many fascinating moments in the transcript of Fed Chairman Powell’s recent press conference, but especially this question: “I’m wondering if a 6.1 percent unemployment rate for African-Americans is consistent with full employment or whether it would need to be lower as part of your inclusive-growth strategy."
Pew Research has an informative new blog post on how the US census determines Who is Hispanic.
In Strong Towns, Daniel Herriges considers the question of scale in development: “Big developers enjoy economies of scale; small developers succeed by knowing their community intimately and thus unlocking value in ways that are illegible to a larger, more streamlined enterprise."