Social Science “Labs”

highvariance:

When most people think of labs, they imagine scientists in white coats staring into microscopes, carrying around beakers of bubbling chemicals, and holding test tubes over Bunsen burners. In social science, the reality is much more mundane. It’s usually just a room full of computers with software that may or may not be useful and may or may not be up to date. Even less compelling are the labs associated with statistical methods classes. The last couple years my own classes have been the worst case scenario–I just get up and lecture about how my students should use some particular piece of software to apply the methods we’ve been learning in the “lecture” part of the class. It doesn’t have to be this way.

Over the next few months I will have the opportunity to teach two new methods classes and completely re-invent how I incorporate labs. I had lunch with Mayur Desai the other day and I think he does a great job with labs in his classes and he’s inspired most (but not all) of the ideas here. This is what I’m thinking:

  1. No lectures. None. Students enter the lab and get their assignment and spend the rest of the class trying to complete it.

  2. Each assignment starts with a data set (preferably real) and a blank screen–that is, I don’t give them any code. Their job is to answer a substantive question by applying methods we’ve covered recently to the data.

  3. Students work in pairs and take turns driving. I think this keeps students focused and they can teach each other. It also means only half the class has to have laptops if I want to implement a lab in a regular classroom.

  4. I’m around to answer questions. In this way, it’s very different from a problem set where getting stuck on something dumb for hours at a time is a common occurrence. Struggling with problem is good for learning, but banging your head against a wall isn’t an efficient use of time.

  5. The end product should be similar to results they might find in a published paper. Sometimes I’ll provide an empty table they must fill in and other times they will produce their own tables of results from scratch.

  6. There should be opportunities for quicker/more advanced students to do more. One size does not fit all.

While it’s possible to use any statistical analysis tool in a lab successfully, I do think some packages are better than others. Most students already know Microsoft Excel and doing basic analyses (even regression) using it is easy, but you really hit a wall when you want to do anything even a little sophisticated. SAS is powerful, but there is a steep learning curve. My plan is to use Stata. You can browse your data in a spreadsheet style interface. You can play with commands through the menus and when you choose one, it shows you the command-line equivalent. You can work interactively at the command-line or build programs (using those same commands) in an editor. And the documentation is excellent and available online.

I’ll let you know how it goes!

leakygrammar:

“”Human social science is stereotyped as the land of fuzzy concepts and fuzzier minds, with hydra-headed jargon lurking in the shadows that will paralyze you with the poison of vagueness and ambiguity before you even have a chance to try and figure out what in the hell they might mean…The stereotype has some truth to it. But in part - only in part - the world salad is excusable. The real world, filled as it is with variable human intentionalities, is messy…If the phenomena are messy, so will be their representation in the language of science, however noble is the scientist’s struggles to be precise. Ambiguity and vagueness are characteristics of the phenomena. Without those characteristics, the human social world would implode.””

— (The Lively Science: Michael Agar, 2013, p. 71)

doctorimpostor:

““Like others of my generation, for me a Ph.D. in the social sciences meant that results were only meaningful if full of numbers, chi squares, and cluster diagrams that had a statistical significance of .05. Although there was something very seductive about artfully uncovering elegant patterns in this matter, the relative trust in a scientific method and distrust of the ‘art’ of studying human behaviour never sat well with me. I watched my scientist housemate start an experiment by getting rid of the “noise.” Yet I found that the noise, the outliers that blew away my 0.05 level of confidence, was where some of the most interesting information lay. I felt an almost tangible beauty in the patterns, especially ones that outliers helped foreground; surely they were part of the story””

— Ellen Pader (p. 161) in Dvora Yannow and Peregrine Schwartz-Shea, eds. “Interpretation and Method: Emperical Research Methods and the Interpretive Turn.” Armonk, NY: M.E. Sharpe. 

It seems to me that, rather than trying to answer questions when you don’t have the necessary data to do it, perhaps you should ask different questions. Certainly, we all do the best with the data we can get, but it is never alright to draw conclusions that your data don’t support—and Regenerus’ data simply cannot answer the question he set out to ask. And when your research questions the legitimacy of people’s families—my family—I demand higher research standards.

In Sociology Lens, an insightful blogger known only as Amanda critiques the new study by sociologist Mark Regenerus. Regenerus has published a paper arguing that children from heterosexual parents are better off than children raised by lesbian and gay parents. I recently posted that academic research actually shows that this is not true. Studies actually show that children of LGBTQI families are slightly better off than kids from heterosexual families with respect to aspiring to more progressive gender roles. In other respects they are similar, when you factor in class differences. 

Amanda notes that Regenerus’ research on gay and lesbian families has produced contradictory findings due to the study’s poorly conceived methodology. Simply put: Regenerus’ methods for data collection do not match his research questions (meaning the methods are invalid). Regenerus defines homosexuality according to anyone who has had a same-sex experience, without taking into account their subjective identities or family experiences. Regenerus has not controlled for the fact that some children from gay and lesbian families are being raised in single parent households. This generally puts any child at an economic disadvantage when compared to dual parent households. Amanda argues Regenerus’ findings are tinged with homophobia, possibly influenced by Regenerus’ ties to the Christian site that hosts his blog.

Sociologists are not above having their politics influence their research interests - including you, me and everyone else. We do not have to agree with one another; however we are trained to make our assumptions explicit and to have our methods match our research questions. I know many sociologists who conduct studies that go against my political and personal beliefs and yet I can engage in useful and challenging discussion because the data and methods warrant attention. Crappy science still warrants attention, but for all the wrong reasons. What a shame that Regenerus’ lax methodology will only fuel public fear and misunderstanding, rather than making a contribution to empirically-informed debate. 

Read Amanda’s excellent article at Sociology Lens.

Currently being haunted by my own writing… Duncan Watts wrote a wonderful piece on the myth of common sense for Freakonomics.com. Here’s part of what I wrote about that last year:

—-

What resonated most for me [about Watts’ argument] was the challenge that sociology faces in making our public contribution valued. Watts points out that sociologists deal with everyday social experiences that are familiar to many people – such as family, gender, social networks, fame and success, popular culture and so on. Due to the familiarity of these topics, most people think they can explain sociological phenomena using their common sense. Watts argues that common sense is problematic because the people we have around us have similar worldviews and this does not necessarily make informal observations valid. The problem with sociology is that unlike other sciences, such as physics or mathematics, sociologists do not offer up concrete answers or predictions…

Nevertheless, Duncan includes some great examples about the strengths of sociology being its methodological tools, which provide a way to understand the complexity of social behaviour and social change. Duncan writes:

"Clearly we’re a long way from a world in which cause and effect in social and economic systems can be established with the level of certainty we’ve come to expect from the physical sciences. In fact, the world of human behavior is sufficiently complicated and unpredictable that no matter how long or hard we try, we will always be stuck with some level of uncertainty, in which case leaders will have to do what they’ve always done and make the best decisions they can under the circumstances.

It sounds like a lot of effort for an uncertain payoff, but curing cancer has also proven to be an enormously complex undertaking, far more resistant to medical science than was once thought, yet no one is throwing up their hands on that one. It is time to apply the same admirable resolve to understand the world—no matter how long it takes—that we display in our struggles to address the important problems of physical and medical science to social problems as well”.

—-

Duncan’s words remain all too relevant for those of us working outside academia. Proving sociology’s relevance in the face of ‘common sense’ is no easy feat.