Scattered thoughts on operationalization in digital research (and more fiddling with #ldsconf tweets).

A few days ago, I presented my final project for digital humanities seminar I’ve been blogging about all semester. The project was built around a Leaflet map that I built in R (thanks to a fantastic package designed for that purpose) to display one of the Twitter datasets that I’ve been working with recently. I’d like to post the map once I’ve had a chance to improve it some, but what I’m writing about tonight is actually an experience that I’ve had adapting that map for a different dataset. One of the happy difficulties I’ve had in this seminar is not knowing what to concentrate on for my final project, since I’ve got a few different projects and corresponding datasets that I have access to, so I thought it might be nice to write up a post presenting a similar map for my #ldsconf dataset (especially since I’ve just been revisiting it).

Well, I did come up with the map I was aiming for, but in doing so, I learned something about how important operationalization (i.e., “defining the measurement of a phenomenon“) when one is doing digital research. Ever since I learned the term in one of my first-year classes, I’ve been fascinated with the idea of operationalization. Researchers train themselves to explain the work that they do in a concise and engaging way; for example, I might explain that I study teachers’ participation in professional development spaces on Twitter. However, translating that short statement into actual variables takes some thinking, defining, and assuming (in short, operationalizing). In fact, my research colleagues and I have spent quite a bit of time discussing how to operationalize “participation”. We can measure tweets and retweets, and we’re even making some progress with measuring “likes.” Do those all count as participation? Are we leaving out any kinds of participation (e.g., more passive reading) because of the methods we’ve picked? How we operationalize something can have a pretty big impact on the results we get and the claims we can make.

Let me take a step back and explain how this all came about tonight. The foundational idea supporting my recent work on Twitter is that Twitter hashtags can serve as social spaces—or affinity spaces—on the Web. In this sense, #ldsconf is no different than the educational hashtags that I’ve been examining; however, what sets #ldsconf apart for me isn’t so much the different content area (i.e., religion rather than teacher professional development) as it is the contested nature of that social space. Although the hashtag was presumably created for people to discuss the LDS Church in a positive light and in orthodox terms, a freely-accessible social space like a hashtag doesn’t filter for that, so there are many people who use the hashtag to criticize the Church, express their frustrations with Mormonism, or address contemporary or historical controversies. (As a side note, #ldsconf and its cousin #twitterstake were fascinating to watch in the week leading up to the November 2016 US Elections—many tweets during that time didn’t even address Mormonism so much as Mormons themselves, trying to convince them to vote for Hillary Clinton, Evan McMullin, or Donald Trump, depending on the account in question).

So, I thought the most interesting thing to do for this map might be to try to differentiate “pro-LDS Church” Twitter accounts from “anti-LDS Church” ones (for lack of better terms). In hindsight, a map wasn’t really the best way to represent this, but it did get me thinking about this idea of operationalization. How could I clearly and reliably identify these different kinds of accounts? More importantly for this “proof of concept” phase, how could I delegate this task to a few lines of computer code so that I didn’t need to read thousands of Twitter profiles on my own? In some cases, this was pretty easy, but in other cases, it was more difficult. Was having retweeted something from an explicitly “anti-LDS Church” account enough to qualify another account as “anti-LDS Church”? Is it safe to assume that attending Brigham Young University makes one “pro-LDS Church”? Suffice it to say that I did not make much progress on these questions tonight!

However, thinking about this issue reminded me that I’d actually been meaning to write about operationalization earlier in the semester. Several weeks ago, we visited the Digital Humanities and Literary Cognition lab on campus and learned about some their efforts to integrate neuroscientific research with questions germane to the humanities (see Phillips and Rachman, 2015). My notes from this class session are a flurry of short questions, most of which revolve around this question of operationalization. How do you interpret fMRI scans in the context of reading literature? I was (and still am) fascinated with this question, despite the fact that—being neither a neuroscientist or a scholar of literature—I have no idea how to answer it. Another snippet from my notes: How do humanists operationalize irony? This one intrigued me even more, because I wasn’t even sure that scholars in the humanities use the term “operationalization” in their work (especially outside the realm of the computational and digital). And yet, isn’t all scholarship, regardless of discipline, focused on making compelling arguments based on compelling evidence? Isn’t all human knowledge dependent on some kind of operationalization?

One of my favorite feelings to experience is a sense of wonder at something I had previously taken for granted, so while I wish I had somewhere deeper or more specific to go with these thoughts, I’d like to close this post with this feeling: this sort of awe I’m experiencing at the task that is before us as researchers, whatever our arguments, methods, or disciplines. It’s easy for me to make casual claims about #ldsconf or any of the other Twitter spaces I study—I’ve done so several times in this blog post alone. For those claims to mean anything, though, is going to take a lot of work and a lot of thinking. This particular project is no great contribution to the sum total of human knowledge, but if I do it right, I’ll be participating in a long and important tradition of humanity’s discovery of the world around us, and that’s an important thing to remember… especially during a stressful end of semester!

References

Phillips, N., & Rachman, S. (2015). Literature, neuroscience, and digital humanities. In P. Svensson & D. T. Goldberg (Eds.), Between humanities and the digital (pp. 311-328). Cambridge, MA: MIT Press.

Leave a Reply

Your email address will not be published. Required fields are marked *