Mar 24 2013

Why Won’t Anyone Talk to Me?

Leave a comment

Dear Dr. Phoebe,

I conducted my first interview today – and it went badly. My project assesses the 2008 NHS reform and involves interviewing many of the key decision-makers from that period. I started with former Secretary of State for Health, Alistair Carrington, who agreed to meet me in the House of Commons. The interview didn’t go as expected, and now that the word has gotten out, no one wants to talk to me. Maybe I took the wrong approach. I’ve attached an excerpt from the transcript – maybe you could look at and tell me where it all went wrong?

Desperate in Docklands,

Sam Bookantiqua, E16

 

# Interview Transcript
Sam Bookantiqua: SB
Alistair Carrington: AC

SB: Very nice to meet you Mr. Carrington. I’d like to start the interview with a couple of demographic questions. Is that ok?
AC: Yes.
SB: Alistair Carrington, what is your name?
AC: Alistair Carrington.
SB: What’s your gender?
AC: I would have hoped this was obvious but… why don’t you try to guess.

Continue reading

Posted by: Posted on by LSE Methodology

Dec 12 2012

My Qualitative Coders Keep Disagreeing

Leave a comment

Dear Dr. Phoebe,

I have hired two supposed “expert” coders to read a sample of newspaper articles and code each paragraph according to a coding scheme that I have carefully developed as part of my project in media studies. But the coder interreliability (reported by NVivo) is only .15. Should I be concerned? 

Vernon Szarhazy, SW12

 

Dear Vernon,

I’m afraid you have a problem with the reliability of your coders, your coding scheme, or both. What NVivo has reported is a kappa score, ranging from 0 (no agreement) to 1.0 (perfect agreement). Kappa is similar to a correlation or percentage agreement, but also takes into account chance agreement (which will small anyway if you have lots of possible codes). Landis and Koch (1977, “The measurement of observer agreement for categorical data”, Biometrics 33:159–174) have a much overused table interpreting 0.15 as slight agreement, but let’s just say if that were your grade for MY429 Qualitative Content Analysis, you’d be resitting the course. You need to take a hard look at your coding scheme and ask whether it can be simplified. Training your coders better, or choosing more consistent coders, would also be a good idea.

Sincerely,

Dr. Phoebe

Posted by: Posted on by LSE Methodology

Dec 5 2012

Girlfriend Terrified I’ll Sleep with Mafia Princess

Leave a comment

Dear Dr. Phoebe,

I’ve just had an epiphany after a month of wracking my brains for a research project that uses the cool ethnographic methods I’ve learned from MY426. The idea came after my new girlfriend forced me to watch a night of Johnny Depp movies, starting with something about a drunken possibly gay pirate that looks a bit like Michael Jackson. But that was followed by a Donnie Brasco, the film that gave me my research idea using participant observation methods. In the film – based on a true story – Depp plays an FBI undercover agent who infiltrates the Bonanno crime Family in New York. He finds the gate-keepers, he records lots of information, he becomes one of the most trusted hit-men of the family and, eventually, he helps the FBI to arrest all the Mafia big bosses. The movie is great because it shows how Donnie is very close to ‘turn native’ at some point (like to become a true gangster); it also shows that gaining access, building trust, and taking notes undercover is not easy of course, but still feasible.

So my idea is to go undercover, like Donnie Brasco, to investigate how hierarchical relationships work out within the Sicilian Mafia. For fieldwork, I’d go to Palermo and take up work in a restaurant frequented by the Mafia. There I will become familiar with the crime bosses, and also secretly record all that they say using microphones hidden under my shirt. Building upon that, I will try to infiltrate higher strata of the Mafia and will study their inner power relationships. Like Brasco, I’ll deeply penetrate their organization, even if this means having participate and not simply observe their activities. But my girlfriend is worried sick, saying I might have to commit crimes or sleep with some mafioso’s daughter to gain their trust. I haven’t told my supervisor about any of this, since he’d probably just worry. I think I’ll be fine, since two summers ago I went on a backpacking tour through Sicilily and just loved it. The data I get will be original and incredible, and the project really seems exciting too. Is this a sound and valid ethnographic research design?

Nick Picklewaite-Busby, SW19

 

Dear Nick,

Fuggedaboudit.

Continue reading

Posted by: Posted on by LSE Methodology

Nov 28 2012

My Variables Have an Unhealthy Relationship

Leave a comment

Dear Dr. Resmeth,

I am wrestling with a political economy model that includes both regime type and socio-economic level as explanatory variables. The problem is that these variables are strongly (positively) correlated, making it very hard to separate their independent effects on my outcome. And the problem gets worse: I also need to test an interaction of the two variables, but their high correlation makes it impossible to find any statistically significant effect for the interaction. A friend has suggested that if I mean center the variables, it will solve the multicollinearity of the interaction term. Can the solution be so simple? Please help, collinearity is making me desperate.

Yours, Francine Needlebush, WC2R

 

Dear Francine,

Unfortunately, such relationships between independent variables are a common problem in social science data analysis. The crux of your difficulty is that it is very hard for techniques such as multiple regression analysis to separate the independent effects of two covariates that are highly related. When you compound this problem by multiplying them as an interaction effect, the problem usually just gets worse. But because they are correlated, leaving one out and estimating the effects of the other separately causes an even nastier problem known as omitted variable bias.

Continue reading

Posted by: Posted on by LSE Methodology