Dear Dr. Phoebe,
I conducted my first interview today – and it went badly. My project assesses the 2008 NHS reform and involves interviewing many of the key decision-makers from that period. I started with former Secretary of State for Health, Alistair Carrington, who agreed to meet me in the House of Commons. The interview didn’t go as expected, and now that the word has gotten out, no one wants to talk to me. Maybe I took the wrong approach. I’ve attached an excerpt from the transcript – maybe you could look at and tell me where it all went wrong?
Desperate in Docklands,
Sam Bookantiqua, E16
# Interview Transcript
Sam Bookantiqua: SB
Alistair Carrington: AC
SB: Very nice to meet you Mr. Carrington. I’d like to start the interview with a couple of demographic questions. Is that ok?
SB: Alistair Carrington, what is your name?
AC: Alistair Carrington.
SB: What’s your gender?
AC: I would have hoped this was obvious but… why don’t you try to guess.
Dear Dr. Phoebe,
I have hired two supposed “expert” coders to read a sample of newspaper articles and code each paragraph according to a coding scheme that I have carefully developed as part of my project in media studies. But the coder interreliability (reported by NVivo) is only .15. Should I be concerned?
Vernon Szarhazy, SW12
I’m afraid you have a problem with the reliability of your coders, your coding scheme, or both. What NVivo has reported is a kappa score, ranging from 0 (no agreement) to 1.0 (perfect agreement). Kappa is similar to a correlation or percentage agreement, but also takes into account chance agreement (which will small anyway if you have lots of possible codes). Landis and Koch (1977, “The measurement of observer agreement for categorical data”, Biometrics 33:159–174) have a much overused table interpreting 0.15 as slight agreement, but let’s just say if that were your grade for MY429 Qualitative Content Analysis, you’d be resitting the course. You need to take a hard look at your coding scheme and ask whether it can be simplified. Training your coders better, or choosing more consistent coders, would also be a good idea.
Dear Dr. Resmeth,
I am wrestling with a political economy model that includes both regime type and socio-economic level as explanatory variables. The problem is that these variables are strongly (positively) correlated, making it very hard to separate their independent effects on my outcome. And the problem gets worse: I also need to test an interaction of the two variables, but their high correlation makes it impossible to find any statistically significant effect for the interaction. A friend has suggested that if I mean center the variables, it will solve the multicollinearity of the interaction term. Can the solution be so simple? Please help, collinearity is making me desperate.
Yours, Francine Needlebush, WC2R
Unfortunately, such relationships between independent variables are a common problem in social science data analysis. The crux of your difficulty is that it is very hard for techniques such as multiple regression analysis to separate the independent effects of two covariates that are highly related. When you compound this problem by multiplying them as an interaction effect, the problem usually just gets worse. But because they are correlated, leaving one out and estimating the effects of the other separately causes an even nastier problem known as omitted variable bias.