An issue that has arisen is that of how well a survey (in this case, a quantitative measure) 'fits' if you will, with individual perception. What do we do, for example, when we have chosen our valid and reliable measure that operationalizes our construct of interest only to find that when we actually give the assessment, our participants seem to balk at certain questions. Participants may, for exmaple, want to/want you to rephrase the question or perhaps give you a non-score (e.g. a 0/zero) when the response format ranges from +3 to -3 with no zero to be found.
This is a challenge we all face. While it is tempting to want to change such measures take a moment to think about doing so and what this really means. We have chosen our reliable and valid measures for that very reason...they are tested with solid psychometric properties. If we change the response format or questions in any way - we do indeed change our psychometrics and therefore should have much less confidence in our findings and have concerns about potentially spurious results.
Is it possible, therefore, that our items and response format can be 'good enough' and that we therefore must leave these items as-is, for the sake of our data collection? I would argue yes. What we can do, however, is help the participant understand the essence of the question, relative to the construct at hand. So if a question in our resilience or LOC scale asks us to rate a statement on our leadership skills and our participant states 'I am not a leader' - instead of getting caught up in semantics, it might be helpful to step back a moment and help the participant understand that in this case (and this depends of course, on the construct and specific operationalization thereof...) we are talking generally or even hypothetically - if the participant cannot identify a specific area of leadership.
Sadly, our psychological assessments are far from perfect. When we discover issues like this - it serves to higlight the importance of assessment development. The other nice thing about this is that once we discover this, we have uncovered a potential gap in the literature - an area that may warrant additional research attention.
AES
Dr. Sickel,
ReplyDeleteYou wrote, "An issue that has arisen is that of how well a survey (in this case, a quantitative measure) 'fits' if you will, with individual perception."
Isn't this where the mixed method will use a qualitative interview without the construct of specific questions?
You further wrote, 'What we can do, however, is help the participant understand the essence of the question, relative to the construct at hand. So if a question in our resilience scale asks us to rate a statement on our leadership skills and our participant states 'I am not a leader' - instead of getting caught up in semantics, it might be helpful to step back a moment and help the participant understand that in this case (and this depends of course, on the construct and specific operationalization thereof...) we are talking generally or even hypothetically - if the participant cannot identify a specific area of leadership."
Now, in helping a participant with a construct with which to identify an area of leadership, in the case of the elder lifespanners and health, leadership might translate to ~ if the participant is still driving ~ picking up Sally or Joe and taking them shopping. I have seen elder lifespanners taking still elder lifespanners shopping using the golfcarts they do here in Peachtree City, GA. The driver could consider themselves a leader in that sense.
Now, further, I would imagine the construct of leadership is different for everyone, as no two brains process, interpret, or apply information in quite the same way. Given this, if the interviewee is helping [italics] a participant think about leadership, should there not be a script for the scope of leadership in adjectival terms to consider relevant to healthcare?
Perhaps the Discussion section or Conclusions might consider these types of questions for future study of the elder lifespanners. Thank you for reading my comments. Enjoy your day.
Hi Cherb,
ReplyDeleteIn my example, I was specifically discussing a chosen quantitative measure - not a qualitative one. There may be qualitative measurement of locus of control but for this study's purposes, we chose a quantitative measure.
I used the term 'leader' as an example, but let me now get more precise. Our LOC measure states: "Whether or not I get to be a leader depends on whether I’m lucky enough to be in the right place at the right time" and we are asked to get the -3/+3 rating on this item.
You are quite right, how we/one conceptualize/conceptualizes the 'leader' construct can be quite different. Most quantitative measures, to the best of my knowledge, do not provide scripts for how to interpret the terms. Rather, they rely on our knowledge of the construct at hand - which presumably we have researched prior to using the measure.
So it then is our job to help the participant understand the questionnaire’s conceptualization. Regarding the item I mentioned about leadership, the individual *can* define it any way he/she chooses; if you look at the statement it is looking to measure the individual’s attitude toward leadership. If I strongly disagree with the notion that for me, being a leader is being in the right place at the right time, I would give that a -3. Here, we are not asking participants how much of a leader are you? Or how lucky are you? – rather, we want to know if they see any of their own leadership as a function of luck.
Thanks very much for your comments!
AES
Dr. Sickel, thanks for your response ;)
ReplyDeleteAgreed, the LOC measure, as many studies have indicated, seems a best possible reliable tool.
Thanks again.
My pleasure - thanks very much for the dialogue ;-)
ReplyDeleteAES