Member Check and Respondent Validation in Qualitative Research
Member checking, also called respondent validation, is a qualitative research technique where researchers and study respondents collaborate to ensure data accuracy.
This technique is often used to verify qualitative data gathered from interviews, semi-structured interviews, or focus groups. Essentially, member checks are when a researcher gives data back to respondents to cross-check their own initial data gathering or data analysis.
While member checks are not error-proof methods of data validation, their ultimate purpose is to offer findings that are real, unique, and trustworthy.
What is Member Checking in Qualitative Research?
Stahl and King (2020) note that member checks are important because “the degree of trust one has in the person telling the tale has much to do with the degree of trust attributed to the telling.” That is, it is a good way for others to validate and assess the trustworthiness of the storyteller.
Validating qualitative data is particularly difficult because “the researcher is often both the data collector and data analyst, giving potential for researcher bias.” (Miles & Huberman, 1994)
Lincoln and Guba (1985) recommend member checking as a way to minimize this bias. They suggest that giving data back to participants helps check for accuracy and completeness. We prefer this straightforward definition as their paper is widely considered the gold standard for establishing trustworthiness in qualitative research.
Member checking covers a range of activities[1], including:
Returning interview transcripts to respondents
Return interview transcripts to respondents
Ask them to verify facts and confirm their original words
May result in respondents adding new information
They may also delete unwanted data – changing the results
Return analyzed data to respondents
Return analyzed to respondents
Present it in an accessible format so that anyone can understand it
Participants should be able to identify their experiences in the synthesized themes
If you want to collect additional data, explain and create space for participants to participate as simply as possible
Usually happens several months after the data is collected (after analysis)
Harvey (2015) used a synthesized data approach and checked with participants before the third (out of four) total member check interviews
Can lead to overcorrection of results or respondents changing their minds, resulting in a deviation from the original interview data (Grbich, 2006).
Member check interview using transcript
Each participant receives their interview transcript
Discuss the interview transcript with the respondents
Focuses on confirming or modifying transcript
Can potentially enable the addition of new data
May also lead to them changing details they dislike
Member check interview using analysis of a single participant’s data
Each participant receives your synthesized notes of their interview
Focuses on confirming or editing that interpretation
Could enable the addition of new data
Need to transcribe and analyze this interview
One note before we continue, member checking is just one of several data validation techniques. Another example is to measure intercoder reliability, which is where multiple researchers measure how much the team agrees when coding the same data set.
As a general concept, all data validation is basically a trust-building exercise for research results.
Why Conduct a Member Check?
In quantitative analysis, researchers rely on numerical data to confirm findings. As a result, researcher bias is less of a concern since you can calculate validation or invalidation.
But generating trustworthy data in qualitative analysis is more difficult. As Miles & Huberman introduce above, researchers need to account for their potential biases and the subjective nature of the data itself. That’s where member checking and other forms of data validation can help.
Member checking is particularly helpful in qualitative research that involves interviews or qualitative observation, such as phenomenological research or grounded theory research.
While member checks are not requisite, more and more qualitative researchers now consider this an “ethical imperative.” In short, researchers use human subjects, so their feedback on the accuracy and completeness of the data is both important and valuable (Birt et. al., 2016).
Lastly, beyond validating data and ethical concerns, this collaborative technique may also increase the overall richness of data by adding first-hand information or insight to a study.
Different Types of Member Checks
Within the broader category of member checking, researchers use different types of respondent validation techniques to measure their accuracy. These include narrative accuracy checks, descriptive validity, interpretive validity, and theoretical validity (Stanley, 2011).
Narrative accuracy check
Used to verify the accuracy of a participant's account of an experience or event.
A summary or transcript of the narrative is presented to the participant for review.
The goal is to ensure the researcher's interpretation matches the participant’s experience.
Descriptive validity
Refers to the accuracy and objectivity of data collected in a research study.
Researchers should record information accurately and without bias.
Low descriptive validity if data collected is not accurate and objective.
Interpretive validity
Checks how well a researcher understands a participant's behavior and viewpoint.
Tests are designed to assess knowledge specific to the research question.
Participants should also agree that the test achieved its goals.
The degree to which these two perspectives align determines interpretive validity.
You can improve upon these types of member checks with other methods of data validation. Intercoder reliability is one option but we will also provide additional examples later on.
Advantages & Disadvantages of Respondent Validation
While many researchers use member checks to establish trustworthiness, others doubt their effectiveness. Typically, this is because respondents tend to view their experiences as unique and not generalizable, which conflicts with the researcher's aim of generating common themes. [2]
Another issue is that the more respondents edit their original words, the less candid and honest they may actually become. Basically, researchers often feel this confirmatory process retracts from the honesty and straightforwardness of the respondents’ original words.
Due to the contention around member checks, it might be better to think of them as a way to decrease errors rather than verify research findings. That is not to imply that member checks are not enough on their own, just to highlight that their efficacy is a matter of debate.
Here are more specific advantages and disadvantages of member checking [3]:
Advantages of member checks
Enhances validity and credibility
Improves accuracy and completeness
Establishes rapport and builds relationships with respondents
Reflects a participatory, collaborative approach
Provides opportunities for clarification and elaboration of findings
Can lead to new insights and perspectives that were not previously considered
Increases the transparency and accountability of a study
Disadvantages of member checks
Can be time-consuming and resource-intensive
May lead to biased feedback
Participants may feel reluctant to offer negative feedback
Can lead to disagreements or conflicts between researchers and respondents
Can be difficult to implement in studies with a large number of respondents
Need to consider respondents’ literacy, language skills, and cognitive abilities
May not be suitable for studies with sensitive or controversial topics where participants may feel uncomfortable providing feedback
How to Do Member Checking? A Short Guide
Member checking is conducted during or after the interview process. You can also use it multiple times as Harvey suggests, depending on the time and number of participants available.
While there are no strict guidelines for how or when to perform member checking, here’s an outline of how to apply the range of member check activities mentioned earlier:
Share data with research participants: Share the research findings with respondents and ask them to review and confirm the accuracy and completeness of the data collected.
Get feedback: Get direct feedback from the participants to identify any gaps or errors in the data you collected. This feedback may or may not change your initial data.
Incorporate feedback: Incorporate the feedback received from research participants into the research findings.
At this stage, you can also do another round of member checking or use synthesized data for your first round of member checking.
Report results: Finally, write up the results of the entire process to include in your final results. Providing these details makes it easier for others to validate your work.
Other Types of Data Validation in Qualitative Analysis
As mentioned above, member checking is not a fail-safe method of data validation. However, it can reduce the risk of inaccuracies and misinterpretations in qualitative data.
To build upon member checks, Birt et al. (2016) suggest that “it should be used judiciously and in combination with other forms of validation." Beyond intercoder reliability and member checks, other validation techniques may include:
Birt implies that the more methods used, the more rigorous the results. Just remember that each additional method extends how long it will take to complete your research.
Key Takeaways
To summarize, member checking is a valuable tool for enhancing research credibility, establishing trustworthiness, and building relationships with study participants. However, its limitations and appropriateness should be carefully considered in each individual study.
Try Delve Qualitative Analysis Tool
Delve computer-assisted qualitative data analysis software (CAQDAS) is cloud-based, collaborative, and easy to learn. It includes free tutorial videos, responsive customer support, and flexible payment options.
See why researchers like Jess are switching to our easy-to-use, drag-and-drop coding software.
“It is easy to use, doesn't take a long time to pick up on how to use it, there are handy videos also to help you and it is a well thought out and efficient piece of software well worth using as a student or researcher.”
References
Birt L, Scott S, Cavers D, Campbell C, Walter F. Member Checking: A Tool to Enhance Trustworthiness or Merely a Nod to Validation? Qual Health Res. 2016 Nov;26(13):1802-1811. doi: 10.1177/1049732316654870. Epub 2016 Jul 10. PMID: 27340178.
Mays, N., Pope, C. (2000). Assessing quality in qualitative research. British Medical Journal, 320, 50-52.
“Member Checks.” RWJF - Qualitative Research Guidelines Project | Member Checking | Member Checks, http://www.qualres.org/HomeMemb-3696.html
Miles, M. B., & Huberman, A. M. (1994). Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks, CA: Sage Publications.
Lincoln, Y.S., & Guba, E.G. (1985). Naturalistic inquiry. Sage. https://doi. org/10.1016/0147-1767(85)90062-8
Stahl, Norman & King, James. (2020). Expanding Approaches for Research: Understanding and Using Trustworthiness in Qualitative Research. 44. 26-28.
Thomson, Stanley. (2011). Qualitative Research: Validity. Joaag. 6.
Johnson, R. Burke. "Examining the validity structure of qualitative research." Education, vol. 118, no. 2, winter 1997, pp. 282+.
Cite This Article
Delve, Ho, L., & Limpaecher, A. (2023c, May 03). Member Check and Respondent Validation in Qualitative Research https://delvetool.com/blog/member-check-respondent-validation