At a Glance
When a survey and a set of interviews are done on the same program, the temptation is to write one smooth story. This page shows a different habit: lay the results side by side, record who and when each covers, and name how they relate. The general design choices are in mixed methods and triangulation. The interviews come from the case in from interview notes to findings.
At a Glance
| Relationship | Meaning | What to Do |
|---|---|---|
| Agree | Both strands point the same way for comparable people | Check that they can fail in different ways before raising confidence |
| Partly agree | Same direction, different degree or scope | State the difference and the reason it may arise |
| Silent | One strand does not address the topic | Do not read silence as absence; plan to ask |
| Conflict | Strands point in different directions | Check population, period, measure and meaning before explaining |
Worked Case
Worked Case: Survey and Interviews That Differ
Everything here is fictional. The program, the counts and the quotations are invented to show the reasoning, not to describe a real program.
A fictional organization runs evening learning circles at three sites. It has two strands of evidence.
Strand 1, a short survey. In Term 2, the organization asked participants still attending in week 6 to answer a few agree-or-disagree items. Fifty-five of the 70 still attending answered. People who had already left were not surveyed. Item results: 44 of 55 agreed the session time suits them; 49 of 55 agreed the reading level is right for them; 33 of 55 agreed they feel safe traveling to and from sessions. There was no item on caring for children.
Strand 2, interviews. In Term 1, at one site, six participants were interviewed, four who had left and two who stayed. Their accounts are traced in the other worked case: a clash between the start time and work or care for several people, difficulty with the material for one, and concern about the walk home in the dark for two.
The joint display lines these up by question.
| Question | Survey (Term 2, Three Sites, Still Attending at Week 6) | Interviews (Term 1, One Site, Four Left, Two Stayed) | Relationship | Not Yet Known |
|---|---|---|---|---|
| Is the session time a problem? | 44 of 55 agreed the time suits them | Several accounts describe a clash with work or care (E1, E3, E6); one stayer arrives late (E2) | Conflict, with a likely coverage reason | Whether leavers in Term 2 would answer like the Term 1 leavers |
| Is the material too hard? | 49 of 55 agreed the level is right | One leaver left because it was too hard and did not want others to see (E5) | Conflict, with a likely coverage reason | How many leavers felt this; a survey of still-attending people cannot show it |
| Is travel safe? | 33 of 55 agreed they feel safe | Two accounts raise the walk after dark (E4, E8) | Partly agree | Which participants, which route, and what time |
| Do care duties limit attendance? | No item | Raised by two leavers (E3, E6, E7) | Silent in the survey | Whether it applies at the other two sites |
Reading the Joint Display
The rows show several things at once.
Different people. The survey reaches people who stayed to week 6. Four of the six interviewees had left. A survey of those who remained cannot show why people left, so the first two rows are not a direct clash. Fifteen of the 70 still attending did not answer, and the survey cannot say how they would have answered. The strands may both be accurate for the people they cover.
Different periods and places. Term 1 at one site and Term 2 at three sites. Changes between terms, or differences between sites, could explain part of the gap, and neither strand tests that.
Different questions. "The time suits me" is a judgment by someone who has already adapted. The interviews describe a clash that led some to leave. The wording of the items shapes what the survey can show.
Silence is not absence. The survey has no item on care. That is a gap in the instrument and says nothing about how common care duties are.
What the display cannot tell you. It cannot show that the survey is right, that the interviews are right, or how common any reason is among all participants. The interviews were a purposive set and the survey a count among those who stayed. The display shows what each strand can speak to.
Handling Conflict
What to Do with Conflicting Results
Fetters and colleagues describe confirmation, expansion and discordance, and suggest handling discordance by looking for sources of bias, examining methodological assumptions, gathering more data, reanalyzing existing data, drawing on theory, or questioning whether the constructs match. Farmer and colleagues' triangulation protocol similarly addresses completeness, convergence and dissonance. Apply them to the case:
Check who each strand covers. Here, a main source of conflict is who was surveyed.
Check what each measure means. "Time suits me" versus "time was why I left."
Look for what would settle it. A short exit survey for those who leave, using the same items, would speak to the people the first survey missed. A care item could be added.
Keep the unresolved rows unresolved. If you cannot explain the conflict, say so.
Do not give the larger strand the last word. A larger count is not a more complete view when it leaves out the people of interest.
O'Cathain and colleagues' triangulation protocol, following a thread and mixed methods matrix offer ways to carry out these steps. Moran-Ellis and colleagues remind us that combining methods and the claims made for the combined knowledge are separate matters: how the data were combined does not by itself settle what the conclusion can claim.
Reporting the Meta-Inference
A meta-inference is a conclusion drawn across both strands. Write it so it carries the limits: "Among participants who remained to week 6, most agreed the session time suits them. Interviews with leavers in Term 1 at one site describe a clash with work or care as a reason for leaving. These findings are consistent with a timing problem that mainly affects people who leave, but this cannot be shown with the present evidence because the survey did not reach leavers. We recommend adding the same items for leavers." That sentence is less tidy than "timing is the main issue," and it is what the evidence supports.
Using It
Checking an AI-Built Display
A model can assemble a joint display from a survey table and interview themes quickly, and tends to produce a coherent narrative. Check it for: population and period columns that were left out; a conflict presented as agreement; silence presented as a finding; invented numbers; and a meta-inference that is stronger than either strand. Open each cited excerpt and each table figure. See checking AI-assisted qualitative coding.
Common Mistakes
Putting results in one column. Without population and period, readers cannot see why strands differ.
Treating agreement as proof. Two sources can share the same blind spot.
Reading silence as absence. Missing items are limits of the instrument.
Averaging. A midpoint between "most agree" and "some left because of it" is not a finding.
Explaining the gap with a story you did not test. Label it as a possibility.