Participant recruitment is a crucial success factor in research on human subjects, yet its significance for the validity of research outcomes is commonly glossed over, at least in public.
“Recruitment is an issue that tends to be underplayed”, commented Kristrún Gunnarsdóttir when looking back at some of the hurdles faced by the HomeSense field trial.
“… and it is something that should be more openly discussed.”
The reasons, you could say, are understandable. In a world of limited resources of time and budget, recruiting participants will always be a challenge. As with any form of recruitment, it also requires a deep appreciation of the motivations in play to achieve commitment to the cause. Above all, failures in the recruitment stage lead to research outcomes that are either poor (lack of participant commitment, biased sample) or incomplete (not many enough participating).
In a 1990 paper, Participant recruitment for qualitative research, published by the Society for Applied Anthropology, the authors argue that a few qualitative researchers discuss recruitment methods.
The issue has become more prominent within research communities in recent years, especially in relation to clinical trials. In a 2014 paper on Factors influencing recruitment to research, Newington and Metcalfe, identify the key influences being the infrastructure supporting the research, what the research is about, and what characterises both recruiters and participants. The main reason for accepting participation, they conclude, is altruism, i.e., serving the greater good, while the logistical aspects of participating (e.g. the burden) appear to be the main reason for declining.
In our previous article Taking stock by looking back at the HomeSense field trial, we already describe some of the methods we used to recruit participants. In this article, we go further into some of the more subtle communication lessons.
Wider issues of research recruitment
Increasing participation is a key area of focus in research within the NHS, for facilitating evidence-based policy, improving health outcomes and reducing health inequality. A review of the views of researchers and clinicians involved in clinical research found participant recruitment vital to the success of research studies, while many research projects fail to recruit sufficient numbers of participants.
Similarly, a 2015 North American Primary Care Research Group study reported in the University of Alberta Health Sciences Journal, found that researchers frequently underestimate the time and resources required for participant recruitment while overestimating their availability, willingness and eligibility.
Such misalignments persist, the study claims, in part because there are few benchmarks to estimate the resources required for successful recruitment.
In addition to not achieving target participation numbers (marking studies as incomplete), the underlying issue is under-representation, and consequent bias leading to unrepresentative findings or poor outcomes.
For example, a 2018 survey prepared for the American Association for Cancer Research Annual Meeting, claims that minority groups are under-represented in trials, but want to participate. The study highlights disparities in recruitment, retention and trust, particularly among adults of low socio-economic status, racial and ethnic minority groups and those who live in rural areas.
Likewise, another recent medical study (of patients with complex medical and social needs) found that research recruitment is made more difficult because of mistrust, travelling issues, or because data collection could seem too intensive.
That said, recruitment is not nearly always about numbers or some universal qualifier of representativeness. Qualitative studies typically rely on a few targeted sources or even a single case to be studied in-depth to obtain so-called ‘rich’ data. The primary goal of recruitment in qualitative research is always study-specific so recruitment will first and foremost have to meet the singular objectives of each study design.
It bears keeping in mind in this respect, that common large-scale surveys using population sampling cannot claim to be truly representative of that population. Other factors are always in play, mostly unaccounted for, such as the motivations of those who actually accept or ‘self-recruit’ as opposed to those the recruiters never reach. In other words, there is unavoidably one bias or other in any type of recruitment, which is something researchers live with and they ought to be clear and honest about it in writing up their findings.
Perhaps the most prominent question is why anyone should wish to take part in research.
The HomeSense field trial experience
The HomeSense field trial was designed to trial a method of using sensors, the key research question being to what extent this method is (or is not) of value to social research. Other possible outcomes and curiosities were not really addressed, for example, if sensors in homes are feasible (or acceptable) for assistive purposes in behaviour intervention relating to energy use, self-care and health care, for improved security or other means. The topic only came up in open discussions with participants, in speculating about the future and expressing their views.
The recruitment was not meant to be representative either, except in the sense of loosely matching three categories of UK households: 30% lone occupancy; 30% with children/teenagers (0-18yrs old); and, 40% occupied by two or more adults. The target number was twenty household and it was very important for the study to recruit participants willing to take the whole process seriously. So, achieving the target number and ensuring ‘good’ participation were key issues, but if participating was based on altruism, it was to support new developments in social research in good faith that such research is actually valuable. Otherwise, participants will have misunderstood. Most of them cited interest in the technology and desire to be of assistance.
A total of 60 respondents expressed interest in participating. They got in touch with us and asked for documentation about the study and, thereby, engaged in communication, asked questions and so on. Twenty of these eventually agreed to go through with an initial meeting in their homes for interview and demonstration of the technology, and thereafter have sensors installed. One household withdrew after the data collection had already started, the cited reason being subsequently learning the concerns of family members living abroad.
One could say that the first stage of recruitment — advertising and receiving initial expressions of interest — ‘funnelled’ the process down to the target number of 20. Two-thirds of those who got in touch with us dropped off the radar at some stage in the process of exchanging information, asking and answering questions, and setting up interviews and demonstrations. One-third went through with inviting researchers into their homes.
But this filtering metaphor doesn’t tally with the realities and niceties of the communications that resulted in the target number of twenty. For instance, the team learned early on how long this was taking and the likely recruitment shortfall, which led to concerted efforts to be rather more assertive in their initial communications.
Much more administrative effort than expected was required by the research team to gain the required commitment. More often, than not, to no avail.
“A lot of people got in touch with us and seemed really curious”, said Kristrún.
“Part of the protocol that we followed was that people had the right to stop communicating with us at any time they liked, without having to explain why. But you do get to be curious about why some people changed their minds.”
“They did and, from our point of view, just seemed to vanish”.
A cautious approach at the start
“We had to shift tactics over time, but as a researcher, each time you start a new field trial you are entering uncharted territory”.
“We were using potentially very intrusive technology, and the use of sensors was new to both us and almost all the participants.”
The initial approach to recruitment was to be cautious and not force the pace of the communications.
“People initially responded to learning the project existed from our flyers or having seen us demonstrating at the Festival of Wonder. Plus, we posted flyers through letterboxes, and around campus.”
“We’d then be contacted either by e-mail or phone, in response to which the next step was to send a project information package and consent forms so that prospective participant could see what they’d be letting themselves in for.”
“In that reply I would open up with, ‘Thanks so much for expressing interest’, asking to read carefully, share with other household members, if any, and added that, ‘I’d be happy to discuss any aspects they might wish to inquire about further’. Then I added, ‘You can email me or call me’.”
This approach, claimed Kristrún, “was very much about opening the door to participation”.
The information package was designed to be comprehensive, understandable and transparent about what was intended to be done, including the purpose of the research, and about the devices to be used.
“Our letter would typically go on (very cautiously) to say, ‘In the event that you’ve agreed to take this forward, we hope we can make arrangements soon for an interview and a demonstration of how the sensors work’. I attached a consent form and a contact form, adding ‘If you’re not in a position to print these out, I can post them or bring them with me when we meet’.”
“I then would write, ‘I look forward to hearing from you soon’, or something like that. So our approach was very much taking a cautious tone, and in a few instances I wouldn’t hear anything back at all.”
Following up on silence often led to continued silence
After a wait of a week or two sometimes, we would follow-up on those that didn’t reply.
“Some people would actually just tell you straight to your face, basically, ‘Sorry, this is not for me’.”
“Otherwise, the silence is quite baffling because then you really don’t know the reason why”.
Misconceptions spark intrigue
“Some responses that we received might take the form, ‘Oh, I’m really interested in this. My missus wanted to know a bit more detail on the data to be captured and what the purpose of it is. Can you please give me some detail on that and how the data is planned to be used? Is there any sample of the data that can be shared?’….”
Such replies, where prospective participants admitted concerns or looked for confirmation that they’d understood, could be translated into a positive way forward.
“On the other hand”. said Kristrún,“they might simply say, ‘Okay. No, it’s not convenient’ Or they came up with some diplomatic way of backing out after some back-and-forth.”
“However, conversations that touched upon misconception or misunderstanding are very much among those who actually went through the trial with us.”
“When we got to the stage that a participant signed up and we organised the first visit and demonstration of the technology – and this is before installing it – we’d have a conversation about what it does and what it doesn’t do, and how we see the data and make sense of it. That is when we’d come across misunderstandings. Or, say, people would say, ‘Oh, okay. I didn’t realise that’. Or they would realise something about what we were doing as we demonstrated it. That confirmed to us just how important demonstrations were as part of the consent process.”
Commitment follows consent
For both ethical and practical reasons, it’s difficult for the busy researcher to spare the headspace wondering why potential research participants fail to follow through on initial expressions of interest and even some further exchange.
It would require a separate study, a different ethics agreement, the response rate of which might, understandably, be meagre. “It’d be interesting research to do for the HomeSense project”, pondered Kristrún, “but we don’t really have the right to ask why, in the end, they didn’t want to be a part of the project”.
The recruiter might personally wonder about the silence, but the job to be done is to ensure the approach is valid in terms of meeting the objectives of the field trial and following ethical protocol.
To this end, Archibald and Munce proposed some strategies to facilitate recruitment for qualitative research, with ‘achievability’, in their view, coming down to time and budget.
So, the recruitment process for the HomeSense field trial may have seemed discouraging initially, but dressing up this two-thirds failure rate as a one-third success rate could be much like a glass half full so long as sufficient ‘good’ participants were recruited in the end.
Balancing caution and pressure
Advice from the Market Research Society is that methods of recruitment must always evolve, while the GradHacker blog recommends that researchers keep communications simple, and human.
The key lesson to take from the HomeSense experience, concluded Kristrún, is about finding a balance between over-caution and gently encouraging participation without losing sight of genuine commitment and understanding of what participating is all about.
“As a trained observer, I notice when people have learned something, by how they ask questions. I’m trained to detect the shift from having a loose idea and not being sure, and then ‘seeing’. A good teacher is trained to observe that, to be able to look for evidence of learning in interaction and verbal exchange. So, it’s important also for the researcher to feel confident that the consent that they get is genuinely informed and you can get to that point where you are confident that people actually understand what they’re signing up for. That obviously excludes those who vanish or brush you off with something polite, such as ‘my missus doesn’t want to do it’, or, ‘my husband’s against it’.”
A shift in approach started as the need for more participants became apparent.
“I know how to suggestively guide people into decision-making very quickly, and we needed to speed up the process, so I gradually became slightly more assertive. You do that by drawing people closer to a decision, saying things like, ‘We’re organising interviews next week. I have an opening on Thursday in the afternoon, and all day Friday if that suits you’.”
“You set these little ‘hooks’ to encourage taking the available time slots, so that does press the person to get this over with and make a decision, yes or no.”
Any approach like that is risky, however, and potentially unethical. It is common knowledge that the shorter the time people have to make a decision, the more likely it is that they go along with what’s being suggested to them. But, academic researchers are not sales people, so one should ask, what is the balancing act in getting this right?
“We started trying to speed up the process and get people to sign up or not as quickly as possible. And there you have a downside. If a person agrees to participate without having ‘all the time in the world’ so to speak, chances are that their level of commitment is wanting, so the trial eventually suffers if then participants are not taking their participation seriously.
“It’s a balancing act. You have to commit people to take seriously each step along the way, including the ongoing communications over a number of weeks. Sensors would come offline, for example. The electricity monitoring sometimes had to be rebooted, and things like that. Agreeing to two open-ended interviews takes up time and effort and may not be comfortable, and filling in a 4-day time-use diary is quite a commitment. You want participants to be really well-prepared and happy with the process.”
To sum up, research recruitment is difficult in so many ways but it cannot be approached like a sales funnel. Although it might have seemed like a cat-and-mouse chase just to time-table the initial appointments for the HomeSense field trial, the team had to be very careful avoiding pressurising potential participants, and be very patient in all aspects of recruitment and retention, “because you absolutely cannot come across as a salesperson”.