Involving citizens and patients in research from " how to " to " how? like this! "

Written by: Christiane Grünloh, Ria Wolkorte, Rita Schriemer


As soon as the calls for research arise, the topic of citizen and patient involvement comes up. Without a letter of support from a patient organization or interest group, your application has little chance. And once you have received the letter of support, how do you properly secure the involvement? Participation is not only a matter of ticking off by patients, but also of sparking researchers. Within ICMS (see description below), three colleagues from their own field are working on implementing active and meaningful involvement. Together they spread their knowledge, experience AND conviction. 

The principles of value-driven care and Shared Decision Making are self-evident in the clinic. Can we say the same of the research that precedes healthcare? Are patients, citizens or end-users involved in ICMS studies? The answer is yes. But it can be even better. If it were up to these colleagues, it will become the gold standard that members of the public, be it patients, citizens or end-users of healthcare and technology are involved in all research we do in ICMS. And not just as a participant or tester, but that they have a voice in the design and choices made in the research. Christiane Grünloh (Roessingh Research and Development), Ria Wolkorte (University of Twente) and Rita Schriemer (Sint Maartenskliniek/Radboudumc) share their point of view.

Christiane: “For the three of us, the need to get people actively involved in your research is crystal clear. It is necessary if you want to create impact. You only have impact if your research is relevant. You will achieve relevance if your research addresses the most important questions and problems of the target group. Or when the outcomes, technological or social, contribute to the quality of life.” Ria: “We involve patients in the daily practice of science. They help make decisions.” All three of them do this within their own disciplines and in ways that are customary in their field. Rita: “In clinical research we call this patient involvement. Citizen Science is the term and approach applied to health research at the UT. At Roessingh Research and Development we speak of Human Centered Design.”

Christiane: “These are different concepts and interpretations for approaches that have the same point of departure, namely that the people for whom you conduct your research are also the people with whom you conduct the research.”


Patient Involvement in clinical and fundamental research

Rita: “In clinical research, it is increasingly becoming the standard to actively involve patients in research. Where people were first only involved as participants, patients now also contribute within other phases of the research. This can already be done at the time of the application. Patients can contribute to the relevance of the study and comment on the layman summary. But if you start the interaction earlier, their input can lead to focus differences because certain areas are more relevant to them in daily life than others. We recently experienced this in an application for osteoarthritis research. Two highly engaged patients even attended the interview with the funding organisation. For them, the main reason for committing to the study was the research's holistic view of the disease. They know better than anyone that their condition involves a combination of medical, biological, social, movement and behavioural aspects.

Basic researchers often think that their field of expertise or research is too difficult for laymen, or that involving outsiders is of little use. Nevertheless, the PhD students on rheumatism research at the Sint Maatenskliniek and at the experimental rheumatology lab at Radboudumc all have a patient representative who follows most of the research process. They then have to explain to him or her in layman's terms what they are investigating, how they do that and what the results of the investigations are. In our experience there is a role for patients in every research phase. Patients and members of the public can contribute ideas at different levels and in different research phases. These partnerships are highly valued by both parties.”


Citizen Science

Citizen science is mainly known from ecological research, for example, counting birds. But this view is too limited. Citizen science has been widely applied in health research in recent years. People with a chronic condition are often already looking for solutions to make their lives more pleasant. It is precisely this motivation and self-inquiry that you can use in research. Ria: “In Twente we started working with a group of people with rheumatoid arthritis according to the principles of citizen science. Actively involving citizens in your research means a different division of roles than the traditional one between patient and researcher. With the people involved we made agreements about data management and open science. Who owns the data and who can access it? Only the researcher? Or is the data also from the participating citizen themselves. How should access be arranged? These were nice conversations with clear agreements, which do normally not take place with the participants in research.

Based on the choice of people with rheumatoid arthritis, our research theme was formulated quite loosely around fatigue. Together with the participants, whom we call co-researchers, we have further shaped this. We found that it was precisely the capriciousness and unpredictability of fatigue that they wanted to understand better. As researchers, we turned this into a research question “Which factors are associated with experienced fatigue in people with rheumatoid arthritis?”. The co-researchers then decided that it was important to take a longer-term look at the degree of fatigue and factors that can influence it. To this end, they determined the maximum number of measurement moments (namely: 1) per 24 hours, as well as the duration of the measurement (namely: 21 days). This is more limited than we as researchers had in mind, but the feasibility of the research was also important to us. However, it is not only the researcher or the citizens who decide, it is a collaborative process As researchers, we naturally continue to look at what is scientifically justified, and we remain responsible for that. But there is much more a principle of joint decision-making and transparency.”


Participative health technology design

To develop technology that has a high usability and creates positive user experiences, , it is important to know what needs and goals people have, what their characteristics are, what tasks they have and in what context they will use the solutions. This is the starting point for creating initial ideas for possible solutions, which are then continuously tested and improved together with the users.

Christiane: “We recently started a study about eHealth for people with COPD. One important goal for the clinical researchers is to develop a prediction model that predicts when symptoms are getting worse for the patient. To do this, patients have to collect data for a longer period, for example, with a wearable and by answering questionnaires . But by talking with patients, we learned that people also want something in return for providing their data. They also expect feedback from their personal measurements. Data collection alone is not helpful to them, but rather can be a burden, especially for patients with low energy. This is important to take into account when developing the app, because otherwise people will stop using it in the long run and then clinical researchers will not have enough data for their prediction model. We already know these kinds of comments from human-centered design, or from action research, yet I keep seeing this blind spot. For me, research without the active input of end-users and stakeholders is unconceivable. In this way, we bring together all the relevant perspectives and expertise, and learn about the needs of the different user groups, to create health technologies that are useful for all of them.”

Together we can make research more relevant and better. And beyond that, patient involvement, citizen science and human-centered design keep the conversations between science and society going.

Involving citizens, patients and end-users is not "rocket science", but it does require a certain responsibility and care from the researcher. If you have questions about this, we can aid, educate and guide you towards involvement in your design and implementation of the research. You will see that it is very rewarding and becomes self-evident. Would you like to know more about how to involve people in your research? We are happy to think along with you at an early stage!


f.l.t.r: Ria Wolkorte, Rita Schriemer, Christiane Grünloh


Ria Wolkorte

Rita Schriemer

Christiane Grünloh


ICMS (Interdisciplinary Consortium for clinical Movement Sciences & technology) is a unique partnership between the Sint Maartenskliniek (a specialized hospital entirely focused on movement disorders), the Radboud University Medical Centre, the Radboud University, the University of Twente, Roessingh Research & Development (RRD), Roessingh Rehabilitation Centre working together with science driven companies (national and international).

Involving stakeholders as co-researchers: Taking a first step in how to train them

Written by: Kira Oberschmidt


For almost three years now, RRD has been involved in the Pharaon project (Horizon 2020, grant agreement No. 857188). In Pharaon, we use action research to integrate technologies for smart and healthy ageing. At the end of the project, RRD is going to provide guidelines for action research that can be used in other projects.

In our work, we notice that the involvement of non-researcher stakeholders (for example patients and healthcare professionals) becomes increasingly important. Especially in participatory projects that make use of approaches like action research or citizen science, simply involving these stakeholders as participants is not enough. Instead, they should take on a more active and empowered role. However, as there are not formally trained as researchers, more knowledge and preparation might be required.

We see that stakeholder skill training is occasionally mentioned in literature as a way of preparing stakeholders to become co-researchers. However, little practical information about such training is given. This left us wondering:

  • What is being taught?
  • Who are the recipients of the training?
  • When do training activities take place?
  • What are identified training needs (why do we need this training)?
  • How is the training set up and which methods are used?


Workshop at NordiCHI 2022

To answer these questions, we decided to invite researchers with an interest in stakeholder skill training to a workshop. This session took part on 9 October 2022, as part of the pre-conference program at NordiCHI 2022. NordiCHI is a biannual, international conference on Human-Computer Interaction This took place in Aarhus, Denmark this year. The topic of this year’s event was Participative computing for sustainable futures.

During the workshop, our group of organisers engaged in some very interesting discussions with the participants. For example we talked about:

  • How to find a common language between stakeholders?
  • How can we identify training needs?
  • Is training even the correct term for such activities?
20221026 NordiCHI (4)

While working in smaller groups we also took some time to identify barriers and facilitators of good stakeholder skill training. (We were unable to settle on a better term, so for now we kept calling it ‘training’). While there are still a lot of open questions, it was nice to see that many participants took inspiration and ideas for their own practice from the session.

20221026 NordiCHI (1)
20221026 NordiCHI (2)
20221026 NordiCHI (3)

We look forward to continue this discussion with others. Would you like to learn more about this topic? Feel free to reach out to Kira or Christiane.

Kira Oberschmidt

Kira Oberschmidt           


Tel: 088 087 5767

Christiane Grünloh

Christiane Grünloh                        


Tel: 088 087 5723

Extending CE certification under the Medical Device Regulation (MDR) to a new 'intended use' - A real-world example

In recent years, Roessingh Research and Development (RRD) has worked intensively to coordinate a multi-center intervention study. This intervention consisted of a soft-robotic glove, called Carbonhand, which was developed to support hand function for patients with reduced hand pinch strength. Bioservo, the manufacturer of Carbonhand and sponsor of the study, had the intended goal of extending CE certification of the system within the MDR to a new medical application. Carbonhand already had a CE for its operation as an aid in daily tasks. However, the new "intended use" is focused on the therapeutic effect of Carbonhand, or improving hand function after long-term use of the system as an aid during daily activities. RRD was therefore asked to design and conduct a clinical evaluation that could demonstrate whether or not that therapeutic effect is there.

Carbonhand systeem

For this, RRD set up an appropriate research protocol according to the Good Clinical Practice (GCP) guideline, an international ethical and scientific quality standard (ISO14155) for the design, recording and reporting of human-related research. In addition, RRD realized the approval at the Medical Ethics Review Committee (METC), we realized cooperation with 8 Dutch (rehabilitation) centers and trained all involved in the application of the Carbonhand system and the implementation of the research protocol. A monitor from an external party was involved to check the implementation of the GCP guideline and all collected data.


Intensive cooperation leads to success

Within this study 63 participants were included with reduced hand squeezing strength, independent of underlying diagnosis, who used the Carbonhand during a period of 6 weeks at home while performing daily activities. Three baseline measurements took place beforehand, followed by a post-measurement (within 1 week of intervention) and a follow-up measurement (4 weeks after the end of the intervention). Despite intervention of the COVID-19 pandemic, we managed to conduct the study successfully, not least due to the intensive and pleasant cooperation with the participating centers. On Tuesday, September 27, the final measurement took place and the data collection was officially completed. Of course, we could not have reached this milestone without the commitment of the clinical centers: Roessingh Center for Rehabilitation, University Medical Center Groningen, Isala clinics, Rijndam rehabilitation center, Reade center for rehabilitation and rheumatology, Hoogstraat rehabilitation, Sint Maartenskliniek with the location Canisius Wilhelmina Hospital, Klimmendaal.


In the coming period we will focus on the analysis of all outcome measures. Interim results were promising, now we need to see if this remains so after analysis of all data. If you are interested in the results of the study, keep a close eye on our news page for the latest developments.


Do you recognize the above process as a medical device manufacturer and have a similar issue? RRD is happy to advise and support you in the design and implementation of your clinical evaluation under the MDR!


Anke Kottink-Hutten

Tel: 088 087 5733

Looking back at RehabWeek 2022

It was a special RehabWeek this year. It was the first in three years because of the covid pandemic. It was also my first physical conference since covid, and the last one prior to all the lockdowns was RehabWeek 2019. So we came full circle last week. And it was 'at home', in the Netherlands. It felt somewhat like a reunion. And whether it was a psychological post-covid effect, or the longer 3-year period in between two RehabWeeks, or just that a lot of developments are happening, it felt like more news was shared than normally, there was more to catch up on with people, and more plans discussed for future collaborations. All in all, it was great to meet our colleagues again and talk about how they are doing and to hear about their latest developments. The fact that the social programme also involved water cabs and the SS Rotterdam topped it all off.

RehabWeek 2022
Groepsfoto RehabWeek

Showcasing RRD at Rehabweek

When Rotterdam was announced as the location for the next RehabWeek (2021, originally) at the previous conference in Toronto, we said we should have a good representation of RRD at this edition. Well, we kept our promise. RRD presented in 6 workshops, Hans Rietman delivered a keynote and was expert panel member, Erik Prinsen and Hans Rietman were involved in parallel programme of ISPO-NLand NSRM, we had 4 presentations in various conference sessions, RRD chaired 5 sessions, and had 3 posters on display. The poster from Robert Schulte was nominated for Best Poster Award and he pitched his research findings on smart signal processing from the MyLeg project. Additionally, Claudia Haarman presented her work in one of the ICORR sessions about GRIP, a collaboration with Anke Kottink, where Hankamp Rehab and RRD evaluated a very small and smart wearable device that supports key grasp for people with spinal cord injury, for example.

Presentatie RehabWeek Robert Schulte
Presentatie RehabWeek Cindy Rikhof

Inspirational topics

In between latest developments and research findings of rehabilitation robots, electrical stimulation, VR, eHealth applications etc., in various combinations, there were a few transdisciplinary topics that came back in many presentations and discussions. A lot was said about the need and benefits of involving end-users, either patients or therapists or other stakeholders, in early stages of technology development. Although this is definitely not news to RRD (glad to hear that we've been doing the right thing all along), it is nice to see this catching on across the RehabWeek audience. It was fitting that Stephanie Kosterink presented (twice) about using the Societal Return on Investment (SROI) method to elicit the impact of technology in a broad sense, including quality and process improvements. One of our examples for an SROI was the Innovation Lab of Roessingh Centrum voor Revalidatie, which gained a lot of interest as a means to implement rehabilitation technology in clinical practice and to gauge its impact. Another topic that stood out was the Medical Device Regulation (MDR). Its challenges was a frequent topic of discussion on the exhibition floor with developers of rehabilitation technology. Besides this, we had two presentations about it's practical implications for research with medical devices; once in relation to safety requirements for rehabrobots and using instrumented dummy limbs to solve this, once in relation to following MDR during a multi-center clinical evaluation of the effect of a soft-robotic glove (Carbonhand) used unsupervised at home. We hope our contributions gave some inspiration for how to deal with the discussed challenges.

Looking forward to the next RehabWeek, next year in Singapore!

Are you interested in the above mentioned topics, but missed our contributions at Rehabweek?
Feel free to contact us! Tel +31 (0)88 0875 777 / E-mail:

Cheers, Gerdienke

Foto Gerdienke Prange

Evaluating E-Health: How do you go about it?

In practice and in science, there is still much ambiguity about how to properly evaluate eHealth. Especially when the technology (underlying the eHealth application) is still in development. Roessingh Research and Development (RRD) has years of experience in guiding such developmental processes, including the evaluation of innovative eHealth technologies. Under the leadership of senior researcher Stephanie Jansen - Kostering, RRD has developed its own method for eHealth evaluations and published it in Digital Health.


Continuous evaluation of eHealth

RRD's approach is characterized by the continuous evaluation of eHealth starting from the maturity of the technology with attention to different perspectives. These perspectives are; the user perspective, the clinical perspective and the societal perspective. The maturity of the technology is determined by the Technology Readiness Levels (TRLs). The 9 TRL levels (TRL 1 = low and TRL 9 = high) can be distinguished into three phases; TRL 1 to 3 is the research phase, TRL 4 to 6 is the development phase and TRL 7 to 9 is the implementation phase. In the research phase, for example, the user perspective is central and formative user testing in a laboratory is important to have a good alignment between the user and the technology. The paper gives several examples of studies in the different TRL phases.

E-Health evalueren

Method of e-Health Evaluation

The RRD methodology for e-Health evaluation is published in Digital Health. Interested? Read the publication here: "Time to act mature - Gearing eHealth evaluations toward technology readiness levels"

Are you or will you be conducting an evaluation study of an e-Health application?

RRD is happy to think along with you. You can contact us for advice and/or to conduct evaluation studies.

For questions about evaluation studies, please contact Sr. researcher Stephanie Jansen - Kosterink (  - 0880 875 717).


PhD Defence of Marian Hurmuz: eHealth - in or out of our daily lives?

While there are many different eHealth services (being) developed, its use among the target population is still low. Marian Hurmuz aimed within her PhD to increase our understanding about the (non-)use of eHealth services among the target population in a real-world setting. After a bit more than 3 years, she finalised her PhD thesis and, last week Thursday, she had her PhD defence.

Her thesis is available online on our website here. The studies described within this thesis cover the following topics:

  • Exploring demographics and personality traits of older adults which can predict dropping out of an eHealth service.
  • Investigating which determinants of the Technology Acceptance Model explain older adults' use and intention to continue using a gamified eHealth service.
  • A case study which shows how you can evaluate an eHealth service in a real-world setting with mixed methods.
  • Qualitatively investigating barriers and facilitators adults with neck and/or low back pain perceive when using an mHealth app.
  • Identifying the reasons of potential end-users to participate in eHealth studies, the influence of these reasons on the use of eHealth, and their expectations about these studies.

This thesis ends with a general discussion about the main findings. This discusses the use of eHealth and the intensity of use among different eHealth services, the different aspects which can be used to measure eHealth use, how eHealth use can be improved, recommendations on improving summative eHealth evaluations, and finally it discusses topics for future studies.

We are very proud of Marian that she completed her work at RRD in this fantastic way. In recent years she has done a lot of work in various European projects and we are grateful for that.

With the ends of her PhD, we are looking forward to see how she will go on working on this important topic!

MicrosoftTeams-image (4)

Festive farewell to Hermie Hermens: Twente pioneer in care innovation

On April 1, Roessingh Research and Development (RRD), together with many (former) RRD staff members and other colleagues from the Roessingh group, inaugurated the retirement of Hermie Hermens. Hermie was (together with Gerrit Zilvold) one of the founders of RRD and has since its inception been one of the pillars on which the current success of RRD is built.

In all these years, Hermie has contributed to leading research in various fields (from electromyography to serious gaming, and from prosthetics to clinical decision support systems).

It is thanks to Hermie that RRD is a well-known player within the European innovation landscape; he has also put RRD on the map at the local level, for example through his appointment as professor in the Biomedical Signals & Systems group at the University of Twente, and as co-founder of organizations such as Vitaal Twente and the Technology and Care Academy. The fact that everyone in Twente involved with healthcare technology knows who Hermie is shows how great the impact of his work has been and how closely he has been linked to the region.

His farewell was centered on the things Hermie has always attached great importance to: interest in your colleagues, good food, and star wars. For the time being, Hermie will remain connected to the University of Twente for a few days a week.


 © Copyright - Roessingh Research and Development