Skip to main content

Citizen involvement in research on technological innovations for health, care or well-being: a scoping review

Abstract

Citizen science can be a powerful approach to foster the successful implementation of technological innovations in health, care or well-being. Involving experience experts as co-researchers or co-designers of technological innovations facilitates mutual learning, community building, and empowerment. By utilizing the expert knowledge of the intended users, innovations have a better chance to get adopted and solve complex health-related problems. As citizen science is still a relatively new practice for health and well-being, little is known about effective methods and guidelines for successful collaboration. This scoping review aims to provide insight in (1) the levels of citizen involvement in current research on technological innovations for health, care or well-being, (2) the used participatory methodologies, and (3) lesson’s learned by the researchers.

A scoping review was conducted and reported in accordance with the PRISMA-ScR guidelines. The search was performed in SCOPUS in January 2021 and included peer-reviewed journal and conference papers published between 2016 and 2020. The final selection (N = 83) was limited to empirical studies that had a clear focus on technological innovations for health, care or well-being and involved citizens at the level of collaboration or higher. Our results show a growing interest in citizens science as an inclusive research approach. Citizens are predominantly involved in the design phase of innovations and less in the preparation, data-analyses or reporting phase. Eight records had citizens in the lead in one of the research phases.

Researcher use different terms to describe their methodological approach including participatory design, co-design, community based participatory research, co-creation, public and patient involvement, partcipatory action research, user-centred design and citizen science. Our selection of cases shows that succesful citizen science projects develop a structural and longitudinal partnership with their collaborators, use a situated and adaptive research approach, and have researchers that are willing to abandon traditional power dynamics and engage in a mutual learning experience.

Peer Review reports

Background

Many technologies are developed to address health, care or well-being related challenges, such as mental health, manage health at home, or coping with an understaffed healthcare sector [1]. These technologies can include new diagnostic or therapeutic methods and devices, ambient assisted living technologies, and eHealth such as monitoring, regulatory, or advisory apps. There is a lot of research into the design, development, implementation, and evaluation of new technologies. However, the scoping review of Krick et al. showed that implementation of new technologies remains a challenge due to lack of in-depth knowledge about the care environment or users [2].

Citizen science, or the use of scientific principles and methods by non-professional scientists, may be a powerful method to improve public participation in research as well as public health and is regarded as a promising research approach to improve implementation and adoption of technological developments in health, care or well-being by collaborating with the intended users as experience experts [3]. The initial drivers to develop citizen science methods were the increase of research capacity [4], and the possibility to respond more effectively on complex societal problems due to the addition of lay, local or traditional knowledge to scientific knowledge [5, 6]. Besides the positive contributions of citizen science to the scientific processes or knowledge, it has potential advantages for lay people being co-researchers of a project. In health related research as co-researcher could result in a broader understanding of their own health and well-being and acquisition of knowledge about a healthy lifestyle [7]. A list of 10 benefits for co-researchers was developed by Haywood [8]. These benefits include enhancing science knowledge and literacy, enhancing understanding of scientific methods, improving access to scientific information, increasing scientific thinking, improving interpretation skills, diminishing the gap between science and people, strengthening connections between people and environment, empowering co-researchers, increasing community-building, changing attitudes, influencing policy, and gaining access to broader networks [8]. King et al. showed similar benefits in several citizen science case studies, where the main benefits concerned scientific literacy [9].

Currently, there is no uniform/ broadly accepted definition of citizen science. So far there is no consensus, and it is even discussed whether a definition is necessary [10, 11]. This non-existence of a definition is accepted and allows stakeholders working in citizen science to use different terms and methodologies [12]. In this paper we define citizen science as research (partly) executed by non-professional scientists, volunteers, citizen scientists or co-researchers. Often the citizen scientists work together or are guided by academic researchers or governmental organisations. Thus, a citizen should be taking active part in any of the research activities and ideally collaborate with professional researchers. We take a similar perspective as Heigl et al. and exclude projects from our definition that use participants only at as providers and resources of knowledge without any active involvement in scientific activities [11].

Citizen science is an approach which comprises a range of participatory approaches that are embraced as collaborative practices and show promising results [3]. Although citizen science in the healthcare domain is a relatively new and rare phenomenon, it is already an established approach in research fields such as ecology, conservation, and biology [5, 13]. Projects with citizen science approaches attract hundreds of thousands of participants, who are involved in different tasks varying in complexity [14]. The inclusion of citizens is valuable for science [15], for example, when the expertise of the general public or a specific population of citizens is needed to understand and solve a problem [16]. The collective intelligence of the citizens has led to scientific discoveries such as protein folding [17] and air pollution [18]. However, citizens science projects also deal with challenges when it comes to the selection of participating citizens, the needed and available competences of citizens, and the credibility of knowledge gathered by or with the citizens [5, 19].

TOPFIT Citizenlab is a three-year research and innovation program aiming at increasing citizen involvement and participation in researching, testing, modifying and implementing technological innovations for prevention and health promotion. One of the objectives withing Citizenlab is to explore and understand useful methodologies of citizen science in health and well-being research. There is limited knowledge on the benefits of citizen science, and the normative assumptions are often taken for granted [20, 21]. Examples of citizen science do not show a thorough analysis of successes and failures, therefor, little is understood about the advantages or pitfalls [21]. In the field of citizen science, to our knowledge nine literature reviews are performed by Carpini et al., Irvin and Stansbury, McGuire, Conrad and Hilchey, Garau, Kimura and Kinchy, Kullenberg and Kasperowski, Peter et al., and Ianiello et al. [13, 21,22,23,24,25,26,27,28]. These reviews focus on the use of citizen science to encourage public discourse, public decision-making, public management, or environmental research. To our knowledge, there were two literature reviews on citizen science with a focus on health, care or well-being published by Domecq et al. and Malterud and Elvbakken [29, 30]. These review focus on patient engagement in healthcare research and show that it is used in many settings. Often the research methods are adapted to make engagement possible for the citizens, and patient engagement is low during the data collection phases or rather tokenistic. Authors of both reviews argue that further research is needed to determine which valuable collaborative methods can achieve more active involvement of patients and define ideas and strategies for citizen science in health research [29, 30].

In our scoping review we analysed the citizen science methodologies applied in technology development for health, care or well-being. We were specifically interested in which innovation phases citizens were involved, and in the level of citizen involvement. The aim was to provide insight in (1) the levels of citizen involvement in current research on technological innovations for health, care or well-being, (2) the used participatory methodologies, and (3) lesson’s learned by the researchers.

Research phases and citizen involvement

To reach the aim, we started by distinguishing three research phases, namely: (1) preparation of research, which included the design of a research, the recruitment process, and design of materials such as interview guides, (2) data collection, which included all (iterative) steps of technology design, data collection and use of feedback throughout these iterations, and (3) data analysis and evaluation, which included methods of analysis as well as reporting of research data and outcome. With this distinction it is possible that one record might describe one, two or three phases in which citizens were involved.

Second, we assessed the level of citizen involvement. We proposed levels of citizen involvement based on the participation ladder of Arnstein and citizen science terminology defined by Hakley [31, 32]. Arnstein’s participation ladder is more used in citizen science research to discuss the methodology and possibilities of citizen involvement, for example by van Leersum et al. [2020] about the involvement of citizens in the need of long-term care during the development of a tool for self-assessment, and by Kotus to explore the level of participation of citizens in the urban policy making of a Polish city [33, 34]. The ladder consists of eight steps including manipulation, therapy, informing, consultation, placation, partnership, delegated power, and citizen control [31]. Each step on the ladder characterizes the role of the participant and the researcher. The higher participants are placed on the ladder, the more influence they will have. This ladder configuration shows that one level is building on the previous level, but there is no logical progression from one to another [35]. The steps manipulation and therapy are considered non-participation. The steps of informing, consultation, and placation are considered tokenism, in which the citizens have superficial involvement. From the informing step in which the citizens are informing the researchers about their personal situation or experiences, towards the placation step in which citizens advise the researchers about needed adjustment, research or innovations. The highest steps of partnership, delegated power, and citizen control are called citizen power, in which the citizens have more and more in-depth control and influence. In partnership the relation and tasks of the citizens and researchers are equal, and with citizen control, the citizens lead the research and researchers are available to support [31].

Hakley uses four different terms to distinguish involvement in citizen science research [32]. First, crowdsourcing, in which cognitive abilities of citizens are not needed and the citizens use digital devices to collect data that are automatically send and analysed by researchers at a later stage. Second, distributed intelligence, in which the citizens get a certain task such as interpretation or classification of data. Third, participatory science, in which citizens are formulating research questions or problems, and are participating in data collection, but analysis is often done by the researchers. Fourth and last, collaborative science, also referred to extreme citizen science, in which a project is carried out by citizens without researchers being actively involved [32]. Although it might seem from terminology and categorisations of citizen involvement that the higher the level of involvement the better, we will not make this argument in this scoping review. All forms of citizen science could be valuable and often citizen science research could not be defined to one level alone [32, 36]. It should be up to the team “to determine the best design specifications for their own unique context, enabling citizen science to achieve its full potential” [10]. Also, as Ferro and Molinari [2009] argue, researchers need to be aware of the complexity of citizens [37]. There is a considerable variation across the population of citizens who might be interested to be involved in different forms. This variation could be in terms of interest as well as educational level or technological skills. For some citizens, these differences make it more difficult or less desirable to participate on higher steps of the ladder. Researchers should be more aware of the variation and should know when to make a step towards the citizens rather than to expect the citizens to move towards them [37].

In our analysis we combine the terminology as used by Arnstein (1969) and Hakley (2013) and define four levels of citizen involvement [31, 32]. The first level is the level of non-involvement, this level includes manipulation, therapy, and crowdsourcing. The second level is sharing knowledge, including informing, consultation, placation, and distributed intelligence. The third level is collaboration, which includes partnership, delegated power, and participatory science, and the fourth level is citizens in the lead, including citizen control, and collaborative science. These four levels of citizen involvement were considered in view of the different research phases. Based on the scoping review, we highlighted five cases, and explored the variation in citizens, how they are involved, and what worked well or what was experienced problematic.

Method

Methodological approach

A scoping review, following PRISMA-ScR guidelines, was performed to examine how citizens are involved in conducted research in the field of health, care or well-being [38]. Scoping review is a reviewing method to map a research area and identify all relevant literature regardless of the study design. This approach is developed to contextualize knowledge and identify the current state of understanding of a specific topic, and to assists in identifying the aspects which are investigated and the aspects which are less or not investigated [39]. A scoping review was chosen due to the broad topic of citizen science and the application of many different study designs within this topic [39]. We investigated citizen science methodologies applied in innovation processes for health, care or well-being. In this article we provide insight in (1) the levels of citizen involvement in current research on technological innovations for health, care or well-being, (2) the used participatory methodologies, and 3) lesson’s learned by the researchers as described in identified records.

Search strategy

First, we developed a search string that included many combinations of search terms within four research fields: ‘how’, ‘why’, ‘what’, and ‘who’ (Table 1). How considers the involvement of citizens, why is health, care or well-being related, what contains the technological innovations, and who are the citizens. For each of these fields specific search terms were defined. For example, the search string for how:

Table 1 The main fields of the search string and all search term

TITLE-ABS-KEY (“citizen science” OR “participatory research” OR “user involvement” OR “citizen engagement” OR “citizen knowledge” OR “patient and public involvement” OR “community-based participatory research” OR “community engagement” OR “community participation” OR “citizen participation” OR “user-centred design” OR “co-creation” OR “co-design” OR “participatory design” OR “co-production”)

As proposed by Jackson and Waters, different combinations of search terms were tested and discussed by the research team and an information specialist [40]. The records acquired with each combination were compared and based on the included and excluded records by addition or removal of search terms, the final search terms were chosen. The final search string was formed by using AND in between the how, why, what, and who, and the terms within each part were connected using OR. To conduct the search, the final search string was entered in January 2021 into the database SCOPUS. The same information specialist that assisted the research team in testing and finalizing the search string, recommended the use of SCOPUS for its suitability to our scoping review's wide-ranging interest and aim. SCOPUS, being a large international and multi-disciplinary database of peer-reviewed literature, such as scientific journals, books, and conference proceedings, and indexes a broad spectrum of studies in the field of science, technology, medicine, social sciences, and arts and humanities, aligning closely with the scope of our review.

Selection of records

For the selection of the records several inclusion criteria and restrictions were chosen. The inclusion criteria were (1) empirical studies, (2) studies describing a method in which citizens were actively involved, (3) health related topics, and (4) technological innovation. Published records were not eligible if:

  1. 1)

    The record did not present the execution of an empirical study. Records describing exclusively the methodology, lessons learned, study design, or reviews are excluded.

  2. 2)

    The used method did not actively involve citizens. Citizens had to be involved to a larger extent than 'just' as subjects. Citizens could be professionals if they are the end-users of the technological innovation.

  3. 3)

    The study did not aim to improve health, care or wellbeing.

  4. 4)

    Technological innovation was not part of the record. We used the definition of Behney “Medical technologies include the drugs, devices, and medical and surgical procedures used in medical care, and the organizational and supportive systems within which such care is provided” [41].

Furthermore, we restricted our search to records that were published in the last five years, were peer reviewed or conference papers, and published in English. There were no restrictions on type of research or study design.

The first 200 titles and abstracts were screened by the four researchers. The exclusion criteria for the first screening consisted of no citizen involvement, no health, no technology, and not empirical. The four researchers discussed the differences of inclusion after screening the first 100 records. Then repeated the process for the next 100 records. The remaining records were split and both sets were screened by two researchers, each researcher couple discussed their differences. In case of doubt, the titles and abstracts were also discussed with the other researcher couple.

Ranking citizen involvement

To classify the used methodologies of the included records, we developed a matrix to rank the citizen involvement within the described research phases. The citizen involvement levels were based on the participation ladder of Arnstein and the citizen science terminology of Hakley [31, 32]. We defined four levels:

  1. 1.

    Non-involvement: there is no active involvement of citizens.

  2. 2.

    Sharing knowledge: the citizen receives information from the researcher, the citizen provides (advisory) information to the researcher, there is a dialogue in which decisions were made by the researcher.

  3. 3.

    Collaboration: the citizen and researcher are partners, equal in dialogue and equal in taking decisions, the researcher assists in all research activities.

  4. 4.

    Citizens in the lead: the citizen is leading and taking decisions, very minimal assistance of the researcher is provided, only on request of the citizen.

The methods sections of the full text records were screened and ranked on research phase and level of citizen involvement. The first 20 full text rankings were performed by four researchers. A record was directly excluded when it did not fit anywhere in the matrix. The four researchers discussed the differences of inclusion and ranking after reading the first 20 full texts. Then repeated this process for the next 10. The remaining records were divided among the researchers. In case of doubt, another researcher read the full text as well to align the analysis.

Data analysis

The research phases and the level of citizen involvement was screened. The analysis was based on the descriptions of the methodology in the records. We decided to take only records ranked on the levels of collaboration or citizens in the lead in the final steps of the analysis. The following information was extracted from the records: authors, year of publication, country of research, used methodology, health, care or well-being research topic, technological innovation, number and characteristics of involved citizens, and level of citizen involvement. Besides these details, five records were described in more detail as example cases. The presented scoping review was performed and reported on in compliance with the PRISMA-ScR guidelines.

Results

Selection and characteristics of records

In total 3846 records were retrieved (Fig. 1), no duplicates were found. After screening the titles and abstracts of the records, 2983 records were removed based on the inclusion criteria: empirical study, citizen involvement, focus on health, care or well-being, and technological innovation. Full texts were obtained, and the methods sections of the remaining 861 records were screened and ranked. During methods screening and full text reading 667 records were removed due to no citizen involvement. From the 194 remaining record, 110 records were put-aside based on citizen involvement solely on the levels of sharing knowledge or non-involvement. A total of 83 records were included for further analysis and are presented in Table 2. The research team used the Population, Context, and Concepts (PCC) criteria to describe characteristics of the included studies. These criteria are utilized in the following text and example cases, as well as in Table 2. More specifically, Table 2 displays, for each record, the year of publication, the country where the study was conducted, the terminology used for research design, the health, care, or well-being topic, technological innovations, the involved citizens, and the level of involvement during various research phases.

Fig. 1
figure 1

The search, reported following PRISMA-ScR guidelines, delivered 3846 records with no duplicates. After title and abstract screening 2983 records were excluded. Method screening resulted in the exclusion of 667 records. In 111 records, the study solely involved citizens on the level of sharing knowledge or non-involvement, therefore, 83 records were finally included

Table 2 All 83 included records in alphabetical order, reported following PRISMA-ScR guidelines

From the 83 included records, eight were published in 2016, 14 in 2017, 15 in 2018, 17 in 2019, and 29 in 2020. Most studies were conducted in the United States (N = 17) [44, 46,47,48, 52, 54, 71, 72, 76, 89, 94, 98, 100, 104, 107, 119, 123], followed by Canada (N = 14) [42, 43, 45, 59, 65, 69, 70, 81, 86,87,88, 92, 95, 105], the United Kingdom (N = 12) [51, 56, 57, 67, 80, 90, 96, 102, 103, 111,112,113], Denmark (N = 6) [77, 78, 85, 106, 114, 120], and Australia (N = 6) [83, 91, 108, 110, 116, 118]. Five studies were conducted in multiple countries [53, 97, 99, 109, 121], three studies were conducted in Sweden [Erlingdottir et al. ; Gardsten; Wikman et al.], and Norway [74, 75, 115], and 2 in Austria [61, 66], the Netherlands [34, 68], and Ireland [79, 117]. One study was conducted in Belgium [49], India [50], Italy [58], Brazil [62], Spain [64], Germany [73], Korea [82], France [84], Peru [93], and South Africa [101].

One of the 83 records was based on a quantitative research design, all other included record had a research design based on qualitative methodologies. Participatory design was most prominent (N = 24) [42, 44, 45, 50, 57, 61, 63, 66, 69, 74, 75, 77, 78, 80, 82, 83, 85, 91, 100, 104, 106, 117, 119, 123], followed by co-design (N = 16) [55, 64, 65, 72, 76, 81, 86, 88, 96, 99, 101, 103, 105, 110, 113, 121]. Other often used terms were community based participatory research (N = 9) [48, 52,53,54, 56, 70, 71, 93, 94], co-creation (N = 8) [43, 46, 58, 62, 73, 97, 107, 120], public and patient involvement (N = 7) [67, 79, 90, 102, 109, 111, 114], participatory action research (N = 5) [59, 68, 108, 112, 122], and user centered design (N = 4) [34, 89, 92, 115]. Citizen science was only used in 2 records [49, 118], and usability study [47], co-production [51], human centered participatory design [84], social Justice design [87], symposium [95], community oriented approach [98], and participation in research [116] were all only used once to describe the research design.

Besides the term citizen, participant, patient, actor or (end) user, there are different terms used to describe the involved citizens. These include key informants [45], co-leadership [46, 81], co-designers [45, 65, 86, 89], co-researchers [51], co-creators [58, 73], co-facilitators [102], co-investigators [109], inventors [123], design/research partners [117, 122], citizen scientist [118], community advisory committee [98], research coalition [105], steering group [71, 90, 112], (expert) advisory group [51, 108, 113, 116], and development group/team [34, 59, 74, 120]. Fifty-nine of 83 records included adults, 11 records included younger adults (13 to 25 years old) [43, 51, 58, 59, 67, 78, 94, 98, 99, 119, 120], eight records included children (12 years or younger) [61, 69, 70, 72, 104, 117, 121, 123], and nine records included older adults [59, 68, 73, 83, 84, 89, 91, 97, 118]. The number of involved citizens is variable among all records, most studies involved between 30 and 40 citizens, with a highest amount of 6246 [49] and the lowest of five citizens [76, 82, 84, 114].

Research phases and citizen involvement

Full texts were read of 194 records. A total of 111 records involved citizens solely on the levels of sharing knowledge or non-involvement. In these records researchers were in the lead during all research phases and decisions. Sixty-five records were considered to have citizen involvement on a collaboration level in one or more research phases. There were 8 records analysed as having citizens in the lead in one research phases. These records had citizens who were responsible and made decisions regarding a development process or were themselves the initiators of a project. Most records involved citizens during the data collection phase. Forty of 83 records had user inclusion in all three research phases, 11 records involved citizens in the data collection and evaluation phase, ten records in the preparation and data collection phase, and one record in the preparation and evaluation phase. Thirteen records described citizen involvement only in one research phase. Table 3 shows the level of citizen involvement in the different research phases of all 194 included for full text reading. Five records are further discussed in the next section as example cases of citizen involvement in research regarding health, care or well-being.

Table 3 Number of records for each level of citizen involvement and within each research phase

Example cases with high level of involvement in multiple phases

I In this section five example cases are elaborated upon to show a diversity of topics, methods, and citizen inclusion. These cases are chosen based on inclusion of citizens in multiple research phases and at a level of collaboration or higher in one or more research phases. The aim of these cases is to analyse the citizen science method during the innovation process in the domain of health, care or well-being. How were citizens involved, who were involved, what worked well, what kind of problems were met, and what kind of motivation do researchers or citizens have?

Case 1. An online self-management tool for spinal cord injury [45]

The technology in this case is an online tool to promote self-management in order to avoid rehospitalization of patients after spinal cord injury (SCI). The data was collected in 2015 and 2016 involving Canadian citizens of different provinces. This case is chosen because it shows a clear description of the organisation and collaboration between the different stakeholders. The authors included several teams demonstrating a divers and broad network (see Box 1 for a case description).

Box 1. Case description:

Online self-management tool for spinal cord injury

Self-management tools are available for many chronic conditions. The use of these tools is linked to a decrease in rehospitalization rates. However, satisfaction with most self-management tools is low, and there is no known online tool for users with SCI. The goal in this study was to apply a participatory design process in order to develop an online self-management tool for users with SCI. This process was chosen by the authors to define design constraints and solutions. Sixteen individuals, seven researchers and nine persons with SCI, participated in ten different meetings of the participatory design process. Experiences with self-management were discussed in the first meetings (exploration). Thereafter, features for a new online tool were defined (discovery). These features were translated and embedded in a prototype which the group discussed in following meetings, and feedback was used to make iterations (prototyping). The design process and collaboration with potential users was valuable to engage end users in the design of a tool to promote self-management after SCI

In this case, the participatory design process consisted of three steps: exploration, discovery, and exploration [124]. The design process was described as an interactive design in which the knowledge of end-users was acknowledged and taken into the design by involving users. The process should include expressing tacit knowledge and encouraging sensitivity [45]. The design process included a core co-design and co-development team meeting on regular basis. The record clearly shows the connections between this core team and the council. This council consisted of a consumer advisory group of users with SCI and a product advisory group of researchers and a clinician [45]. Several other records described a collaboration between different groups, for example Wikman et al. included parent research partner group and an expert research partner group [122], Russ et al. made use of a steering group in which public representatives collaborated with researchers [112], and Mrklas et al. had a research coalition [105]. A difference with the study of Allin et al. , quite often these advisory boards did not include the actual users, but healthcare professionals [45].

The minimal involvement of citizens in the evaluation phase could be seen as a limitation of this case, but in all other phases it was very strong. Furthermore, they recognize possible bias due to a low number of involved citizens, and we noticed a lack of involvement of citizens with a lower educational level. Allin et al. discuss that all involved citizens had access to high-speed internet, which is associated with a high self-reported health status, and therefore these involved citizens are maybe less in need of self-management than those who are not online [45].

Beforehand the authors motivated their choice to use a participatory design, because it might empower and educate the citizens, and encourages them to adopt the developed application. The three benefits reported afterwards include “elicitation and consideration of diverse accessibility considerations”, “prioritization of features and identification of core design concerns”, and “co-creation of acceptable strategies and techniques to mitigate identified concerns” [45]. Although the expected benefits considered the citizens, the experienced benefits show the value for the development process.

Case 2. Digital stories for health promotion [59]

In this case, digital stories are the technologies, which are at the same time the technologies under development as well as the technologies used to perform the research. Sharing knowledge on healthy lifestyle and increasing self-esteem are the health goals with the stories. From 2012 to 2014, 18 workshops on digital stories were organised in communities from Victoria to Ahousat in Canada. This second case was chosen due to intensive groupwork and collaboration between younger and older adults within the community as researchers. A mutual learning process was present between the younger and older adults to develop the digital stories (see Box 2 for a case description).

Box 2. Case description:

Digital stories for health promotion

Aboriginal youth in Canada experience lower health and well-being due to different challenges they face. Digital stories can emphasize on community connection and cultural continuity, which are factors influential to improve health and well-being. The aim of the study was to create and share digital stories to support health promotion and encourage interactions within a community. This creates a place for younger and older adults where knowledge on healthy lifestyle could be shared, where mentorship could be facilitated, and where self-esteem could be increased by celebrating identity and community. Participatory action research was used to engage younger and older adults in the community. A youth research team was formed with eight younger adults. Next to this team there were 60 core younger participants, 170 younger workshop participants, and 14 older adults involved. At the start, the youth research team was trained to create digital stories. After workshops in which the older adults taught about their healthy lifestyle, the younger and older adults together created mini stories with the use of cameras, audio recorders, and story boards. These stories were evaluated during follow-up sessions, and a small group of younger adults were trained to lead other communities in creating their digital stories

Fletcher and Mullett [2016] refer to McIntyre, but do not give a definition for their research method [59, 125]. The authors argue that participatory action research was beneficial because it was a non-prescriptive approach. Other records using similar participatory action research methods do give a definition. For example Wikman et al. based their design on Kindon et al., “which is a collaborative process of knowledge production and co-learning, placing people with lived experience at the centre of the process” [122, 126], and Rabba et al. [2020] based their method on Baum et al., “the stakeholders are integral to all parts of the research process, participants’ active role in the outcome sets participatory action research apart from traditional research methods where the participants tend to be more passive in their receipt of research outcomes” [108, 127]. Often a participatory action research is divided in phases. Rabba et al. used planning, action, observation, reflection, and using the new learning to plan further steps in their research [108], and Haufe et al. used the two phases of understanding the current and developing a tool [68]. All participants of the study by Fletcher and Mullet were involved in the phases planning, designing, and evaluation of the community workshops [59].

The research process gave the community members together with the researchers the opportunity to create a unique, broad, and holistic concept of healthy lifestyle, including personal wellness in body, mind, and spirit. Clear outcome and follow-up include ownership of the stories by the communities, the youth took on leadership in training other communities to develop digital stories, and the older adults took a teaching role in ecology, traditional food preparation, cleansing rituals, and medicinal plants [59]. There were more records discussing the need for public health interventions and the involvement of community members in creating digital stories based on their community and personal stories. Although in most records the creation was done by the community members, this creation was more seen as a needs assessment activity and the researchers chose which stories were used in discussions [93].

It seems that the aim to use a participatory action approach with digital stories to improve health and well-being was positive. The creation of stories by community members was relevant to capture a personal voice and made explicit which public health themes were recognized. The workshops were fun and engaging, and the youth experienced a process with the older adults giving them a voice and a sense of belonging. However, Fletcher and Mullett also acknowledge that the participants were very active in the creation of stories, presenting, training, and workshops, but there was a much lower interest in planning and analysis [59].

Case 3. Design social play things [61]

This case is about technology for social play for mixed groups of neurodivergent and neurotypical children. The well-being goal is to scaffold and support the development of complex social skills through technology-facilitated social play. In the ‘SocialPlayTechnologies’ project the researchers collaborated over the course of three years with two inclusive mainstream primary schools in Vienna, Austria. Three social play technology prototypes were designed together with three groups of children, aged 7 to 12 years. We selected this case due to researchers' effort to give children the lead in the collaborative design process, their flexibility in adapting to the children’s needs, and to showcase how citizen science can be executed with challenging target groups (see Box 3 for a case description).

Box 3. Case description:

Design social play things

Social play is an essential part of children’s healthy development. For neurodivergent children engaging in social play with neurotypical peers that have different interaction styles can be challenging, often leading to a preference for solitary play. This study sought to explore how digital technologies can facilitate and support social play among mixed groups of neurotypical and neurodiverse children. The aim was to create technologies that are engaging for both target groups, interactive, open-ended, robust, embedded in their natural play context, extendable, and non-normative. The researchers engaged in a long-term participatory design (PD) process with three groups of children aged 7 to 12 years in. In total 16 children participated as co-designers in this study. The PD process consisted of weekly or bi-weekly series of design workshops (50 in total across the three groups) with at least two research team members as facilitators. Groups were completely free in what social play technology they would design. The first phase of the PD process focussed on exploring children’s perception of playfulness. This phase was followed by various technology immersion activities to introduce the children to sensors and actuators. In the next phase, researchers provided new input through material, scenarios, and narratives, while also narrowing the design space to agreed-upon ideas. When concepts were more concrete, researchers started to bring in first prototypes or groups collaboratively designed prototypes. Three distinct social play technologies were designed: 1) a concept with coloured fabrics embedded with interactive lighting called LightSpaces, 2) a set of interactive pads that trigger sounds called MusicPads, and 3) a concept consisting of a reading lamp with a Raspberry Pi, a camera and a projector called PictureStage. Towards the end of the design process researchers worked on increasing the maturity of the prototypes for the purpose of evaluation. This evaluation phase is escribed in another paper

The researchers applied a PD to reach their design goal of creating social play technologies suitable for the different ways neurodivergent and neurotypical children make sense of the world, can only be achieved by including these children directly in the design process [61]. As neurotypical adult researchers they felt unequipped to imagine the children’s distinct sense making process in social play. Similar reasoning on the value of PD is given by [42] who argue that PD ensures that a design solution meets the target groups needs and beliefs, and is culturally relevant. Besides understanding perspective and producing relevant design solutions, PD can provide other benefits to participants, such as fostering agency and teaching useful skills [123]. The researchers’ design process was built on previous PD work with children including co-operative Inquiry [128]. Co-operative inquiry has three key elements: (1) children are regarded as research partners and involved throughout the design process, (2) design activities with low-tech materials, and (3) technology immersion in which children explore the design space of novel technologies. Yarosh and Schueller also worked with children and showed methodological similarities by having co-operative inquiry as a framework to build their PD process, and having long-term engagement since they met with the children twice a week for 14 design sessions [123].

Another concept, central in the researchers’ design process, is ‘Handlungsspielraum’ (in English: room to act) [129]. To open up Handlungsspielraum it was important for the researchers to create design activities that balance structure and freedom through the used materials, planned activities, physical setting and role of the facilitator. While structure provides safety, guides participants, and offers stimuli to participant, freedom allows participants to explore their creativity and ideas. Researchers found that this balance between structure and freedom had a similar effect on creativity within play. Hence, this balance was also reflected in the final prototypes through modularity, manipulability and the option to use the prototypes without the technological component. The researchers in this case tried to let children lead the design process. This is reflected in the fact that each group came up with a different social play technology. Moreover, while the design process followed the same overall blueprint across design groups, the overall pace and specific content of each design session was different for each group and evolved based on the children’s interest and conceptions of playfulness. On the other hand, the researchers noted that they sometimes made design decision contrary to the wishes of the children to limit the complexity of the designed technologies.

Researchers in the current case indicated that working with a mixed group of neurotypical and neurodivergent children was challenging, especially in the groups where no teacher was present. Researchers had to act as mediators when conflict occurred between children and prepared for this task by drawing on previous literature about appropriate design frameworks for this target group. Moreover, they interviewed the teachers at the start of the design process to learn about the group dynamics. Some team members received a training from a special needs educator to learn how to deal with conflict.

Case 4. New voices to design exercise technology [78]

The technology in this case is a health promoting digital application to support vulnerable young adults in exercising and physical activities. The ‘Pulse Up project’ is a collaboration of a general practitioner (GP), community workers, and citizens in a neighbourhood in Denmark. Eighteen young adults, aged 18–30, suffering from depression and/or anxiety, and having a low level of physical activity, participated in the exploration and manifestation of design ideas. This case is chosen because of the situated PD process, the creative PD activities, and the inclusion of vulnerable citizens in the choice of research activities, design ideas, analysis, and evaluation of each step in the process (see Box 4 for a case description).

Box 4, case description

New voices to design exercise technology

Most research into digital technologies to support exercise is centred on people already engaged in exercise activities, while those who are challenged to exercise and are at risk of ill health due to low physical activity level have a limited voice in technology design. This research presents a PD process with young adults suffering from depression and/or anxiety, living in a neighbourhood identified as a high-risk health zone, and motivated but faced with challenges to exercise. The design process consisted of two interventions. The first intervention focussed on developing appropriate participatory methods for bringing new voices to the design of exercise technology and to gain initial insight to inform future design. The second intervention focussed on exploring and manifesting design ideas. Participants met twice a week for eight weeks in the neighbourhood’s local gym to exercise with an instructor and participate in a total of 16 PD activities. Eleven young adults participated in the first intervention. Four continued in the second intervention, and seven new participants were recruited. Exercise itself was not challenging, but the challenge was in the sub-activities to complete before exercise. The participants created three visions to meet with this challenge: 1) dragging friends to exercise, 2) keeping an eye on the activities of group members, and 3) a ‘join’ functionality as an easy way to join activities. The participants’ visions on exercise technology were low since they did not prioritize tracking and competition, health information or exercise instructions. However, based on their challenge, they created visions for digital support

To bring the voices of vulnerable young adults into the design process of digital technologies to support health promotion, Kanstrup et al. chose for a situated PD [78]. The process combined physical exercises with PD activities. Although the researchers initially planned to do the PD activities in a classroom setting, the participants expressed a dislike of classrooms and preferred activities in the gym. Therefore, the PD activities were integrated with the physical activities in the gym. To change the setting, they made rules for the activities to emphasise on the need for professionalism, effectiveness, and the ability to trigger design. This formed the foundation for the development of a design cycle with progressive stages, supported by game-like PD activities, continuous analysis and evaluation of results. All activities were developed as a fusion of training sessions with music and exercises, and PD activities. In addition, Kanstrup et al. developed a variation of artefacts to trigger exercise and technology design [78].

The previous case had a similar approach in their research of social playthings. Both the location and the integration of playful activities became part of the design process. Yarosh and Schueller also reported that they deviated from their original plan of putting the researchers in charge of video-recording design sessions [123]. They decided to give the participants control over the recording process to change the perceived power-imbalance between researchers (as data-collector) and participants (as data-objects). These studies show the benefits of situated research in which group activities are part of the research process. Although this situated approach can be in a physical place, it can also be interpreted as a safe space for open discussions, as is visible in the study of Dewa et al. [2020]. In their study with young people with mental health, a WhatsApp-group became the place for open discussions between the co-researchers [51].

Each PD intervention was evaluated by the participants, who were engaged in an iterative analytical process. Visualisations of analytical results were shared with the participants between 2 interventions, and they were invited to reflect on the presented analysis and to reconstruct any misinterpretations, missed insight or future visions. After the second intervention, the co-researchers conducted a final analysis of the visions, in which they identified design elements for exercise technology. The created visions move beyond an understanding of exercise technology for vulnerable groups as simply information apps, towards an ambition to design mediators of community health resources in which digital communication is the mediator for physical activities. This requires critical insight about adolescent use of digital communication and the potential importance of messaging, apps, gaming, wearable technology, and rapid changes in youth communication and use of digital technology in developing adolescent physical activity health promotion. This is visible in the study of Livingood et al. in which they develop a digital communication intervention to reduce adolescent obesity [94].

Case 5. A person-centred patient portal [81]

This study focuses on a patient portal app for patients with cancer. A patient portal is a secure extension of an electronic medical record (EMR) of a health care institution where patients and professionals have access to [130]. In the study of Kildea et al. participatory stakeholder co-design was applied, involving patients and health care providers [81]. Data were collected between 2015 and 2018. This study was selected because of the interface between patients and the medical world, and the authors strongly show their used participatory stakeholder co-design (see Box 5 for a case description).

Box 5. Case description

Design and development of a patient portal (eHealth)

Patient portals represent a real-world example of patients facing electronic health (eHealth). Patient portals become important in daily care because they provide patients with personal health information and contribute to patient engagement and empowerment. A participatory stakeholder co-design was applied involving patients and health care providers. Six core elements were used: equal co-leadership, patient preference determination, security, governance, and legal input, user evaluation and feedback, continuous staff input, and end-user testing. Regarding ‘equal co-leadership’, all three co-leads, a patient, a radiation oncologist, and a medical physicist, were equally involved in decisions and in constant communication. ‘Patient preference determination’ was conducted by a voluntary convenience sampling survey, to obtain input from people receiving cancer treatment. For ‘security, governance and legal input’ purposes a Security and Governance team provided guidance regarding the security and confidentiality of patient data and compliance with applicable regulations. Three patients were participated in a focus group to provide ‘user evaluation and feedback’. The prototype was demonstrated, and participants were observed. A second focus group was conducted before pilot release, consisting of five members of the cancer centre’s patients committee. ‘Continuous staff’ input was ensured by a clinical co-lead, radiation therapists, staff ranging from health care providers to senior management, medical physicists, oncology nurses, radiation oncologists, administrative assistants, and the board of directors of the institution. Finally, to ensure ‘end-user testing’, students and real patients tested the first version of the prototype

In this case the differences between co-design, patient-centred design and person-centred design were described. In co-design, patients help identify the process or project that needs to be designed based on personal experience. Patient-centered design focuses on fulfilling the needs of the patients, but the project may not have been identified by patients or involve patients. Person-centered design focuses on patients’ need as a whole person and as an equal partner in their care [131, 132]. Kildea et al. described that the term patient co-design is often confused with patient-centered design and person-centered design [81]. Therefore, the term participatory stakeholder co-design was preferred as term in this study. Patients and clinicians were involved as equal partners, and both actively participate in all parts of the design process [81]. Some other studies involving both patients and health care providers used the term PD [42, 57, 74, 77, 85, 100, 104, 106, 108].

Furthermore, in this case an agile development approach was applied, which consisted of rapid prototyping and testing of various features [133]. A team of developers, students in computer science and medical physics, and co-leads worked together. This led towards a person-centered, clinician acceptable, and informatics feasible patient portal. Comparable to this case, in the study of Leeming and Thew a patient data platform was developed involving different stakeholders as part of a steering group. Public and patient views were explored during design sessions, and the steering group highlighted the key learnings from the feedback to guide the design [90].

Kildea et al. described several challenges during the process of this study [81]. For example, the equal co-leadership, where patients are on an equal level as clinicians. This was often not clear to people outside of the immediate design and development team because the hierarchical health care system assumed that projects must be led by a clinician. Also, challenges regarding security, governance, and legal input, especially time-consuming legal issues, were encountered, for example intellectual property, liability, and the contents of the patient disclaimer form. However, as a result of this study a final design of the patient portal smartphone app for patients with cancer was developed from scratch within the health care system based on the principles of person-centeredness design.

Discussion

Main findings

The aim was to provide insight in (1) the levels of citizen involvement in current research on technological innovations for health, care or well-being, (2) the used participatory methodologies, and (3) lesson’s learned by the researchers as described in identified records. From the 194 records included after full text reading, 83 records involved citizens on a collaboration level in at least one research phase. The level of citizen involvement differed across research phases. In contrast to the review of Domecq et al. in which citizen involvement was lacking or tokenism in the data collection phase, in our scoping review most citizens were involved in the data collection, and less in preparation or evaluation [29].

The metrics of these 84 records show that there was a two-fold increase in one year between 2019 and 2020. This shows a growing interest in citizen science for health, care or well-being in the last years. Health, care or well-being research was originally considered less adequate for citizen science, but exceptions are arising [13]. The increase is also visible in the scientific journals that are picking up citizen science as a separate field. The journal ‘Citizen Science Theory and Practice’ was a first journal on citizen science, with a first issue in 2016 [134]. Other journals, such as PlosONE, include citizen science as a search or subject area. In the domain of health, care or well-being the use of citizen science was also incorporated recently, considering the entry of the Medical Subject Heading (MeSH) in 2020. Looking at the countries where the citizen science studies were performed, most of the 83 records in our sample originated from developed countries. In the entire dataset (N = 3846), there is a lot of community-based research in developing countries, and in most research the citizens are involved, but most records were excluded due to lack of technological innovation [135,136,137,138]. The included studies show a variation in the number of citizens involved, but most studies did not include more than 20. Although the number of citizens in the studies is often argued as sufficient or data saturation was reached, it introduces the risk of recruiting only a minority of the population, often the well-educated and most affluent citizens [37].

Apparent from all records was the terminology used to describe the research methodology or the citizen involvement. The term citizen science was only used in two records and participatory design was most prominent in the included records. Identical authors give different terms to their research design across studies [42, 43, 46, 47, 86, 87], and when using similar terms, the authors base the research method on different sources [86, 88]. Inclusion of all the different research methods is due to our broad search string, however, this is in line with continuing discussions on the definition for citizen science [11], and the acceptance of working in citizen science with different terms and methodologies [12]. Den Broeder et al. also wrote “some approaches in health research strongly resemble citizen science, one of these is participatory action research” [5]. However, they argue that there is a main difference between participatory action research and citizen science, and that difference is in the action part, which refers to the need to act, having change, addressing a specific problem, or developing an intervention. Citizen science can also be a method used to do research without a focus on any action. Furthermore, participatory action research aims towards a strong involvement of citizens and in citizen science, the citizens can be engaged in a less intensive way [5]. Regarding the two records naming their research method citizen science, the level of involvement was diverse. One aimed to reach a large audience with a citizen science-mass experiment with no focus on action [49], and the other trained citizen scientists to become agents of change in their own environment which had a focus on action [118]. These examples show that there is no consensus on the use and terminology of citizen science and related methodologies.

These different views and approaches towards citizen science raise questions regarding the value and benefits of citizen science. Greenwood et al. argue that active involvement of citizens in the preparation phase ensures that the most relevant research questions will be asked [139]. Also, successful involvement of citizens causes sensitivity and understanding of needs to incorporate in design and implementation [61, 140]. Thereby increasing the chance of adoption and sustainable implementation. Many records in this scoping review consider a form of citizen science to raise awareness on health, care or well-being issues. This is in line with the benefits of citizen science for the citizen scientists [8, 9]. However, little evaluation was done in the included empirical studies. Some discussed the possible benefits for the citizen scientists, such as improved personal health. However, these issues were not obtained from the citizens, but expected or experienced by the researchers. One record discussed that the development team had the feeling of being heard and all ideas were considered in the development [34], but it is often unknown how citizens have experienced their contributions and if they would recognize their ideas as part of a prototype [141].

Compared to traditional science, citizen science asks for different research approaches and skills from researchers. The selected cases imply that successful citizen science project develope a structural and longtidutinal partnership with their collaborators [45, 59, 61, 78, 81]. Citizen science requires an open attitude, flexibility and context sensitivity from researchers. They have to give-up traditional power dynamics, step away from their own expectations and pre-conceptions, be sensitive to the signals of their co-researcher and be willing to engage in a mutual learning experience. For example, researchers might have preconceived theoretical knowledge about the topic of interest and probably about the research population, but this does not always align with the views of the co-researchers. Yarosh and Schueller and Gobl et al. both underline that it is important that the researchers learned how to engage and learn from the mental models and views of their co-researchers [66, 123]. Being sensitive to the context and the co-researcher often means that pre-planned methods and approaches have to be adapted on the fly to create a comfortable and stimulating environment for citizens to actively engage in research [61, 78].

Conclusion

In this scoping review, our main interest was (1) the levels of citizen involvement in current research on technological innovations for health, care or well-being, (2) the used participatory methodologies, and (3) lesson’s learned by the researchers. The increasing number of publications per year in the period 2016–2020, shows the growing interest in citizens science in the field of health, care or well-being. The records that surfaced in this review confirm the diversity of research methods that are applied in citizen science for health care or well-being, mainly applying qualitative methods, often tailored to the specific research problem and group of co-researchers. The five example cases showcase how the contextuality of citizen science as a situated research approach has an impact on the chosen research methods, analysis, and research results. Citizen science in this context requires an open attitude towards how citizens are involved, who the involved citizens are, which methods are used and what kind of topics are researched.

Based on this scoping review, the topic of the required flexibility of researchers in citizen science projects is an opportunity for further research. Although citizen science projects could take more time, it has a meaningful impact on researchers and citizen scientists [Dewa et al. 2020]. Researchers need to engage with co-researchers and find a balance between control and an open structure. The records in this review show some directions to adapt and relinquish control based on learning experience during the citizen science project. Explore this learning process in more depth could provide further understanding and recommendations for researchers as well as citizen scientists involved in citizen science projects. Where is guidance of researchers asked, where is it necessary to give citizens more space, and when would you exert more supervision?

Another follow-up could include exploration of collaboration with vulnerable target groups. The cases and other records in this review show that it is possible to actively involve and collaborate with vulnerable target populations, with or without the use of proxies. Furthermore, based on the minimal findings considering the value of citizen science for the involved citizens, motivation of researchers to start collaborating and motivation of citizens to become active in a project, this could be an interesting direction to explore. Connected to this question on the value of citizen science is the question on the need of a high level of citizen involvement. This scoping review showed that the involvement of citizens in citizen science projects could differ in each research phase. From the included records, it cannot be discussed whether this difference in level of involvement effects the project, research outcome or contribution of the citizens. Despite the lower levels of involvement in one research phase or another, the contribution of citizens in other phases was worthwhile. Therefore, it might be argued that a high level of involvement is not always necessary to execute a valuable citizen science project, but we cannot draw firm conclusions without empirical evidence.

Availability of data and materials

The datasets supporting the findings and conclusions are available from the corresponding actor on request.

Abbreviations

eHealth:

Electronic health

EMR:

Electronic medical record

GP:

General practitioner

MeSH:

Medical Subject Heading

PD:

Participatory design

SCI:

Spinal cord injury

References

  1. Liu JX, Goryakin Y, Maeda A, Bruckner T, Scheffler R. Global health workforce labor market projections for 2030. Hum Res Health. 2017;15(1):11. https://doi.org/10.1186/s12960-017-0187-2.

    Article  Google Scholar 

  2. Krick T, Huter K, Domhoff D, Schmidt A, Rothgang H, Wolf-Ostermann K. Digital technology and nursing care: a scoping review on acceptance, effectiveness and efficiency studies of informal and formal care technologies. BMC Health Serv Res. 2019;19(1):400.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Wiggins A, Wilbanks J. The rise of citizen science in health and biomedical research. Am J Bioeth. 2019;19(8):3–14. https://doi.org/10.1080/15265161.2019.1619859.

    Article  PubMed  Google Scholar 

  4. Rosner H. Data on wings. Sci Am. 2013;308(2):68–73. https://doi.org/10.1038/scientificamerican0213-68.

    Article  PubMed  Google Scholar 

  5. Den Broeder L, Devilee J, Van Oers H, Schuit AJ, Wagemakers A. Citizen Science for public health. Health Promot Int. 2018;33(3):505–14. https://doi.org/10.1093/heapro/daw086.PubMed.

    Article  Google Scholar 

  6. Irwin A. Citizen science: a study of people, expertise, and sustainable development. New York: Routledge; 1995.

    Google Scholar 

  7. Ceccaroni L, Woods SM, Sprinks J, Wilson S, Faustman EM, Bonn A, et al. Citizen science, health, and environmental justice. Cham: Springer; 2021.

    Book  Google Scholar 

  8. Haywood B. A “sense of place” in public participation in scientific research. Sci Educ. 2013;98:64–83.

    Google Scholar 

  9. King AC, Winter SJ, Sheats JL, Rosas LG, Buman MP, Salvo D, et al. Leveraging citizen science and information technology for population physical activity promotion. Transl J Am Coll Sports Med. 2016;1(4):30–44.

    PubMed  PubMed Central  Google Scholar 

  10. Auerbach J, Barthelmess EL, Cavalier D, Cooper CB, Fenyk H, Haklay M, et al. The problem with delineating narrow criteria for citizen science. Proc Natl Acad Sci USA. 2019;116(31):15336–7. https://doi.org/10.1073/pnas.1909278116.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Heigl F, Kieslinger B, Paul KT, Uhlik J, Dorler D. Opinion: Toward an international definition of citizen science. Proc Natl Acad Sci USA. 2019;116(17):8089–92. https://doi.org/10.1073/pnas.1903393116.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Eitzel MV, Cappadonna JL, Santos-Lang C, Duerr RE, Virapongse A, West SE, et al. Citizen science terminology matters: exploring key terms. Citiz Sci. 2017;2:1–20.

    Google Scholar 

  13. Kullenberg C, Kasperowski D. What is citizen science?–a scientometric meta-analysis. PLoS One. 2016;11(1):e0147152. https://doi.org/10.1371/journal.pone.0147152.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Huang J, Hmelo-Silver CE, Jordan R, Gray S, Frensley T, Newman G, et al. Scientific discourse of citizen scientists: models as a boundary object for collaborative problem solving. Comput Hum Behav. 2018;87:480–92. https://doi.org/10.1016/j.chb.2018.04.004.

    Article  Google Scholar 

  15. Rafner J, Gajdacz M, Kragh G, Hjorth A, Gander A, Palfi B, et al. Revisiting Citizen Science Through the Lens of Hybrid Intelligence. arXiv. 2021:34. abs/2104.14961.

  16. Danielsen F, Burgess ND, Coronado I, Enghoff M, Holt S, Jensen PM, et al. The value of indigenous and local knowledge as citizen science. In: Hecker S, Haklay M, Bowser A, Makuch Z, Vogel J, editors., et al., Citizen science: innovation in open science, society and policy. London: UCL Press; 2018. p. 110–23.

    Chapter  Google Scholar 

  17. Koepnick B, Flatten J, Husain T, Ford A, Silva DA, Bick MJ, et al. De novo protein design by citizen scientists. Nature. 2019;570(7761):390. https://doi.org/10.1038/s41586-019-1274-4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Snik F, Rietjens JHH, Apituley A, Volten H, Mijling B, Di Noia A, et al. Mapping atmospheric aerosols with a citizen science network of smartphone spectropolarimeters. Geophys Res Lett. 2014;41(20):7351–8. https://doi.org/10.1002/2014gl061462.

    Article  Google Scholar 

  19. Wright J, Parry J, Mathers J. Participation in health impact assessment: objectives, methods and core values. Bull World Health Organ. 2005;83(1):58–63.

    PubMed  PubMed Central  Google Scholar 

  20. Burton P. Conceptual, theoretical and practical issues in measuring the benefits of public participation. Evaluation. 2009;15(3):263–84. https://doi.org/10.1177/1356389009105881.

    Article  Google Scholar 

  21. Ianniello M, Iacuzzi S, Fedele P, Brusati L. Obstacles and solutions on the ladder of citizen participation: a systematic review. Public Manag Rev. 2019;21(1):21–46. https://doi.org/10.1080/14719037.2018.1438499.

    Article  Google Scholar 

  22. Carpini MXD, Cook FL, Jacobs LR. Public deliberation, discursive participation, and citizen engagement: a review of the empirical literature. Annu Rev Polit Sci. 2004;7:315–44. https://doi.org/10.1146/annurev.polisci.7.121003.091630.

    Article  Google Scholar 

  23. Conrad CC, Hilchey KG. A review of citizen science and community-based environmental monitoring: issues and opportunities. Environ Monit Assess. 2011;176(1–4):273–91. https://doi.org/10.1007/s10661-010-1582-5.

    Article  PubMed  Google Scholar 

  24. Garau C. Citizen participation in public planning: a literature review. Int J Sci. 2012;1(12):21–44.

    Google Scholar 

  25. Irvin RA, Stansbury J. Citizen participation in decision making: is it worth the effort? Public Admin Rev. 2004;64(1):55–65. https://doi.org/10.1111/j.1540-6210.2004.00346.x.

    Article  Google Scholar 

  26. Kimura AH, Kinchy A. Citizen science: probing the virtues and contexts of participatory research. Engaging Sci Technol Soc. 2016;2:331–61.

    Article  Google Scholar 

  27. McGuire M. Collaborative public management: assessing what we know and how we know it. Public Admin Rev. 2006;66(S1):33–43. https://doi.org/10.1111/j.1540-6210.2006.00664.x.

    Article  Google Scholar 

  28. Peter M, Diekotter T, Kremer K. Participant outcomes of biodiversity citizen science projects: a systematic literature review. Sustainability-Basel. 2019. https://doi.org/10.3390/su11102780.

    Article  Google Scholar 

  29. Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. Bmc Health Serv Res. 2014. https://doi.org/10.1186/1472-6963-14-89.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Malterud K, Elvbakken KT. Patients participating as co-researchers in health research: a systematic review of outcomes and experiences. Scand J Public Health. 2020;48(6):617–28. https://doi.org/10.1177/1403494819863514.

    Article  PubMed  Google Scholar 

  31. Arnstein SR. Ladder of citizen participation. J Am I Planners. 1969;35(4):216–24. https://doi.org/10.1080/01944366908977225.

    Article  Google Scholar 

  32. Haklay M. Citizen science and volunteered geographic information: overview and typology of participation. Dordrecht: Springer; 2013.

    Google Scholar 

  33. Kotus J. Position of the Polish city on the ladder of public participation: are we going the right way? The case of Poznan. Cities. 2013;35:226–36. https://doi.org/10.1016/j.cities.2013.08.001.

    Article  Google Scholar 

  34. van Leersum CM, Moser A, van Steenkiste B, Reinartz M, Stoffers E, Wolf J, et al. What matters to me—a web-based preference elicitation tool for clients in long-term care: a user-centred design. BMC Med Inform Decis Mak. 2020;20(1):57. https://doi.org/10.1186/s12911-020-1067-6.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Connor DM. A new ladder of citizen participation. Natl Civ Rev. 1988;77(3):249–57. https://doi.org/10.1002/ncr.4100770309.

    Article  Google Scholar 

  36. Heyen NB. Self-tracking as knowledge production: quantified self between prosumption and citizen science. Wiesbaden: Springer VS; 2016.

    Google Scholar 

  37. Ferro E, Molinari F. Making Sense of Gov 2.0 Strategies: ’No Citizens, No Party’. Prosser A, Parycek P, editors. Vienna. 2009. https://doi.org/10.29379/jedem.v2i1.13

  38. Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. Bmc Med Res Methodol. 2018. https://doi.org/10.1186/s12874-018-0611-x.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. https://doi.org/10.1080/1364557032000119616.

    Article  Google Scholar 

  40. Jackson N, Waters E, Guidelines for Systematic Reviews in Health P, Public Health T. Criteria for the systematic review of health promotion and public health interventions. Health Promot Int. 2005;20(4):367–74. https://doi.org/10.1093/heapro/dai022.

    Article  CAS  PubMed  Google Scholar 

  41. Behney C. Medical technology—contributions to health care, the future of health in America. Washington, D.C.: U.S. Government Printing Office; 1989.

    Google Scholar 

  42. Abbass-Dick J, Brolly M, Huizinga J, Newport A, Xie F, George S, et al. Designing an eHealth breastfeeding resource with indigenous families using a participatory design. J Transcult Nurs. 2018;29(5):480–8. https://doi.org/10.1177/1043659617731818.

    Article  PubMed  Google Scholar 

  43. Abbass-Dick J, Chyzzy B, Newport A, Huizinga J, Xie F. Designing an eHealth breastfeeding resource with young mothers using a participatory design. J Transcult Nurs. 2021;32(3):295–303. https://doi.org/10.1177/1043659620957065.

    Article  PubMed  Google Scholar 

  44. Ahmed R, Toscos T, Rohani Ghahari R, Holden RJ, Martin E, Wagner S, et al. Visualization of cardiac implantable electronic device data for older adults using participatory design. Appl Clin Inform. 2019;10(4):707–18.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Allin S, Shepherd J, Tomasone J, Munce S, Linassi G, Hossain SN, et al. Participatory design of an online self-management tool for users with spinal cord injury: qualitative study. JMIR Rehabil Assist Technol. 2018;5(1): e6. https://doi.org/10.2196/rehab.8158.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Arevian AC, O’Hora J, Jones F, Mango J, Jones L, Williams PG, et al. Participatory technology development to enhance community resilience. Ethn Dis. 2018;28(Suppl 2):493–502. https://doi.org/10.18865/ed.28.S2.493.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Arevian AC, O’Hora J, Rosser J, Mango JD, Miklowitz DJ, Wells KB. Patient and provider cocreation of mobile texting apps to support behavioral health: usability study. JMIR Mhealth Uhealth. 2020;8(7): e12655. https://doi.org/10.2196/12655.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Ceasar J, Peters-Lawrence MH, Mitchell V, Powell-Wiley TM. The communication, awareness, relationships and empowerment (C.A.R.E.) model: an effective tool for engaging urban communities in community-based participatory research. Int J Environ Res Public Health. 2017. https://doi.org/10.3390/ijerph14111422.

    Article  PubMed  PubMed Central  Google Scholar 

  49. De Cocker K, Chastin SFM, De Bourdeaudhuij I, Imbo I, Stragier J, Cardon G. Citizen science to communicate about public health messages: the reach of a playful online survey on sitting time and physical activity. Health Commun. 2019;34(7):720–5. https://doi.org/10.1080/10410236.2018.1433955.

    Article  PubMed  Google Scholar 

  50. DeRenzi B, Dell N, Wacksman J, Lee S, Lesh N. Supporting Community Health Workers in India through Voice- and Web-Based Feedback. CHI Conference on Human Factors in Computing Systems; New York, NY, USA: Association for Computing Machinery; 2017. p. 2770–81.

  51. Dewa LH, Lawrence-Jones A, Crandell C, Jaques J, Pickles K, Lavelle M, et al. Reflections, impact and recommendations of a co-produced qualitative study with young people who have experience of mental health difficulties. Health Expect. 2021;24(Suppl 1):134–46. https://doi.org/10.1111/hex.13088.

    Article  PubMed  Google Scholar 

  52. Doty JL, Brady SS, Monardez Popelka J, Rietveld L, Garcia-Huidobro D, Doty MJ, et al. Designing a mobile app to enhance parenting skills of latinx parents: a community-based participatory approach. JMIR Form Res. 2020;4(1): e12618. https://doi.org/10.2196/12618.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Edwards C, Costello E, Cassidy N, Vick B, Russell AM. Use of the patientMpower app with home-based spirometry to monitor the symptoms and impact of fibrotic lung conditions: longitudinal observational study. JMIR Mhealth Uhealth. 2020;8(11): e16158. https://doi.org/10.2196/16158.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Elk R, Emanuel L, Hauser J, Bakitas M, Levkoff S. Developing and testing the feasibility of a culturally based tele-palliative care consult based on the cultural values and preferences of Southern, Rural African American and White Community Members: a program by and for the community. Health Equity. 2020;4(1):52–83. https://doi.org/10.1089/heq.2019.0120.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Erlingsdottir G, Rydenfalt C, Persson J, Johansson G. Conceptual Design of E-health Services by, and for Support of, Home Care Staff. Proceedings of the 20th Congress of the International Ergonomics Association (Iea 2018), Vol Vii: Ergonomics in Design, Design for All, Activity Theories for Work Analysis and Design, Affective Design. 2019; 824: 793–9. https://doi.org/10.1007/978-3-319-96071-5_83.

  56. Evans C, Turner K, Suggs LS, Occa A, Juma A, Blake H. Developing a mHealth intervention to promote uptake of HIV testing among African communities in the conditions: a qualitative study. Bmc Public Health. 2016. https://doi.org/10.1186/s12889-016-3278-4.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Ferguson M, Leighton P, Brandreth M, Wharrad H. Development of a multimedia educational programme for first-time hearing aid users: a participatory design. Int J Audiol. 2018;57(8):600–9. https://doi.org/10.1080/14992027.2018.1457803.

    Article  PubMed  Google Scholar 

  58. Ferri F, D’Andrea A, D’ulizia A, Grifoni P. Co-Creation of e-learning content: the case study of a MOOC on health and cyber-bullying. J Univ Comput Sci. 2020;26:200–19.

  59. Fletcher S, Mullett J. Digital stories as a tool for health promotion and youth engagement. Can J Public Health. 2016;107(2):e183–7.

    Article  Google Scholar 

  60. Fraco Z, Hooyer K, Rizia R, Kowser Patwary ABM, Armstrong M, Semaan B, et al. Crisis Warning Signs in mHealth for Military veterans: A Collaborative Design Approach. Proceedings of the ISCRAM 2016 Conference; Rio de Janeiro, Brazil: Tapia, Antunes, Bañuls, Moore and Porto de Albuquerque; 2016.

  61. Frauenberger C, Kender K, Scheepmaker L, Werner K, Spiel K. Desiging Social Play Things. Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society; New York, NY, USA: Association for Computing Machinery; 2020. p. 1–12.

  62. Garcia FE, Brandao RP, Mendes GCD, Neris VPD. Able to Create, Able to (Self-) improve: how an inclusive game framework fostered self-improvement through creation and play in alcohol and drugs rehabilitation. Human-computer interaction—interact 2019, Pt I. 2019;11746:337–58. https://doi.org/10.1007/978-3-030-29381-9_21.

  63. Gardsten C, Mortberg C, Blomqvist K. Designing an ICT self-management service: suggestions from persons with type 2 diabetes. Health Technol-Ger. 2017;7(2–3):197–206. https://doi.org/10.1007/s12553-016-0176-9.

    Article  Google Scholar 

  64. Garmendia M, Epelde G, Zabala B, Sucre A, Etxeberría A, Cajaraville G, et al. ONKOVIS, Co-designed Oncology Follow-up Visualisation Tool. HEALTHINF; Prague, Czech Republic. 2019.

  65. Giroux D, Tremblay M, Latulippe K, Provencher V, Poulin V, Giguere A, et al. Promoting identification and use of aid resources by caregivers of seniors: co-design of an electronic health tool. JMIR Aging. 2019;2(2):e12314. https://doi.org/10.2196/12314.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Gobl B, Hristova D, Jovicic S, Chevron MF, Slunecko T, Hlavacs H. Fostering Social Media Literacy through a Participatory Mixed-Methods Approach: Discussion of Workshop Findings. Ieee Int Conf Seriou. 2019

  67. Grant C, Widnall E, Cross L, Simonoff E, Downs J. Informing the development of an E-platform for monitoring wellbeing in schools: involving young people in a co-design process. Res Involv Engagem. 2020;6:51. https://doi.org/10.1186/s40900-020-00219-0.

    Article  PubMed  PubMed Central  Google Scholar 

  68. Haufe M, Peek STM, Luijkx KG. Matching gerontechnologies to independent-living seniors’ individual needs: development of the GTM tool. BMC Health Serv Res. 2019;19(1):26. https://doi.org/10.1186/s12913-018-3848-5.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Hernandez HA, Khan A, Fay L, Roy JS, Biddiss E. Force resistance training in hand grasp and arm therapy: feasibility of a low-cost videogame controller. Games Health J. 2018;7(4):277–87. https://doi.org/10.1089/g4h.2017.0193.

    Article  PubMed  Google Scholar 

  70. Hill ME, Bodnar P, Fenton R, Mason B, Bandoh G. Teach our children: stroke education for indigenous children, first nations, Ontario, Canada, 2009–2012. Prev Chronic Dis. 2017;14:E68. https://doi.org/10.5888/pcd14.160506.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Hiratsuka VY, Moore L, Avey JP, Dirks LG, Beach BD, Dillard DA, et al. An internet-based therapeutic tool for American Indian/Alaska native adults with posttraumatic stress disorder: user testing and developmental feasibility study. JMIR Form Res. 2019;3(4): e13682. https://doi.org/10.2196/13682.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Hong MK, Lakshmi U, Olson TA, Wilcox L. Visual ODLs: Co-Designing Patient-Generated Observations of Daily Living to Support Data-Driven Conversations in Pediatric Care. Proceedings of the 2018 Chi Conference on Human Factors in Computing Systems (Chi 2018). 2018. https://doi.org/10.1145/3173574.3174050

  73. Jarke J, Gerhard U, Kubicek H. Co-creating digital public services with older citizens: Challenges and opportunities. In: David K, Geihs K, Lange M, Stumme G, editors. INFORMATIK: 50 Jahre Gesellschaft für Informatik – Informatik für Gesellschaft; Kassel, Germany: Bonn: Gesellschaft für Informatik e.V.; 2019. p. 51–64.

  74. Jessen S, Mirkovic J, Nes LS. MyStrengths, a strengths-focused mobile health tool: participatory design and development. JMIR Form Res. 2020;4(7): e18049. https://doi.org/10.2196/18049.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Jessen S, Mirkovic J, Ruland CM. Creating gameful design in mHealth: a participatory co-design approach. JMIR Mhealth Uhealth. 2018;6(12): e11579. https://doi.org/10.2196/11579.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Jo E, Toombs AL, Gray CM, Hong H. Understanding Parenting Stress through Co-designed Self-Trackers. Proceedings of the 2020 Chi Conference on Human Factors in Computing Systems (Chi'20). 2020. https://doi.org/10.1145/3313831.3376359.

  77. Jorgensen TS, Skougaard M, Taylor PC, Asmussen HC, Lee A, Klokker L, et al. The Parker model: applying a qualitative three-step approach to optimally utilize input from stakeholders when introducing new device technologies in the management of chronic rheumatic diseases. Patient. 2018;11(5):515–26. https://doi.org/10.1007/s40271-018-0306-8.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Kanstrup AM, Bertelsen P. Bringing new voices to design of exercise technology: participatory design with vulnerable young adults. Participatory Design Conference; New York, NY, USA: Association for Computing Machinery; 2016. p. 121–30.

  79. Kearns A, Kelly H, Pitt I. Rating experience of ICT-delivered aphasia rehabilitation: co-design of a feedback questionnaire. Aphasiology. 2020;34(3):319–42. https://doi.org/10.1080/02687038.2019.1649913.

    Article  Google Scholar 

  80. Kenning G, Treadaway C. Designing for Dementia: iterative grief and transitional objects. Des Issues. 2018;34(1):42–53. https://doi.org/10.1162/DESI_a_00475.

    Article  Google Scholar 

  81. Kildea J, Battista J, Cabral B, Hendren L, Herrera D, Hijal T, et al. Design and development of a person-centered patient portal using participatory stakeholder co-design. J Med Internet Res. 2019. https://doi.org/10.2196/11371.

    Article  PubMed  PubMed Central  Google Scholar 

  82. Kim SI, Jo E, Ryu M, Cha I, Kim YH, Yoo H, et al. Toward becoming a better self: understanding self-tracking experiences of adolescents with autism spectrum disorder using custom trackers. Int Conf Per Comp. 2019. https://doi.org/10.1145/3329189.3329209.

    Article  Google Scholar 

  83. LaMonica HM, Milton A, Braunstein K, Rowe SC, Ottavio A, Jackson T, et al. Technology-enabled solutions for Australian mental health services reform: impact evaluation. JMIR Form Res. 2020;4(11): e18759. https://doi.org/10.2196/18759.

    Article  PubMed  PubMed Central  Google Scholar 

  84. Lan Hing Ting K, Dessinger G, Voilmy D. Examining usage to ensure utility: co-design of a tool for fall prevention. Irbm. 2020;41(5):286–93. https://doi.org/10.1016/j.irbm.2020.03.001.

    Article  Google Scholar 

  85. Larsen SB, Sorensen NS, Petersen MG, Kjeldsen GF. Towards a shared service centre for telemedicine: telemedicine in Denmark, and a possible way forward. Health Informatics J. 2016;22(4):815–27. https://doi.org/10.1177/1460458215592042.

    Article  PubMed  Google Scholar 

  86. Latulippe K, Hamel C, Giroux D. Co-design to support the development of inclusive eHealth tools for caregivers of functionally dependent older persons: social justice design. J Med Internet Res. 2020;22(11): e18399. https://doi.org/10.2196/18399.

    Article  PubMed  PubMed Central  Google Scholar 

  87. Latulippe K, Hamel C, Giroux D. Integration of conversion factors for the development of an inclusive ehealth tool with caregivers of functionally dependent older persons: social justice design. JMIR Hum Factors. 2020;7(3): e18120. https://doi.org/10.2196/18120.

    Article  PubMed  PubMed Central  Google Scholar 

  88. Latulippe K, Tremblay M, Poulin V, Provencher V, Giguere AM, Sevigny A, et al. Prioritizing the needs of caregivers of older adults to support their help-seeking process as a first step to developing an eHealth tool: the technique for research of information by animation of a group of experts (TRIAGE) method. JMIR Aging. 2019;2(1): e12271. https://doi.org/10.2196/12271.

    Article  PubMed  PubMed Central  Google Scholar 

  89. Lee HR, Sabanovic S, Chang WL, Nagata S, Piatt J, Bennett C, et al. Steps Toward participatory design of social robots: mutual learning with older adults with depression. Acmieee Int Conf Hum. 2017. https://doi.org/10.1145/2909824.3020237.

    Article  Google Scholar 

  90. Leeming G, Thew S. DataWell: public involvement in the development of a federated platform for shared patient records in Greater Manchester. UK Stud Health Technol. 2017;244:48–52. https://doi.org/10.3233/978-1-61499-824-2-48.

    Article  Google Scholar 

  91. Leong TW, Johnston B. Co-design and robots: a case study of a robot dog for aging people. Lect Notes Artif Int. 2016;9979:702–11. https://doi.org/10.1007/978-3-319-47437-3_69.

    Article  Google Scholar 

  92. Leslie M, Khayatzadeh-Mahani A, MacKean G. Recruitment of caregivers into health services research: lessons from a user-centred design study. Res Involv Engagem. 2019;5:17. https://doi.org/10.1186/s40900-019-0150-6.

    Article  PubMed  PubMed Central  Google Scholar 

  93. Limaye NP, Rivas-Nieto AC, Carcamo CP, Blas MM. Nuestras historias- designing a novel digital story intervention through participatory methods to improve maternal and child health in the Peruvian Amazon. PLoS ONE. 2018;13(11): e0205673. https://doi.org/10.1371/journal.pone.0205673.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  94. Livingood WC, Monticalvo D, Bernhardt JM, Wells KT, Harris T, Kee K, et al. Engaging adolescents through participatory and qualitative research methods to develop a digital communication intervention to reduce adolescent obesity. Health Educ Behav. 2017;44(4):570–80.

    Article  PubMed  Google Scholar 

  95. Lo B, Zhang T, Leung K, Mehta R, Kuziemsky C, Booth RG, et al. Identifying best approaches for engaging patients and family members in health informatics initiatives: a case study of the group priority sort technique. Res Involv Engagem. 2020;6:25. https://doi.org/10.1186/s40900-020-00203-8.

    Article  PubMed  PubMed Central  Google Scholar 

  96. Lockerbie J, Maiden N. Modelling the quality of life goals of people living with dementia. Inf Syst. 2022. https://doi.org/10.1016/j.is.2020.101578.

    Article  Google Scholar 

  97. Lu Y, Valk C, Steenbakkers J, Bekker T, Visser T, Proctor G, et al. Can technology adoption for older adults be co-created? Gerontechnology. 2017;16(3):151–9. https://doi.org/10.4017/gt.2017.16.3.004.00.

    Article  Google Scholar 

  98. Maertens JA, Jimenez-Zambrano AM, Albright K, Dempsey AF. Using community engagement to develop a web-based intervention for Latinos about the HPV vaccine. J Health Commun. 2017;22(4):285–93. https://doi.org/10.1080/10810730.2016.1275890.

    Article  PubMed  Google Scholar 

  99. Martin A, Caon M, Adorni F, Andreoni G, Ascolese A, Atkinson S, et al. A mobile phone intervention to improve obesity-related health behaviors of adolescents across Europe: iterative co-design and feasibility study. Jmir Mhealth Uhealth. 2020. https://doi.org/10.2196/14118.

    Article  PubMed  PubMed Central  Google Scholar 

  100. Matthews M, Abdullah S, Gay G, Choudhury T. Detecting and Capitalizing on Physiological Dimensions of Psychiatric Illness. Physics: Proceedings of the 3rd International Conference on Physiological Computing Systems. 2016:98–104. https://doi.org/10.5220/0005952600980104.

  101. Mburu CW, Wardle CJ, Joolay Y, Densmore M. Co-designing with Mothers and Neonatal Unit Staff: Use of Technology to Support Mothers of Preterm Infants. Proceedings of the Second African Conference for Human Computer Interaction: Thriving Communities (Africhi). 2018:118–27. https://doi.org/10.1145/3283458.3283487

  102. McMillan B, Fox S, Lyons M, Bourke S, Mistry M, Ruddock A, et al. Using patient and public involvement to improve the research design and funding application for a project aimed at fostering a more collaborative approach to the NHS health check: the CaVIAR project (better care Via improved access to records). Res Involve Engage. 2018. https://doi.org/10.1186/s40900-018-0101-7.

    Article  Google Scholar 

  103. McNaney R, Vines J, Mercer J, Mexter L, Welsh D, Young T. DemYouth: Co-Designing and Enacting Tools to Support Young People's Engagement with People with Dementia. Proceedings of the 2017 Acm Sigchi Conference on Human Factors in Computing Systems (Chi'17). 2017: 1313–25. https://doi.org/10.1145/3025453.3025558

  104. Miller AD, Pollack AH, Pratt W. Bursting the information bubble: identifying opportunities for pediatric patient-centered technology. AMIA Annu Symp Proc. 2016;2016:894–903.

    PubMed  Google Scholar 

  105. Mrklas KJ, Barber T, Campbell-Scherer D, Green LA, Li LC, Marlett N, et al. Co-design in the development of a mobile health app for the management of knee osteoarthritis by patients and physicians: qualitative study. JMIR Mhealth Uhealth. 2020;8(7): e17893. https://doi.org/10.2196/17893.

    Article  PubMed  PubMed Central  Google Scholar 

  106. Naeemabadi M, Sondergaard JH, Klastrup A, Schlunsen AP, Lauritsen REK, Hansen J, et al. Development of an individualized asynchronous sensor-based telerehabilitation program for patients undergoing total knee replacement: participatory design. Health Informatics J. 2020;26(4):2492–511. https://doi.org/10.1177/1460458220909779.

    Article  PubMed  Google Scholar 

  107. Payton FC. Cultures of participationfor students, by students. Inform Syst J. 2016;26(4):319–38. https://doi.org/10.1111/isj.12086.

    Article  Google Scholar 

  108. Rabba AS, Dissanayake C, Barbaro J. Development of a web-based resource for parents of young children newly diagnosed with autism: participatory research design. Jmir Pediatr Parent. 2020. https://doi.org/10.2196/15786.

    Article  PubMed  PubMed Central  Google Scholar 

  109. Rai T, Morton K, Roman C, Doogue R, Rice C, Williams M, et al. Optimizing a digital intervention for managing blood pressure in stroke patients using a diverse sample: integrating the person-based approach and patient and public involvement. Health Expect. 2021;24(2):327–40. https://doi.org/10.1111/hex.13173.

    Article  PubMed  Google Scholar 

  110. Rathnayake S, Moyle W, Jones C, Calleja P. Co-design of an mHealth application for family caregivers of people with dementia to address functional disability care needs. Inform Health Soc Ca. 2021;46(1):1–17. https://doi.org/10.1080/17538157.2020.1793347.

    Article  Google Scholar 

  111. Reade S, Spencer K, Sergeant JC, Sperrin M, Schultz DM, Ainsworth J, et al. Cloudy with a chance of pain: engagement and subsequent attrition of daily data entry in a smartphone pilot study tracking weather, disease severity, and physical activity in patients with rheumatoid arthritis. JMIR Mhealth Uhealth. 2017;5(3):e37. https://doi.org/10.2196/mhealth.6496.

    Article  PubMed  PubMed Central  Google Scholar 

  112. Russ S, Latif Z, Hazell AL, Ogunmuyiwa H, Tapper J, Wachuku-King S, et al. A smartphone app designed to empower patients to contribute toward safer surgical care: community-based evaluation using a participatory approach. JMIR Mhealth Uhealth. 2020;8(1): e12859. https://doi.org/10.2196/12859.

    Article  PubMed  PubMed Central  Google Scholar 

  113. Sin J, Henderson C, Woodham LA, Sese Hernandez A, Gillard S. A multicomponent eHealth intervention for family carers for people affected by psychosis: a coproduced design and build study. J Med Internet Res. 2019;21(8): e14374. https://doi.org/10.2196/14374.

    Article  PubMed  PubMed Central  Google Scholar 

  114. Skovlund PC, Nielsen BK, Thaysen HV, Schmidt H, Finset A, Hansen KA, et al. The impact of patient involvement in research: a case study of the planning, conduct and dissemination of a clinical, controlled trial. Res Involv Engage. 2020;6:43. https://doi.org/10.1186/s40900-020-00214-5.

    Article  Google Scholar 

  115. Solem IKL, Varsi C, Eide H, Kristjansdottir OB, Borosund E, Schreurs KMG, et al. A user-centered approach to an evidence-based electronic health pain management intervention for people with chronic pain: design and development of EPIO. J Med Internet Res. 2020. https://doi.org/10.2196/15889.

    Article  Google Scholar 

  116. Synnot AJ, Hawkins M, Merner BA, Summers MP, Filippini G, Osborne RH, et al. Producing an evidence-based treatment information website in partnership with people affected by multiple sclerosis. Health Sci Rep-Us. 2018. https://doi.org/10.1002/hsr2.24.

    Article  Google Scholar 

  117. Tsvyatkova D, Storni C. Designing an educational interactive eBook for newly diagnosed children with type 1 diabetes: mapping a new design space. Int J Child-Comput Interact. 2019;19:1–18.

    Article  Google Scholar 

  118. Tuckett AG, Freeman A, Hetherington S, Gardiner PA, King AC, Burnie Brae Citizen S. Older adults using our voice citizen science to create change in their neighborhood environment. Int J Environ Res Public Health. 2018. https://doi.org/10.3390/ijerph15122685.

    Article  PubMed  PubMed Central  Google Scholar 

  119. Vacca R. Brokering Data: Co-Designing Technology with Latina Teens to Support Communication with Parents Leveraging Cultural Practices of Latinx Youth through Co-Design. Proceedings of Acm Interaction Design and Children (Idc 2019). 2019; https://doi.org/10.1145/3311927.3323142

  120. Vallentin-Holbech L, Guldager JD, Dietrich T, Rundle-Thiele S, Majgaard G, Lyk P, et al. Co-creating a virtual alcohol prevention simulation with young people. Int J Env Res Pub He. 2020. https://doi.org/10.3390/ijerph17031097.

    Article  Google Scholar 

  121. Vilarinho T, Floch J, Stav E. Co-designing a mHealth application for self-management of cystic fibrosis. Lect Notes Comput Sc. 2017;10515:3–22. https://doi.org/10.1007/978-3-319-67687-6_1.

    Article  Google Scholar 

  122. Wikman A, Kukkola L, Borjesson H, Cernvall M, Woodford J, Gronqvist H, et al. Development of an internet-administered cognitive behavior therapy program (ENGAGE) for parents of children previously treated for cancer: participatory action research approach. J Med Internet Res. 2018. https://doi.org/10.2196/jmir.9457.

    Article  PubMed  PubMed Central  Google Scholar 

  123. Yarosh S, Schueller SM. “Happiness inventors”: informing positive computing technologies through participatory design with children. J Med Internet Res. 2017. https://doi.org/10.2196/jmir.6822.

    Article  PubMed  PubMed Central  Google Scholar 

  124. Spinuzzi C. The methodology of participatory design. Tech Commun-Stc. 2005;52(2):163–74.

    Google Scholar 

  125. McIntyre A. Participatory action research. Los Angeles, CA: Sage Publications; 2007.

    Google Scholar 

  126. Kindon S, Pain R, Kesby M. Participatory action research approaches and methods: connecting people, participationand place. New York: Routhledge; 2008.

    Google Scholar 

  127. Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Commun Health. 2006;60(10):854–7. https://doi.org/10.1136/jech.2004.028662.

    Article  Google Scholar 

  128. Druin A, editor Cooperative Inquiry: Developing New Technologies for Children with Children. SIGCHI Conference on Human Factors in Computing Systems; 1999: ACM, New York, NY, USA.

  129. Makhaeva J, Frauenberger C, Spiel K. Creating creative spaces for co-designing with Autistic children—the concept of a “Handlungsspielraum”. 14th Participatory Design; 2016: ACM Press, Aarhus, Denmark.

  130. Irizarry T, DeVito DA, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res. 2015;17(6): e148. https://doi.org/10.2196/jmir.4255.

    Article  PubMed  PubMed Central  Google Scholar 

  131. Bate P, Robert G. Experience-based design: from redesigning the system around the patient to co-designing services with the patient. Qual Saf Health Care. 2006;15(5):307–10. https://doi.org/10.1136/qshc.2005.016527.

    Article  PubMed  PubMed Central  Google Scholar 

  132. Ekman I, Swedberg K, Taft C, Lindseth A, Norberg A, Brink E, et al. Person-centered care–ready for prime time. Eur J Cardiovasc Nurs. 2011;10(4):248–51. https://doi.org/10.1016/j.ejcnurse.2011.06.008.

    Article  PubMed  Google Scholar 

  133. Beck K, Beedle M, van Bennekum A, Cockburn A, Cunningham W, Fowler M, et al. Principles behind the agile manifesto 2001. http://agilemanifesto.org/principles.html. Accessed Sep 2022.

  134. Bonney R, Cooper C, Ballard H. The theory and practice of citizen science: launching a new journal. Citizen Sci. 2016;1(1):1. https://doi.org/10.5334/cstp.65.

    Article  Google Scholar 

  135. Healey J, Conlon CM, Malama K, Hobson R, Kaharuza F, Kekitiinwa A, et al. Sustainability and scale of the saving mothers, giving life approach in Uganda and Zambia. Glob Health Sci Pract. 2019;7(Suppl 1):S188–206. https://doi.org/10.9745/GHSP-D-18-00265.

    Article  PubMed  PubMed Central  Google Scholar 

  136. Hodes R, Doubt J, Toska E, Vale B, Zungu N, Cluver L. The stuff that dreams are made of: HIV-positive adolescents’ aspirations for development. J Int AIDS Soc. 2018;21(Suppl):1. https://doi.org/10.1002/jia2.25057.

    Article  Google Scholar 

  137. Iles IA, Nan XL, Ma ZX, Feldman R, Butler J, Wang MQ, et al. Self-affirmation and defensive processing of graphic cigarette warning labels by African American smokers: a community-based study. Health Educ J. 2019;78(3):301–14. https://doi.org/10.1177/0017896918805129.

    Article  Google Scholar 

  138. Kalokhe AS, Iyer S, Katendra T, Gadhe K, Kolhe AR, Paranjape A, et al. Primary prevention of intimate partner violence among recently married dyads residing in the slums of Pune, India: development and rationale for a dyadic intervention. Jmir Res Protoc. 2019. https://doi.org/10.2196/11533.

    Article  PubMed  PubMed Central  Google Scholar 

  139. Greenwood DA, Litchman ML, Ng AH, Gee PM, Young HM, Ferrer M, et al. Development of the intercultural diabetes online community research council: codesign and social media processes. J Diabetes Sci Technol. 2019;13(2):176–86. https://doi.org/10.1177/1932296818818455.

    Article  PubMed  PubMed Central  Google Scholar 

  140. Ostrow L, Penney D, Stuart E, Leaf PJ. Web-based survey data collection with peer support and advocacy organizations: implications of participatory methods. Prog Commun Health Partnersh. 2017;11(1):45–52. https://doi.org/10.1353/cpr.2017.0006.

    Article  Google Scholar 

  141. Bratteteig T, Wagner I. What is a participatory design result? 14th Participatory Design Conference; 2016: ACM Press, Aarhus, Denmark.

Download references

Acknowledgements

The authors thank all colleagues of TOPFIT Citizenlab for their regular support and collaboration. Special thanks to R. Wolkorte and L. Heesink for their involvement at the start of this scoping review and K.E. Konrad and M.E.M. den Ouden for their supervision.

Funding

This study was powered by the Twente Regional Deal and received financial support from the central government’s regional budget, the province of Overijssel, the region of Twente, and the Twente Board. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

All authors designed the study protocol. JvdZ contributed to the data collection. All authors contributed to the data analysis and data interpretation. CvL undertook the drafting of the first manuscript. All authors contributed and critically reviewed the manuscript, and all approved with the final version.

Corresponding author

Correspondence to Catharina Margaretha van Leersum.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

van Leersum, C.M., Jaschinski, C., Bults, M. et al. Citizen involvement in research on technological innovations for health, care or well-being: a scoping review. Health Res Policy Sys 22, 119 (2024). https://doi.org/10.1186/s12961-024-01152-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12961-024-01152-4

Keywords