Development and psychometric validation of an age-friendly health system assessment tool in the Iranian context

Abstract

Evaluating health system interventions for elderly care necessitates a context-specific, credible and dependable instrument. This research was dedicated to creating and validating a tool to assess the health system’s age-friendliness. The study unfolded in two pivotal stages: the generation of items utilizing a hybrid model and the psychometric appraisal of the tool, encompassing both validity and reliability assessments. From an initial pool of 522 items derived from a systematic scoping review and qualitative analysis, a concise assessment tool emerged, featuring 52 items across 6 domains: governance, information, resources, service delivery, community engagement and outcomes. The 52-item tool offers a straightforward, substantiated and robust means to gauge age-friendliness, pinpoint health system deficiencies and facilitate strategic planning on the basis of its insights.

Introduction

Population ageing is a natural and inevitable process that is occurring all over the world [1], with the proportion of individuals aged 65 years and older projected to increase from 10% in 2022 to 16% by 2050 [2]. This demographic shift is expected to intensify the prevalence of chronic diseases; [3, 4]. Current statistics indicate that 88.7% of those older than 65 years have at least one chronic condition, 71.7% have two and 3.51% have three or more. Additionally, the likelihood of health issues such as falls, Alzheimer’s disease and disabilities is rising, leading to greater healthcare demands and hospitalizations among the elderly [5].

In response to these trends, the concept of an age-friendly health system (AFHS) has been introduced by international entities and health systems in countries with significant elderly populations [6]. The AFHS are designed to optimize care for older adults by reducing healthcare-associated risks and enhancing satisfaction with the care they receive. These systems strive to provide the highest value of care by proactively promoting the health and wellbeing of older individuals, addressing potential health concerns, preventing avoidable injuries and improving the quality of care for those with serious illnesses or nearing the end of their lives. Additionally, AFHS extend support to family caregivers, recognizing their crucial role in the care process [7]. The goal is to address the specific health needs of this population [8]. AFHS aim to minimize healthcare-related harm, enhance patient satisfaction and ensure the elderly receive the highest quality of care [7].

The initiative for AFHS is grounded in key aspects of geriatric care models, including leadership committed to ageing concerns, the application of evidence-based care, specialized training for caregivers of the elderly and effective care teams that prioritize measurable outcomes. It emphasizes a coordinated approach to care, involving collaboration with other organizations and engagement with patients, their families and caregivers [9].

Evaluating health systems for their age-friendliness and identifying factors that influence their performance is of utmost importance. This study aimed to develope and psychometrically evaluate a measurement tool specifically tailored to assess AFHS in the context of Iranian society. While previous research in Iran has explored age-friendly characteristics within specific elements of the healthcare system, such as hospitals or pharmacies, these studies employed limited checklists or researcher-devised instruments [10]. However, a comprehensive evaluation of the health system as a whole, considering the unique context of Iran, remains lacking. The existing studies have limitations, including restricted data sources, a small number of indicators and a lack of consideration for the opinions and preferences of elderly individuals in the design of measurement tools. Our study seeks to address these gaps by developing a robust and contextually appropriate assessment tool.

Methods

Study design

This exploratory sequential mixed method design was conducted in two main phases, including item generation and psychometric evaluation, which will be described in detail. For item generation, after conducting a systematic scoping review, a qualitative study was performed. In addition, in the second phase, a psychometric evaluation of the assessment tool was conducted.

Phase 1: item generation

Systematic scoping review: development of the conceptual model, themes and items

In this phase of study, for the development of the AFHS concept, according to the results of a systematic scoping review conducted on the basis of Arksey and O’Malley’s model, an AFHS conceptual framework was designed, and its article was published [11]. On the basis of the results of this stage, 139 items were extracted.

Conducting qualitative study

To delineate and ascertain the concept of an AFHS, along with its domains and components within Iranian society, a qualitative study employing content analysis was undertaken in 2021. This phase adhered to the guidelines proposed by Graneheim [12]. Interviewees were chosen through purposeful sampling, and the interviews were carried out and recorded. Subsequent to the extraction of semantic units, a deductive classification was applied, anchored in the conceptual framework established in the preceding phase. The procedural steps of the qualitative study are outlined below:

Study participants

The concept of an AFHS is inherently complex and requires a multidisciplinary approach. To thoroughly explore this concept and its facets, it was essential to select interviewees capable of covering all domains and components. Initially, the research team developed guidelines for conducting the interviews, which were then carried out as semi-structured, in-depth discussions with both experts in Iran’s health system and older adults. The experts chosen for this study were seasoned professionals with extensive knowledge and experience in gerontology and the health of older adults. Input was sought from specialists across various sectors, including the Geriatric Health Department of the Ministry of Health, academic institutions with geriatric disciplines (e.g. geriatric medicine, nursing and gerontology) and individuals involved in planning for the elderly. Additionally, contributions were gathered from experts at health centres in Tehran and other universities, policy-makers from the Ministry of Health and health centre managers. A systematic and intentional sampling method was employed to select these experts, whose ages ranged from 40 to 70 years and whose healthcare experience spanned 12–30 years. Specifically, the distribution of work experience among the interviewees was as follows: three individuals with 12 years, two with 15 years, one with 13 years, two with 16 years, one with 18 years, two with 20 years and one with 30 years of experience. Furthermore, 10 older adults from the community, aged between 61 and 85 years, were also interviewed to provide a comprehensive perspective.

Data collection and processing

To ensure uniformity in data collection and interviewing, the research team crafted a guide with specific questions. This guide underwent expert review to confirm its validity. The initial interview was conducted as a pilot, overseen by a professor experienced in qualitative research. Once the guide, questions and interviewees were finalized, interviews were arranged and carried out in adherence to interviewing principles, such as clarifying study goals, maintaining confidentiality, securing consent and respecting participants’ time. Expert interviews were both in-person and virtual, lasting between 45 and 70 min. For elderly participants, interviews took place in open, public settings and were shorter, ranging from 20 to 45 min, to accommodate their needs. The process continued until data saturation was achieved, meaning no new information was gleaned from additional interviews, a state referred to as “informational redundancy” [13]. This is when interviewers consistently hear the same feedback from participants [14]. Each interview was transcribed and coded immediately, resulting in 383 distinct codes. The demographic details of the experts and older adults, all of whom contributed to the full psychometric validation, are outlined below in Tables 1 and 2:

Table 1 Demographic characteristics of experts who participated in the interviews

Table 2 Demographic characteristics of older people who participated in the interviews

Data analysis

Data collection and analysis were conducted concurrently. Following Graneheim’s qualitative research guidelines, the transcribed interviews were meticulously examined to extract semantic units. Each unit was then recoded and assigned a unique code. These codes were subsequently grouped by identifying similarities, aligning them with the previously established conceptual framework. To ensure the codes’ validity and reliability, they were validated with select interviewees and independently reviewed by two researchers.

Integration of the results of qualitative and quantitative stages (pool items)

In this phase, the item pool was created by integrating indicators from the scoping review and qualitative study. Some indicators were similar or merged, suggesting the potential for further refinement. The goal was to eliminate duplicates to form a unique, non-repetitive set of indicators. The draft tool underwent multiple reviews by the research team and experts, who consolidated overlapping questions. After incorporating feedback and making necessary adjustments, the research team approved the initial version of the tool. Consequently, a preliminary assessment tool with 128 questions was established to evaluate the AFHS. The subsequent steps will involve assessing the tool’s validity and reliability.

Phase 2: psychometric evaluation

Face validity

The face validity of the AFHS assessment tool was scrutinized through both qualitative and quantitative lenses. For the qualitative aspect, drafts were disseminated to 20 experts across various fields for feedback on each item’s clarity, relevance and potential for confusion. Their insights led to subsequent revisions.

Quantitatively, the impact score (IS) for each item was determined. The same panel of experts rated the significance of each item on a scale from 1 (strongly disagree) to 5 (strongly agree). The IS was then computed using the formula: impact score = frequency (%) × importance [15]. An IS exceeding 1.5 was deemed acceptable [16].

Content validity

To ascertain the content validity of the AFHS assessment tool, a blend of qualitative and quantitative methods was employed. Initially, 20 specialists critiqued the grammar, phrasing, item placement and scaling for qualitative content validity [17]. Their feedback prompted revisions to each item. For quantitative content validity, the content validity ratio (CVR) and content validity index (CVI) were calculated. The CVR was determined by asking specialists to rate each item’s necessity on a 3-point scale, with the formula:

$${\text{CVR}}\,{ = }\,\left( {\text{nE - N/2}} \right){\text{\% N/2}}$$

where nE is the number of experts who labelled an item as Essential, and N is the total number of experts [18]. A CVR of 0.42 or higher is considered satisfactory for 20 specialists [19]. For the CVI, experts rated each item’s relevance on a 4-point scale. The CVI was computed by dividing the sum of relevant but needs revision and completely relevant scores by the number of experts. A CVI score above 0.79 indicates acceptable content validity [20].

Participants and setting

Given the specialized nature of the AFHS assessment tool, it was administered to a select group of 20 health system managers. These individuals were chosen from various departments, including senior positions such as the vice president of treatment, deputy of health, deputy of development and human resources and directors of the health deputy focussing on elderly health at the Ministry of Health. Additionally, representatives from the Secretariat of the National Aging Council, Welfare Organization and insurance organizations were included. The questionnaire was presented in person at the offices of these managers, who were then asked to evaluate each question using a Likert scale ranging from 0 to 10.

Reliability

The reliability of the AFHS  assessment tool was evaluated by using the test–retest stability approach with the Wilcoxon test. Accordingly, 20 specialists were asked to complete the assessment tool twice with a 2-week time interval. Then, the interclass correlation coefficient (ICC) was calculated. ICC values greater than 0.4 [21] and greater than 0.7 were considered acceptable.

Results

Phase 1: item generation

From the initial 522 codes – 139 from the scoping review and 383 from qualitative interviews – a reduction process was undertaken to eliminate duplicates and merge similar items. Expert feedback helped narrow down the pool to 159 unique, non-repetitive indicators, resulting in a tool with 224 distinct indicators. The draft underwent multiple reviews by the research team and experts, who consolidated items with overlapping meanings. This collaborative effort led to a consensus on the tool’s content, aligning it with the AFHS conceptual framework. The finalized tool, comprising 128 items, was now ready for psychometric evaluation.

Phase 2: psychometric evaluation

Face validity

The initial assessment tool, featuring 128 items, was presented to 20 experts who had contributed to the foundational conceptual framework. They were requested to evaluate the items for clarity, simplicity and understandability, as well as their organization within the tool. Following their feedback, the tool was refined, resulting in a reduction to 115 items. These experts were then asked to complete the tool to establish its quantitative face validity. On the basis of the impact scores (IS) calculated, 30 items were removed. The outcome was a streamlined tool with 85 questions, ready for the subsequent phase of content validity determination (Table 3).

Table 3 Demographic characteristics of experts participating in the psychometric evaluation process

Content validity

During the qualitative validity phase, the tool’s items were refined with input from experts. Subsequently, the quantitative phase involved calculating the content validity ratio (CVR) for each item using the Lawshe table. With 20 experts, items scoring below 0.42 were eliminated. No new items were suggested by the experts, resulting in a tool with 60 items.

The content validity index (CVI) was then determined on the basis of each item’s relevance, as rated by the experts. Items with a CVI score above 0.79 were retained, leading to a final selection of 52 items. These items span six domains: governance, information, resources, service delivery, social participation and outcomes, forming a comprehensive tool to assess the age-friendliness of health systems (Table 4).

Table 4 Domains and items of assessment tool for age-friendly health systems (AFHS)

Reliability

The reliability of the AFHS  assessment tool was verified through two distinct methods:

Test–Retest Reliability The tool was distributed to 20 managers twice, with a 2-week interval. The Wilcoxon signed-rank test, a non-parametric method suitable for ordinal data and small sample sizes, was employed due to the data not meeting the assumptions required for parametric testing [22]. The lack of significant differences in scores over the two administrations (p-value = 0.59) suggests the tool’s stability and consistency over time.

Inter-rater reliability Three evaluators independently assessed the AFHS using the tool. The intraclass correlation coefficient (ICC) and Cronbach’s alpha were calculated to measure observational reliability. An ICC of 0.86 and a Cronbach’s alpha of 0.94 indicate a high level of internal consistency and agreement among the evaluators’ assessments.

These results collectively demonstrate the tool’s reliability, ensuring that it produces stable and consistent evaluations when applied by different users or at different times.

Discussion

The study successfully developed an AFHS  assessment tool with robust psychometric properties. The finalized tool comprises 52 items across 6 domains: governance, information, resources, service delivery, community involvement and outcomes. Each item is rated on a 10-point Likert scale, yielding a total score range of 0 –520, where higher scores indicate a more AFHS.

Governance. This domain assesses the health system’s leadership in addressing the health needs of older adults, emphasizing the importance of a well-informed management, multi-sectoral policy-making and the inclusion of specialists, older adults and their families in decision-making processes.

Information. It focusses on the necessity of an integrated medical information system for the elderly, accessible across all health centres, alongside a functional referral system and a performance monitoring system that utilizes standard indicators to identify and address system weaknesses.

Resources. Every health system needs proper supply and allocation of resources to perform its tasks properly. The AFHS  also needs to build infrastructures that are suitable for the needs of older people. The needs of older people for long-term care centres and day care centres should be met. This dimension of the AFHS emphasizes that the provision of mobility aids and equipment for adults is one of the duties of this system. Additionally, considering the high costs of health services for elderly individuals, financial access to health services for all the elderly without catastrophic costs should be provided through the establishment of long-term care insurance and proper rehabilitation. In addition, proper training and employment of various specialists in the field of geriatrics is one of the important components evaluated in this domain of the AFHS.

Service delivery. This domain evaluates the health system’s infrastructure and resource allocation, ensuring they cater to the needs of older adults, including long-term care facilities, mobility aids and financial accessibility to health services without incurring catastrophic costs.

The service delivery domain underscores the importance of timely preventive services and screenings to mitigate disabilities and diseases among the elderly. It also emphasizes the need for acute and inpatient services to cater to the unique needs of older adults, advocating for the proper implementation of clinical guidelines by healthcare professionals. Additionally, this domain considers the benefits of home care and efforts to minimize hospital stays for the elderly.

Community involvement. reflects the essential role of societal sectors, including families, NGOs, charities and the private sector, in providing comprehensive health services to meet the diverse needs of older adults.

Outcomes. The outcomes domain focusses on the tangible results of an AFHS, such as the satisfaction of the elderly with the quality of health services and insurance coverage, as well as their contentment with long-term care facilities.

For future research, it is recommended that this tool be utilized to evaluate health systems across various populations and cultural contexts, identifying strengths and areas for improvement. Additionally, studies exploring the tool’s potential applications in health system planning and evaluation could provide valuable insights into its effectiveness.

This comprehensive approach ensures that the health system is responsive to the evolving needs of an ageing population, aiming to enhance their quality of life and healthcare experience.

Conclusions

The development of this AFHS assessment tool represents a significant step forward in evaluating healthcare services for the elderly in Iran. Its comprehensive nature, grounded in cultural and local contexts, allows for a nuanced understanding of the system’s effectiveness. The tool’s robust psychometric validation process and high reliability ensure that it can serve as a reliable instrument for policy-makers and healthcare providers to identify areas of strength and opportunities for improvement within the health system.

The tool’s six domains – health governance, information, resources, service delivery, community involvement and outcomes – cover a broad spectrum of factors that contribute to an age-friendly health system. This breadth ensures that the tool not only assesses the current state, but also guides future enhancements tailored to the needs of the elderly population. By incorporating indicators that resonate with the mental and functional realities of Iranian society, the tool also offers a culturally relevant framework that can be used to compare AFHS across different regions and cultures. Such comparisons could lead to a deeper understanding of global best practices and the adaptation of successful strategies to local contexts.

Overall, the tool’s design and validation process underscore its potential as a comprehensive and adequate measure of the AFHS, particularly within the unique cultural setting of Iran. It sets a precedent for future research and application in the ongoing effort to optimize healthcare for older adults.