Skip to main content

Cross-cultural adaption and psychometric investigation of the German version of the Evidence Based Practice Attitude Scale (EBPAS-36D)

Abstract

Background

The implementation of evidence-based practice (EBP) in mental health care confers many benefits to patients, and research into factors facilitating the implementation of EBP is needed. As an important factor affecting the implementation of EBP, service providers’ attitudes toward EBP emerged. The Evidence-Based Practice Attitude Scale (EBPAS-36) is an instrument with good psychometric characteristics that measures positive and ambivalent attitudes toward EBP. However, a German version is missing. The present study therefore aims to provide a validated German translation of the EBPAS-36.

Methods

The scale was translated and back-translated as recommended by standard procedures. German psychotherapists were recruited to participate in an online survey. They provided demographic and professional information, completed the EBPAS-36, the Implementation Climate Scale (ICS) and the Intention Scale for Providers (ISP). Standard item and reliability analyses were conducted. Construct validity was evaluated with exploratory (EFA) and confirmatory factor analyses (CFA) in two subsamples (random split). Convergent validity was tested by predicting a high positive correlation of the EBPAS-36D with two scores of attitudes of the ISP and an interest in EBP score. It was tested whether the EBPAS-36D predicts the intention to use EBP.

Results

N = 599 psychotherapists participated in the study. The item analyses showed a mean item difficulty of p= 0.64, a mean inter-item correlation of r = 0.18, and a mean item-total correlation of ritc = 0.40. The internal consistency was very good for the total scale (α = 0.89) and ranged from adequate to very good for the subscales (0.65–0.89), indicating high reliability. The original factor structure showed an acceptable model fit (RMSEA = 0.064 (90% CI = 0.059–0.068); SRMR = 0.0922; AIC = 1400.77), confirming the 12-factor structure of the EBPAS-36. However, a second-order factor structure derived by the EFA had an even better model fit (RMSEA = 0.057 (90% CI = 0.052–0.062); SRMR = 0.0822; AIC = 1274.56). When the EBPAS-36D was entered in a hierarchical regression model with the criterion Intention to use EBP, the EBPAS-36D contributed significantly to the prediction (Change in R2 = 0.28, p < 0.001) over and above gender, age and participants’ report of ever having worked in a university context.

Conclusions

The present study confirms good psychometric properties and validity of a German version of the EBPAS-36 in a sample of psychotherapists.

Peer Review reports

Contributions to the literature

  • The article provides a rigorously conducted German translation of the Evidence-Based Practice Attitudes Scale (EBPAS-36).

  • The EBPAS-36D was tested in a large sample of German psychotherapists and demonstrated good item characteristics and internal consistency.

  • The EBPAS-36D predicted the intention to use evidence-based practices.

  • The examination of the factor structure of the EPBAS-36D advances theoretical considerations regarding the underlying constructs and cross-cultural consistency of provider’s attitudes toward EBP.

Background

The Institute of Medicine defines evidence-based practice (EBP) as “the integration of best research evidence with clinical expertise and patient values” ([1, 2], p. 147). In order to maintain and improve the effectiveness of health care, the implementation of EBP in routine care is a major objective [3]. If successful, EBP implementation may result in several advantages, including participation in informed health care decisions [4], and better outcomes for patients [5], guidance for the development of treatment plans for practitioners [6], and increased cost-effectiveness of interventions for the health care system [7,8,9]. Although facilitating the uptake of EBP is in the interest of all stakeholders (including government agencies and insurance companies), a substantial gap between research and practice is evident [10]. This gap results in a large proportion of patients who receive interventions that are not justified in terms of safety, efficacy or cost-effectiveness [11,12,13]. Accordingly, in the past 20 years, enormous efforts have been made to disseminate and implement EBP in mental health care [14,15,16]. This gave rise to a rapidly growing research interest in implementation of EBP in health care [17].

Previous research efforts have identified several determinants of successful implementation of EBP. The Consolidated Framework for Implementation Research (CFIR) [18] provides a typology for the complex and interacting constructs associated with successful implementation. The major domains comprise (a) intervention characteristics, (b) outer setting, (c) inner setting, (d) characteristics of individuals and (e) the implementation process. Regarding the organizational factors of implementation success (b and c), the implementation climate of the organization in which EBP should be established is an important determinant, which can be measured with the Implementation Climate Scale [19]. Perceived barriers to adopt EBP in psychotherapy were found [20] and more positive attitudes toward EBP were linked to higher organizational support [21, 22].

The Exploration, Preparation, Implementation, Sustainment (EPIS) framework identifies provider characteristics and attitudes as important in the uptake of EBPs [23, 24]. These characteristics are most relevant in the inner context of organizations where mental health services are provided [23,24,25]. Specifically, attitudes can influence the initial decision to consider EBP, how it is implemented and whether it is sustained beyond the implementation phase [24, 26,27,28]. Demographic factors seem inconsistently related to attitudes toward EBP among mental health care providers [21, 29,30,31,32,33]: Higher age was associated with more positive attitudes in [34,35,36,37], with less positive ones in [38,39,40,41]; women were reported to show more positive attitudes in [34, 36,37,38], whereas other studies found no sex differences [26, 39, 40].

An important part of the implementation research agenda is the development of pragmatic measures capturing potentially important implementation determinants, mechanisms, and outcomes [42, 43] that promote or obstruct dissemination and implementation [44]. Martinez, Lewis and Weiner [45] identify several challenges for such instruments including the use of frameworks and theoretical models including consistent construct definitions and appropriate assessments of psychometric properties [45].

The Evidence-Based Practice Attitude Scale (EBPAS) [26] is an instrument that has been identified as a psychometrically strong measure assessing positive as well as ambivalent attitudes toward EBP [26, 34, 44]. It was developed specifically for the target group of mental health care providers, but has since been employed in broader contexts [42]. In line with suggestions put forward in the literature [44, 46], it is based on mental health dissemination and implementation theories [47,48,49] and has been developed in collaboration with service providers and researchers [26, 27, 50]. The original 15-item version showed strong psychometric properties including high validity in various settings and samples from the US, Norway, Greece and the Netherlands [34, 37, 38, 40]. In an effort to incorporate additional relevant dimensions, the 15-item EBPAS was expanded to 50 items and 12 dimensions through formative work, which included researcher input, focus groups with program managers and clinicians, and subsequent data reduction [27]. In the interest of rendering the measure more brief and pragmatic [46], the 50-item version was reduced to 36-items while retaining 12 dimensions [52]. On the one hand, the domains assess positive attitudes toward EBP: the intuitive Appeal of EBP, the willingness to adopt EBP given the Requirements to do so, providers’ Openness to new practices and manualized interventions, the Fit of EBP with values and needs of providers and clients, and providers’ perceptions of an increased Job Security provided by learning EBP, of the Organizational Support for learning EBP and of receiving Feedback [27]. On the other hand, the following domains assess ambivalent attitudes toward EBP: the Divergence between research-based interventions and current practice, the Limitations of EBP due to not addressing client needs, negative perceptions of Monitoring by supervisors, the perceived Balance of clinical skills and science in therapy, and the Burden of learning EBP [27]. The EPBAS-36 has shown good psychometric properties and cross-cultural validity in US and Norwegian samples [51].

As argued by Kien et al. [52], German instruments assessing implementation science constructs are scarce and psychometric properties rarely reported. This stands in contrast with the growing relevance and increasing efforts of implementation research in German-speaking countries [53,54,55,56,57,58]. Over the last decades, psychotherapy in these countries has experienced a significant professionalization and focus on EBP. These countries may benefit from reliable and valid instruments in implementation science. In Germany, a law (‘Psychotherapeutengesetz’) regulates the practice of psychotherapy since 1999, stating that only state approved practitioners may offer treatment [59,60,61]. In September 2019, the German parliament approved an adapted law that aims to further align the postgraduate training for psychotherapy to the structure of medical education [59]. Learning about German psychotherapists’ attitudes toward EBP may help to inform the psychotherapy training. To the best of our knowledge, two independent German translations of EBPAS-15 exist [62, 63], but no translation of the EBPAS-36 is available. Therefore, the present study aims to present a German translation of the EBPAS-36 and evaluate its psychometric properties.

Methods

Ethics

The cross-sectional online survey study was approved by the Internal Review Board of the University of Marburg (approval number: 2019-58 k). Participants received study information and provided informed consent before they were able to access the survey. Data were collected anonymously. All raw data were stored securely at the Department of Clinical Child and Adolescent Psychology at Philipps University in Marburg, Germany.

Participants

Eligible participants were licensed psychotherapists and psychiatrists for adults, children and adolescents as well as psychotherapists and psychiatrists enrolled in postgraduate training to obtain such a license. No exclusion criteria were applied.

Procedure

Translation. The translation was carried out in accordance with the WHO recommendations (www.who.int/substance_abuse/research_tools/translation/en/), including the following steps: (1) Forward translation, (2) Expert panel back-translation, (3) Pre-testing and cognitive interviewing, and (4) Final version. The EBPAS-36 [51] was translated into German by the second author (JT) (step 1) and back-translated by the bilingual English-speaking senior author (AB). The back-translation was reviewed by the original authors of the scale (MR, GAA) who provided feedback to assure the items represented the meaning and original constructs (step 2). The original and back-translated versions were then reviewed in a consensus meeting of the translating authors. Relevant items of previously available German translations of the EBPAS-15 were compared and the translations showed a good match. The consensus version was then reviewed by a group of German clinical psychotherapists (in training) and researchers (n = 26) as well as a graduate linguist for comprehensibility and wording (step 3). Their revisions were discussed and considered by the translating authors in a second consensus meeting, resulting in a final German version of the scale (step 4) (see Additional file 1, Additional file 2).

Recruitment and data collection. Data were collected (14/11/2019–27/04/2020) via an openly accessible online survey, using the scientific survey platform SoSci Survey (www.soscisurvey.de). The link was widely distributed via e-mail lists of professional psychotherapy organizations that all licensed psychotherapists are members of, universities, training institutes, and psychiatric in- and outpatient institutions as well as Facebook groups of psychotherapists and psychiatrists. On the first page of the survey, potential participants received study information and were required to provide informed consent before they were able to proceed with the survey. Additional information on the survey is found in the 'Checklist for Reporting Results of Internet E-Surveys' (CHERRIES) [64] in the Additional file 3.

Measures

Demographics and information on training and profession. Participants gave standard demographic information and professional information (university degree, license status, therapy orientation, and current occupation).

Evidence Based Practice Attitudes Scale (EBPAS-36D). The EBPAS-36D is an instrument to assess mental health providers’ attitudes toward adopting EBP [51]. The 36 items of the EBPAS-36 load on 12 subscales of three items each: Requirements, Appeal, Openness, Divergence, Limitations, Fit, Monitoring, Balance, Burden, Job security, Organizational support, and Feedback. Respondents are asked to rate their agreement with statements on a 5-point Likert scale ranging from 0 (‘not at all’) to 4 (‘to a very great extent’). Most items are worded in such a way that a higher total score indicates a more positive attitude toward the adoption of EBP; 15 items are scored reversely. A mean of the subscales can be computed to create a total scale. The German instrument can be found in the Additional file 1, Additional file 2.

Implementation Climate Scale (ICS). The ICS is an 18-item instrument measuring the implementation climate in organizations and work groups [19]. The original English version was translated into German by the first author (KS) and back-translated by the bilingual English-speaking senior author (AB). In order to adapt the scale for psychotherapists in private practice, a parallel version was constructed that captures the implementation climate in the health system. Respondents are asked to rate their agreement with statements describing how the respondents perceive the climate in the institution they work at with regard to the implementation of evidence-based interventions. A 5-point Likert scale ranging from 0 (‘not at all’) to 4 (‘to a very great extent’) was used. Six subscales can be calculated: Focus on EBP, Educational Support for EBP, Recognition for EBP, Rewards for EBP, Selection for EBP, and Selection for Openness toward EBP. Means of the subscales are computed to create a total scale. In the present study, the internal consistency for the ICS regarding organizations was Cronbach’s α = 0.91 for the total scale and between α = 0.77 (Selection for Openness) and α = 0.92 (Focus on EBP) for the subscales. For the ICS with respect to the health system, Cronbach’s α was α = 0.90 for the total scale and between α = 0.77 (Rewards for EBP) and α = 0.88 (Educational Support for EBP) for the subscales. (A separate manuscript for this measure is in preparation.)

Intention Scale for Providers (ISP). The ISP is a 70-item instrument assessing individual behavioral intentions for EBP use [65] based on the theory of planned behavior [66]. The original English version was translated into German by the first author (KS) and back-translated by the bilingual English-speaking senior author (AB). Responses are given on 7-point and 4-point rating scales. Seven subscales can be calculated. Direct measurement scales of attitudes (A-D, 5 items, α = 0.69), subjective norms (SN-D, 3 items, α = 0.85), perceived behavioral control (PBC-D, 4 items, α = 0.69) and behavioral intention (BI-D, 4 items, α = 0.89) are generated by calculating the average subscale scores. Indirect measurement scales of attitudes (A-ID, 22 items, α = 0.80), subjective norms (SN-ID, 18 items, α = 0.89) and perceived behavioral control (PBC-ID, 14 items, α = 0.89) are created by multiplying and summing up the belief and influence items (e.g., Normative Beliefs and Motivation to Comply).

Global assessments. As a subjective self-assessment measure, participants were asked to rate their interest in EBP on visual analogue scales for nine questions, e.g. “How great is your interest in evidence-based treatment methods?” (see all items in Additional file 4). A total score was computed (α = 0.84). At the end of the survey, participants were asked to rate the honesty of their responses (‘How honestly did you answer the questions of this study?’) and their self-reported tendency toward social desirability when answering the survey (‘Did social desirability play a role in the survey?’) on visual analogue scales.

Before answering the EBPAS-36D, ICS and ISP, participants were provided the following definition of evidence-based methods: “Evidence-based methods are treatment or intervention methods (in psychotherapy, e.g., certain therapy manuals; in physical medicine, e.g., medications or surgical procedures) whose effectiveness has been empirically demonstrated in various scientific studies. This can be done, for example, by demonstrating the efficacy of a psychotherapy over that of a waiting list condition or an alternative treatment.”

Statistical analysis

All statistical analyses were performed using IBM SPSS 26 for Windows (Chicago, IL, USA). For the confirmatory factor analysis (CFA), SPSS AMOS version 26.0.0 was used. P values < 0.05 were set as thresholds for statistical significance in all analyses. For the EBPAS-36D, means were computed if there was a maximum of one missing item per scale. Otherwise, respondents were excluded from analyses. For item analyses, item difficulties, corrected item-whole correlations and Cronbach’s alpha if item is deleted were calculated. To obtain internal reliability coefficients of the scales and subscales, Cronbach’s alpha was calculated. Values above 0.70 are regarded as acceptable, higher than 0.80 as good, higher than 0.90 as excellent. In order to assess construct validity, the factorial structure of EBPAS-36D was investigated by dividing the total sample randomly into two samples: With the first subsample, we conducted an exploratory factor analysis (EFA), followed by a confirmatory one (CFA) with the other subsample. Differences between both samples regarding age, gender distribution and the EBPAS-36D total scale and subscales were examined with independent t-tests. The suitability of data for EFA was assessed with the Kaiser-Meyer-Oklin (KMO) sample adequacy measure [67, 68] and Bartlett’s test [69]. To determine the number of components for the EFA, Horn's parallel analysis and Velicer's MAP test were conducted using the SPSS programs available online (https://people.ok.ubc.ca/brioconn/nfactors/nfactors.html) and the results compared [70]. Since parallel analysis of principal factor analysis tends to over-extract factors [71], parallel analysis of principal component analysis was conducted with raw data permutation and 1000 datasets. The EFA was conducted using principal axis factoring analysis with promax correlated factors rotation method. Subsequently, a CFA was conducted to test and compare the original 12-factor structure of EBPAS-36 against a second-order factor structure derived by the EFA, merging the EFA components 4 (Constraints by the institution), 5 (Monitoring) and 6 (Burden) into one second-order factor, and another second-order factor solution that was proposed by Rye et al. [41]. Maximum likelihood estimations were used. Since the Mardia-test for multivariate normal distribution is significant (z = 19.16) and all variables exceed either the limits for skewness or for excess as postulated by West et al. ([72], skewness < 2, excess < 7), an increased χ2 value was expected and the Bollen-Stine bootstrap procedure (1000 samples) was performed. The chi-square test statistic the χ2/df ratio, the root mean square error of approximation (RMSEA), the standardized root mean squared residual (SRMR), the comparative fit index (CFI) and the parsimony-adjusted comparative fit index (PCFI) were reported as fit indices. To assess the convergent validity of EBPAS-36D, the following hypotheses were tested by calculating Pearson correlation coefficients: The EBPAS-36D total scale shows a high positive correlation with the ISP direct and indirect scale scores of attitudes (A-D, A-ID), as well as the interest in EBP score. According to Cohen [73], r = 0.50 indicates high correlations. To test whether the EBPAS-36D total scale is an incremental predictor of the direct scale of behavioral intention to use EBP of the ISP (BI-D), a hierarchical linear regression analysis with the method ENTER was conducted. Gender and age (block 1), having ever worked in science (block 2), and the EBPAS-36D total scale (block 3) were successively included in the regression model to assess incremental improvements of model fit. Mean differences across gender and professional groups on the EBPAS-36D were assessed with independent t-tests. Pearson coefficients were calculated to assess correlations between age as well as demographic/professional variables and the EBPAS-36D. Lastly, Pearson correlations between ICS and EBPAS-36D were assessed. The findings are reported following the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) guideline [74] and informed by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) taxonomy [75].

Results

Participants

The link to the online survey was clicked 2.417 times. Overall, 913 participants continued after informed consent. Of these, 863 met the inclusion criteria (i.e., profession). A total of 261 participants were excluded due to drop-out before completion of the EBPAS-36D, two due to implausible answers (for example being 99 years old), one due to conspicuous response patterns in EBPAS-36D (e.g., straight-lining despite reverse coded items). Of the remaining 599 participants, 502 were female (83.8%) and their age ranged from 23 to 82 years (M = 36.62, SD = 11.26). Roughly half of the sample (56.26%) stated being in postgraduate training to become psychotherapists or psychiatrists. 42.7% of the German psychotherapists reported ever having worked in science. Further information on profession is presented in Table 1.

Table 1 Demographics and information on profession

Item analysis

Detailed information on valid n and missing values for the EBPAS-36D items are found in Table 2. Item difficulties of EBPAS-36D ranged between pi = 0.21 (item 28) and pi = 0.93 (item 13) with a mean difficulty of pi = 0.64. The mean inter-item correlation was r = 0.18. The item-total correlations of the individual items with the total scale ranged from ritc = 0.07 (item 26) to ritc = 0.62 (item 2) with a mean item-total correlation of ritc = 0.40. Eight items showed item-total correlations under 0.30 (see Table 2). Considering the subscales, the correlations of the individual items with their subscales ranged from ritc = 0.44 (item 5) to ritc = 0.87 (item 9 and 29).

Table 2 Item analyses of EBPAS-36D

Reliability

The internal consistency of the EBPAS-36D total scale was α = 0.89 and would not have benefitted from removing any item. Internal consistencies of the EBPAS-36D subscales were Requirements α = 0.89; Appeal α = 0.69; Openness α = 0.75; Divergence α = 0.65; Limitations α = 0.82; Fit α = 0.68; Monitoring α = 0.77; Balance α = 0.65; Burden α = 0.81; Job security α = 0.89; Organizational support α = 0.85; Feedback α = 0.76.

Subscale correlations

The correlation coefficients between the EBPAS-36D total scale and the 12 subscales are presented in Table 3. The highest correlation was between the total scale and the Openness subscale (r = 0.689). On subscale level, high correlations were between the Appeal and Fit subscales (r = 0.609), the Divergence and Limitations subscales (r = 0.550), the Openness and Divergence subscales (r = -0.531), the Appeal and Openness subscales (r = 0.514) and the Job Security and Organization Support subscales (r = 0.547).

Table 3 Pearson correlations of EBPAS-36D total scale, EBPAS-36D subscales and age

Validity

For analyses regarding factorial validity, 25 (4.2%) of the participants were excluded as they had more than one missing item in at least one subscale of the EBPAS-36D, so that no means could be calculated. Of those 25 excluded participants, 80.0% dropped out of the survey during the EBPAS-36D after item 15. Of those included in analyses (n = 574), six participants (1.0%) had missing information on only one item of EBPAS-36D, none had more than two items missing. More than 98.9% of the included participants answered all items. Split samples for EFA and CFA showed no group differences on mean age, gender, and EBPAS-36D total scale or subscales (see Additional file 5).

EFA. The sample adequacy measure (KMO = 0.844) and significant Bartlett’s test (χ2 (630) = 5616.83, p < 0.001) indicated suitability of data (n = 296) for analysis. Parallel analysis and MAP test both recommended the extraction of six factors. The six extracted factors accounted for 57.56% of variance. The rotated factor matrix is found in the Additional file 6. Eleven items loaded on factor 1 and explained 24.33% of the variance (factor loadings from 0.357 to 0.844). Ten items loaded on factor 2 and explained 9.41% of the variance (0.410 to 0.773). Factor 3 comprised six items that explained 7.92% of the variance (0.343 to 0.969). Three items loaded on factor 4 and explained 5.93% of the variance (0.735 to 0.946). Three items loaded on factor 5 and explained 5.27% of the variance (0.549 to 0.815). Factor 6 comprises three items that explained 4.71% of variance (0.693 to 0.841). With the exception of item 2 of the Openness subscale, items of the original subscales loaded on the extracted factors together. Item 3 of the Openness subscale showed nearly equally high factor loadings on factor 1 (0.357) and factor 2 (-0.335).

CFA. The path diagrams of the original 12-factor structure (model A), a second-order 4-factor structure derived by the EFA (model B) and the second-order 3-factor model established by Rye et al. [41] (model C) are shown in Figs. 1, 2 and 3, respectively. Although the model fit of the original factor structure was adequate, both second-order models showed even better model fits (see Table 4). For all three models, all regression weights were significant.

Fig. 1
figure1

Model A: original 12-factor structure

Fig. 2
figure2

Model B: second-order 4-factor structure derived by EFA

Fig. 3
figure3

Model C: second-order 3-factor model established by Rye et al. [41]

Table 4 CFA: model fit indices

Correlation analyses. The EBPAS-36D total scale correlated with the direct scale of attitudes (A-D, r = 0.663, p < 0.001, n = 574) and the indirect scale of attitudes (A-ID, r = 0.531, p < 0.001, n = 126) of the ISP. The global 9-item assessment of participant’s interest in EBP correlated with the EBPAS-36D total scale (r = 0.529, p < 0.001, n = 566).

Regression analysis. The EBPAS-36D total scale was included in a hierarchical regression model to predict the Behavioral Intention Scale of the ISP as a third block, subsequent to the predictors gender and age (block 1) and ever having worked in science (block 2). The inclusion improved the model fit (Change in R2 = 0.28, F = 267.32, p < 0.001) and the significant regression coefficient of the EBPAS-36D total scale (ß = 2.13; t = 16.35; p < 0.001) indicated incremental prediction beyond the previous predictors (see Tables 5 and 6).

Table 5 Model summary
Table 6 Regression coefficients

Group differences and correlations

Compared to women, men scored lower on the EBPAS-36D total scale (t (570) = 2.59; p = 0.010) and the subscales Requirements (t (592) = 2.91; p = 0.004), Appeal (t (595) = 3.73; p < 0.001), Fit (t (595) = 2.91; p = 0.004) and Organizational Support (t (579) = 2.66; p = 0.008). Age was correlated with the EBPAS-36D total scale and all subscales (see Table 3), indicating more negative attitudes toward EBP with increasing age. Consistent with this, licensed psychotherapists differed from psychotherapists in training on all subscales and the total scale of the EBPAS-36D (t (445.82) = − 7.581; p < 0.001; d = − 0.72), indicating more positive attitudes of psychotherapists in training, which is consistent with prior findings regarding professional development level and attitudes toward EBP [26].

The EBPAS-36D total scale was associated with the total score of the ICS on organizations and work groups (r = 0.432, p < 0.001, n = 408), but not with the total scale of the ICS on the health system (r = 0.138, p = 0.080) that was exclusively completed by licensed psychotherapists and psychiatrists working in private practices (n = 161). The self-rated honesty when answering the survey showed a small correlation with the EBPAS-36D total scale (r = 0.146, p = 0.001, n = 556).

Discussion

The present study is the first to present a German version of the EBPAS-36 and investigate its validity and psychometric properties. In a sample of psychotherapists and psychiatrists, the original factor structure was confirmed and the EBPAS-36D demonstrated good item properties, internal consistency and convergent validity.

Rising implementation research efforts in German-speaking countries necessitate the development and psychometric examination of German instruments assessing implementation research constructs [52]. Regarding characteristics of individuals, additional well-suited instruments have been translated and validated during the course of our study, for example the Evidence-based Practice Inventory (EBPI) questionnaire [76]. The EBPI assesses health care providers’ adherence to EBP as well as barriers and facilitators for the use of EBP. A total of 26 items load on five domains: attitude, subjective norm, perceived behavioral control, decision making and intention and behavior. A German version was adopted and its reliability was examined in a nationwide online survey [76]. The EBPI and EBPAS-36D could thus complement each other well in future implementation research studies, with the latter focusing on attitudes and capturing diverse aspects of positive and ambivalent attitudes toward EBP. The evaluation of providers’ attitudes toward EBP with help of both instruments might inform about successful strategies in implementation efforts in research in German-speaking countries as well as potential targets for improvement in clinical training and practice. For example, The Leadership and Organizational Change for Implementation (LOCI) strategy can be used to improve workplace climate for EBP, which should then influence provider attitudes toward, and use of, EBP with fidelity [77, 78]. Thus, attitudes could be considered a mechanism by which an implementation strategy has its effects on clinical practice. This is consistent with an implementation science approach where it is recommended to identify and integrate the use of implementation frameworks and strategies to address implementation determinants, mechanisms, and outcomes [79, 80].

Regarding the psychometric properties of the individual items, most item difficulties were in the medium range. In the context of attitude measurements, a high item difficulty translates into low endorsement of the item. Medium difficulty is desirable as it is optimal to differentiate between respondents with different attitudes. Items of the subscales Fit, Feedback and Appeal received high approval (subscale means > 3.2), whereas items of the subscales Burden, Job Security and Divergence were less strongly endorsed (subscale means < 1.3). Removing any item would not have improved the internal consistency of the total scale and could result in poor content validity. While correlations between items and their subscales were at least in the medium range, item-whole correlations demonstrated considerable variability. Consistent with this, the subscales Fit, Monitoring, Burden and Feedback showed only moderate correlations with the EBPAS-36D total scale and only few subscales showed high inter-correlations, namely the subscales Appeal and Fit, Divergence and Limitations, Openness and Divergence, Appeal and Openness, and Job Security and Organizational Support. This accords well with previous results in US and Norwegian examinations of the EBPAS-36 that demonstrated high inter-correlations only between the Appeal and Organizational Support subscales (US) and between the Limitations and Divergence, and the Job Security and Organizational Support subscales (Norway) [51].

The internal consistency of the EBPAS-36D total scale obtained is good and comparable to those found for the US and Norwegian versions. The internal consistencies of the subscales ranged from acceptable to good, with the subscales Appeal, Divergence, Fit and Balance demonstrating the lowest internal consistencies, as seen in the Norwegian sample [51]. Due to their limited reliability, these subscales should be interpreted with caution. In consideration of the extreme brevity of the subscales (3 items), the overall reliability of the EBPAS-36D subscales can be considered high.

The CFA confirmed the 12-factor structure of the original EBPAS-36 by demonstrating adequate model fit. Nonetheless, two second-order factor structures, one derived by EFA in the present sample and one that was proposed by Rye et al. [41], showed even better model fits. Therefore, second-order models might map the actual underlying construct of attitudes toward EBP even better than the original factor structure. The four second-order constructs we found are: Positive alignment with EBP, consisting of the Openness, Appeal, Fit and Feedback subscales, Reservations toward EBP, consisting of the Divergence, Limitations and Balance subscales, Institutional Endorsement, consisting of the Job Security and Organizational Support subscales and Constraints by the institution, consisting of the Requirements, Monitoring and Burden subscales.

As expected, the EBPAS-36D showed high correlations with two other scales assessing attitudes toward the use of EBP, the direct and indirect measurement scales of attitudes of the ISP. This confirms the convergent validity of the scale. Accordingly, a high correlation was found between the EBPAS-36D and a global assessment of participant’s interest in EBP, a nine-item subjective self-rating of one’s interest in research on psychotherapy and clinical psychology, clinical guidelines, EBP and treatment manuals with high face validity. Moreover, the EBPAS-36D allowed incremental prediction of the intention to use EBP as assessed with the Behavioral Intention Scale of the ISP beyond gender, age and working in science.

Responders of the 50-item version of the scale commented on being annoyed and fatigued by answering the items [51]. This might result in response biases and missing answers, limiting the validity of the scale. The acceptability of the shorter EBPAS-36D appears to be high as indicated by a low amount of missing answers in the current study. Further, the EBPAS-36D is 28% shorter than the prior 50-item version and is consistent with calls for brief and pragmatic measures in implementation research [46]. With its 36 items and 12 first order subscales, the EBPAS-36D is a complex instrument and the reader may wonder about its feasibility. However, evidence-based practice itself is a complex construct and the attitudes of health professionals may vary on a number of dimensions. In our view, it is helpful to represent these dimensions in a detailed fashion on a measurement instrument. The most likely contexts of measurement will be evaluations of existing services and its stakeholders prior to the implementation of changes. In these contexts, a fine-grained assessment of the attitudes and views of the professionals may offer information about problem areas that might need attention in transformation processes (i.e. providing training, identifying obstacles). Given that the target group who will complete the questionnaire are health-professionals who are generally acquainted with such instruments, we feel confident that the length and complexity should not present any barriers.

In the present sample of German psychotherapists, higher age was associated with less favorable attitudes toward EBP. Consistent with this, licensed psychotherapists reported more negative attitudes compared to psychotherapists in training. While this result is in contrast to two previous studies reporting higher scores on the Requirements and Openness subscales of the EBPAS with increasing age [34, 35], it is in accordance with other studies [38,39,40,41]. As assumed by van Sonsbeek et al. [37], professionals may rate their own clinical experience higher than EBP with increasing age and experience. However, it should be noted that as yet, age effects may be confounded by cohort effects, since psychotherapy training underwent considerable changes in Germany over the last 30 years. Concerning sex differences, women reported more positive attitudes toward EBP in the present study. Sex differences were apparent for the total scale and the subscales Requirements, Appeal, Fit and Organizational Support. This result adds to other research demonstrating more positive attitudes toward EBP in women [34, 36,37,38, 41]. Still, these sex differences were not found consistently [26, 39, 40], which is why possible moderators should be investigated in future research.

As demonstrated in previous studies [27, 35], provider’s attitudes toward EBP were associated with organizational climate for the implementation of EBP. Since psychotherapists working in private practices were unable to rate the implementation climate of any organization or work group, those participants were asked to rate a parallel version of the ICS, capturing the implementation climate in the German health system. Interestingly, individuals’ attitudes toward EBP were not associated with their evaluation of the implementation climate of the health system. One reason for this might be that the German health system does not provide support for EBP to a similar extent as some organizations do, thus not leading to more positive attitudes toward EBP [28]. Another reason might be that psychotherapists with a positive attitude toward EBP are able to choose organizations with a better implementation climate for EBP or affect the implementation climate of the organization they work at, while they are probably unable to choose a health system according to its implementation climate or affect the implementation climate of the health system.

When interpreting the results of the present study, some limitations must be borne in mind. All data are based on self-reports in a cross-sectional online survey. A large proportion of the sample reported ever having worked in science and having a cognitive behavioral therapy approach. The findings refer to a convenience sample that is most likely self-selected for interest in EBP and not representative [81] of the population of mental health providers. This does not affect the evaluation of the psychometric properties of the EPBAS-36D; still future studies with representative samples should seek to confirm the results. Although the dropout rate in the present study can be considered as average for online surveys [82], a selection bias cannot be precluded with only particularly interested participants completing the survey [83]. The total number of items of the survey would have been significantly reduced if we have had the EBPI available when we planned our study. It would have been fruitful to use it instead of the ISP to investigate the convergent validity of the EBPAS-36D not only due to its smaller item number but because of the availability of a German version instrument that has been psychometrically examined [76]. In future studies, this well-suited questionnaire might be used to further validate the EBPAS-36D. Finally, although a definition of EBP preceded the questionnaire, some comments indicated that individual participants may have been uncertain about the exact meaning of EBP. Future research should assess the comprehensibility of the definition to ensure that all participants rate the same construct.

Conclusions

The consideration of providers’ attitudes toward EBP in implementation research might inform about successful strategies to address their readiness to implement EBP, while in clinical practice this could point to important targets for addressing in training and supervision. Therefore, reliable instruments assessing attitudes toward EBP could be useful for researchers, training directors, and supervisors. Although further validating research is required, the present study confirms good psychometric properties and validity of a German version of the EBPAS-36 in a sample of psychotherapists. The proposed second-order model of attitudes toward EBP may initiate further research on the construct of attitudes toward EBP.

Availability of data and materials

The datasets analyzed in the current study are available from the corresponding author on reasonable request.

Abbreviations

AIC:

Akaike information criterion

CBT:

Cognitive behavioral therapy

CFA:

Confirmatory factor analysis

CFI:

Comparative Fit Index

CFIR:

Consolidated Framework for Implementation Research

CHERRIES:

Checklist for Reporting Results of Internet E-Surveys

COSMIN:

COnsensus-based Standards for the selection of health Measurement INstruments

EBP:

Evidence-based practices

EBPAS:

Evidence-based Practice Attitudes Scale

EBPI:

Evidence-based practice inventory

EFA:

Exploratory factor analysis

EPIS:

Exploration, preparation, implementation, sustainment framework

ICS:

Implementation Climate Scale

ISP:

Intention Scale for Providers

PCFI:

Parsimony-adjusted Comparative Fit Index

RMSEA:

Root Mean Square Error of Approximation

SRMR:

Standardized Root Mean Squared Residual

STROBE:

STrengthening the Reporting of OBservational studies in Epidemiology

US:

United States

References

  1. 1.

    Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. Washington: National Academy Press; 2010. https://www.nap.edu/catalog/10027/crossing-the-quality-chasm-a-new-health-system-for-the.

  2. 2.

    Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71–2.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  3. 3.

    American Psychological Association. Presidential task force on evidence-based practice. Evid Based Pract Psychol Am Psychol. 2006;61(4):271–85.

    Google Scholar 

  4. 4.

    Adams JR, Drake RE. Shared decision-making and evidence-based practice. Community Ment Health J. 2006;42(1):87–105.

    PubMed  Article  Google Scholar 

  5. 5.

    Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50.

    PubMed  Article  Google Scholar 

  6. 6.

    Cook SC, Schwartz AC, Kaslow NJ. Evidence-based psychotherapy: advantages and challenges. Neurotherapeutics. 2017;14(3):537–45.

    PubMed  PubMed Central  Article  Google Scholar 

  7. 7.

    Andrews G, Issakidis C, Sanderson K, Corry J, Lapsley H. Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders. Br J Psychiatry. 2004;184(6):526–33.

    PubMed  Article  Google Scholar 

  8. 8.

    Fortney JC, Pyne JM, Burgess JF. Population-level cost-effectiveness of implementing evidence-based practices into routine care. Health Serv Res. 2014; 49(6):1832–51. https://doi.org/10.1111/1475-6773.12247.

  9. 9.

    Hoagwood KE, Olin SS, Horwitz S, McKay M, Cleek A, Gleacher A, et al. Scaling up evidence-based practices for children and families in New York State: toward evidence-based policies on implementation for state mental health systems. J Clin Child Adolesc Psychol. 2014;43(2):145–57.

    PubMed  PubMed Central  Article  Google Scholar 

  10. 10.

    Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20.

    PubMed  PubMed Central  Article  Google Scholar 

  11. 11.

    Haines A, Kuruvilla S, Borchert M. Bridging the implementation gap between knowledge and action for health. Bull World Health Organ. 2004;82(10):724–31; discussion 732.

    PubMed  PubMed Central  Google Scholar 

  12. 12.

    Münzer A, Rosner R, Ganser HG, Naumann A, Plener PL, Witt A, et al. Usual care for maltreatment-related pediatric posttraumatic stress disorder in Germany. Z Kinder Jugendpsychiatr Psychother. 2018;46(2):135–41.

    PubMed  Article  Google Scholar 

  13. 13.

    Ganser HG, Münzer A, Plener PL, Witt A, Goldbeck L. Kinder und Jugendliche mit Misshandlungserfahrungen: Bekommen sie die Versorgung, die sie brauchen? Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. [Children and adolescents after child abuse and neglect: Do they receive appropriate treatment?] 2016;59(6):803–10.

  14. 14.

    Beidas RS, Williams NJ, Becker-Haimes EM, Aarons GA, Barg FK, Evans AC, et al. A repeated cross-sectional study of clinicians’ use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia. Implement Sci. 2019;14(1):67.

    PubMed  PubMed Central  Article  Google Scholar 

  15. 15.

    Chambless DL, Ollendick TH. Empirically supported psychological interventions: controversies and evidence. Annu Rev Psychol. 2001;52:685–716.

    CAS  PubMed  Article  Google Scholar 

  16. 16.

    McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. Am Psychol 2010; 65(2):73–84. https://psycnet.apa.org/record/2010-02208-010?doi=1.

  17. 17.

    Dearing JW, Kee KF, Peng T-Q. Historical roots of dissemination and implementation science. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. 2nd ed. New York: Oxford University Press; 2012.

    Google Scholar 

  18. 18.

    Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

    PubMed  PubMed Central  Article  Google Scholar 

  19. 19.

    Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implement Sci. 2014;9(1):157.

    PubMed  PubMed Central  Article  Google Scholar 

  20. 20.

    Cook JM, Biyanova T, Coyne JC. Barriers to adoption of new treatments: an internet study of practicing community psychotherapists. Adm Policy Ment Health. 2009;36(2):83–90.

    PubMed  Article  Google Scholar 

  21. 21.

    Powell BJ, Mandell DS, Hadley TR, Rubin RM, Evans AC, Hurford MO, et al. Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study. Implement Sci. 2017;12(1):64.

    PubMed  PubMed Central  Article  Google Scholar 

  22. 22.

    Barnett M, Brookman-Frazee L, Regan J, Saifan D, Stadnick N, Lau A. How intervention and implementation characteristics relate to community therapists’ attitudes toward evidence-based practices: a mixed methods study. Adm Policy Ment Health. 2017;44(6):824–37.

    PubMed  PubMed Central  Article  Google Scholar 

  23. 23.

    Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23.

    PubMed  Article  Google Scholar 

  24. 24.

    Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework. Implement Sci. 2019;14(1):1.

    PubMed  PubMed Central  Article  Google Scholar 

  25. 25.

    Lengnick-Hall R, Willging C, Hurlburt M, Fenwick K, Aarons GA. Contracting as a bridging factor linking outer and inner contexts during EBP implementation and sustainment: a prospective study across multiple US public sector service systems. Implement Sci. 2020;15(1):43.

    PubMed  PubMed Central  Article  Google Scholar 

  26. 26.

    Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the evidence-based practice attitude scale (EBPAS). Ment Health Serv Res. 2004;6(2):61–74.

    PubMed  PubMed Central  Article  Google Scholar 

  27. 27.

    Aarons GA, Cafri G, Lugo L, Sawitzky A. Expanding the domains of attitudes towards evidence-based practice: the evidence based practice attitude scale-50. Adm Policy Ment Health. 2012;39(5):331–40.

    PubMed  Article  Google Scholar 

  28. 28.

    Aarons GA, Sommerfeld DH, Walrath-Greene CM. Evidence-based practice implementation: the impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implement Sci. 2009;4(1):83.

    PubMed  PubMed Central  Article  Google Scholar 

  29. 29.

    Beidas RS, Stewart RE, Walsh L, Lucas S, Downey MM, Jackson K et al. Free, brief, and validated: standardized instruments for low-resource mental health settings. Cogn Behav Pract. 2015; 22(1):5–19. http://www.sciencedirect.com/science/article/pii/S1077722914000145.

  30. 30.

    Connors EH, Schiffman J, Stein K, LeDoux S, Landsverk J, Hoover S. Factors associated with community-partnered school behavioral health clinicians’ adoption and implementation of evidence-based practices. Adm Policy Ment Health. 2019;46(1):91–104.

    PubMed  Article  Google Scholar 

  31. 31.

    Farahnak LR, Ehrhart MG, Torres EM, Aarons GA. The influence of transformational leadership and leader attitudes on subordinate attitudes and implementation success. J Leadersh Organ Stud. 2020;27(1):98–111.

    Article  Google Scholar 

  32. 32.

    Locke J, Lawson GM, Beidas RS, Aarons GA, Xie M, Lyon AR, et al. Individual and organizational factors that affect implementation of evidence-based practices for children with autism in public schools: a cross-sectional observational study. Implement Sci. 2019;14(1):29.

    PubMed  PubMed Central  Article  Google Scholar 

  33. 33.

    Okamura KH, Hee PJ, Jackson D, Nakamura BJ. Furthering our understanding of therapist knowledge and attitudinal measurement in youth community mental health. Adm Policy Ment Health. 2018;45(5):699–708.

    PubMed  Article  Google Scholar 

  34. 34.

    Aarons GA, Glisson C, Hoagwood K, Kelleher K, Landsverk J, Cafri G. Psychometric properties and US National norms of the evidence-based practice attitude scale (EBPAS). Psychol Assess. 2010;22(2):356–65.

    PubMed  Article  Google Scholar 

  35. 35.

    Aarons GA, Sawitzky AC. Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychol Serv. 2006;3(1):61–72.

    PubMed  PubMed Central  Article  Google Scholar 

  36. 36.

    Aarons GA. Transformational and transactional leadership: association with attitudes toward evidence-based practice. Psychiatr Serv. 2006;57(8):1162–9.

    PubMed  PubMed Central  Article  Google Scholar 

  37. 37.

    van Sonsbeek MAMS, Hutschemaekers GJM, Veerman JW, Kleinjan M, Aarons GA, Tiemens BG. Psychometric properties of the Dutch version of the Evidence-Based Practice Attitude Scale (EBPAS). Health Res Policy Syst. 2015;13:69.

    PubMed  PubMed Central  Article  Google Scholar 

  38. 38.

    Egeland KM, Ruud T, Ogden T, Lindstrøm JC, Heiervang KS. Psychometric properties of the Norwegian version of the Evidence-Based Practice Attitude Scale (EBPAS): to measure implementation readiness. Health Res Policy Syst. 2016;14(1):47.

    PubMed  PubMed Central  Article  Google Scholar 

  39. 39.

    Gray MJ, Elhai JD, Schmidt LO. Trauma professionals’ attitudes toward and utilization of evidence-based practices. Behav Modif. 2007;31(6):732–48.

    PubMed  Article  Google Scholar 

  40. 40.

    Melas CD, Zampetakis LA, Dimopoulou A, Moustakis V. Evaluating the properties of the Evidence-Based Practice Attitude Scale (EBPAS) in health care. Psychol Assess. 2012;24(4):867–76.

    PubMed  Article  Google Scholar 

  41. 41.

    Rye M, Friborg O, Skre I. Attitudes of mental health providers towards adoption of evidence-based interventions: relationship to workplace, staff roles and social and psychological factors at work. BMC Health Serv Res. 2019;19(1):110.

    PubMed  PubMed Central  Article  Google Scholar 

  42. 42.

    Lewis CC, Boyd MR, Walsh-Bailey C, Lyon AR, Beidas R, Mittman B, et al. A systematic review of empirical studies examining mechanisms of implementation in health. Implement Sci. 2020;15(1):21.

    PubMed  PubMed Central  Article  Google Scholar 

  43. 43.

    Stanick CF, Halko HM, Dorsey CN, Weiner BJ, Powell BJ, Palinkas LA, et al. Operationalizing the “pragmatic” measures construct using a stakeholder feedback and a multi-method approach. BMC Health Serv Res. 2018;18(1):882.

    PubMed  PubMed Central  Article  Google Scholar 

  44. 44.

    Lewis CC, Weiner BJ, Stanick C, Fischer SM. Advancing implementation science through measure development and evaluation: a study protocol. Implement Sci. 2015;10(1):102.

    PubMed  PubMed Central  Article  Google Scholar 

  45. 45.

    Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implement Sci. 2014;9:118.

    PubMed  PubMed Central  Article  Google Scholar 

  46. 46.

    Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. Am J Prev Med 2013; 45(2):237–43. http://www.sciencedirect.com/science/article/pii/S0749379713002651.

  47. 47.

    Glisson C. The organizational context of children’s mental health services. Clin Child Fam Psychol Rev. 2002;5(4):233–53.

    PubMed  Article  Google Scholar 

  48. 48.

    Schoenwald SK, Hoagwood K. Effectiveness, transportability, and dissemination of interventions: what matters when? Psychiatr Serv. 2001;52(9):1190–7.

    CAS  PubMed  Article  Google Scholar 

  49. 49.

    Simpson D. A conceptual framework for transferring research to practice. J Subst Abuse Treat. 2002;22(4):171–82.

    PubMed  Article  Google Scholar 

  50. 50.

    Ajzen I. The theory of planned behaviour: reactions and reflections. Psychol Health. 2011;26(9):1113–27.

    PubMed  Article  Google Scholar 

  51. 51.

    Rye M, Torres EM, Friborg O, Skre I, Aarons GA. The Evidence-based Practice Attitude Scale-36 (EBPAS-36): a brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples. Implement Sci. 2017;12(1):44.

    PubMed  PubMed Central  Article  Google Scholar 

  52. 52.

    Kien C, Schultes M-T, Szelag M, Schoberberger R, Gartlehner G. German language questionnaires for assessing implementation constructs and outcomes of psychosocial and health-related interventions: a systematic review. Implement Sci. 2018;13(1):150.

    PubMed  PubMed Central  Article  Google Scholar 

  53. 53.

    Härter M, Bermejo I, Ollenschläger G, Schneider F, Gaebel W, Hegerl U et al. Improving quality of care for depression: the German Action Programme for the implementation of evidence-based guidelines. Int J Qual Health Care. 2006; 18(2):113–9. https://academic.oup.com/intqhc/article/18/2/113/1844349.

  54. 54.

    Strauß B. Aktueller Stand der Reform der Psychotherapieausbildung: Zwischen Aufbruchstimmung und Verlustängsten. [Current situation of the reform of psychotherapy training: Between spirit of optimism and fear of separation.] Psychotherapeut. 2013;58(2):184–90.

  55. 55.

    Ullrich C, Mahler C, Forstner J, Szecsenyi J, Wensing M. Teaching implementation science in a new Master of Science Program in Germany: a survey of stakeholder expectations. Implement Sci. 2017;12(1):55.

    PubMed  PubMed Central  Article  Google Scholar 

  56. 56.

    Christiansen H, Bauer A, Fatima B, Goodyear M, Lund IO, Zechmeister-Koss I, et al. Improving identification and child-focused collaborative care for children of parents with a mental illness in Tyrol, Austria. Front Psychiatry. 2019;10:233.

    PubMed  PubMed Central  Article  Google Scholar 

  57. 57.

    Berthoud L, Turcotte M, Dzemaili S, Ducraux D, Zumstein-Shaha M, Ortoleva BC. Facilitators and barriers in implementing clinical nursing assessment in mental health care for older people: a multimethod approach. Nurs Health Sci. 2020. https://doi.org/10.1111/nhs.12796.

    Article  Google Scholar 

  58. 58.

    Rosner R, Barke A, Albrecht B, Christiansen H, Ebert DD, Lechner-Meichsner F, et al. BEST FOR CAN—bringing empirically supported treatments to children and adolescents after child abuse and neglect: study protocol. Eur J Psychotraumatol. 2020;11(1):1837531.

    PubMed  PubMed Central  Article  Google Scholar 

  59. 59.

    Melcop N, Werder T von, Sarubin N, Benecke A. The role of psychotherapy in the German Health Care System: training requirements for psychological psychotherapists and child and adolescent psychotherapists, legal aspects, and health care implementation. Clin Psychol Eur. 2019; 1(4):1–12. https://cpe.psychopen.eu/index.php/cpe/article/view/2549.

  60. 60.

    Strauss B. The psychotherapeutic professions in Germany. Society for Psychotherapy Research; 2008. (Reports on psychotherapeutic professions, SPR conference in Barcelona.). https://www.psychotherapyresearch.org/page/SPRCultureReport.

  61. 61.

    Strauss B. Patterns of psychotherapeutic practice and professionalisation in Germany. Eur J Psychother Couns. 2009;11(2):141–50.

    Article  Google Scholar 

  62. 62.

    Frantz I, Heinrichs N. Implementation von in der Forschung untersuchten Präventionsprogrammen in die Praxis: Akzeptanz und Barrieren. [Implementation of Evidence-Based Prevention Programs Into Practice: Attitudes and Barriers] Z Klin Psychol Psychother. 2015;44(1):56–61.

  63. 63.

    James S, Lampe L, Behnken S, Schulz D. Evidence-based practice and knowledge utilisation—a study of attitudes and practices among social workers in Germany. Eur J Soc Work. 2019;22(5):763–77.

    Article  Google Scholar 

  64. 64.

    Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6(3):e34.

    PubMed  PubMed Central  Article  Google Scholar 

  65. 65.

    Burgess S, Propper C, Ratto M, Tominey E. Incentives in the Public Sector: Evidence from a Government Agency. Econ J. 2017; 127(605):F117–F141. https://academic.oup.com/ej/article/127/605/F117/5069467.

  66. 66.

    Ajzen I, Fishbein M. The prediction of behavior from attitudinal and normative variables. J Exp Soc Psychol. 1970; 6(4):466–87. http://www.sciencedirect.com/science/article/pii/0022103170900570.

  67. 67.

    Kaiser HF. A second generation little jiffy. Psychometrika. 1970;35(4):401–15.

    Article  Google Scholar 

  68. 68.

    Kaiser HF. An index of factorial simplicity. Psychometrika. 1974;39(1):31–6.

    Article  Google Scholar 

  69. 69.

    Bartlett MS. A note on the multiplying factors for various chi square approximations. J R Stat Soc. 1954;16:296–8.

    Google Scholar 

  70. 70.

    O’Connor BP. SPSS and SAS programs for determining the number of components using parallel analysis and velicer’s MAP test. Behav Res Methods Instrum Comput. 2000;32(3):396–402.

    CAS  PubMed  Article  Google Scholar 

  71. 71.

    Buja A, Eyuboglu N. Remarks on parallel analysis. Multivar Behav Res. 1992;27(4):509–40.

    CAS  Article  Google Scholar 

  72. 72.

    West SG, Finch JF, Curran PJ. Structural equation models with nonnormal variables: problems and remedies. In: Hoyle RH, editor. Structural equation modeling: concepts, issues, and applications. Thousand Oaks, CA: Sage Publications; 1995. p. 56–75.

    Google Scholar 

  73. 73.

    Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hoboken: Taylor and Francis; 1988.

    Google Scholar 

  74. 74.

    von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Int J Surg. 2014;12(12):1495–9.

    Article  Google Scholar 

  75. 75.

    Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737–45.

    PubMed  PubMed Central  Article  Google Scholar 

  76. 76.

    Braun T, Ehrenbrusthoff K, Bahns C, Happe L, Kopkow C. Cross-cultural adaptation, internal consistency, test-retest reliability and feasibility of the German version of the evidence-based practice inventory. BMC Health Serv Res. 2019;19(1):455.

    PubMed  PubMed Central  Article  Google Scholar 

  77. 77.

    Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol. Implement Sci. 2017;12(1):29.

    PubMed  PubMed Central  Article  Google Scholar 

  78. 78.

    Egeland KM, Skar AM-S, Endsjø M, Laukvik EH, Bækkelund H, Babaii A, et al. Testing the leadership and organizational change for implementation (LOCI) intervention in Norwegian mental health clinics: a stepped-wedge cluster randomized design study protocol. Implement Sci. 2019;14(1):28.

    PubMed  PubMed Central  Article  Google Scholar 

  79. 79.

    Moullin JC, Dickson KS, Stadnick NA, Albers B, Nilsen P, Broder-Fingert S, et al. Ten recommendations for using implementation frameworks in research and practice. Implement Sci Commun. 2020;1:42.

    PubMed  PubMed Central  Article  Google Scholar 

  80. 80.

    Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.

    PubMed  Article  Google Scholar 

  81. 81.

    Nübling R, Jeschke K, Ochs M, Schmidt J. Zur ambulanten psychotherapeutischen Versorgung in Deutschland: Eine Befragung von Psychotherapeutinnen und Psychotherapeuten in fünf Bundesländern als ein Beitrag zur psychotherapeutischen Versorgungsforschung. [On outpatient psychotherapeutic care in Germany: a survey of psychotherapists in five federal states as a contribution to mental health care research]. 2014. https://www.lpk-bw.de/sites/default/files/fachportal/fachartikel/nuebling-et-al-2014-ambulante-pt-versorgung-ergebnisbericht.pdf.

  82. 82.

    Bosnjak M, Tuten TL. Classifying response behaviors in web-based surveys. J Comput Mediated Commun. 2001. https://doi.org/10.1111/j.1083-6101.2001.tb00124.x.

    Article  Google Scholar 

  83. 83.

    Galesic M. Dropouts on the web: effects of interest and burden experienced during an online survey. J Off Stat. 2006; 22(2):313. https://search.proquest.com/docview/1266792615?accountid=14571

Download references

Acknowledgements

None.

Funding

Open Access funding enabled and organized by Projekt DEAL. This work was funded by the German Federal Ministry of Education and Research, Project BESTFORCAN (Bringing empirically supported treatments to children and adolescents after child abuse and neglect, grant numbers: 01KR1804B, 01KR1804C), Project BETTER CARE (Better Care for Unaccompanied Young Refugees: Comparing a Stepped Care Model with Usual Care, grant number: 01EF1802AC). The German Federal Ministry of Education and Research had no role in the design and conduct of the study, including the collection, management, analysis, interpretation of the data, or the preparation, review, or approval of the manuscript and decision to submit the manuscript for publication.

Author information

Affiliations

Authors

Contributions

KS, AB and HC designed the study. KS, JT and AB translated the instruments. MR and GAA provided feedback on the translations. KS collected the data. KS and AB analyzed the data and drafted the manuscript. JT provided a first draft of the introduction. HC, MR and GAA commented on the initial draft and critically revised it. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Katharina Szota.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Internal Review Board of the University of Marburg (approval number: 2019-58 k). Participants received study information and provided informed consent. Data were collected anonymously.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

EBPAS-36D (German).

Additional file 2:

EBPAS-36D Scoring Instructions (German).

Additional file 3:

CHERRIES Checklist.

Additional file 4:

Global assessments.

Additional file 5:

Split sample t-tests.

Additional file 6:

Rotated factor matrix.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Szota, K., Thielemann, J.F.B., Christiansen, H. et al. Cross-cultural adaption and psychometric investigation of the German version of the Evidence Based Practice Attitude Scale (EBPAS-36D). Health Res Policy Sys 19, 90 (2021). https://doi.org/10.1186/s12961-021-00736-8

Download citation

Keywords

  • Evidence-based practice
  • Evidence-based treatments
  • Interventions
  • Implementation
  • Attitudes
  • Therapists
  • Mental health