Valuing the scholarship of integration and the scholarship of application in the academy for health sciences scholars: recommended methods

In the landmark 1990 publication Scholarship Reconsidered, Boyer challenged the 'teaching verses research debates' by advocating for the scholarship of discovery, teaching, integration, and application. The scholarship of discovery considers publications and research as the yardstick in the merit, promotion and tenure system the world over. But this narrow view of scholarship does not fully support the obligations of universities to serve global societies and to improve health and health equity. Mechanisms to report the scholarship of teaching have been developed and adopted by some universities. In this article, we contribute to the less developed areas of scholarship, i.e. integration and application. We firstly situate the scholarship of discovery, teaching, integration and application within the interprofessional and knowledge exchange debates. Second, we propose a means for health science scholars to report the process and outcomes of the scholarship of integration and application with other disciplines, decision-makers and communities. We conclude with recommendations for structural and process change in faculty merit, tenure, and promotion systems so that health science scholars with varied academic portfolios are valued and many forms of academic scholarship are sustained. It is vital academic institutions remain relevant in an era when the production of knowledge is increasingly recognized as a social collaborative activity.


Background
Over 25 years ago, Mensah [1] identified many dysfunctional manifestations in the process of tenure and promotion that result from judging an academic scholar's ability and worthiness using a yardstick that measures only the success rate of research grant applications and the quota of publication outputs. She noted there was little value placed on one's success in classroom teaching or on service activities that made a difference outside the academic community but that directly flowed from academic work. Further, she claimed that the pressure to publish had influenced a rise in 'intellectual dishonesty, academic espionage, academic plunder, the me-orientated attitude, disinterest in students and teaching, as well as a lack in collegiality' [1]. She argued that links between the ethic of publish or perish and market forces had shifted the focus from the major responsibility of teaching. More recently, Ernest Boyer [2] reported that more than half the faculty working in academic institutions believed the pressure to publish diminished teaching quality. Faculty reported feeling torn between competing demands of research work and teaching courses and felt compelled to 'take short cuts in their research or rely heavily on teaching assistants -an arrangement that is often less than satisfactory for both student and professor' [2].
In the landmark 1990 research report, Scholarship Reconsidered, Boyer reconfigured academic scholarship into four dimensions: discovery, teaching, integration, and application. Boyer advocated a return to the broader view of scholarship but stressed the four dimensions should be evaluated as separate yet overlapping areas [2]. Glassick and colleagues continued the work in the area of evaluation and quality measurement in the 1997 publication, Scholarship Assessed [3]. While considerable work has been undertaken to define the scholarship of teaching and the methods to document its value (see for example: [4,5]), sustained change in emphasis within the health sciences academy is a contested and complex process.
In this article, we contribute to the less developed areas of the scholarship of integration and the scholarship of application in the health sciences academy. We consider Boyer's theory of scholarship for health sciences scholars and an approach to evaluate the scholarship of integration and application using the six standards developed by Glassick and colleagues [3]. We discuss a participative framework to guide institutional change and propose a method to report and evaluate the scholarship of integration and application as part of our recommendations. We conclude with a call for structural and process change in the merit, tenure and promotion policies to value all health science scholars' varied forms of academic scholarship; this change must include equitable evaluation strategies.

Creating a sense of urgency for participation in the health sciences
There is increasing interest in the challenges and mechanisms related to reporting non-traditional academic scholarship productivity and contributions to applied scholarship for health services research [6,7]. Today's academy is stressed by reduced operating budgets which have influenced financial planning decisions by university administrators. This fiscal climate has resulted in rigid rules in terms of career advancement, tenure and promotion that directly link to scholars' performance in research and publishing records [1,8] while teaching and service are disproportionately undervalued [9]. In 2000, Gros Louis asserted that academic institutions still maintain a traditional approach in what is counted in reward, promotion and tenure systems that remains puzzling to those outside its walls and remains contentious to those within the system. Questions commonly asked about this issue include: "How much research is enough? How good does it have to be? Does it have to be funded, at what level, and by whom? How widely cited and read? How do we know if someone is or is not an excellent teacher? What counts and what does not count as service to the profession and to the university?" [10]. Asking the questions about research, teaching and service is a start, but obtaining unambiguous answers and linking these answers to rewarding the scholarship of discovery, teaching, integration and application in the academy is the more difficult work.
The current view of scholarship as accrued research grant dollars and publications (the scholarship of discovery) runs counter to academic institutions' obligations to serve society and improve the health and wellbeing of communities using applied and integrated scholarship. In this sense, discovery scholarship fails to reflect increasing awareness of the contextualized nature of knowledge production [11]. This is especially evident in the health sciences academy. Increasingly blurred lines between science and society require that we re-think how academic contributions are valued both within and beyond the academy. Publishing in journals with a high impact factor is viewed as a measure of quality, but most certainly does not guarantee dissemination of scholarship activities to community stakeholders [12,13]. New rules of the game should strike a balance between the current demands to publish or perish and the newly required emergent call for the academy to 'participate or perish' [11]. Recognizing and weighting applied scholarly activities for tenure and promotion involves conversations between researchers and administrators in faculties and departments in the academy [6].

Health science scholars and interrelated areas of scholarship
In academic institutions, scholars most often pursue the scholarship of discovery with specific emphasis on thinking and developing knowledge with other scholars [14,15]. Boyer [2] proposed that a scholar works in the four interrelated areas of scholarship in the pursuit of knowledge that is responsive to human problems and societal needs. This broader perspective of interdependence between academic institutions and society in the application of new knowledge to address societal problems makes the case for the scholarship of application and integration. Academic institutions also benefit because there is potential to fulfill their societal obligations through the interrelated areas scholarship [16].

The scholarship of discovery
The scholarship of discovery is understood as original research that expands or challenges current knowledge in a discipline. Boyer [2] defined discovery as the creation of knowledge for knowledge sake; its purpose is to contribute not only to knowledge but also to the intellectual climate of academic institutions. Questions asked by scholars of discovery include: What should be known?; and, What has yet to be found? [2]. New knowledge is vetted and regulated through peer-evaluation via publications. While this commodity is most important in the merit, promotion and tenure reward systems in the academic institutions, this traditional view of scholarship marginalizes other forms of scholarship and is a powerful disincentive to those who are pursuing tenure and promotion but who are more active in teaching, integration, and applied scholarship.

The scholarship of integration
The scholarship of integration is closely related to the interprofessional debates; it relates to making connections across disciplines and shaping a more coherent and integrated use of knowledge. Integration work is creative connectedness, interpretation and synthesis, so is closely related to discovery, but poses somewhat different questions in terms of meaning and impact. This form of scholarship interprets meaning to isolated facts and creates new perspectives that can answer questions not originally possible to answer. Health science scholars engaged in integration require innovative thinking to be able to integrate knowledge from different disciplines and create new and different perspectives on significant ideas and theories [17]. Such scholars ask questions that require critical analysis and interpretation such as questioning what the research findings mean and whether it is possible to interpret what has been discovered in ways that provide a larger, more comprehensive understanding [2].
Previously located on the margins of academic endeavor, the scholarship of integration is now central because it is definitely best equipped to respond to contemporary problems at both an individual and societal level [2]. Researchers are locating their discovery work, or that of others, into broader intellectual patterns, thus moving beyond the disciplinary silos to build interdisciplinary partnerships with capacity to respond to multi-focal, complex human problems. Moreover, funding bodies are increasingly supportive of collaborative, integrated partnerships and teams as a way to generate knowledge and new approaches.

Scholarship of application
In application scholarship, health science scholars build bridges and collaborative relationships with other disciplines, decision and policy-makers and communities in order to apply theory to solve every-day problems. Application scholarship directly links other forms of scholarship with practice [14,17,18]. This process involves dynamic engagement and the translation of new knowledge in practical interventions that solve problems or improve the difficulties experienced by individuals and society [2,16]. Hall states this scholarly activity allows dynamic creativity, allows new public policies, allows theory and practice to renew each other and allows "the aca-demic world to climb down from its ivory tower" [14]. Health science scholars engaged in applied scholarship seek to understand how knowledge can be responsibly and ethically applied to consequential problems and how it can be helpful at micro (individual), meso and macro levels (society, government, institutions), as well as seek to learn how social problems themselves can define an agenda for scholarly investigation [2].

The scholarship of teaching
The scholarship of teaching must extend beyond simply transmitting information to a process that is also transforming and extending the learning of students and scholars [2]. In this sense, the scholarship of teaching involves stimulating active learning, critical thinking and the commitment to life-long learning. Recent debates have centred on how to differentiate between the scholarship of teaching and teaching excellence and the relationship of this scholarly pursuit to other forms of scholarship [19]. Moreover, considerable weight is now placed on student evaluations of teaching received which may reflect their personal satisfaction related to grades assigned to their work, rather than the merit of the teaching by the scholar and the curriculum.

Evaluating the different areas of academic scholarship
Soon after Scholarship Reconsidered was published in 1990, questions arose about how to evaluate and reward Boyer's four forms of scholarship. What worked to assess the quality of discovery scholarship in the past could not meaningfully assess the new fields of scholarship [18]. Borne out of this critique was the development of six assessment standards for scholarly work: (1) clear goals; (2) adequate preparation; (3) use of appropriate methods; (4) achievement of significant results; (5) effective presentation and communication of results; and (6) reflective critique of one's work [3]. These six standards provide clear criteria for excellence and a framework for reporting the application and integration forms of scholarship as well as their combined contribution as 'community scholarship'. Further, these standards can be considered a springboard for Faculty in developing their own interpretations of scholarship [20].
While there is general agreement that the scholarship of discovery and teaching are relatively straightforward to evaluate, Calleson and colleagues [21] argue that applied and integrated scholarship are not as easily quantified. Glassick and colleagues [3] proposed specific questions to evaluate the six standards of scholarship. Their work was extended by Aday & Quill [22] who developed specific questions to evaluate integration and application scholarship. Maurana and colleagues [23] contributed questions to evaluate community scholarship and engagement. This literature is succinctly presented in Table 1. Evaluation of the scholarship of integration in the health sciences academy can review whether academic publications written for 'nonspecialist' audiences demonstrate analytical and literary ability to translate complex scholarship into accessible, relevant messages. Evaluation can focus on whether this work demonstrates a clear understanding of the disciplines involved and whether key issues have been defined and creative insights presented. Further, how well the essential message has been delivered and how the non-specialist has responses to the work can be examined (e.g., in what ways has the public or other discourse been advanced?) [2]. In the context of academic institutions' obligations to improve the human experience and society, the legitimacy of this scholarly work is clearly obvious.
Evaluating the scholarship of application is predicated upon identifying examples of applied scholarly activities and developing comprehensive reporting methods. There must be a direct correlation between the intellectual work of the scholar and the applied work, such as consultation, evaluation and analysis [2]. Reporting applied scholarship should include the evaluation by the scholar, and by the recipients of the service, such as decision and policymakers and communities. Those assessing applied scholarship should ask whether the activity is directly related to the academic expertise of the scholar and whether project goals have been defined, whether procedures have been well planned and whether actions have been carefully recorded. Further, evaluation should include how the endeavor has not only benefited the community, but also added to the scholar's own understanding of their health sciences expertise [2]. Calleson et al. [21] also recommend that 'applied products' and 'community dissemination products' be assessed as scholarship contributions alongside peer-reviewed articles. Valued by communities are applied products to improve health, such as programs, guidelines, policies, resource materials, technical assistance and training. Community dissemination products include media reports, websites, and presentations. A similar concept of community scholarship, engagement, and productivity was proposed by Maurana and colleagues [23] who suggested the work must be responsive to the needs and problems defined by the community and result in enhanced health. However, work with communities will only be defined as scholarship when there is clear evidence of links with current research findings and discipline specific knowledge, when the products are peer reviewed and are available for public scrutiny, use, evaluation [23].
Scholarly integration and application activities can be reported in a mechanism such as a Creative Professional Activity Dossier [6,24]. The dossier can work as a reflective document in which scholars draw links between theories and evidence, provide a description of their applied scholarly activities, the underlying guiding rationale/evidence, their objectives, the evidence of impact, and a range of supporting documentation [6].
Those evaluating the dossier could consider: • the creativity and quality of the integration and application activities and the degree of impact on the profession, which is considered more important than the quantity of activities and achievements; • whether the activities are current and able to be sustained; • in what way the contributions and activities are relevant to the institutions; and • whether there has been a resulting change in policy, organizational-decision making, or clinical practice [24].

Recommendations supporting the broader research perspective
Academic work in the health sciences field has been traditionally categorized as research, teaching and service. Service is understood as doing good works in the university such as committee work and citizenship activities to benefit the community but it is important to distinguish between social and civic service (citizenship) and scholarship. As stated earlier, to be considered as applied scholarship, service activities must directly relate to health scholars' specific area of knowledge and professional expertise and exhibit intellectual rigor [2,17]. A more inclusive and integrated view of what it means to be a health sciences scholar is needed. This view must value new knowledge generated through research, teaching synthesis, and practice [2]. An open, social process of inquiry that represents a synthesis of academe and community influences would facilitate greater reflexivity of the complexities, risks and benefits [11].
The health sciences academy is well equipped and positioned to attend to the interests of the larger community and to serve as a bridge between academia and society by linking theory and practice through applied and integrated scholarship in addition to teaching and discovery scholarship. To prevent further disengagement from applied scholarship, academic institutions must take the necessary risk to legitimize and to equally reward all forms of scholarship for the benefit of scholars who engage in each area of scholarship [25,26]. Although much has been written about how traditional reward systems need to be restructured in order to recognize a broader definition of scholarship, it is unclear whether or not the professoriate in general will embrace their new interdisciplinary roles and address complex social issues in pursuit of their careers [25].
We strongly advocate for the following changes and elaborate more specifically on the final two recommendations: • Health science faculties to embrace diversity and creativity in scholarship and to take any necessary risks involved in changing the status quo of privileging traditional scholarship, and those who generate it.
• Allowing scholars to focus on two or three areas of scholarship (i.e. discovery, teaching, integration and/or application) during an agreed period and evaluating the scholarly work over that agreed period of years.
• Support is unequivocally provided to scholars engaged in interdisciplinary partnerships within academic institutions and communities, and rewarded in merit, tenure and promotion systems.
• Naturally evolving communities of scholars are valued and mutually respected by colleagues.
• A framework to guide change be adopted for encouraging participation by all involved in the change.
• A method to compile evidence in a portfolio format that reports the scholarship of integration and application be widely adopted.

A framework to guide change
The inconsistency between mission statements of academic institutions and tenure and reward structures is socializing health science scholars away from the scholarship of integration and application. This reality must change given the position of governments, policy makers and professional bodies that premises healthcare system renewal on scholars and community-based professions working together in interprofessional teams for societal benefit [27][28][29]. For this to occur, however, there needs to be greater congruence between the institutional mission statements that claim to value all four scholarship dimensions and the explicit policies about tenure-track appointment, incentives, evaluation, rewards, merit, promotion, and tenure in health science faculties. This process is best achieved by academies and their faculties reaching consensus about their own definitions of each dimension of scholarship and develop agreed examples in each area [30].
We suggest adopting a participative approach using Kotter's eight-step change framework to encourage participation by all involved in the change within and beyond the academy [31].
Step 1 is about developing a sense of urgency for change which we suggest is gathering momentum in the health sciences literature.
Step 2 is about creating criteria to report and evaluate the scholarship of integration and application. We consider that Steps 3 to 8 proposed by Kotter [31] are best developed between scholars in the academy and with relevant stakeholders in the community. These steps encourage faculty to adopt a broad scholarship reporting mechanism, to leverage support from influential players, and to foster the cultural change that accompanies formal structural change in tenure and promotion systems.

A Portfolio method
The Portfolio method is designed to explain the objectives of the scholarship of integration and/or application activities, the rationale thinking that underpinned each activity, and provide evidence of impact [6]. The compilation of evidence aims to address a range of evaluative questions (illustrated in Table 2). These evaluative questions serve to firstly guide the scholar in organizing relevant evidence and second, serve as a guide for the evaluator in the process of assessing the evidence. The process of developing the Portfolio contributes to scholarship through selfreflection, self-evaluation, and self-development. The Portfolio is a means to report on knowledge transfer and exchange activities and includes products produced in collaboration with others as supporting documentation. The Portfolio could be five to eight pages in length, plus supporting documentation (appendices) and organized in four sections: (1) philosophy and clear goals; (2) contributions to the scholarship of integration and application; (3) reflections and assessment; and (4) supporting documentation presented as appendices.

Conclusion
It is vital that academic institutions and their scholars are relevant to global communities and change in tenure and promotion systems to support this relevancy are long overdue. Academic institutions have had research, teaching and service in their mandate, but evaluation guidelines, structures and systems have not reflected the multiple forms of scholarship. In this paper, we presented methods for reporting and evaluating two specific dimensions of scholarship, i.e. integration and application. Academic institutions the world over have responsibilities to develop unambiguous interpretations and criteria to evaluate the scholarship of discovery, teaching, integration, and application. Faculty members with enhanced and expanded understandings of scholarship can support academic institutions in their obligations to improve the health and wellbeing in communities. This enlightenment and change could improve the experience of health sciences scholars, create fair merit and promotion processes, strengthen the contributions of academic institu-

Scholarship of Integration and Scholarship of Application Portfolio
The scholarship of integration gives meaning to specific discoveries by making connections within and between disciplines, locating knowledge in a broader context, making connections and synthesizing knowledge. The scholarship of application is focused on engagement with the broader community and the use that might be made of knowledge to address societal problems 1,2,3,7 The Portfolio is designed to explain the objectives of the scholarship of integration and application activities, the thinking that underpinned it, and to demonstrate impact 1,3,6 . Developing the Portfolio contributes to scholarship through self-reflection, self-evaluation, and self-development. The Portfolio is a yearly record and a cumulative record of process and outcomes. Yearly records contribute to a cumulative record of historical data that will be useful when compiling award, grants, tenure and promotion applications 4 . Scholars include rationale for and description of each activity, and provide evidence of impact. The Portfolio reports knowledge transfer and exchange activities and products produced in collaboration with communities (engagement) 23  In what ways has the work not only benefited the recipients of such activity, but also added to the professor's own understanding of their academic field? How can knowledge be responsibly applied to consequential problems? How can it be helpful to individuals as well as institutions? Can social problems themselves define and agenda for scholarly investigation? 2 3.4 Future Plans -Outline short-term goals (within one year) and long-term goals (two to five years) to further develop your scholarship program. 4. Supporting Documentation: eligible activities and products included as appendices: Appendix A • documents to illustrate your philosophy and goals. Appendix B • Funding awards, objectives of projects and activities.
• Examples of activities such as teaching interprofessional courses with colleagues from other disciplines, interprofessional curriculum development; seminars, advising students from other disciplines, supervision of an integrated or applied scholarship practicum; scholarship that contributed to the achievement of awards or employment for students. Cross-reference if such evidence is reported in your Scholarship of Teaching Portfolio.

Appendix C
• Description of efforts to improve knowledge, skills and methods on scholarship, e.g., seminars, lectures, workshops, and conferences attended. Reflection on your learning. Appendix D Include details such as names of committees, dates, and the nature of your contribution.
• Activities concerned with scholarship undertaken as a member of a faculty, department, or cross disciplinary committee, sub-committee, ad hoc committee, or task force, accreditation committees, program review committees, interdisciplinary curriculum development.
• Faculty resources developed, workshops, conferences organised. Use of your scholarship materials in other faculties, colleges, or universities.
• Participate in orientation sessions for new faculty, seminars, or invited presentations within and outside of the University about your knowledge transfer and exchange activities.
• Invited to consult by scholars in other faculties to improve applied scholarship effectiveness.

Appendix E
• Innovative dissemination strategies.
• Peer reviewed documents. Details of books (chapters in books, edited books); articles (refereed, solicited, or non-refereed); papers in conference proceedings (refereed or non-refereed); bibliographies; unpublished professional and technical reports.
• Applied products -innovative programs, policy development, training materials, resource manuals and technical products 21 • Community dissemination products -newsletters; posters; workshop presentations; community forums, websites, media 21 Appendix F • Peer reviews from members in the community that your work was meant to benefit. • Results of evaluations or questionnaires designed by you to obtain feedback about the effectiveness and impact of your activities.
• Solicited and unsolicited letters to attest evidence of your impact and effectiveness.