Send Orders for Reprints to Reprints@benthamscience.ae Training Dietetic Interns to Do Outcomes Research/management Using Community-based Participatory Research

Dietetic Interns (DIs) need effective supervised practice opportunities to apply the principles of program development, implementation, evaluation, and particularly, outcomes management. Outcomes management tracks data to improve practices, outcomes, effectiveness and efficiency. The Nutrition Care Process guides outcomes management in dietetics. Merging it with Community-based Participatory Research (CBPR) can insure more appropriate and sustainable outcomes and processes. Syracuse University's dietetic internship provides a concentration in outcomes management. Through a three-credit graduate class, DIs learn CBPR, specifically the PRECEDE-PROCEED model. The model covers how to collaborate with their community, collect/analyze data, design an objective-driven outcomes management project, and to marshal available resources for meeting the identified health needs of the community. A preceptor and advisory panel collaborate with a team of DIs for each project. This paper describes the structure of the class used to teach DIs to do outcomes management using CBPR. DIs and graduate students completed a survey designed for this research, which assessed their perceptions of the effectiveness and their satisfaction with the class, rotation/experience, and the barriers, benefits and lessons learned using CBPR. Although participants found learning and implementing PRECEDE-PROCEED to be complicated, they rated the experience as effective for: 1) building appropriate entry-level dietetic competencies, 2) collaborating with the community and 3) designing effective and sustainable outcomes management projects. They were satisfied with their abilities, effort and outcomes. The benefits, challenges and proposed improvements for the experience and training are discussed.


INTRODUCTION
Outcomes research is the study of the result of healthcare interventions and policies in terms of safety, effectiveness, efficiency, equity, patient-centeredness, and timeliness [1].Outcomes management is the utilization of the most appropriate intervention/procedure based on evidence of the effectiveness of the alternatives for similar patients or groups [2].Dietitians should use outcomes management methods to track and analyze outcomes data to improve their practices.[3].Within dietetics, the Nutrition Care Process (NCP) [3,4] is the decision making system for outcomes management.A dietitian uses it to assess and diagnose problems and to design, monitor and evaluate interventions.It is typically applied to a clinical setting working with individual patients, via Medical Nutrition Therapy (MNT), but it does include language for the community setting.Dietitians are encouraged to use the NCP to facilitate communication across the health care system using this standardized language and to demonstrate evidence-based practice [5].We should monitor the success of implementing the NCP, as an outcomes collaboration with the community to assess the complex behavioral and environmental factors that influence health and quality of life.Mapping and prioritizing the relevant predisposing, enabling and reinforcing factors, PRECEDE phases guide development of aligned interventions and policies.During PROCEED, process evaluations ensure effective implementation, and impact and outcome evaluations are tied to measurable objectives determined in each PRECEDE phase of the process.CBPR and PRECEDE-PROCEED have been used extensively to design and develop effective community [9] and recent nutrition-focused [10][11][12][13][14] interventions.
Why pair CBPR with Outcomes Research/Management? CBPR approaches have been repeatedly recommended as a key strategy for increasing and diversifying participation in clinical trials and enhancing their relevance and quality [15,16].Some randomized controlled trials (RCT) have been designed based upon CBPR [17][18][19] with effective results [20][21][22].Seifer and colleagues [15] specifically advocate for a new model for designing and implementing cancer clinical RCTs using CBPR.
CBPR research with the highest degree of community-based participatory approaches realized the best community health outcomes [23] and developed sustainable intervention systems [24][25][26].CBPR has also effectively improved the health of the community through ecological approaches.CBPR has guided environmental changes [27] or new policies [28,29] in support of community health outcomes management.CBPR has also been used for outcomes management as a community chronic care model to manage diabetes [30].
Dietitians are encouraged to participate in the research process to maintain and expand their expertise in the health care system [31][32][33][34].Participation in research is an effective way to learn scientific and evidence-based methodology necessary to do outcomes management to provide effective service in an outcomes-based health care system.The revised core competencies for entry level dietitians are grounded in solid research skills to ensure scientific and evidence-based practice [35].Unfortunately, most dietitians beyond academia lack the self-efficacy and capacity for conducting research [36][37][38] and therefore few are engaged in research [39,40].Dietetic Interns (DIs) need effective supervised practice opportunities to develop and challenge their research /outcomes management skills.Service learning and internship programs have been effective avenues for engaging medical [41] and dietetic [42,43] students in the community health projects, but little has been done to engage them in the research process [32,44].A few dietetic internship programs have a concentration in research or outcomes management, and even fewer programs have evaluated the effectiveness of outcomes research/management training for dietitians [32,45].

Objective
This study explains and evaluates the merging of CBPR and the NCP to train DIs to do outcomes management in supervised practice.

Sample
Syracuse University DIs and students enrolled in NSD 658: Participatory Program Planning between 2011-2013 were invited to voluntarily participate in this study to assess their experience with learning/using PRECEDE-PROCEED to implement community-based outcomes management projects.NSD 658 is a graduate class for our DIs, but other graduate and undergraduate students are encouraged to enroll in the class to give them a competitive edge for applying to the internship or graduate school.Enrollment of students outside the internship program diversifies the students working on the teams in terms of time availability and talent.

Intervention: NSD 658 Participatory Program Planning Class
Syracuse University's dietetic internship provides a concentration area of outcomes management to challenge and shape the skills of our DIs for the future demands of the profession [35,46].Instead of designing a researcher-driven approach to outcomes research/management, experience [18,19,22,43,47] led us to use a community-based participatory approach.
Through a three-credit graduate class which merges the NCP and CBPR (see Table 1), DIs/students learn to apply CBPR in the form of the PRECEDE-PROCEED model [7].Through this class, they learn how to: 1) collaborate with their community, 2) investigate existing literature and sources of data, 3) collect/analyze needs assessment data from the PRECEDE stages, and 4) map out an objective-driven outcomes management project for the PROCEED stages.To the degree possible within the given timeframe, they apply available resources for meeting the needs of the community.A preceptor and advisory panel (community members and/or program employees) collaborate with a team of three-four DIs/students for each project.
The full class meets one night each week.The theory/content is covered intensively at the start of the semester, so students can focus on applying what they learn through the later portions of the semester.As the semester progresses, students submit incremental drafts, then finished portions, of their work and discuss their progress in class.The discussions in class across projects are instructive, stimulating and supportive for each group project.Toward the end of the semester, DI teams prepare an abstract/poster for a state dietetics meeting to present the process (but not the data due to IRB guidelines) they used to work with their community.At the end of the semester, they present their process and findings to the community agency preceptors.
DIs are placed in the regular rotations Tuesday-Friday, so Mondays during the spring semester are reserved for CBPR project work.DIs are expected to spend five hours per week in the concentration.DI and students use their time flexibly to meet with their community and preceptors and to work on their projects.Starting with the 2013 year, the training was started toward the end of the fall semester (November for a total of four weeks), allowing the participants more time in the spring for project work.

Community Projects
Examples of community projects include:

Materials
The process evaluation survey regarding NSD 658 was designed for the study.It was pilot-tested with two professionals and refined before administering.Participants indicated their project agency, status (DI versus student), how often they visited their community/agency, and estimated the hours/week they spent on this project.Participants rated the class and experience on the following measures, using Likert scales: a) How well the learning activities built on their entry level competencies regarding collaboration, needs assessment, short/long-term goals, policy review/development/advocacy, outcome measures and data, leadership/advocacy, and professionalism (15 items) (1=Not well, 2=Somewhat, 3=Very well and 4=Extremely Well and X= Unsure).b) Effectiveness of aspects of the schedule and time, class logistics, instructor/ preceptor input, team's abilities, and potential changes suggested (14 items) (1=Very ineffective, 2=Ineffective, 3=Neutral, 4= Effective and 5= Very Effective and X=NA).c) Satisfaction with respect to their performance on the entry level competencies (8 items), how they worked together, the didactic class, and what they accomplished, and learned (1=Very dissatisfied, 2=Dissatisfied, 3=Neutral, 4=Satisfied, 5=Very Satisfied, X=NA.).d) Self-assessed their abilities, preparedness, project successfulness/sustainability, and future use of CBPR (1=Strongly disagree, 2 = Disagree, 3=Agree, 4=Strongly Agree, X=Not sure.).Participants also recorded responses to the following open-ended questions: benefits to using PRECEDE-PROCEED, barriers to using PRECEDE-PROCEED, what they learned through the process, and what they would change or leave the same about the experience.

Analysis
Data was recoded to remove Unsure or NA responses.Statistics were applied to determine frequencies, mean, and standard deviations.Differences in perception were examined by status and year using t-tests.T-tests were used to compare mean scores for each question against the desired benchmark [48], which was comparable to a "B" grade: 3.4 was the benchmark for answers on a 1-4 Likert scale and 4.2 was the benchmark for answers on a 1-5 Likert scale.All analyses were conducted using the Statistical Package for the Social Sciences (version 21, IBM, Chicago, IL, 2012), and significance was set at P < .05.All open-ended questions were summarized by themes.

RESULTS
Of the 43 NSD 658 Participatory Program Planning DI/students between 2011-2013, 22 participated for a 51% response rate.The highest participation rate was in 2012 with 83% participating.A majority of the participants were DIs (86.4%) and the remaining 13.6% (n=3) were graduate students.Participants indicated spending the following average time per week on these projects: 9% 1-2 hrs/wk; 45.5% 3-5 hrs/wk; 27.3% 6-9; and 18.2% >10 hrs/wk.Generally, the participants rated the learning activities of the class/experience to effectively build upon their entry level competencies, however a maximum of 13.6% of participants were unsure and did not rate indicated questions (see Table 2).The highest means, indicating the most perceived progress in building competencies, were noted in: a) establishing collaborative relationships, b) teamwork, and c) conducting a research project with the most appropriate methods and procedures.On the other hand, the participants rated the effectiveness of building their competence for participation in public policy activities significantly below the benchmark score of 3.4 (P < 0.05).There were no differences by year or status for how the participants rated the effect of these learning activities on their competencies.
In terms of the effectiveness of the class structure, the highest average perceived effectiveness ratings were for the flexibility of the schedule and the professor's feedback during the process (Table 2).The class training schedule, quality of the PRECEDE-PROCEED training, and feedback from their  preceptor were significantly lower than the benchmark score of "4.2" (P<0.05).Additionally, DIs rated the following as significantly more effective than did the graduate students: dedicated Monday time to be at sites (4.16 ± .89vs 2.0 ± 1.4) and the blocked weeks set aside in DIs rotations to be at sites (4.0 ± 1.14 vs 2.0 ± 1.4) (P<0.05).The items -start the training and literature reviews for the projects in the fall were perceived as potential highly effective changes to the class/structure, hence allowing more time in the spring for their projects.
The participants generally thought that they could describe the model and form collaborative relationships.None of the questions scored significantly lower than the benchmark score of 3.4.Although 18% of the participants were unsure and therefore did not rate their perception, a majority of the participants agreed or strongly agreed that the project is sustainable (77.2%) and 72.8% indicated they would use PRECEDE-PROCEED in the future.The graduate students indicated significantly higher degree of agreement that the program was a success and sustainable and that they felt supported by the DI program to do the project.
Generally participants were very satisfied with their effort and what they learned, and satisfied with their preceptor's effort, the class, the overall experience and what they accomplished (Table 3).On average, the graduate students were more significantly satisfied than the DIs with their personal effort and with their preceptor's effort (P<0.5).All participants were very satisfied with their ability to apply each of the entry level competences.Table 4 summarizes participants' comments regarding the benefits and barriers of using PRECEDE-PROCEED for outcomes management projects.Many of the benefits cite the effectiveness of collaborating and the structure/guidance provided by PRECEDE-PROCEED.One participant was very enthusiastic regarding the benefits of using PRECEDE-PROCEED: "It forces you to look at your target population and how successful/effective what you are trying to implement might work.I think it is very important to use this model.I think you can be more successful trying to say accomplish a few goals which are appropriate rather than some elaborate plan your target population is not ready for.PRECEDE/PROCEED helps you get to know your population and what is important to them and what they might be ready for."The following barriers were identified by the indicated percentages of participants as likely to affect their work in the community: 100%-time; 50% -financial resources; 18%-community members' knowledge of project/model and 13%-community interest in working on project/model.From their open ended comments, the barriers appear to consist of the complexity of the model, challenges of limited resources, and difficulties of working with others.
Many of the lessons learned were about the effectiveness of this process and the realities of community work, but also included were lessons about professional development (table 5).One participant wrote the following: "That as a DI, I'm now on the cutting edge of where this profession is headed.… Being able to show outcomes will allow for better insurance reimbursement and improve our healthcare system from one that takes care of the sick to one that advocates for wellness.All because the evidence-based proof has been obtained through the PRECEDE-PROCEED process."See Table 5  As shared in Table 6, most of the participants' suggestions for change were directly about class structure and the class experience and partners.A few of the participants took a more personal approach and described what they would change about how they engaged in the process, thus providing some advice that can be made available to future students.A few of the statements participants wrote about aspects of the class/experience which should be maintained included: the sample papers and outlines for each phase were extremely helpful; continue to work with real communities; the professor; working in groups; and try to maintain the continuity of projects over multiple years to fully implement the projects and show effective outcomes.

DISCUSSION
This is one of the few studies to evaluate the effectiveness of research training in the field of dietetics [32,45,49,50], and the first to focus on the effectiveness of training for outcomes management using CBPR.A number of recent nutrition-related studies have used PRECEDE-PROCEED to design health promotion projects [12][13][14], and one was specifically implemented through service learning with dietetic students [10].This study evaluated DI and graduate students' perceptions of the effectiveness of using PRECEDE-PROCEED to collaborate with a community to design and implement an outcomes management project.
This study found DIs and graduate students perceived that the training in, and use of, PRECEDE-PROCEED was effective to improve their dietetic competencies for outcomes management.It enhanced their ability to collaborate with the community and helped them to implement and design an outcomes management tracking system to implement the most appropriate interventions and policies.They perceived the projects to be successful and sustainable.
CBPR is effective for empowering the community [29].The benefits of using PRECEDE-PROCEED listed by the DIs and graduate students were similar to what community members' have reported for a long-standing CBPR project [51].Similar to findings in other studies, Downey and   colleagues [52] outline approaches to building leadership in community and training community members to be partners in research.DIs and graduate students reported that the projects with the most community participation realized the most effective outcomes [23].The DIs/graduate students recognized the importance of building community capacity, but found it difficult to do within their limited time frames.
Although the mean scores indicated that students judged the PRECEDE-PROCEED training and experience to be less able to build upon their competencies for public policy activities, including legislative and regulatory initiatives, a few of the projects actually resulted in policy changes (i.e.obesity tracking and referrals, total parental nutrition initiation) rather than educational interventions.Additionally students reported being satisfied with their abilities to develop and review policies and discuss the impact of policy decisions on their programs.The results of this study of the PRECEDE-PROCEED model is supported by other results regarding the effectiveness of CBPR for advocating for policy changes [28] or other environmental supports [27].
The DIs and graduate students found PRECEDE-PROCEED to be complex and difficult to learn and implement.Yet they were satisfied with their effort the results achieved with the method once they mastered it.Most indicated they were able to describe the model and planned to use it in the future.The DIs and graduate students' moderately favorable perception of the effectiveness of the class and their satisfaction with it is a concern.This training specifically focused upon the CBPR process and may have assumed that the students' "basic" research skills from undergraduate training were sufficient.Therefore students' research self-efficacy should be assessed [51] to insure they have adequate baseline skills for conducting outcomes management before adding this more complex CBPR process to their skill set.In fact, a few of the DIs reported opinions that research should not be part of the dietetic internship, and that it is only necessary for graduate students.Given students' limited ability to deal with abstract processes, the debate continues regarding how much training to provide pre-experience versus simultaneously with the experience.Some students indicated they wanted more training prior to the community experience and appreciated learning from the examples of past projects, while others indicated they learned PRECEDE-PROCEED by implementing it.Learning to implement CBPR can be difficult regardless of your comfort with the research process.DiGiroliamo and colleagues [53] found that even a sample of current academic researchers required substantial training to acquire CBPR skills and identified difficulties particularly with obtaining funding, partnership development, evaluation and dissemination of CBPR projects.Many of the difficulties or barriers mentioned by the DIs and graduate students have also been reported by academics working on a CBPR project [54].
It is challenging and time consuming to accurately implement CBPR [55].The structure of this class and experience is actually not the most conducive to implementing CBPR.Four-five months is a very tight timeline for learning the model, collaborating with a community to effectively assess its problems, and design and implement an outcomes management project.Most projects take years.Projects made varying degrees of progress toward implementation, based upon the quality of the collaboration, effort of the team, and the attitude toward outcomes management.Students' perceptions of the average effectiveness of preceptors' feedback may be attributable to the fact that the students were more thoroughly trained in CBPR than the preceptors.Preceptors themselves are just starting to adopt outcomes management practices and also may have had pre-conceived ideas about what they wanted the DI projects to accomplish.At annual advisory board meetings, the PRECEDE-PROCEED model is discussed in hopes of increasing preceptor commitment, knowledge, and support for using CBPR to do outcomes management.
The mix of DIs and graduate students in the class provided challenges but was outweighed by the benefits.The DIs' rotation schedule is structured such that they are at their regular sites Tuesday-Friday, so then could only meet with their communities on Mondays or nights/weekends (the latter of which was atypical).In contrast, students' schedules were more flexible through the week, but they weren't always available for the full day on each Monday.Blocking specific weeks in the rotation schedule for these outcomes management projects had mixed reviews, and may not have been an effective use of time.Graduate students rated the schedule and blocked weeks less effective, because they could not take a whole week or two off to be at sites.Even though DIs thought the blocked weeks were effective, there was limited likelihood that the week assigned aligned well with actual work in the community.For effective teamwork, not all members need to be present at all times, and yet dividing the project among group members and merging the parts leads to inefficiency/frustration and disjointed and ineffective outcomes.Some groups still struggled with using their time wisely and effective team work principles.Finding the best mix of dedicated time for projects given other class/internship rotation responsibilities continues to be a challenge.
So given the challenges, was it better that the DIs and graduate students were at least exposed to and attempted a CBPR-designed, rather than a research-driven, outcomes management project?From their comments regarding the benefits of the model and what they learned, it would appear so.The DIs and students were enlightened by what was really important to their communities and the barriers they face.The students realized the benefits of carefully listening to members of their communities, by discovering that many of their assumptions about what they thought would have been the most appropriate interventions were debunked through the PRECEDE-PROCEED process.This simple exposure to PRECEDE-PROCEED motivated them to want to collaborate with their communities, to take the time necessary to do so and empowered them to design a more appropriate and sustainable outcomes management project.Exit surveys for the dietetic internship from 2012-2013 indicated that many students choose the Syracuse University dietetic internship for the outcomes management concentration and wanted to work closely with a population to develop an effective intervention (personal communication, D. Connolly 2013).
PRECEDE-PROCEED provides an effective structured process for conducting CBPR, but is often perceived as too complicated.There are other CBPR models, each with advantages and disadvantages [56] which might be used instead of PRECEDE-PROCEED in the design of outcomes management projects.On one level, less intense qualitative methods exist to connect with the population and gain their perspective [57,58]; one such example is Photovoice [59,60].Other methods for community decision making include: the problem/solution-tree [61], concept-mapping [62] and group data processing methods [63].
Beyond the students' perceptions of the effectiveness of using CBPR, this type of study also needs to assess the effectiveness of using CBPR from the perspectives of preceptors and community members [51].Additionally, the effectiveness of the collaboration with the community should be assessed to determine to what degree the "research" is CBPR.A published review of CBPR studies assessing how involved the community was in each study conveyed an average score of 2.3 out of 3 (range 1.4 to 3) [64].A variety of tools have been created to evaluate the CBPR process/dynamic [65][66][67] and to test this balance between research-driven versus community-driven projects.

CONCLUSIONS
PRECEDE-PROCEED is an effective method for teaching dietetic interns how to engage with communities to design and implement effective outcomes management projects.We found DIs and graduate students perceived they were able to design and implement outcomes management projects using CBPR.To engage more dietitians in the research and the outcomes management process, we need to build these skills early in their education/career.As shown here and by others [25], this can be accomplished in creative ways.By engaging DIs in the CBPR process to do outcomes management, they will develop their appreciation for the need for it in the health care system.As they experience the process of creating more relevant and sustainable outcomes management projects, it might prove to be a less intimidating process.By building their self-efficacy, we might influence the future of how dietetics is practiced.

CONFLICT OF INTEREST
There was no financial support for this study nor are there any conflicts of interest to report.


To work with a team is the only way to succeed  How to listen to and collaborate with a population."What is important to them?"  That PRECEDE-PROCEED is not as daunting as I had thought-it is an extremely easy and useful tool!  Why it is so important to know your population; contemplated what I had tried to implement in the past  That time is a limiting factor  That many assumptions were challenged by using the PRECEDE-PROCEED model  Importance of really getting to know your target population, refining our assessment methods  "Look before you leap"-fully understand the problem and what the population wants before creating a solution  Interventions are most successful if you first dig into research literature and interact with the target population  How to effectively use outcomes measures  Remain professional while working in a stressful group situation

Table 1 . NSD 658 -PRECEDE-PROCEED combined with the Nutrition Care Process. Program defined DI Supervised Practice Competencies PRECEDE-PROCEED Timeline*
Develop/Refine health promotion program/ outcomes management project according to set objectives.Use research/existing programs/policies to identify potential tools/programs/policies.

Table 2 . Contd…. Rate how well completion of this course/experience built on the following entry level competencies?
For the DI concentration, "the learning activities must build on entry level competencies." 2 Scales a 1=Not well, 2=Somewhat, 3=Very well and 4=Extremely Well b 1=Very ineffective, 2=Ineffective, 3=Neutral, 4= Effective and 5= Very Effective c 1=Strongly disagree, 2 = Disagree, 3=Agree, 4=Strongly Agree 3 Number of participants checking unsure for the item

Table 3 . Perceived satisfaction with effort, abilities and outcomes. Rate your satisfaction with: 1 Mean ±SD Range
for