Integral Counseling Psychology Assessment Processes

ICP Assessment Fall 2012.rtf 10/23/12 11:51 AM

ICP Assessment Overview September 2012

Philip Brooks, Associate Professor and Assessment Liaison

There are two fundamental purposes for assessment in ICP. One is to provide timely, accurate, comprehensive, accessible feedback to our students as to their progress in the program. The other is to elicit, solicit, analyze and utilize feedback about how well we're doing as a program.

Student Evaluation

With respect to the first purpose, I think we're doing a better job. Years ago our assessment efforts were almost wholly focused on problematic students. In recent years we have tried to be more proactive, in part to avoid the situation in which we were putting out fires regarding "high needs students," who often had advanced quite far in the program. The timely, compassionate assessment of problematic students is a high priority, as we know that one difficult student can severely impact the learning experience for whole class.

Some key contact points for student evaluation are:

The initial advisee meeting with new students during the first semester. While not officially an assessment session, this is a time when we can get to know a little bit about a student, and possibly surface "red flag" issues. Further, I believe in the saying "connect before you correct," and this is often a time for relatively stress-free connections, and the goodwill generated in this meeting may be called on in the future, when an advisor may be asked to sit in on a less pleasant evaluative session involving that student. We are currently questioning if this should be strongly recommended or required.

Completion of pre-practice evaluation forms. Writing up these forms not only gives us a hard copy data source for our pre-practicum meeting with students, but also orients us to evaluation throughout our classes. Hopefully, we are also giving oral feedback to those students who particularly need it. We all know stories about students who have been passed along from class to class on questionable grounds. Also, we need to be alert to the potential for "weak students" to miss out on valuable and necessary feedback when being taught by adjunct faculty, who may be less aware of the need for consistent attention to taking notice of/ communicating with them. See "assessment reports" link.

The pre-practicum advising meeting. This relatively new meeting is a critical one in providing students with interim feedback as they move through the program. Here we go through their files (especially their pre-practicum evaluation forms), follow up on any evaluations which may be questionable, carefully look at their answers to the pre-practicum questions, and get an in-person, felt sense of their readiness for practicum.

The practicum experience should provide the richest and most meaningful information. The ICC directors and staff will be assessing students' capacities to successfully sit with clients. Also, supervisors will be giving the student feedback, and will be providing information to us for inclusion in their file. My concern here is that some supervisors take a laissez-faire attitude toward the written evaluation process, and I fear that this attitude might extend into their work with students in the supervision hour. See attachment B.

The last major evaluation point comes in the Integrative Seminar class. Here professors evaluate students' in-class presentations and final integrative papers, assessing their readiness to graduate and represent our program in the world.

Program Evaluation

Evaluation of ourselves as a program lags behind student evaluation. Once again we can easily point to improvement in this area, but we also need to improve data collection, and even more important, utilization of the information we receive. Perhaps the faculty is inappropriately resting on our laurels as the program has historically done well in recruitment, student satisfaction, and performance on BBS exams.

Page 1 of 3

ICP Assessment Fall 2012.rtf 10/23/12 11:51 AM

That said, there is an ongoing need for program evaluation. We are required to do capstone assessment each semester as a primary source of "hard data". Additionally, I imagine all of us quite simply care about how are doing. We are collecting data on our teaching, our advising, our supervision, our counseling centers, and our program as a whole.

Teaching

Informal evaluation of what's working and what's not working in our classes through asking for ongoing feedback from our students, from soliciting input from our TA's, and from honest self reflection continues to be a rich source of information.

Mid-semester evaluations can give us important information that can help us adjust the trajectory of our class, and help expose needs that we are unaware of.

End of semester evaluations give us good (though not necessarily timely) information, and are more comprehensive than the mid-semester feedback sessions. One important concern, which warrants our attention, is that they often seem to be skewed to the positive, as students may be reluctant to express their genuine opinions. I'm not sure if this is still true, and if it is I don't know why.

Collegial feedback doesn't often occur, but if we follow up on the idea of sitting in one another's classrooms, we could tap a rich source of inspiration, creative new ideas, and needed adjustments.

Advising

According to the latest graduation survey, ICP has greatly improved its advising. Having Daniela handle the nuts and bolts, and making contact with students early and often makes a positive difference. Mark's work with the content of advising meetings is superb, as it provides a useful framework for our time with students. This should help us continue to improve in the future.

Supervision

We have forms for students to assess their supervisors, but I'm not sure if or how these are being used. This is something we should take up in our faculty meeting. In the past, we have intervened only when there were major problems with a supervisor. While I think our supervisors provide great service for a small fee, it seems important to assess their work.

Integral Counseling Centers

We need to explore whether we want assessment of all our counseling centers on a regular basis (add to the agenda at the faculty meeting).

ICP As A Whole

Several of the data collection points for students are also opportunities to gather information about how we are doing. For instance, in our meeting with students in their first semester, we can find out what we're doing well and what we might need to do better. The pre-practicum evaluation meeting is another time when we can collect useful information.

Our student representatives have historically provided some of the most critical information that we've used to make program changes.

A major source of program-wide information is the Integrative Seminar class which provides quantitative information on the program. And while the evaluation instrument is imperfect, it can be improved. We need that data for program review discussions in the faculty meetings. I think an even more robust source of

Page 2 of 3

ICP Assessment Fall 2012.rtf 10/23/12 11:51 AM

information is the verbal feedback students in this course provide about the program. I'm not sure if this is true of all Integrative Seminar classes, but in the one I taught, students were quite forthcoming with what helped and what hindered in their education. I particularly hope that this information can be collected, and communicated to the faculty as a whole. See "assessment reports" link for a discussion of the Capstone Evaluation and for the Spring Semester results compared to earlier semesters taken as a whole.

The graduation survey provides information each year about how we're doing. Once again the challenge is to make time to discuss the results and trends we find.

We survey our alumni to find out how they are doing, but I think we can be more creative in trying to get a greater sample size. In the past we did a focus group for alums that yielded useful data, perhaps we should do this again.

This year we have the gift of two external evaluators who will offer fresh perspectives on the program. We look forward to hearing from them.

(See "Assessment Reports" link for more information).

Page 3 of 3