Outcomes Assessmen, Center for Innovation in Teaching and Learning, University of Illinois at Urbana-Champaign

Student Outcomes Assessment

Background

In 1997 all academic units were required to develop plans to assess student achievement in their undergraduate and graduate programs. In the Unit Assessment Plans all academic programs identified (a) the process followed to develop the plan, (b) desired student outcomes, (c) measures and methods for assessing student achievement of the outcomes, and (d) plans to use the assessment results for program improvement. In 1998 Unit Assessment Plans were submitted to the Provost and posted on the campus’ assessment website.

In 2000 the Provost required all units to report their assessment activities and any changes/improvements made in the units based upon their assessment results. Following submission of the 2000 progress reports, units were encouraged to continue implementing their assessment plans. Unfortunately, it wasn’t until 2007 when the next follow-up was conducted. It became apparent that without a recurring process for requiring assessment updates, such as a program review program, formal student assessment activities get lost in administration changeovers, budgetary concerns, and competing priorities.

To re-emphasize the importance of continuing assessment efforts, the Provost asked all academic units in Summer 2007, to once again identify an assessment coordinator, report on their use of recent assessment results, and to revise their Unit Assessment Plans for future implementation. A series of assessment workshops were offered for coordinators by the Center for Innovation in Teaching and Learning. Units submitted their updates and revised plans in Spring 2008. All plans are made available to the campus on the Units Outcomes Assessment Plans webpage at http://www.cte.illinois.edu/outcomes/unit_assess.html.

2008-09 Unit Assessment Plans

It was encouraging to see from 80 unit assessment plans collected how many units continued to assess student outcomes from 2001 to 2008 without a formal process for reporting results to the administration. Table 1 identifies the most common unit changes/improvements resulting from outcomes assessments in recent years.

Table 1
Reported Unit Improvements from Outcomes Assessments: 2000-2008
80 Academic Units Reporting

Unit Improvement                                                                Number of Units

Revised/redesigned curriculum
28
Added new course(s)
25
Revised course(s)
17
Changes requirements
17
Improved outcomes assessments
14
Developed./Revised program
14
Improved monitoring of student progress
10
Modified qualifying exam procedures
8
Developed new concentration or minor
7
Increased/improved student recruitment
7
Provided or improved student research opportunities
7
Provided more student/faculty interaction
7
Added new faculty
7
Improved classroom assessment procedures
6
Modified unit committee structure
6
Improve lab/classroom space
5
Dropped course(s)
5
Added field experiences
4
Hired curriculum/student advisor
4
Added honors opportunities
4
Improved advising
3
Created/revised student handbooks
3
Added or improved internships
3
Increased opportunities for community service
2
Added Senior Thesis
2
Held faculty retreat
2
Added more writing in courses
2
Improved course placements
2
Improved supervision of TAs
2
Added capstone course
2
Created alumni database
1
Improved student spaces in unit
1
Developed strategic plan
1


Examples of Unit Assessment Activities


A complete listing of planned assessment activities across units is presented in 2008-09 Assessment Measures for Student Outcomes Assessment - Developed at the Unit Level and 2008-09 Assessment Measures for Student Outcomes Assessment - Existing/Standardized Measures. Below are brief descriptions of some selected unit activities.

The School of Social Work

The School of Social Work has engaged in a number of assessment activities in recent years. Several of these include a bi-annual survey of recent graduates, focus groups with current students, and analyses of ICES student ratings. Results of these assessments (shared with all faculty and reviewed by both the MSW Program Committee and each Specialization Committee), have resulted in the development of new courses and the development of Problem Based Learning case scenarios for each specialization. These cases are given as a class assignment toward the end of students’ internships and are graded by the instructor. Instructors then provide a sample of these to the Specialization committees who in turn discuss their findings with the MSW Program Committee.

Department of Industrial and Enterprise Systems Engineering

Similar to some other engineering departments, the Department of Industrial and Enterprise Systems Engineering uses direct measures of student learning to assess desired program outcomes. Curriculum mapping first identifies courses addressing each of the program outcomes. Exam items assessing the outcomes are embedded in course exams or class assignments. Instructors record student achievement and report back to the college any changes they make to increase the level of student learning. All data (courses, outcomes, embedded items, instructor actions) are stored in a custom-developed database and reviewed by the Continuous Improvement Committee (ACI). The ACI committee has the responsibility to review the course outcome data, as well as the recommendations/actions of the instructor. This closes the loop to make sure that appropriate actions are made to maintain the high quality of the educational experience in each required course in the program.

The Department of History

In assessing their graduate programs the History Department brought previous graduate students back to campus to discuss the effect of their training on their current careers, and engaged departmental forums on the success and needs of the graduate program. These discussions generated a number of areas where the program could be strengthened, especially (1) first-year cohort-building and introduction to historical scholarship and (2) an improved set of milestone checks on student progress through program requirements in order to lessen time-to-degree by clearly mapping expectations and achievements. Specifically, a “prelim preparation” coursework option was developed, whereby a student could contract for a special writing assignment, such as an annotated bibliography, to replace the standard long paper in ONE course taken to satisfy ONE of the three preliminary examination fields.

The Department of Dance

The Department of Dance uses several mechanisms for assessment such as the sophomore review, post-production evaluations of concerts, mid-term evaluation meetings and exit surveys. Among other things, results indicated a large majority of students felt procedures for casting roles in productions were unfair or biased. To address this, the department revised their casting procedures for departmental productions in order to give more students a range of performing experiences before graduation. It was also determined that a faculty or guest choreographer would be assigned to choreograph a piece for the senior class each year that would be performed in one of the main stage concerts, ensuring that every dance Other unit changes from assessment results included, hiring a physical therapist to evaluate majors in both programs yearly, tracking postural, muscular and mechanical features and potential weaknesses of every student, revising the curricular sequencing of courses, adding two more required courses in choreography and a course specific to mentoring graduate students through the Thesis requirement, plus revisions to the Student Handbook.

The Department of Special Education

All undergraduates in the Department of Special Education are required to complete an electronic portfolio which includes a multimedia teaching philosophy, integrated reflections that address the major standard areas required to be a special education teacher, and results of assessments and or assignments they have completed in order to meet multiple indicators within the standards. The e-portfolio is developed across several semesters with guidance from Special Ed Department faculty and the student’s advisor. A formal e-portfolio presentation attended by the advisor and e-portfolio coordinator occurs prior to the student’s graduation from UIUC. Written and verbal feedback from cooperating professionals and university supervisors also provide data regarding each student’s dispositions, attitudes, knowledge, and skills.

The e-portfolios have provided an opportunity to reflect on the program knowledge and performance standards. As a result, the department has identified standards and indicators that have not been adequately addressed in the program and subsequently made several changes in the courses or program experiences.

Department of Statistics

The Department of Statistics made significant changes in its undergraduate and graduate programs as the result of assessments since 2000. These were in response to indicators of student demand, exit interviews with graduates, post-degree communications with alumni, feedback from industry on the MS program, and faculty assessment of the training of our students and trends in the field. The major developments included:

  • A new first course for Statistics majors was developed, Statistics 200, “Statistical Analysis.”
  • The requirements of the B.S. in Statistics were revised to include more exposure to data analysis and computing earlier in the undergraduate experience,
  • Electives available to upper level Statistics majors were expanded to incorporate new courses developed in the past several years.
  • A new Minor in Statistics was developed in response to interest on the part of students majoring in other subjects who frequently take several of our courses to enhance their research skills.

Department of Business Administration

The Department of Business Administration is using several case write-ups to assess student learning objectives in their capstone course (BAdm 449). The criteria used to grade the case write-ups matches the language of the learning objectives, such as:

Critical Thinking & Problem Solving
Students will demonstrate the ability to analyze an unstructured problem, formulate solutions, identify strengths and weaknesses of those solutions, and make an appropriate recommendation.

Results of the cases are collected by the college assessment coordinator and shared with the faculty and curriculum committees.

Department of Philosophy

A “Comment Review” is being planned by the Department of Philosophy based on the faculty practice of regularly write comments on students’ papers. Each faculty member will be requested, for each Philosophy major in his or her class, to make a copy of any set of comments for the student’s file. A group of faculty would then review comments from a range of semesters from their seniors’ folders, looking for evidence of problems that their students are having or evidence that students have improved in various respects over the semesters.

Department of Linguistics

The Linguistics faculty is currently discussing the incorporation of one or more capstone courses for Linguistics majors to take in their senior year. A capstone course would be focused around a research project in which students would collect linguistic data that they would then analyze drawing on theories of grammar and language use introduced in the courses required for the Linguistics major. At the end of the course, participants would present their research projects to the broader UIUC linguistics community at a Linguistics Senior Symposium, held on the UIUC campus. An evaluation will be administered at the end of the capstone course, to capture students’ evaluations of their own learning outcomes over the major program of study. The faculty member(s) directing the capstone course will also be surveyed for their input about specific skills and areas of knowledge in which students demonstrate strengths and/or weaknesses. This data will inform faculty decisions about courses and teaching methods in upcoming semesters.

Department of Journalism

Journalism pre-majors will be asked to complete an ungraded test at the start of Introduction to Journalism (Jour 200) or Journalism Ethics and Diversity (Jour 250) concerning their awareness and understanding of core principles. Majors then will be tested again during their last semester before graduation. Results from the latter test will be compared with the earlier test to gauge student learning. Master’s students will be asked to complete a similar test during their initial orientation or otherwise upon admittance to our graduate program. They then will be tested again at or near the completion of their final Master’s projects. Again, the earlier and later results will be compared.

The department will also collect student work (for example, final projects or audition tapes that serve as portfolios for those particular classes as well as final Master’s projects) and have selected professionals review a sample of that accumulated work. These professionals will provide comments through an evaluation form which requires the evaluator to rank the effectiveness of the work in presenting the 11 student outcomes relevant to the sample area.

Future Assessment Activities

All units are asked to continue implementing their assessment plans and making use of their results for program improvement. In Fall 2009 all units will be receiving a survey from the Provost asking about their progress in implementing their plans. What have they done so far? Have they made any changes based on their assessment results? What are their future assessment plans? How can the university help in the implementation? Unit responses will be reviewed and the summary of the results will be posted on this website.