Selecting Learning Outcome(s) for Assessment
Ensure you have selected a learning outcome, rather than a goal for your assessment project. Goals specify what an individual course, a program, or the university aims to deliver; outcomes specify what students actually learn.
Though they will likely be related, assessment projects and reports should focus on the program’s learning outcomes rather than DePaul’s learning outcomes or strategic plan.
Assessment projects should be based on one or more program-level student learning outcomes, rather than a particular course. While some learning outcomes are addressed more in certain courses, it is important to first select a learning outcome, then identify the particular class(es) in which it is assessed. Students generally master program-level outcomes over the course of several classes.
Ensure you are assessing each learning outcome within the ten-year cycle of APR. Limiting the number of program learning outcomes to no more than nine should make this easier.
Complex learning outcomes can be broken down into components, then assessed in parts over multiple years.
Collecting Evidence
It is usually best if assessment projects are not “one-off,” but are clearly tied to unit goals, past assessment projects, and plans for the future of the program.
Multiple sources of evidence that can be triangulated generally give the best evidence of student achievement of program-level student learning outcomes. (See "Triangulation" in the Methodology page for more information).
Student learning gains in a program may be best measured using a pre- post- design, where a similar assessment project is conducted in a course students take at the beginning of the program and in a course students take at the end of a program, such as a capstone course.
Avoid assessing entry-level courses every year. In undergraduate programs, these courses may carry Liberal Studies credit (which has its own learning outcomes); but more importantly, are more likely to contain a significant number of students who are not majors. While you may hope non-major students are still achieving your unit’s identified learning outcome(s) in a particular course or courses, the overall evidence of achievement of your learning outcomes may be weakened because these students are not exposed to your entire curriculum.
Assessing student work in core courses may provide a better understanding or clearer picture of what your majors are learning.
Avoid assessing all of your learning outcomes at once. Assessment projects that focus more specifically on one or two outcomes generally provide better, more actionable results.
Collecting, Analyzing, and Interpreting Data
Instructors of individual courses from which student work was collected may not be best-suited to evaluate student work for purposes of program assessment. It may be better if other program faculty do this. Program-level assessments are not about the assignment, grading, or any particular student’s attributes (for example attendance, engagement in class, punctuality in turning in an assignment), but focus on demonstration of the program-level learning outcome(s) being assessed.
Student work evaluated for program assessment purposes should be anonymous. The focus of program-level assessment is on aggregate student performance rather than individual student performance. The evaluation of anonymous work is less likely to be unconsciously affected by faculty knowledge of individual student attributes.
Recommendations based on Assessment Results
Recommendations should be realistic, clear, and directly tied to the results of the assessment project.
It is best to include a timeline for implementing recommendations in the assessment report to ensure everyone in the unit is on the same page. This makes the recommendations more actionable for the unit and therefore increases the likelihood they will be implemented.
It is important to discuss any barriers to implementing recommendations. While the existence of barriers should not dissuade a department from making the recommendation(s), reporting on barriers is important context for interpreting recommendations and setting realistic timelines for implementing them.
Writing the Report
Although your assessment project may address multiple program learning outcomes, choose one to report for your institutional annual assessment report. This eliminates confusion in reporting results, recommendations, and planned actions in the assessment report template.
Use the assessment report template. Certain information from all institutional assessment reports are aggregated each year. Use of a different format for assessment reporting may make it difficult or impossible to include information from your assessment project in this aggregated data.
Be sure to address all portions of the assessment report template. It is very rare that a portion of this template will not be applicable to your project and each section provides the evidence of the assessment process our regional accrediting body (The Higher Learning Commission) requires.
Reports should be written for a naïve audience without a lot of jargon or acronyms. It is helpful to define field-related terms and acronyms, and to include examples of disciplinary-related concepts.
Writing the Abstract
An abstract is a critical component of a report. An abstract gives a brief overview of the purpose and content of the report. The abstract should simply state the contents of the report without evaluating any of its information. An abstract should be written in the present tense and use active words. A good abstract is accurate, self-contained, concise, and specific.
To ensure accuracy, the information in the abstract should match the information within the body of the report.
To ensure the abstract is self-contained:
- All abbreviations and unique terms and phrases should be defined the first time they are used.
- Paraphrase text, instead of using quotations.
An abstract should be no more than 250 words and should begin with the most important information. It should include the purpose of the report, learning outcome explored and results from the assessment project.
To keep the abstract concise and specific:
- Use digits for all numbers, unless numbers are at the beginning of a sentence.
- Use abbreviations and acronyms generously; explain abbreviations or acronyms when first used in the abstract.
Source: Publication manual of the American psychological association. (2001). (5th ed.) Washington, D.C.: American Psychological Association.
Rationale for Reporting on Only One Learning Outcome
Although you may have assessed multiple learning outcomes in your annual assessment projects, we ask you to complete the assessment report for just one of those outcomes.* We do this for two reasons:
- Institutionally, we aggregate certain information from all assessment reports. Specifically, we aggregate the information from the learning outcome assessed and results section. Because each of your learning outcomes is aligned to particular DePaul University Learning Goals and Outcomes, we can combine this information with all other assessment reports submitted during a particular academic year to gather information about how well DePaul is achieving its institutional learning goals and outcomes. When an annual assessment report describes the results for multiple learning outcomes, it is not always clear which results pertain to each particular learning outcome.
- When DePaul’s Assessment Advisory Board revised the annual assessment report template, one of their primary goals was to simplify the assessment reporting tasks for individual units and programs. Our hope is that asking for reports to focus on only one learning outcome simplifies the reporting task.
*Although we ask for reports to focus on just one learning outcome, we understand that it often makes sense to assess multiple learning outcomes at once. We encourage you to do so and to use all of the information you gain from these assessment projects internally within your units or programs.
Determining What is "Acceptable" Performance for a Program Learning Outcome
DePaul University’s assessment report template asks for the number of students with “acceptable” or better performance, but how do we determine what is acceptable?
Determination of Acceptable Performance
Program faculty should determine what acceptable performance is for given a particular learning outcome. As defined by
HLC’s criteria for assessment, assessment should involve the substantial participation of faculty and other instructional staff members.
Resources for Determining Acceptable Performance
Review the program’s curriculum, including curriculum map, if you have one. Looking at the curriculum, with particular emphasis on the core curriculum, can be a useful exercise in breaking down the components of program learning outcomes. This will also give faculty a realistic idea of what students should have learned in regards to a particular learning outcome across the entire program.
Determine what students need to know after they leave
All academic programs prepare students to leave DePaul University. It may be useful to discuss not only the knowledge, skills, and abilities students need to possess to move onto their next steps (graduate school, full-time employment in the field, fellowships/post-doc work), but the level at which students’ knowledge, skills, and abilities should be to successfully take the next steps in their educations and/or careers. One useful resource to reference may be the professional organizations within your particular field. These groups generally define standards for professionals in their fields and may be able to inform your faculty’s decisions about what one might consider acceptable performance for each of your program learning outcomes.