Frequently Asked Questions

Administrators in charge of OTE for an academic unit (College, Department, Program) that administers its own evaluations (as opposed to one which delegates administration to its own subunits) have essential quarterly responsibilities:

1. At the beginning of each quarter, when prompted by an email from the system, the admin must initiate the evaluation process for the unit. This is done by accessing the system at Campus Connect > For Employees > Online Teaching Eval-Admin > Term Evaluation Tools > Initiate Course Evaluation Process, checking the box next to the term of interest and clicking the Save button. For more information on this topic see also question 17 in the General Question section below (“How do administrators know what to do at the beginning of each quarter?”)
2. At the beginning of each quarter the admin must enter any special class that needs to be evaluated according to a schedule different from the standard one. This is done by accessing the system at Campus Connect > For Employees > Online Teaching Eval-Admin > Term Evaluation Tools > Define Special Classes, and following the detailed instructions found on that page.
3. If and when prompted by an email from the system, the admin must claim any class that belongs to the unit and that the system flagged as a cross-listed exception. This is done by accessing the system at Campus Connect > For Employees > Online Teaching Eval-Admin > Term Evaluation Tools > Manage Cross-listed Exceptions, and following the detailed instructions found on that page. For more information on this topic see also question 8 in the General Questions section below (“How does OTE handle combined (cross-listed) classes?”)
4. During the evaluation period the admin should monitor rates of response in the unit and intervene with the faculty members whose rates are significantly low. This is done by accessing the system at Campus Connect > For Employees > Online Teaching Eval-Admin > My Evaluation Reports > Rate of Response Report

In a regular 10 weeks quarter OTE administrators must initiate the process, exclude classes, enter special classes and resolve cross-listed conflicts by the end of the 6th week of the quarter. No changes made to excluded and special classes after the beginning of week seven will be loaded in the system.

Administrators in charge of OTE for an academic unit (College, Department) that delegates the administration of evaluations to its own subunits (Departments, Programs) have one essential quarterly responsibility:

At the beginning of each quarter the admin must update the owners of records for all the subunits in the system. For example, when a new Chair of a Department or a new Director of a Program begins to serve as such, their information needs to be updated in the system. This is done by accessing the system at Campus Connect > For Employees > Online Teaching Eval-Admin > Set-up Tools > Edit Setting & Security, Evaluation Settings tab, clicking on Continue. Click on Edit next to the subunit whose owner needs to be updated, enter the First and/or Last Name and/or Emplid of the new owner and click Search. Click Select next to the new owner. Click Save.

1. First week of the quarter: OTE sends emails to remind about initiation of the process,and need to attend to special classes and excluded classes
2. End of week six: Deadline for initiating the process, entering special classes, entering excluded classes, managing cross-listed conflicts, changing end dates for student evaluation period
3. Beginning of week seven: The system opens to Faculty to allow creation of individualized questions (where applicable)
4. End of week eight: Deadline for faculty to enter individualized questions (where applicable)
5. Beginning of week nine: The system opens to students to perform class evaluations. Admins should start monitoring rates of response.

Administrators in charge of OTE for an academic unit (College, Department, Program) that administers its own evaluations (as opposed to one which delegates administration to its own subunits) have three essential one-time responsibilities:

1. The admin must create instrument(s) for the unit. This is done by accessing the system at Campus Connect > For Employees > Online Teaching Eval-Admin > Set-up Tools > Create/Edit Academic Unit Evaluation and clicking on create new evaluation. The system allows a unit to use different instruments for face-to-face and online classes. If no special instrument is entered for online classes the same instrument entered for face-to-face classes will be deployed to online classes as well.
2. The admin must decide if faculty in the unit will have the option of customizing their instruments with up to five individual questions. This is done by accessing the system at Campus Connect > For Employees > Online Teaching Eval-Admin > Set-up Tools > Edit Settings &Security, Evaluation Settings tab, checking the box next to “Allow Faculty to add up to 5 questions to their course evaluations”.
3. The admin must input the necessary parameters for the system to operate.This is done by accessing the system at Campus Connect > For Employees > Online Teaching Eval-Admin > Set-up Tools > Edit Settings & Security, clicking on Continue. The required parameters are:
1. Numerical and percentage thresholds for evaluation availability and report generation. These parameters can be changed at any time.
2. End date for the evaluation period for students. and clicking on create new evaluation. To change this date for a specific quarter, the change must be entered before the end of week 6.

Online Teaching Evaluation (OTE) is a web-based application that allows:

1. Academic units (colleges, departments, programs) to manage and activate online their own forms for course evaluations;
2. Students to perform the evaluations online;
3. Faculty members to:
1. Personalize the evaluations with up to five course-based additional questions if their academic unit so chooses;
2. Access online the results of their evaluations and comparative statistics of their academic units;
4. Admin teams of academic units to access a variety of reports on the results of the evaluations.

No.

The system gives complete control of the evaluation form to each College and its internal units.

The only exception concerns the 5 questions approved by faculty council in 2006, which are automatically added at the beginning of the evaluation form of each unit. Find here more information on these questions.

Upon request of a few units within the University, IS developed an OTE 1.0 system in March 2007 (pilot program) with the system in production in May 2007. The following Colleges-Schools used OTE 1.0:

Driehaus College of Business
School of Music
College of Law
School of Public Service

A decision to re-design and improve the application, working towards OTE 2.0, was made in 2008. The new version was launched in Fall 2009.

OTE consists of:

An administrative web tool, accessible through Campus Connect, that allows academic units to:

independently and autonomously join the system;
enter their evaluation forms;
set customized values for a number of system parameters;
access reports for their units.

A delivery module that will:
Generate the evaluation forms putting together the 5 university questions, questions inputted by academic units (Colleges-Schools, Departments, Programs), possible additional questions added by individual faculty, possible additional questions for courses that are special in some way (CBSL,….);
Deliver the evaluation to students via email messages with periodic reminders;
Collect and compile the data.

A faculty web tool accessible through Campus Connect, that will allow faculty to:
add personalized questions (up to 5) to evaluation forms for their classes each quarter;
access results of their evaluations in various forms.

OTE 2.0 is built around a hierarchy of users articulated in 5 levels.

1. Think of this as the CENTRAL/TECHNICAL level
2. Think of this as the COLLEGE/SCHOOL LEVEL
3. Think of this as the DEPARTMENT LEVEL
4. Think of this as the PROGRAM LEVEL
5. FACULTY LEVEL

Each College/School may or may not choose to take advantage of levels 3 and 4 of the hierarchy, thus allowing both a centralized and a decentralized model of course evaluation.

The labels "College/School", "Department", and "Program" are used here simply as examples. Each College/School and other high-level unit (the Liberal Studies Program for example) communicated its own internal structure to the OTE team.

The OTE team implemented the structure in People Soft using the Academic Organizations tree.

OTE 2.0 is built around a hierarchy of users articulated in 5 levels:
CENTRAL/TECHNICAL LEVEL
COLLEGE/SCHOOL LEVEL
DEPARTMENT LEVEL
PROGRAM LEVEL
FACULTY LEVEL

Here are the most important thing you need to know about the role of the hierarchy of users.

Each level of the hierarchy has one owner. For example, The Deans are owners of level 2, chairs in LAS and COE are owners of level 3.

The owner of each level is designated by the owner of one level up. Example: The Deans (or their proxies-see below) will communicate to the system who the owners of all their departments (if any) are, the Chairs (if any) will communicate to the system who the owners of all their programs (if any) are.

Each owner has full control of the evaluation process in her/his unit: S/he communicates to the system that her/his unit will avail itself of the system itself in a specific quarter/semester. S/he enters questions (if applicable) to be used in the evaluation process. S/he sets all parameters necessary for the system to work. S/he decides (if applicable) if s/he wants to delegate the administration of the evaluation process to his/her sub-units.

Each owner can give access to the system to as many other people from his/her administrative team as s/he wishes. Owners can give three different level of access to other people:
Admin and reports: this choice creates a full proxy for the owner.
Individuals with this level of security can perform all administrative functions (set parameters, decide whether to allow subunits to add questions….) and have full access to reports of evaluations results for all faculty members in your unit.
Admin only: Individuals with this level of security can perform all administrative functions (set parameters, decide whether to allow subunits to add questions….) but will not have access to any report of evaluation results of your unit.
Reports only: Individuals with this level of security will have full access to reports of evaluations results for all faculty members in your unit.

Classes are attached to one of the nodes created at the various level. For example, all PSY classes will attached to the "Department of Psychology" node at the Department level in the College of LAS tree, all School of Continuing and Professional Studies undergraduate classes will be attached to the "Undergraduate" node at the Department level in the SCPS tree.

Statistics for a group of classes are computed only if the group is represented in the hierarchy as a node at one level of the tree.

The system allows three types of questions: Open ended, Yes/No, and Likert-scale.

There is technically no limit for the number of questions you can enter for an evaluation, although you should be mindful of the fact that very long evaluations might turn students off.The following table summarizes all existing upper bounds of which you must be aware. Item and maximum allowed:
Text of each question (no matter what type)... 130 characters
Text of students' answer to each open ended question... 3999 characters
Number of values in a Likert scale... 7 values
Text of name/description of each value in a Likert scale... 30 characters

If two or more classes are cross-listed within a unit (for example a graduate and undergraduate version of a class) and thus share the same evaluation form, OTE 2.0 will simply deliver to the students the unit's evaluation form with (if applicable) the instructors added questions.

If two or more classes are cross listed across different units (different departments, different colleges...), OTE will do the following:
Beginning on the first day of the quarter, OTE checks daily who the primary instructor of record for the combined class is.
OTE then checks payroll data to determine the academic unit to which the instructor of record belongs.
OTE then considers the combined class as belonging to the academic unit established in 2.2. and will deliver to the students the appropriate evaluation.

If payroll data is not available for the instructor of record, OTE cannot complete step 2.2. In this case an email is sent automatically by the system to ALL administrators involved with the combined class at hand, asking them to come to a consensus on which academic unit "owns" the class. Admins can then communicate the decision to the system via the web-tool. If a conflict is not resolved by the end of week 6, the class involved will not receive an evaluation.

No.

OTE 2.0 associates the primary instructor of record in People Soft with each class. Only this faculty member will have access to results of the evaluations through the system.

Several units have expressed a desire for this feature. It will be considered during future revisions.

Yes.

Many units have classes whose duration is non-standard.

OTE 2.0 refers to these classes as "Special."

The system gives each unit complete control on the evaluations for these classes. Each unit specifies when faculty can add questions (if applicable) and when students can conduct the evaluations for special classes.

Each unit may have a few special classes that might not be amenable to be evaluated by the current capabilities of the system. Once you are sure that one or more of your classes cannot possibly be served by OTE 2.0, you can communicate to the system that they need to be "Excluded."

A simple process accessible via the admin web-tool allows you to enter this information.

Students will receive a personalized email reminder, as in the sample below, at the beginning of the evaluation period.

The email will be triggered by the availability of evaluations for any course in which the student is enrolled.

The message will list ALL courses in which the student is enrolled and for which the evaluation period is active. The message will indicate if any of the courses has already been evaluated by the student.

Email reminders will be sent every other day for the first week of the evaluation period, and then every day for the remaining amount of time during which evaluations are open. For special length classes, if the evaluation period is longer than two weeks, email reminders will follow the same pattern as the ones sent for regular quarter length classes. If the evaluation period is less than two weeks long, students will receive email reminders daily.

If at the same time a students has an active regular quarter class and a special class, the email will be sent with the highest of the two frequencies.

Email reminders will stop when all available evaluations have been completed.

At the beginning of week seven of a quarter, faculty members will receive a personalized message as in the image below. The message will contain an active link to the system, where the faculty member will be able to add questions to the evaluations of their classes.

The system generates the following reports, each of which is available in PDF, Excel, or HTML format via a web tool accessible through Campus Connect:

1. A basic statistical report, containing means and standard deviation for each question on the evaluation that allows a numerical answer. Faculty are also provided with the means and standard deviations of their program-department-college, whenever these make sense. The means are also be represented via a histogram. An example of the PDF version of this type of report is found here.
2. A detailed report divided into two sections.
3. The first section lists all students' responses to each open-ended question, grouped by question. Each respondent is assigned an anonymous ID number that can be found in the left-hand column of this report. This is not the student's DePaul ID number. It is a randomly assigned value that is only used in the current evaluation report.
4. The second section shows responses grouped by student. This section displays responses to all questions in the evaluation—both open-ended and Likert scale. This allows you to view each student's responses to all questions in one place in the same way you would view a single student's entire paper evaluation. An example of this type of report can be seen here.
5. A frequency report, containing the frequency of answers to each non open-ended question.

Three families of reports are available to admins:

1. Rate of response report
2. Faculty Reports
1. Statistical
2. Detailed responses
3. Frequency
3. Instructor Effectiveness
4. Aggregate reports

Classes are attached to one of the nodes created at one of the levels of the hierarchy of the system. The system computes means and st-devs over each group of classes represented by a node at one of the levels of the hierarchy.

At the beginning of each quarter administrators receive an email message to alert/remind them of the need to attend to OTE business.

The body of the message depends on whether the academic unit of interest has sub-units or not specified in the hierarchy structure of the system.

Information on the process that led to the adoption by Faculty Council of these five questions in May 2006 can be found on the Teaching Commons.

The OTE working group agreed on a reordering of the questions. They will appear at the beginning of every evaluation, in the order listed here below.

1. Given your experience at DePaul, rate the overall quality of the course.
1. 5 Among the best
2. 4 Better than average
3. 3 About average
4. 2 Worse than average
5. 1 Among the worst
2. Given your experience at DePaul, rate the instructor's overall teaching effectiveness.
1. 5 Among the best
2. 4 Better than average
3. 3 About average
4. 2 Worse than average
5. 1 Among the worst
3. Overall, this course increased my knowledge or skills:
1. 5 More than almost all other courses
2. 4 More than many courses
3. 3 About the same as other courses
4. 2 Less than many courses
5. 1 Less than almost all other courses
4. I found this course to be:
1. 5 Among the most challenging
2. 4 More challenging than many courses
3. 3 About as challenging as other courses
4. 2 Less challenging than other courses
5. 1 Among the least challenging
5. The instructor stimulated interest in the subject.
1. 5 Almost always
2. 4 Most of the time
3. 3 About half of the time
4. 2 Some of the time
5. 1 Almost never

Each unit has the ability to enter two separate evaluations: a standard evaluation for for face-to-face classes and an optional evaluation for OL classes.

These two evaluations can be completely different if the unit so chooses.

If a separate evaluation for OL classes is not entered AND the unit has OL classes scheduled, the standard evaluation is delivered to OL classes as well as face-to-face ones.