Creating Assessment Plans

Video-Introduction: PLO Program Assessment - Cycles and Report Template

Video-Introduction: Undergraduate Equity Dashboards

Useful Links:

Assessment Overview Slideshow - Helpful Worksheet ("Equity Assessment Guiding Questions")

VPN Access Setup (Required for viewing Tableau Dashboards) 

Undergraduate Equity Tableau Dashboard - Flow of Major Diagram - Course Grade Distribution

Graduate Dashboards (no login required) - UCSB Student Enrollment, Time to Degree, Advancement; UCOP Exit Survey, Grad Statistics, Doctoral Rates, Doctoral Program

WHAT EXACTLY ARE YOU LOOKING FOR?

Assessment projects are designed to support equitable achievement of your learning outcomes, across students in your major. This is achieved by looking at your PLOs through a lens of disaggregated data about student learning across effectors of opportunity, compared with the program’s observable goals/benchmarks for student learning. The Council on Assessment is looking for the following in an Assessment Report and Plan:

SECTION I

  • A concise summary of your findings from the last cycle of assessments
  • What the program learned from them (i.e., is X meeting your standards? would you like to see Y improve?)
  • What the department has done/is doing with that information. 

Here, we can see whether your prior plan was (a) realistically achievable and (b) practically useful for your program.

SECTION II

  • Some current information about your program (degree name, student enrollment, faculty involved in this assessment) and a review of your PLOs and curriculum map. 

This is your opportunity to describe any changes made to the program in the last three years, as well as to request assistance in updating your PLOs, if you would like.

SECTION III

  •  An equity-focused plan that looks at data about student learning, to formulate questions and collect evidence regarding the program's explicit goals for student learning (i.e., your Program Learning Outcomes).
  • This plan should utilize institutional data about your program (department dashboards, equity dashboards, etc: provided to your program by our assessment team), examine direct evidence (i.e., actual student work) as well as indirect evidence (e.g., surveys and focus groups about students' experiences), include a criterion-based rubric / assessment tool specific to your inquiry, and a offer a timeline of when/where data will be collected and who/how it will be assessed.

Your plan focus can be either the same PLO as the prior cycle (if you need further data or need to adjust your method of investigation) or a different PLO (if that inquiry is complete, or the program currently wants to prioritize another program outcome). 

We have created a Template for PLO Assessment Plans that lays out all of these parts in question-answer format, for your convenience! 

Midway through the second year of implementing your plan, we will request a brief Progress Report, mainly to ensure that—if you needed to make changes to your initial plan, or would like any support in completing it—we can help you in a timely manner.

And that’s it! Just those two documents, every three years. PLO Assessment achieved.

 

Step-by-Step: Equity-Focused Assessment

As outlined on UCSB’s Institutional Accreditation page, equity-focused assessment of your program’s learning outcomes begins by looking at Institutional Data: department profiles, equity dashboards, and course grade distributions on Tableau, as well as any other pertinent data from UCSB/UCOP.

First, formulate pressing questions (aim for 5) about undergraduate and graduate learning in your program. Then list details in these data (4-7 things) that you noticed, with reasons why you found them interesting, what they made you wonder about. Next, prioritize this list based on which of these areas of interest you see as most important to equitable achievement of student learning in your program. For each of these prioritized items, look at the dashboards and list other useful data related to this topic of interest.

Finally, use these reflections to develop a PLO Assessment focus: 2-3 questions about an item on your prioritized list, related to courses and/or instruction in your department, focused on achieving a specific program learning objective. Consider what direct evidence (criteria-based quantitative data and expert assessments) you can collect to better understand what is happening with student learning in the program, as well as what indirect evidence (surveys, focus groups, interviews about learner experience) you can collect to understand why it is happening.

 

WHEN DO YOU WANT IT?

UC Santa Barbara operates under a three year cycle for assessing program learning outcomes.  Every three years departments report on the recent PLO assessment, and create a plan for the next PLO assessment. 

Is your program in Group 1, 2, or 3? Here is a list of Assessment Groups/Cycles by Department.

Assessment Groups/Cycles by Department

At the beginning of the three year cycle (in January), your department must report on its assessment findings from the prior cycle AND its plan for the next three year PLO assessment.

Midway through year two of your three year PLO Assessment Plan (in October), a progress report is due. This is a required document.

Email all documents to Josh Kuntzman, our assessment coordinator.

Three-Year Assessment Plans

A successful PLO Assessment Report and Plan will include (a) a report of findings and outcomes from the prior PLO assessment cycle; (b) any updates to the program’s PLOs or Curriculum Map; and (c) a new plan, focused on equitable achievement of your Program Learning Outcomes (PLOs) which references institutional data, defines an equity-oriented question related to a specific PLO, describes data to be collected, includes a rubric or scoring guide to be used in evaluating this data, and offers a timeline detailing when/where/who will be doing this work.

We strongly recommend that you submit this current Template for PLO Assessment Plans to ensure that your plan includes all pertinent details, and will be approved by the Council on Assessment.

PLO Assessment Plan - Template

Plans that are not initially approved will receive specific recommendations for revision from the CoA, and programs will coordinate with our assessment coordinator Josh Kuntzman on submitting a revised plan for CoA to review and approve. So:

  • Your program's Assessment Team will submit a PLO assessement plan to the Assessment Coordinator, who will distribute it to Council on Assessment members for review and vote;
  • The CoA will send a Memo to your program, either approving your plan or requesting specific revisions; 
  • If revisions are requested, your program will submit a revised plan to the Assessment Coordinator, who will work with you to ensure that all CoA issues are adequately addressed;
  • The CoA will send you a memo approving your plan for implementation over the next three years of the assessment cycle. 

Of course, you can reach out to the Assessment Coordinator, Josh, at any point during your planning process! He can help you review draft-plans, craft PLO-specific rubrics for your program, and answer any other questions about how to make your plan work best for you, your faculty, and your program.

Here are some important points to remember:

  • These Assessments target Programs: focus on required courses in the major/program (as opposed to electives or GEs).
  • These Assessments are Equity-Oriented: examine the diverse factors affecting students' educational access and success in your program.
  • These Assessments are Practical: focus on what you can do (in courses/curriculum) to improve student learning; select PLOs and guiding questions/theories accordingly.
  • Plans need to be Coherent: make a clear connection between your target-PLO(s), your research question(s), your direct/indirect evidence, and your rubric/scoring guide criteria.
  • Plans need to be Specific: reviewers are not from your departments! So, when in doubt, spell it out - why are you collecting data at This point in your curriculum, why is this a Sufficient sample-size for your question, why is your Standard for success __% or __/5 achievement etc., in this plan? Let us know your reasoning.

For inspiration, feel free to look through some of these prior plans from Graduate and Undergraduate Programs across departments at our University.

Sample Assessment Plans

Humanities & Fine Arts:

Social Sciences:

Math & Hard Sciences:

Engineering:

Other Colleges:

 

Progress Reports

Submitting a Progress Report midway through the second year of your assessment benefits you in two ways. 

  1. If you have deviated from your initial plan, or need to make changes going forward, you can use the progress report to describe these changes and get feedback/support from the CoA as needed. 

  2. Alternately, if everything is going according to plan, you can provide an initial report of these findings, which can then be used as a rough draft for your final report at the end of your assessment’s third year.

Progress Report - Template

Below are a few sample Progress Reports, to give you an idea of what these check-in documents might look like.

Sample Progress Reports