| Time Codes |
Visuals |
Audio |
| 00:00:01-00:00:28 |
Slide 1: "Fundamentals of Student Learning Outcome (SLO) Assessment, Part I
Eight Frequently Asked Questions about writing SLO Statements by Joe Cullen-Connecticut State Community College, Director of Program Review and Assessment |
Welcome to this presentation on the Fundamentals of Student Learning Outcome Assessment Part one. This presentation will focus on eight questions frequently asked about writing Student Learning Outcomes Statements. My name is Joe Cullen, director of Program Review and assessment for the Connecticut State Community College and over the next 20 minutes, I will share my take on best practices in the field of student learning outcome assessment. I will try to be as scintillating as possible.
|
| 00:00:29-00:00:47 |
Slide 2: FAQ 1: Why do we need Student Learning Outcomes (SLO) Assessment?
|
It's been my observation that folks in academia like to know the reasons for things before they invest their time and effort. So the first question we'll consider is why do we need student learning outcome assessment? I will try to answer this question by telling you a story. |
| 00:00:48-00:01:44 |
A: Open Education Movement of the 1970's: A Case Study
-Traditional classrooms stifle children's creativity and natural desire to learn.
-Schools built around open space
Example image of an open education classroom
|
In the 1970s and eighties, there was a reform movement in elementary education called Open Education. Open education was based on the belief that traditional classrooms with their neat rows of desks and teacher centered activities, stifled children's creativity, self-direction and natural desire to learn. As a result, billions of dollars were spent on designing school buildings around freedom and open space, which meant that they had few, if any, interior walls. That's sort of what they look like. They also did not have standardized tests, whole class lesson plans, detailed curricula, or many of the other usual trappings of elementary education. Instead, students worked in little activity centers where they could move around, explore and interact, or on projects in collaborative teams with other students. |
| 00:01:45-00:01:57 |
Slide 3: How did the Open Education model work out? |
So how, you might ask, did the open education model work out? Well, the intentions of the OE movement were certainly noble, but the results were at best mixed. |
| 00:01:58- 0:02:15 |
-student behavior problems; high rates of teacher burn-out; and poor learning outcomes |
Its positive benefits included things like growth in emotional learning and collaborative skills but they were overshadowed by the fact that open classrooms were often chaotic learning environments characterized by increased rates of student behavior problems, high rates of teacher burnout and poor learning outcomes. |
| 00:02:15-00:02:43 |
OE advocates could not defend the model because they did not have:
-Clearly stated, measurable learning outcomes
-well- developed measures of student learning
-Adequate data collection and analysis procedures
|
In addition, students in OE schools did not perform well on state mandated standardized tests. So when it came time to defend the model, advocates for open education really were not able to do so effectively because they had rejected some of the essential features of traditional classroom learning, such as clearly stated measurable learning outcomes, well-developed measures of student learning and adequate data collection and analysis procedures. |
| 00:02:43-00:03:51 |
led to "back to basics" backlash |
Because they couldn't provide data that was good enough to inspire confidence in the model it was widely rejected in favor of a back to basics movement. That movement prioritized student performance on state mandated tests, and as a result, many communities that had built OE schools retrofitted them with interior walls and neat rows of desks. Ironically, the OE movement ended up advancing the stifling educational practices it was designed to eradicate and in the process, discredited educational reform and educational research in general. My point here is obvious. Had the advocates for open education been more rigorous in assessing their model, more objective in interpreting their data, they might have been able to identify its shortcomings and make improvements. In particular, if they had disaggregated their data by ethnicity and culture, that would have found that it worked well in privileged white communities but was a dismal failure and inner city minoritized communities that heavily bought into the model. |
| 00:03:54-00:04:29 |
Slide 4:
The Moral of the Story
Sound SLO assessment is essential to the task of evaluating the effectiveness and fairness of educational programs.
- specifying learning outcomes in measurable terms,
-developing valid and reliable measures,
-assessing a large and representative sample of students,
-disaggregating data to identify any equity issues
-using the results to improve programs and promote the success of ALL students
|
So the moral of the story is that the demise of the OE movement demonstrates the importance of sound SLO assessment. Sound SLO assessment is essential to the task of evaluating the effectiveness and fairness of educational programs and by sound assessment, I mean specifying learning outcomes in measurable terms, developing valid and reliable measures, assessing a large and representative sample of students, disaggregating data to identify any equity issues and using the results to improve programs and promote the success of all students. |
| 00:04:29-00:04:59 |
Slide 4:
(Animated gif on left side of screen, girl looks up at illustrations of lightbulbs flashing around her head)
FAQ 2: So, what, exactly, is Student Learning Outcome (SLO) assessment, anyways?
A1: SLO assessment is NOT:
-a tool used by administration to evaluate faculty or staff
-a set of standards directed by the administration
-based primarly on course grades
-designed to be an onerous burden on faculty
|
So frequently asked Question number two is 'So what exactly is student learning outcome assessment anyway?' I will answer this first of all by telling you what it is not. SLO assessment is not a tool used by administration to evaluate faculty or staff. It is not a set of standards that is dictated by the administration. It is not based primarily on course grades, and it is not designed to be an onerous burden on faculty. |
| 00:05:00-00:05:34 |
(Slide 4 continued)
A2: SLO assessment IS a data collection process that:
- is designed and implemented primarly by faculty
-reveals the extent to which students know what we want them to know after completing a learning activity
-is focused on student success relative to benchmarks
-spotlights effective campus-level practices and brings them to scale
-generates a rich source of information that can be used to improve programs
|
What student learning outcome assessment is, is it's a data collection process that is designed and implemented primarily by faculty that reveals the extent to which students know what we want them to know after completing a learning activity. It’s focused on student success relative to benchmarks, spotlights effective campus level practices, and brings them to scale so that we can all benefit from them and generates a rich source of information that can be used to improve programs. |
| 00:05:36-00:06:32 |
Slide 5:
FAQ 3: What does the SLO assessment process look like?
Seven steps:
1 .developing student learning outcome (SLO) statements
2. mapping SLOs to curriculum
3. developing assessments
4. determining criteria for success
5. collecting and analyzing data
6. closing the loop: planning and executing improvement plans
-documenting those activities to repor to key stakeholders
(Gif with Eugene Levy in scene from Schitt's Creek says "Why don't we take this one step at a time?")
|
So what does the SLO assessment process look like? Is frequently asked Question number three. Well, there are seven generally accepted steps, the first being developing student learning outcomes statements, mapping those statements to the curriculum, developing assessments, determining criteria for success, collecting and analyzing data, then using that data to close the loop and plan and execute program improvements, and then documenting those activities and reporting them to key stakeholders, including accreditation bodies.
Why don't we take this one step at a time? This presentation will focus on step one developing Student Learning outcomes statements. Subsequent presentations will deal with steps two through seven.
|
| 00:06:33-00:07:10 |
Slide 6: Illustration of head with arrows shooting of it
FAQ 4: What do SLOs look like?
(Gif of John Candy getting ready to cut meat pops up)
-Are specific statements of what students should know or be able to do after successfully completing a learning experience.
-Are written in concise, student-centered, measurable terms
-Can be used to describe student learning at any level
|
Frequently asked question number four: What do SLOs actually look like? Well, this gets to the meat of this presentation. SLOs are specific statements of what students should know or be able to do after successfully completing a learning experience. They are written in concise, student centered, measurable terms, and that can be used to describe student learning at any level-institutional, programmatic, departmental, course level, and even the unit or assignment level. |
| 00:07:11-00:08:14 |
Slide 7: Lightbulb illustration
FAQ 5: If we have goals and objectives, why do we need to write SLOs?
Because they are the first step of backward design. Once we have described what students are expected to know or do, then we can:
-Plan appropriate instructional activites, teaching strategies, materials, and assignments.
-More readily determine when instruction is not working
-More effectively align course, program and institutional SLOs
-Show students what they will gain from a learning experience
-Give students a tool for measuring their progress
-Let students know in advance what will be expected of them and how they will be assessed.
|
Frequently asked Question number five is if we have goals and objectives, why do we need to write SLOs? Well, the answer is because they are the first step of backward design. Backward design is the process of developing curriculum by first determining what we want our students to know or be able to do at the end of their learning experience. If we describe what we want them to know or be able to do at the outset, then we can plan appropriate instructional activities, teaching strategies, materials and assignments. We can more readily determine what instruction is not working. We can more effectively align course program and institutional SLOs. We can show students what they will gain from a learning experience. We can give them a tool for measuring their progress, and we can let students know in advance what they will be expected to do as part of their participation in a course or other learning experience. |
| 00:08:14-00:08:49 |
Slide 8
FAQ 6: How do SLO's differ from goals and objectives?
A: Three ways 1) specificity, 2) measurability and 3) student centeredness
Goals=broad statements written from an instructor, prgrmmatic, or institutional perspective that describe th general content and direction of a program, course, or other learning experience.
-"The history curriculum at CSCC seeks to expand student knowledge of world affairs, cultures, societies, and values."
|
How do SLOs differ from goals and objectives is frequently asked question number six and they differ in three ways. Specificity, measurability, and student centeredness. Goals tend to be broad statements. They're written from an instructor, programmatic or institutional perspective. They describe the general content and/or direction of a program course or learning experience.
An example would be the history curriculum at CSCC seeks to expand student knowledge of world affairs, cultures, societies and values.
|
| 00:08:50-00:09:06 |
(Slide 8 continued)
Objectives=more specific than goals; focus on what faculty intend to teach in a course or other learning experience.
-"In History 212 we will explore the historical significance of pre-modern 'non-western' cultures
|
Objectives are more specific than goals, but they tend to focus on what faculty tend to teach in a course or other learning experience. An example would be in History 212 we will explore the historical significance of premodern, non-Western cultures. |
| 00:09:08-00:09:30 |
(Slide 8 continued)
SLOs=specific, measurable statements written in student-centered terms.
-"Upon successful completion of History 212, students will be able to describe three ways that the Greco Roman culture has historically influenced the culture of Western Europe."
|
An SLO by contrast, is a specific, measurable statement written in student-centered terms. An example would be “Upon successful completion of History 212 students will be able to describe three ways that Greco-Roman culture has historically influenced the culture of Western Europe.” |
| 00:09:31- 00:10:00 |
Slide 9
FAQ 7: What are the characteristics of effective SLOs?
They generally take the form of:
"Upon successful completion of [learning experience], students will be able to [measurable verb] + [content statement]."
|
Frequently asked question number seven is “What are the characteristics of effective student learning outcome statements?” Well, first of all, they generally take the form of upon successful completion of X, Y, or Z learning experience, students will be able to add measurable verb and content statement. We will come back to what we mean by a measurable verb in a moment. |
| 00:10:01-00:11:14 |
(Slide 9 continued)
S.M.A.R.T.:
-Specific
-Measurable
-Attainable
-Relevant
-Timely
|
In addition effective SLOs are often described with the acronym SMART, where S stands for specific, which means using precise language, that is, terms that inform the consumer of exactly what you're trying to achieve. So for example, rather than, say, “students in Computer Science 1010 will learn the programing language python” you might say that “students in Computer Science 1010 will use the program language python to complete an independent programing project.”
M stands for measurable meaning that they result in a product or behavior that can be observed and evaluated.
A stands for attainable meaning that they are written within our student’s instructional range challenging but not over their heads.
R means relevant meaning they address knowledge that is germane to the subject matter of the program or course they were taking and where possible skills that can be applied in the real world.
T means timely meaning they should be continuously updated to reflect the ever changing world we live in.
|
| 00:11:15- 00:11:55 |
(Slide 9 continued)
Written in language that students can understand
Manageable-more is not better
(Gif of man pointing to head with word "SMART" overlaid)
|
A couple of additional features of effective SLOs include that they should be written in language that students can understand. If we're going to partner with them to achieve our learning outcomes, then it makes sense to write SLOs in a manner that engages them in that partnership and they should be manageable. More is not better. Quality is preferable to quantity. Now, returning to the form at the top of the slide, the verb used in writing SLO statements tend to come from Bloom's hierarchy of learning. |
| 00:11:57-00:14:25 |
Slide 10:
Measurable Verbs: Bloom's Taxonomy (A.K.A. Hierarchy) of Learning
Image: Graph pyramid with 6 rainbow colored sections shows the different levels of Bloom's Taxonomy as each level. Each level is described verbally (in box to left)
|
Many of you have, no doubt, seen Bloom's taxonomy of learning so if you're familiar with it, feel free to skip the slide. If you're not familiar with it, or you would like a quick review. Bloom's Taxonomy of Learning describes six levels of learning in terms of the depth and complexity of the cognitive activity required of the learner.
At the bottom, we have lower order thinking skills, starting with the most basic skill of remembering, which refers to the simple recall of information, concepts, principles and or ideas. Learning outcomes at this level tend to use measurable verbs such as define, identify, label, list, name, recognize, or recall.
Next step is understanding, which requires the learner to demonstrate that she or he grasped the meaning of concepts and ideas and learning outcomes at this level use measurable verbs like characterize, clarify, discuss, distinguish, explained, or express.
The third level up is applying, which refers to the act of information and or concepts to perform a task or solve a problem. Learning outcomes at this level use measurable verbs like adapt, calculate, demonstrate, determine, illustrate, perform, solve or use.
Next up is analyzing, which refers to breaking down concepts, principles, and or ideas to identify relationships. Learning outcomes at this level use measurable verbs like break down, compare contrast, determine, discriminate, examine, infer or interpret.
Level five Evaluating requires the learner to examine concepts, principles, and/or ideas to make judgments about them. Learning outcomes at this level use measurable verbs like appraise, assess, criticize, critique, grade, judge, rank, rate, test, validate and/or verify.
And at the top of the hierarchy is a level of creating which requires the learner to use information, concepts, principles, and/or ideas to generate something new. Learning outcomes at this level use measurable verbs like build, composed, construct, design, develop, devise, enhance, formulate and/or generate.
|
| 00:14:27-00:15:01 |
Slide 11
Additional Features of Effective Unit/Assignment-Level SLOs
-Contitions -the circumstances under which the skill is to be demonstrated
-Standard (a.k.a. benchmark)-the minimum level of mastery the learner is expected to achieve.
(Gif of girl giving thumbs up. Text overlay says "Bench-marks the spot")
|
There are two additional features that you will often but not always see on your new unit or assignment level SLOs and they include conditions. That is the circumstances under which the skill is to be demonstrated. Examples might include without the aid of notes or without prompting, and standard or benchmark is often incorporated into the SLO, and that is the minimum level of mastery that the learner is expected to achieve. |
| 00:15:01-00:15:43 |
Slide 12: Pyramid illustration
FAQ 8: Can you share some examples of effective SLO statements?
|
FAQ #8 provides an opportunity for me to put together many of the ideas that I've shared with you thus far in this presentation. It asks, “Can you share some examples of effective SLO statements?” And the answer to the question is, “As a matter of fact, I can.” The first example is a multi level example, and it's designed to show that the same concept can be taught at multiple levels of complexity. The concept “Erickson's theory of psychosocial development” comes from my field of developmental psychology. And as an instructor, if I were to teach this concept at the remembering level, the SLO might look something like this. |
| 00:15:43-00:17:09 |
(Slide 12 continued)
Example #1 (Multi-level) Upon the successful completion of PSYC 2010, students will be able to:
-Remembering level-Describe the eight stages of Erikson's theory of Psychosocial development.
-Understanding level-explain the developmental task in each stage of Erikson's theory and how it affects human behavior.
-Applying level- Use Erikson's theory to explain behavior patterns presented in case studies.
-Analyzing level-Compare Erikson's theory of Psychosocial development with Piaget's theory of cognitive development
Evaluating level-Critique Erikson's theory based on evidence from social science research
-Creating level-Generate an original theory of Psychosocial development that is informed by criticism of Erikson's model.
|
Upon successful completion of Psych 210, students will be able to describe the eight stages of Erickson's theory of psychosocial development.
If I were to teach the same content at the understanding level, the SLO might look like this: explain the developmental task in each stage of Erickson's theory and how it affects human behavior.
At the applying level, it might look like this: Use Erickson's theory to explain behavior patterns presented in case studies. At the analyzing level: Compare Erickson's theory of psychosocial development with Piaget’s theory of cognitive development.
At the Evaluating level: Critique Erickson's theory based on evidence from social science research.
And finally, at the creating level, generate an original theory of psychosocial development that is informed by criticism of Erickson's model.
Now, just to clarify, this exercise is for demonstration purposes. It's not to suggest that you teach the same content at multiple different levels of complexity, or that you develop multi-level SLOs for the same concept. It's merely intended to show the bottom line, and that is that Bloom's taxonomy is a classification scheme for different levels of human cognition, not a system for class defying course content.
|
| 00:17:10-00:17:33 |
Slide 13
Additional examples of effective program or course-level SLOs
(illustration of graduates throwing caps in air)
Upon the successful completion of PSYCH 2010:
-(Applying) Students will be able to use the concepts and principles of key developmental theories to evaluate elementary school curricula
|
Some additional examples of effective program or course level student learning outcomes. Returning again to Psych 2010. Upon the successful completion of the course, students will be able to use the concepts and principles of key developmental theories to evaluate elementary school curriculum. That's an example of an applying level student learning outcome. |
| 00:17:35-00:17:45 |
(Slide 13 continued)
Upon the successful completion of Biology 1010:
-(Understanding) Students will be able to explain how the process of aerobic respiration maintains human life.
|
One from Biology1010 would read “Perhaps like this, students will be able to explain how the process of aerobic respiration maintains human life” |
| 00:17:46-00:17:59 |
(Slide 13 continued)
Upon the successful completion of Philosophy 2010:
-(Evaluating) Students will be able to critique an argument using principles of logic.
|
and in Philosophy 2010, upon successful completion of the course, students will be able to critique an argument using principles of logic. |
| 00:18:00-00:19:14 |
Slide 14
Examples of efective unit/assignment-level SLOs
Upon the successful completion of the unit on Cognitive Development:
- (Understanding) Students will be able to explain three key features of Piaget's Concrete Operational Stage of Cognitive Development with at least 70% accuracy.
Upon successful completion on the unit on Cellular Respiration:
-(Remembering/Understanding) Students will be able to label the eight steps of the Krebs cycle and explain that their functions with at least 70% accuracy.
Upon the successful completion of the unit on Logic:
-(Applying/Evaluating) Students will be able to effectively use the 10 Common Logical Fallacies to identify fallacious arguments with at least 70% accuracy.
|
Examples of effective unit or assignment level SLOs might look something like this. Returning to Psychology 2010, upon the successful completion of the unit on Cognitive Development, students will be able to explain three key features of Piaget's concrete operational stage of cognitive development, with at least 70% accuracy. That would be an understanding level SLO upon the successful completion of the unit on cellular respiration and Biology 1010 students will be able to label the eight steps of the Krebs Cycle and explain their functions with at least 70% accuracy. That would be a remembering and possibly understanding level SLOand in Philosophy 2010, upon the successful completion of the unit on Logic, students will be able to effectively use the ten common logical fallacies to identify fallacious arguments with at least 70% accuracy that would be in applying or evaluating level SLO.
Can you see how these are more accurate, more measurable, they incorporate conditions, and they incorporate benchmarks which you typically find in an assignment or unit-level SLOs.
|
| 00:19:13-00:20:42 |
Slide 15
Summary
-The failure of the Open Education movement illustrates the purpose and reinforces the importance of high quality SLO assessment.
-The purpose of SLO assessment is to improve teaching and learning, not evaluate faculty.
-There are seven generally accepted steps to SLO assessment. This presentation covered Step 1 Developing SLO statements.
-SLOs are the first step in backward design.
-SLOs are concise statements of what students will be able to do upon the successful completion of a learning activity, which could be a program, course, unit, or assignment.
-SLOs differ from goals and objectives in three ways: 1) specificity, 2) measurability, and 3) student centeredness.
-SLOs typically contain an action verb aligned withBloom's taxonomy of learning.
-Essential features of a good SLO statement include specificity, measurability, attainability, revelance and timeliness. Also, unit/assignment level SLOs often (but not always include) conditions and benchmarks.
|
So let me conclude by summarizing the main points of this presentation. First of all, the failure of the open education movement illustrates the purpose and reinforces the importance of high quality student learning outcome assessment. The purpose of SLO assessment is to improve teaching and learning, not evaluate faculty.
There are seven generally accepted steps to SLO assessment. This presentation covered only step one-developing SLO statements. SLOs are the first step in backward design, an approach to curriculum development that is highly beneficial to our students. SLOs are concise statements of what students will be able to do upon the successful completion of a learning activity, and that could be a program, course, unit or assignment.
SLOs differ from goals and objectives in three ways. Specificity, measurability and student centeredness. SLOs typically contain an action verb aligned with Bloom's taxonomy of learning. An essential features of a good SLO include specificity, measurability, attainability, relevance and timeliness. In addition, unit and assignment level SLOs often but not always include conditions and benchmarks.
|
| 00:20:42- 00:20:53 |
Slide 16
Thanks for your time!
(Gif of man signing 'thank you')
Please check out Fundamentals of SLO Assessment Parts II-V for additional information.
If you have further questions, please contact Joe Cullen at jcullen@commnet.edu or 860-612-7018.
|
Thank you for your time. Please check out Fundamentals of SLO Assessment parts two through five for additional information. My contact information is on your screen. Please contact me if you have further questions. |
