How to Use A Logic Model For Evidence Based Action Planning

This blog is cross-posted from OhioTeachersForQualityEducation.org

Planning for action around learning goals or building goals can take many forms in the classroom and in your building.  Most traditional action plans include space for defining the goal, documenting what steps you are going to take to achieve it, and who will take the steps.  I have spent the past year working with a Logic Model as an evidence based approach to action planning.  What I like best is the focus not just on planning action, but collecting evidence that the action is complete along with evidence of the impact of that action.  Logic Models encourage you to have conversations around assumptions you are making about the work.  Often, these assumptions, when not discussed or addressed, lead to frustration and communication gaps.  For the past year, I have used Logic Models in planning for whole district initiatives, like our high school chromebook roll-out.  I have also used Logic Models for smaller goals, including keeping a focus on text complexity and writing good text dependent questions.  Logic Models could also be modified to be used as templates for Unit Planning or differentiating instruction for English Learners or Gifted Students.

The first step in starting a Logic Model is to identify your goal.  This might be a set of standards based learning goals that will be the center of an instructional unit.  The goal might also have a broader project or initiative focus.  Ideally, you should be collaborating with your grade level team, building leadership team or district team to frame your goal.  Next, connect that goal to the broader vision.  How does this work fit into the broader work of your classroom, building or district?  Once you have identified your goal, the next step is not to start planning action.  Instead, take time to talk through the assumptions you are making about the students or staff that are going to be impacted by the work.  What prior learning or experience are you assuming they will bring to this work? How are you going to connect this goal to their existing practices or beliefs?  The reflection on assumptions will also help you to identify possible barriers.  Identifying a true barrier that will need to be worked around, or an attitude or mindset that may need to be shifted in order for the goal to succeed, will help in thinking through possible action steps and evidence of outcomes.  Planning the action steps can be done next.  As you think through action steps, identify resources you have or will need.  Resources can be time, materials, human capital, or financial. If you don’t have access to the resources you need, include in your action steps a plan for acquiring that resource.  The real benefit of the Logic Model is in the last two steps.  Take time to identify what evidence will be gathered to show that the action has been completed.  This might be an agenda, minutes, emails, anecdotal records or student work.   More importantly, have a collaborative discussion on what evidence of impact or outcome of the work you will want to see or hear.  What will teachers or students or staff sound like, or act like if the action step in having a impact on moving them toward a goal?  

There are 4 levels of outcomes to think about when deciding on the mindful collection of evidence of impact.

Level 1: Reaction

At this level, evidence of impact might be pushback or a lot of questions on why we are doing this or learning this.  Often this evidence of impact is overlooked, or is seen in a negative light.  Really, it is evidence that the learners are having to re-examine their own thinking or beliefs and seeing how this new information or idea might fit or not fit into this.  Pay attention to the pushback comments or questions.  You may need to adjust or add an action step to help move this group forward.

Level 2: Learning

Evidence of learning might include a shift in questions from “Why are we doing/learning this” to “How do I…?” or “Maybe I could try…”  Much of this evidence will be anecdotal comments heard in a class or in a team meeting.  You might consider using a Google Form or a Reflection document to capture these comments and shifts in thinking.

Level 3: Behavior/Attitude Shifts

By the time students or staff start to show evidence of Level 3 outcomes, they are trying out the new skills, applying the new strategies in a small setting or as a pilot, or starting to grow their own learning around the goal. You might hear students expressing a more sophisticated approach to a task, or hear teachers talking about applying strategies or ideas to an upcoming unit.  

Level 4: Results

This is full buy-in.  There has been a change in attitudes or in skills that is evident across an entire class, grade level, building or district.  Students or teachers regularly demonstrate their new learning or skills in their daily work.  A common vocabulary has been developed and everyone now has incorporated the new skill, strategy or idea into their own belief system.   Evidence at this level might include shifts in district or grade level data, requests for additional “next step training”, increase in student success or the success of a subgroup.  

I have learned a lot about using Logic Models from the team at Bellwether Education Partners and the Collaborative for Student Success Teacher Champions Fellowship.  Using Logic Models has really changed the way I think about planning and following up on my own work, both as a Curriculum Director and as an educator leader.  I have now started to create 2 folders in my Google Drive to help me gather evidence of the work I have done, and evidence of the impact of my work.  Focusing on evidence of impact at all four outcome levels has really nurtured my own positive mindset about my work and the work of the teams I am a part of!

This LINK will take you to a blank logic model in Google Docs.  Feel free to make a copy for yourself by clicking on FILE ---MAKE A COPY.  

This LINK will take you to a Logic Model that has reflection questions for each component.

Using Games As A Final Formative Assessment of Learning

Carnival of Evidence – A Different Approach to End of Year Elementary Portfolios

It is a Friday near the end of the school year.  For Kindergartners in Mrs. Bragg’s room, it is Carnival Day.  The game centers are ready to go, prizes are arranged on the counter, popcorn is ready to be put into bags, and the camera is ready to capture student reactions as they enter the room.   The Carnival Day is a tradition. It is a chance to celebrate all that the students have learned, and give the students a chance to show what they know to their parents.

Mrs. Bragg feels that the time and effort that goes into planning for this day is well worth it.  “My students love it and remember it, but each year I wonder if I have the energy to do it again. And then, I hear from the students comments like, ‘There’s a lot more than I expected’, and ‘This is so fun’, and  that’s why I will do it again.”

What makes Carnival Day a unique way for students to demonstrate their learning are the games themselves.  All are standards based. All have extension questions or challenges to let students really show a deeper level of mastery.  Mrs. Bragg starts by identifying the central standards for the year in math and literacy.  Once she has mapped out the concepts, she takes a look at what game would best let students show evidence of their learning around each concept.    Seven game centers seems to be the just right number. This allows a class of 22 students to be divided into groups of 2-3. She tries to pair students with others who they will work productively with.  Parents help to facilitate the games and have a chance to watch students work with words and numbers. 

Mrs. Bragg has found that each year she has had to add increasingly more complex enrichment questions to each game.  “My students are able to show much higher levels of mastery than when I first started Carnival Day.  For example,  instead of just reading a word, my students are now reading sentences.”   In the Pick A Duck game, students use a fishing rod to catch a carnival duck from the pond. Each duck has a word written on it. Some students may read the word. As enrichment, students may be prompted to name a rhyming word, identify diagraphs or count syllables.  In the Plinko game, students drop 2 tokens into the Plinko board. They then write number sentences using the 2 numbers that the tokens land on.  Extensions include asking students how many more would they need to add to the answer to make 10, how could they right the number sentence another way, or can you change the number sentence by adding 3 to one of the numbers?  Spin to Win is a chance for students to identify numbers, count on to 20, and identify a number that would be 2 more or bigger or less than.  Face painting is a time for students to demonstrate their speaking and listening skills by talking about what they would like to have  painted on their face, and why.  Students collect tickets as they spend approximately 10 minutes at each game. Tickets can be exchanged for small prizes. The students are proud of the skills they have learned.  Parents have a chance to actually watch and listen as their students demonstrate their Kindergarten skills, rather than look through a traditional portfolio of student work.  Because each game has multiple entry points, all students are able to confidently demonstrate their mastery level, in an environment that is celebrating everyone’s learning.

 

The Carnival Day is a unique way to think about a summative assessment of student learning and how to communicate learning to parents.  The concept could certainly be applied to other grade level classrooms.  Carnival games could be replaced with game show style games or a set of problem solving challenges.  Not all parents may be able to attend.  Sending home directions for how to do at home games that would allow students to demonstrate the same skills, along with facilitating questions and a set of parent friendly mastery level descriptors could accomplish the same goal.  

Lessons I Learned From Participating in Ohio Standards Revision Process

I co-authored this blog with Tricia Ebner, the co-chair of the ELA Advisory Group. 

Over the past 18 months, Ohio has been involved in a cycle of standards reviews. Per state law, Ohio’s standards must undergo a revision process every five years. Teams of Ohio teachers, administrators, college professors and content experts have volunteered their time to do this work. In 2016, the math and ELA standards went through this process. During the 2016-17 school year, the science and social studies standards have experienced this same process.  This five year review and revision cycle enables educators and stakeholders to reflect and consider how well the standards are working and what improvements might be necessary.

The process is thoughtful and thorough. The review starts with a period of public comment, where teachers, parents, administrators, college and university faculty, and community members can provide comments, recommend changes, and point to research supporting those comments and changes. Then a revision advisory committee made up of teacher leaders and content experts examines each and every comment, with a goal of coming to consensus on the proposed change.  If the consensus is that the comment is relevant and will potentially clarify or strengthen the standards, it is passed along to the standards operational working group.  This second team of teachers, professors and content experts then work to make the revisions if they agree they are necessary. These revisions are then sent to the public for a second round of feedback, followed by the advisory committee reviewing those comments and sending any standards still needing work back to the working group.

As members of the advisory committee for ELA and the operational working group for science, we have been involved with standards review for the past two years. We’ve gained some insights into Ohio’s standards:

• The vertical progression is key. As educators, it is critical that we know and understand the vertical progression within the standards. In the work with the ELA standards review, it didn’t take long to see that a change made in sixth grade, for example, would have a ripple effect running towards both kindergarten and grade 12. One important strand in the ELA progression is writing opinions/arguments. Standards help to frame the increasing sophistication of students use of evidence to support their argument. In science, this progression helps to map out how students build an understanding of a concept, like force and motion, starting with simple pushing and pulling in kindergarten and going all the way up to calculating force in physics. As a teacher, It is important to understand the foundation students have as they walk into your classroom.  It’s also important to understand that if that foundation is shaky, intervention needs to happen with an eye toward the requirements of the standard in previous grades. Additionally, knowing the vertical progression  of the knowledge and skills students will be working can help educators make decisions regarding students who have already mastered standards at a particular grade level. In this instance, a teacher can make a decision as to whether to broaden the student’s experiences with the skills in that standard, or accelerate the student into the next grade level’s work on that particular standard.

• Knowing the vocabulary is also important. As we worked through the standards review process, it became clear that some terms used within the language arts standards, for example, needed a glossary, so that all educators in Ohio can work from the same definition in addressing those standards. As we prepare to transition into the revised standards, it is important to pay attention to the glossary to ensure each standard is clearly understood. These are the definitions the model curriculum writing teams are using in their work, and because the test blueprints will be developed based on these standards, the assessments will address these terms as defined in the glossary. In math and science, content specific vocabulary was also carefully looked at to be sure that correct terms were used consistently throughout the standards.  In science, the operational working group had many discussions over exactly the right word to use within each standard being reviewed. Many laundry lists of terms were replaced with a focus on a few key terms, keeping the standards language based in the building knowledge of science concepts and skills, not just memorizing lists or tables.  Beyond vocabulary for students, essential vocabulary was also clearly defined or explained as a support for the teachers who will be working with the standards.

• Standards build from grade level to grade level, and they also work in conjunction with other standards at the same grade level.  Part of the work of standards review and revision is to be sure that the standards articulate across grade levels and within grade levels in a way that will make sense to teachers and to students. While we as educators need to break the standards down to understand their component parts, that is not the way we should be teaching our students on a daily basis. The standards aren’t meant to be taught as separate, isolated skills and concepts. While we may need to focus students’ attention on one aspect of a standard to deepen their mastery, it is also critical that we have them then work with the standard as a whole.  One way to look at the Ohio Learning Standards is to think of them as the story of the learning that we would like students to master at each grade level.  Within each story, there are a number of strands. In ELA, these include literature, informational text, writing, foundational reading, language and speaking and listening.  The science standard story begins with the nature of science statements, and weaves in Earth/space, physical and life sciences.  Just like any good story, the standards have connections to each other.  Look closely at the literature and informational text standards for reading, and you will see the writing standards reflected in the wording.  Spend time with the physical science standards and you will see that they can be taught through the lens of life science.  Going even further, it is also possible to teach many of the language arts skills through the context of the science concepts!

• The standards are the floor, not the ceiling, of what students can and should be doing in Ohio classrooms. The standards don’t limit us to only the skills embodied within them. We can stretch beyond those standards. For example, I’ve heard concerns expressed that letter-writing is not specifically named in Ohio’s ELA standards. There is nothing preventing a teacher from addressing letter-writing skills in his or her classroom.  One creative teacher had students write letters to an author, another had students write letters to a story character, from another character.  In science, the working group worked hard to write standard language that would encourage teachers to let students explore the world around them, use authentic data, and find real world situations to build their understanding of science skills. This allows teachers to find science in their local community or their school yard and set students up to become lifelong scientists.  The science working group spent time revising the nature of science descriptions for grade bands k-2, 3-5, 6-8 and 9-12 to be sure teachers would have the flexibility to let students be actively involved in doing science.  

Perhaps the greatest take-away we have had from the work of directly helping to review and revise Ohio’s Learning Standards is the power of teachers from various grade levels and backgrounds working together to really unpack standard language together.  If time could be spent in teacher based teams, grade level teams, professional learning communities having the same kind of focused dialogue, teachers at all grade levels would grow in their own understanding of the the standards, and begin to share best practices for how to help students to master these standards.

Reflecting On Your Assessments - 4 Steps To An Honest Conversation

*This blog is cross posted from https://ohioteachersforqualityeducation.org/

When was the last time you had an honest conversation with yourself about your class assessments for reading and writing? Before you hand out your last test of the school year, take a close look at the passages you have chosen.  Are they worth reading? How about the questions? Are they worth answering? What verbs are at the center of the test?  Here is a 4 step process for reflecting on your assessments.

Step 1: Check That It Is A Text Worth Reading

Read the passage(s) you selected as the base for the assessment. Are there sticking points within a passage that will introduce new information, challenge the students’ beliefs or cause them to dig into the text more deeply? You want students to use the text as the expert they can refer to as they build their argument, write their narrative or gather information. For multiple text passages, ask yourself if the content will support questions that can go deeper than asking students to find the similarity between the texts.  Students need to be able to analyze different points of view or synthesize information to support a claim. It is hard to do this if the texts have only a shallow connection to each other or are simplistic or lack text for students to really struggle with.   Read the text again.  Do you see something new you didn’t consider the first time through? What words are key to understanding the passage or supporting the claim? How would your use questions to point students to those words or key information?  Are you able to think of more than one writing task that could grow from the text(s)? That is an indicator of a rich text worth working with.   

Resources for Identifying Texts Worth Reading

• Assessment Quality Checklist  (Achievethecore.org)
• Toolkit for Text Complexity (Achievethecore.org)
• Examples of Passages Worth Reading On Assessments  (Achievethecore.org)

Step 2: Do A Question Audit

Above all else, do the questions all relate to the central idea or important details of the text and will they provide you with evidence of student mastery of a standard or set of standards? A well designed test has questions that help the students to build their thinking toward a final writing task. In order to do this effectively, the questions should be text dependent. This means that students should be able to go back to the passage(s) to build an answer.  Questions about each passage should help students focus on what is important in the text. Researchers have shown that items that assess peripheral details actually encourage students to do superficial “skim” reading to quickly find answers for filling in blanks rather than reading deeply to find evidence to support ideas.  Highlight the verbs in your questions. Are you asking students to analyze, integrate, determine, and evaluate or are you asking them to list, define, identify or pick from a list?  Take the time to revise your questions. Questions should be a scaffold that leads to a final task, not a series of gotchas and traps to see if a student has memorized information or can remember isolated details from a text.

Resources for Identifying Questions Worth Asking

• Checklist for Identifying Question Quality (Achievethecore.org)
• Questions Standards Alignment Modules (examples & non-examples) (Achievethecore.org)
• Texts with high quality text dependent questions (Achievethecore.org)

Step 3: Questions Should Expect Students To Use Evidence From The Text

An assessment worth giving expects students to demonstrate a command of evidence. How does your assessment consistently require students to use textual evidence  to analyze, evaluate, and make inferences around the use of text structures, author point of view, the central idea or important details? Students who have a command of evidence are adept at selecting details from a text to support their ideas and arguments or write a cohesive informational piece. Ohio’s Learning Standards for ELA/Literacy are meant to help students be college or career ready. The ability to make claims that are developed with appropriate evidence is a college career ready skill.  Not all evidence based questions need to be short answer. It is possible to write evidence based selected response questions. These can be two part multiple choice questions or questions that ask students to select or highlight multiple quotes taken from a passage to support an answer choice.  As you read through your passage and questions, are they written and ordered in a way that supports the use of text based evidence?

Resource That Shows The Use of Evidence

• Examples of Evidence Based Selected Response Questions (Achievethecore.org)
• Text Dependent Question toolkit (Illinois State Univ. College of Education)
• Evidence Based Selected Response Tutorial (ODE OhioStateTests.org)

Step 4: Check Alignment To The Standards

It is worth the time to make a blueprint for any summative test or final exam you are going to be using with your class.  A blueprint is a map of the questions that includes for each item on the test, the standard(s) it is meant to collect evidence of learning on, the type of question, the depth of knowledge required by the standard, and how the question connects to the major focus of the unit or class. Just a reminder, all text dependent questions should allow student to use evidence from the text and should be based on appropriately complex text.  Have you allowed for multiple entry points into your test for students who are at varying levels of mastery of the standards?  You can also look at how your are going to provide feedback on this assessment to students. Will you be using points? Partial Points? A rubric? Will there be opportunities for the students to correct or revise answers to receive full credit? Do not rely on the publisher of your book series or program to create standards based questions. Take the time to review those questions too.  Ohio’s State Tests are matched closely to Ohio’s Learning Standards.  There are blueprints for the tests as well as specific test spec documents.  You can also look at the answer documents for the released test items to see the connection to standards, and rationale for the distractor items.

Resources for Checking Alignment To Standards

• Blueprints for Ohio State Tests (ODE OhioStateTests.org)
• Performance Level Descriptors (ODE OhioStateTests.org) scroll down!
• Scoring Guides for Released Test Items (ODE OhioStateTests.org)
• Creating Your Own Test Blueprint (Holly Lavender and Catherine Shaw -ODE)
• Ohio Learning Standards - ELA (ODE education.ohio.gov)

*This blog is cross posted from https://ohioteachersforqualityeducation.org/