How to Gather Evidence of Positive Outcomes

I don't know about you, but there are times when I feel like I haven't made a bit of difference in my work.  As a leader, how do you gather evidence of outcomes?  I think by default, many of us look only for end results.  We want to see evidence of 100% buy-in of an idea or program.  We look for big shifts in thinking around a topic we have been advocating for.  When we don't see those obvious outcomes, we decide that our work didn't have an impact.

 A few weekends ago I had the chance to work with educators from across the country who are part of the Collaborative for Student Success Teacher Champion Fellowship.  Part of our work that weekend focused on gathering evidence of outcomes from our work.  We worked with a great team from Bellwether Education who helped us recognize 4 levels of outcomes connected to our work.  As I listened to their presentation I had one of those light bulb moments.  Evidence of impact has been all around me, I just wasn't paying attention to it!  Here are some ideas to help you be more mindful of the evidence of all levels of outcomes of your work - with parents, teachers or education leaders.  After looking at the outcome levels below, how might you think about evidence of your impact on student learning through the same 4 levels of outcomes?  How might you structure your formative assessment and instruction to allow you to collect evidence at each level?

Outcome Level 1 - Raised Awareness
Evidence of impact at this level might include sign-in sheets from presentations, push-back if your outcome might include making a shift in thinking or culture, interest in future discussions or collaboration, use of new terminology.

Outcome Level 2 - Learning 
Evidence of impact at this level might include emails asking for more information or posing follow-up questions, Professional Learning Communities or teacher teams discussing the shift or topic, PTA information nights, blog posts or twitter chats that include the shift or topic, requests for additional resources or examples.

Outcome Level 3 - Changing Behaviors/Attitudes
Evidence of impact at this level might include proposed changes in policy at the building, district or state level,  pilot projects, less push-back, shift in funding or budget to support additional professional development around the shift or topic, or purchase/design of new instructional materials.

Outcome Level 4 - Results
Evidence of impact at this level might include enacted policy changes, implementation of revised curriculum or instructional materials, regular use of terms in professional conversations, widespread acceptance of the shift or topic, shared ownership of the shift or topic.

How to Bring Instructional Shifts Into Practice - Focus on Complex Text

 .

There are 6 key instructional shifts that are at the center of Ohio's Learning Standards for literacy and math.   Do you know them? Could you identify them if you saw them in action in classrooms?  What do students sound like, write like, reason like if they are learning standards through these shifts?

As you read through my reflection on this shift, think about your own classroom, building or district.  What does this shift look like and sound like?  What actionable step might you take that might have a direct impact on student learning? How will you share what you have done with your colleagues?

Literacy Shift 1 : Regular Practice With Complex Text And Its Academic Language.



Key resource: Understanding Text Complexity (achievethecore.org)

Complexity is determined holistically by looking at these three features of the text:

• Quantitative Measure 
• What is the academic & content vocabulary demand of the text? 
• Qualitative Analysis 
• What is the structure of the text?
• What is the language demand?
• What prior knowledge of content or culture does the text demand? 
• Matching Reader and Task 
• How is the text to be used by the reader? 
• What is the purpose for reading/listening to the text?

There are a number of tools that can help you to look at the Quantitative Measure of the texts you are using in your classroom. Teachers who are mindful of vocabulary demand will Close Read texts prior to assigning them to students to look for words that may need to be pre-taught, words that may be used uniquely in the context, or words that are keys to a student's ability to unlock the content of the text.  Students might use Frayor Models to help construct meaning for key words.  Interactive word walls in the classroom may contain examples of words being used in context, and images that illustrate meaning or usage. 

• Academic Word Finder - Achievethecore.org    This tool can be used as part of the Close Reading process the teacher utilizes prior to assigning passages to a student.  Look for passages that have a balance of words at, below and above grade level.  Passages with many words above or below grade level may still be appropriate to use with students depending on the purpose for reading the passage and the Qualitative Features of the text. 
• Create a free account on achievethecore.org  to use this tool
• Cut and paste text or type text into the Word Finder.
• Select a target grade and run the tool
• The Word Finder tool will highlight in colors words that are below, at, and above grade level within the text passage. Listed below the passage will be possible definitions of the word. The complete passage is visible with words highlighted in context.
• Lexile Analyzer - Lexile.com  Lexile is one way to look at the Quantitative Measure of a text. Approximate Lexile ranges for each grade level have been included in the literacy standards.   Lexile can be compared to other quantitative measures like AR scores.  
• You can cut and paste text into the Lexile Analyzer, but it needs to have all formatting removed.  
• WordSift.org  WordSift looks more closely at academic vocabulary and content vocabulary. 
• Cut and paste or type text into the tool
• A word cloud will be created, showing the highest frequency words. This is a good way to identify words that may be key to unlocking the content of the text.
• The word lists tool will highlight in colors words that are specific to science, math, social studies and ELA.
• A set of related images will appear for each word that is clicked on in the word cloud. You can use these images to add to your word wall or make visual dictionaries for ELL students or at risk readers.
• Paired Texts by Lexile Range from Readworks.org can be found HERE

Qualitative features of a text can be looked at using a rubric or a checklist.  There are 4 areas to consider.  First, is the text structure simple or more complicated? Remember that text can also be a graphic, so look at the graphic features as well.  Are there text structures that are normally found in a particular content area writing style or in a genre?  When skimming the text on a first Close Reading, is the language more conversational or formal? Are terms contemporary or more unfamiliar? Teachers being mindful of the Qualitative features of the text will also take into consideration the knowledge that a text expects a reader to bring with them into the reading.  This can be cultural or regional experiences, content specific background or individual life experiences.  This particular aspect of text complexity requires the teacher to think carefully about how to scaffold texts for students who may be lacking some or all of the background knowledge a more complex text might require.  Ideally, the text is the expert and students will not need to bring large amounts of prior knowledge into their reading and discussion of the text. 

• Qualitative Measures Rubrics for Informational and Literary Text can be found HERE
• Book excerpt on Qualitative Measures from ASCD 
• Determining Text Structures  Resource List from the West Virginia Dept of Education

Matching reader to task is often overlooked as the third component of complexity.  A text may be moderately or slightly complex, but be a primary source document that is important to helping a student understand the context of an historical event.  On the other hand, a text that is exceedingly complex may be a scientific paper a student is reading to get background information for a project.  All children should be given the opportunity to read a range of complex texts throughout the year. Texts should be high quality, be worth the instructional time to read them, and help students to build knowledge and vocabulary.   One strategy a teacher might consider when selecting informational and literary texts to use in a classroom would be to build expert text sets.  Students build content vocabulary and knowledge when they have an opportunity to read, listen too, or analyze multiple texts on the same topics.  

• Expert Packs grade K-12  achievethecore.org
• NEWSELA Text Sets
• Creating Classroom Text Sets  from Louisiana Believes  
• Guidance and Resources for Text Selection from EngageNY
• Resources for literacy and ELL from Colorin Colorado

Hope Centered Leadership

We all have that one child, that one teacher, that one book that we can say made a difference in  how we perceived ourselves or see the world.  The challenge as a hope centered leader is to do what is necessary to be that one teacher who can change the world for someone else. 

Guiding Questions:

• What book that had an impact on your own views of education can you share with a colleague? What will you do to follow-up with them to talk about it together?
• What is your passion as a teacher? Who might share that passion? How will you connect with that educator? What can you accomplish by connecting and working together?
• How would your instructional plan change if you thought of your class one child at at time? What is the just right next step for learning for your children?  What can you do to help them see their path to change the world? 
• What can you do to share your voice with other educators to encourage them to let their light shine through?

To be entrusted with the children of a community is a perhaps the highest compliment that can be given to an educator. Making instructional choices that will help children build the skills and knowledge that they will need to carry the community into the future is an essential part of all educator's jobs.   The challenge as a hope centered leader is to create a culture within a school or district that is based on collaboration and respect and encourages all involved to act on their hopes and not make decisions based on fear.

Guiding Questions:

• What is one thing you can do this week to strengthen your professional learning community?
• What evidence of learning will you look for to make decisions about how effective an instructional practice has been and what actions to take to continue moving learning forward?
• How are the community's goals reflected in the instructional decisions made in the schools? 
• What small step can you take to continue to build a collaborative environment?

Words can be barriers to learning.  The challenge we face as educators is to select the right words that act as doors, bridges and paths to learning.  Carol Dweck's work on growth mindset, Carol Ann Tomlinson's focus on differentiated instruction, and the work of the Stephen and Jan Chappuis, Susan Brookhart and Rick Stiggins around effective feedback and assessment for learning all point to the positive impact the right words can have on a student's ability to take charge of their own learning.  The challenge as a hope centered leader is to search out and amplify voices that are providing effective, specific feedback to students, and creating learning spaces where all students feel safe to explore their thinking. 

Guiding Questions:

• What do classrooms centered on formative instructional practices look like and sound like?
• What next step will you take to reflect on the research being done by Dweck, Tomlinson, Brookhart, Stiggins or the Chappuis?
• What words will be at the center of the instructional mission of your school or classroom? How will you communicate those words to all stakeholders? 
• What strategies do students need to develop in order to find and select high quality instructional materials that can help them answer questions or explore ideas? 

High expectations for everyone. Holding the bar high. Cliches that are easy to say but difficult to implement.  A hope centered leader needs to be a good listener and a good observer. Identify what supports, resources, encouragement, materials and time students and teachers need to reach higher. Then, be the ladder.  The foundation of hope is in the ability to see not only that there is a step forward, but that you have the ability to take the step.  The challenge as a hope centered leader is to do what is necessary to support a learning culture where hope is at the core of all decision making. 

Moving Learning Forward One Step At A Time

 If you are looking for my testing resource page - you can find it here:
 http://iteachbay.blogspot.com/p/onls-ngas-2014-15.html
"Put one foot in front of the other and soon you'll be walking cross the floor. Put one foot in front of the other and soon you'll be walking out the door."  This song refrain from one of my favorite holiday shows, Santa Claus Is Coming To Town, is appropriate for this start of the school year.

It is easy to get caught up in the procedural side of education and lose sight of what has to happen in a classroom to move learning forward.  What are the steps you can take to help your students grow their learning or grow your own learning? Pick one step to start!

Clean Out Your Assessment Closet

What evidence of learning does this assessment collect for me or for my students?

How does this assessment match to the standards I am teaching

Where does this assessment fall in the continuum of assessment in my classroom - from formative to summative?

What can I do to revise this assessment to build in more stretch?

Resources for Assessment Literacy:

• Achievethecore.org Assessment Evaluation Tool
• Annotated Mini-Assessments For Math k-12
• Writing Text Dependent Questions
• Char's Assessment Literacy Resource Page

Use Informational Text To Help Students Build Knowledge

• What texts, including print, digital,  video, audio and graphics, are you using in your classroom?
• How are you building academic and content area vocabulary through a variety of texts?
• What opportunities are you building into your lessons to allow students to interact with the text and each other?
• Are the texts you are using appropriately complex for your students? 

Resources for Informational Text

• Achievethecore.org Instructional Practices Guide - Literacy
• Tools for Text Complexity
• Char's Close Reading/Informational Text Resource Page

Build A Professional Learning Network For You & Your Students

• What doors can you open that will allow you or your students to connect with others who share similar interests or passions for learning?
• How can you use technology to help you to collaborate on student learning or help students collaborate with each other around their learning?
• What learning networks are you already a part of? How can you continue to make the focus of those discussions around "moving learning forward"?
• What do you want to learn more about? What steps do you need to take to make that happen?

Resources for Professional Learning Networks & Student Collaboration

• Learn more about Google Apps for Education 
• Speaking and Listening Standards 
• Choosing Collaborative Student Groups
• Join a Twitter Education Chat http://cybraryman.com/chats.html
• Participate in a Teacher Leadership Team 

Welcome To Open House 2015-2016!

Char's Testing Resource Page for 2015-2016 can be found here bit.ly/OHTests1516

The start of school brings with it the annual Open House.  This is one of the few chances you may have during the school year  to communicate face to face with large numbers of your parents and family members.  

• What do you want to learn from the parents? 
• What do you want parents to learn from you?   
• What class management information can be shared through a handout or on your website to allow you more time to interact with the parents? 
• How will you reach out to parents or family members who may not be able to attend the Open House?

As we have shifted to new learning standards, more and more parents are wanting to know what they can do to help their child at home.

• What suggestions can you make on ways to read out loud with children, or have children read to parents?  
• What activities, websites or books  could they use together as a family that might connect to topics being learned in class.

Math has become a particular hot button topic as parents try to understand why their child's homework may not look like the math they did when they were in school.

• Suggest some real world math activities that can be done at home to help build a child's understanding of math.  
• Set up learning centers or stations with hands-on math activities for parents to try.  
• What math games  might you recommend they play at home?  
• For older students, consider having parents work through a short math task together, or provide them with some real world scenarios where the math skills the children will be learning can be applied.

If you teach science, social studies, technology, the arts, physical education, world languages or family consumer science, think about how to give parents some first hand experience doing the kinds of learning activities their students will be doing throughout the year.

• How might you help parents to see the connection between what their students will be learning in class and what they may be doing at home or in the community?
• How could you share examples of work students have done in past years to give parents an idea of what typical work looks like for a student in the grade and class?

Testing is another topic parents may have questions about.  Think through how you can help parents understand the role of formative assessments in your planning of day to day instruction to help their students move their learning forward.

• What examples might you use to demonstrate the difference between frequent formative assessment and the more long term summative assessments?  
• Where does state testing fit into the picture? 
• One way to explain state testing might be to make a comparison to a check-up at the doctor's office. The doctor will usually show the parent a graph that places  a child on a line for height and weight, putting them in the context of other children their age. State Tests do the same thing, only using where a child is in their learning of a set of knowledge and skills.  These check-up results can be used to make a long term plan for learning, provide information on whether a child is on track, and help teachers see where a child's strengths and weaknesses are in comparison to a larger group.

Finally, parents are looking to teachers to provide information on state testing.  Think about how you can share what we know right now about state testing with parents.

• Ohio State Tests will be in ELA, math, science and social studies. 
• There will only be 1 test for each subject area.  
• The tests have been shortened. Each child will have up to 3 hours to complete each test, but most will be able to finish in less time. 
• Districts can choose to do the test in one sitting or split it into two 90 minute sittings.  
• The tests will be aligned to our Ohio Learning Standards. 
• There will be one testing window that runs from April through the beginning of May. 
•  Ohio teachers will be helping to select the items that will be on the tests.  
• We will be using the same computer platform that the science and social studies tests were given on last year, so students and teachers have had experience using it.  
• Paper and pencil tests will be available to districts who may choose to use them for all or part of their testing.

Here are some great resource sites to direct parents to:

• http://bealearninghero.org/  Be A Learning Hero is sponsored by the PTA and a number of other national groups that support public education. It has helpful hints for what parents can do at home to help support their student's learning. It includes a Skill Builder Resource that can be used to help understand state test scores and what do with them.
• http://www.ohiopta.org/ This is the homepage of the Ohio PTA.  It includes parent friendly info on Ohio's Learning Standards.
• http://education.ohio.gov/Parents Ohio Department of Education Parent Portal page.
• Ohio's State Test Dates for 2015-2016 
• Roadmap For How Ohio's State Tests in ELA and math are being developed
• https://public.govdelivery.com/accounts/OHED/subscriber/new Parents can subscribe to Ohio Dept. of Education email updates on testing and many other topics related to education in Ohio.
• http://www.readingrockets.org/audience/parents Reading Rockets is a website designed to provide parents with resources and strategies to support literacy at home.
• http://www.colorincolorado.org/families/ Colorin Colorado is available in English and in Spanish and provides resources for parents to support literacy and English language skills at home.

What Has Changed & What Stays The Same 2015-2016

Char's Testing Resource Page for 2015-2016 can be found here bit.ly/OHTests1516

"Be the change." 
There is an undercurrent of change that runs through education and carries along with it the constant need to adjust and update to new classroom conditions, new standards, new assessments, new teaching strategies, new students and new colleagues.

This summer in Ohio brought about a number of changes to the plan for Ohio State Tests.  These changes were based on the input of teachers, parents and community members who wanted a shorter test aligned to Ohio Standards that could be given on one testing platform closer to the end of the school year.   Ohio teachers will be working side by side with the Ohio Department of Education and AIR to build Ohio State Tests for ELA and math.  These new tests will join our existing Ohio State Tests for science and social studies and will be given this year.   The good news is that Ohio teachers and students already have taken tests on the AIR technology platform.  Districts who have been working to add devices and build infrastructure will be able to still give online tests. Paper and Pencil versions of all the tests will also continue to be available.

In addition to the change in the testing platform, new Safe Harbor provisions that put a hold on the use of Ohio State Test results in teacher evaluation,  will allow teachers the opportunity to focus on instruction and on revising classroom assessments to better gather evidence of student learning.  Achievethecore.org has excellent resources for teachers to use to continue to change their instructional practice to meet the depth of our standards and develop assessments to match to these standards.

• Aligned Blog - Achievethecore.org
• Tools to help teachers continue to develop standards aligned classroom materials
• Tips and strategies from teachers who are working with Ohio standards
• Assessment Quality Criteria Checklist - Achievethecore.org
• Use this to develop or revise your own assessments aligned to standards

There are a number of key pieces in Ohio education that are staying the same.  Our Ohio Learning Standards have been in place since 2010 and Ohio teachers continue to locally develop lessons around these more challenging standards.  Instruction that meets the needs of all students will always be the main priority in Ohio classrooms. Good Formative Instruction and Assessment are essential to planning instruction that helps all students succeed. Teachers have been using formative assessment to make changes in instruction to immediately meet the needs of their students.   Parents are partners in the education of their children. The Ohio PTA is working to continue to support parents as districts move forward with the new Ohio State Tests.

All of the work Ohio teachers and administrators did last year to make the shift to the new, computer based, standards aligned tests has helped to build a solid base for the 2015-2016 tests.   This year, we can keep our focus on instruction.

Three Instructional Shifts for Assessments

Making the shift to our new learning standards includes shifting the way we think about assessment too.  Assessment is a part of instruction. Start with an honest conversation with yourself about the role of assessments in your classroom.  How have you remodeled your assessments to match the depth of knowledge expected in the standards? How do you use assessments as tools to collect evidence of student learning? How do your students view your assessments?  What is your vision for assessment? How do you use a balance of Formative and Summative assessments?  These same questions can be asked at your team meetings, staff meetings or district leadership meetings.


Three Instructional Shifts - Assessment   

• Depth of Knowledge [DOK]   Look closely at the new standards and you will find skills like Analyze, Describe, Explain, Infer, Define, Collaborate, Compare, Measure, Evaluate.  Then, look at your assessments, and the lessons you have developed along with those assessments.
  
  What opportunities have you given your students to develop these skills in the context of your content? How have you used formative assessment to give effective feedback throughout their learning to help them refine these skills? How do your assessment items match to the DOK level of the standards?  Try this reflection activity. Sit down with one of your assessment tools and the Depth of Knowledge matrix. Choose your favorite color of ink or font - I like purple.  In the margins, or as comments/revision notes, mark the DOK level that goes with each assessment item. Do these DOK levels match with the standard? If not, how can you revise the assessment item or task?  
• Using Webb's Depth of Knowledge to Increase Rigor - Edutopia Blog by Gerald August Sept. 4, 2014
• Depth of Knowledge Overview Chart w/evidence - Oregon Dept of Education
• Depth of Knowledge Question Stems - Spencer Valley School District
• Karen Hess's Learning Progressions Frameworks Designed for use with CCSS ELA/Literacy gr k-12
• Karen Hess's Learning Progressions Frameworks Designed for use with CCSS Math gr k-12
• Karen Hess's Cognitive Rigor Matrix and Curricular Examples - ELA/Literacy
• Karen Hess's Cognitive Rigor Matrix and Curricular Examples - Math/STEM

• Evidence Assessments are tools to collect evidence of learning. What evidence do your assessments give you and your students? Instead of teaching students strategies for answering specific item types, shift your thinking to selecting specific item types that will help students demonstrate where they are in their learning.
  
    Start with the types of items you choose.  Make the shift from traditional multiple choice questions to two part multiple choice items that allow students to not only answer, but give evidence for why they chose the answer.  Performance tasks let students apply what they know in a real world context and let you assess multiple standards in one task. Try creating items that ask students to find multiple correct answers. This eliminates guessing and lets you see evidence of their depth of understanding. Be intentional about distractors you include. This can give you evidence of misconceptions or misunderstandings. Using text dependent questions allows students to find evidence to support their answers from the text passages you provide. This can give you evidence of comprehension, analysis, and inference.   Beyond item types, also think about how you structure your assessments. How do you scaffold questions to help get student thinking moving in the right direction so they can produce quality evidence of their learning? How can you tier your questions using DOK levels or Blooms' Levels to go from basic knowledge to higher level thinking within one assessment so that you can get evidence of where they are in their learning? 
• Bloom’s Taxonomy Verbs, Question Stems and Activities
• PARCC practice PBA for ELA good example of tiered questions
• Kathleen Scalise -University of Oregon Taxonomy Table of Assessment Item Types - developed pre CCSS, but still very useful when looking at picking the right assessment tool for the task.

• Vision   Having a clear vision for assessment allows all stakeholders to understand the role of assessment in instruction, and what evidence of learning will be used in making instructional decisions or evaluating programs and staff.  There has been much discussion around time spent on assessment or too much assessment. Both discussions are worth having if they take place in the context of creating a shared vision of assessment.  Taken out of this context, they will lead only to disjointed assessment planning that won't benefit teachers or students.
  
   Starting at the classroom level, think about how you find a balance between Formative Assessments [for learning] and Summative Assessments [of learning]. As you plan instruction, how do you include formative and summative assessment in your lesson plans? How do you predict possible misconceptions and plan assessments to help you identify and address them?  How do you balance assessment of content knowledge and higher order thinking skills? How do your assessments mirror the time you spent on instruction? Common assessments, whether they are at the grade, district or state level have value. They serve as touchpoints, allowing teachers and students to compare the evidence of learning gathered by classroom assessments to a common standard.   An assessment vision also must include clear expectations for how evidence of learning growth will be gathered, and also how evidence of learning mastery will be collected. How this evidence will be used is the starting point for discussions about classroom grading practices and teacher evaluation.  One way to begin building an assessment vision is to work with a grade level or district team to make an assessment continuum. 

Building an Assessment Continuum

• On individual post-it notes, write the name/description of each assessment used in a class or building or district (depending on the group who is doing this)
• Arrange the post-its as a continuum from those that are most formative (assessment for learning) entrance/exit slips, thumbs up/thumbs down are examples of most formative ---  to most summative (assessment of learning) - State tests, ACTs tests are examples most summative
• Discuss as table groups or whole group what patterns can be seen in assessment?
• Discuss formative vs summative - should assessments be equally distributed along the continuum or would it be best to have more at one end or the other?
• Discuss any assessments that provide duplicate evidence - is this necessary?
• Discuss how the evidence from the assessments are used - by whom? for what purpose?

 Looking for information on Making Instructional Shifts for ELA/Literacy and Math? Find resources at http://achievethecore.org/ or on my Supporting Ohio's New Learning Standards Homepage http://iteachbay.blogspot.com/p/onls-ngas-2014-15.html  Click here to go directly to my Assessment Literacy Resources http://iteachbay.blogspot.com/p/assessment-literacy.html