Atlanta K12 Design Challenge (@AK12DC) Summit 2015 is underway for day 2. Started early on a Saturday morning with eleven design teams and 60+ educators ready to build on Day 1. The schedule:
We started the day reflecting on the work from day 1. A number of teams shared their learnings from day 1.
- The day provided productive time to work on their prototype.
- They enjoyed the design thinking refresher on “designing the ideal chair,” especially thinking about it from the perspective of different users (Simpson characters)
- What they learned is that different types of chairs were designed depending upon the user they were assigned.
- They felt positive about the storytelling exercise and thinking about their design challenge as a story that has unfolded.
We then went into a session on defining what it means to build a design thinking mindset. What are the elements (see image below) that go into a team or an individual building a mindset that promotes a culture of design thinking?
- Focus on Humans
- Be Obnoxiously Curious
- Be Mindful of Process
- Embrace Experimentation
- Iterate Everything
- Words Matter
- Show Don’t Tell
- Inspire and be inspired
- First and Teach to Fish
The teams were asked to apply their understanding of the mindset elements to a series of hypothetical activities.
The teams ran through a series of three activities. After each activity, we processed what was learned in the conversation. Scott Sanchez (@jscottsanchez), our design facilitator, engaged all teams in the sharing time. One team member from Westminster Schools, Peyten Williams, indicated that the nine elements for a design thinking mindset are an instructive tool to think about how to be a good teacher.
The large group conversation on developing a design thinking mindset was a lively and instructive time for everyone. (see Storify summary of this part of day 2 summit).
Design teams were given time to work on developing their stories for the afternoon session on “storytelling.” Each team will have 5 minutes to construct a story, sharing with the audience their journey. What have been the milestones along the way, what are their learnings, and where are they along the path will be components of their story. The story arc is the template they are using (see below)
After working on constructing their stories, design teams were paired up to test their story on another team and receive feedback. Using the feedback they prepared for their final presentation of their story after lunch. There was a good deal of energy in the room, as well as some nervousness about getting all the data together into a five-minute story. Quite a task!
The following link to a Storify synopsis of Twitter links during the storytelling time (click here) will give you some sense of what was accomplished. We videotaped the whole day and captured all eleven stories, which will be posted on the website in the coming weeks. Here are a few highlights as well.
It was clear from storytelling that all eleven teams accomplished a great deal along their journey. They all recognized that design thinking has been, and will continue to be, a highly successful tool for helping schools innovate the challenges they face. While it may not be the only process a school uses to address challenges in the 21st Century, it is a process that helps a school build empathy with its users, such as parents, students and faculty, and then applying that understanding to defining, prototyping, iterating, and testing solutions to the challenges (see graphic below). In addition, when the process is used with fidelity it builds a collaborative culture in which people find common goal and shared values.
One of our primary goals in AK12DC is to build and sustain a design thinking mindset (see above) with members of our design teams. Preliminary data from Summit 2015 suggests we are making great progress towards this goal.
We wrapped up Summit 2015 with 70+ participants forming a circle and be christened by Scott Sanchez as Stanford d.school design thinkers.
A national conversation continues to take place about teacher evaluation systems, especially in states that submitted a Race-to-the-Top proposal, an Obama Administration initiative designed to fuel innovation in schools. This competitive grant program required states to submit a plan for how they would retool their current evaluation system, making greater use of student achievement data. If you follow this extensive body of literature, then you know that value-added models for using student achievement data, the exact percent that student achievement data would count in a teacher’s evaluation, and whether a teacher’s cumulative score would be made public have created all types of conversations, arguments, and potentially lawsuits.
Kate Taylor, in a recent New York Times article entitled, Cuomo Fights Rating System in Which Few Teachers Are Bad, tells the story of the battle between Governor Cuomo and the state’s teachers’ union. Cuomo wants an evaluation system that tightly aligns a teacher’s performance rating to his or her students’ test scores. The teachers’ union doesn’t believe this system can give a fair assessment of a teacher’s performance. She writes:
Around the state, administrators, teachers and parents have been protesting the governor’s proposals, which would both increase the weight of test scores, to 50 percent of a teacher’s rating, and decrease the role of their principals’ observations.
Cuomo, other governors, educational policy makers, and other political leaders believe that a system that closely ties students’ achievement scores to a teacher’s evaluation rating is a more effective way to commend good teachers, provide growth plans for average teachers, and weed out those that are not effective. Of course, there has been extensive research and commentary on whether value-added models for aligning student achievement data to a teacher’s performance are truly valid models given the plethora of variables that impact student achievement.
One thing is universally true about school districts that have had to retool their evaluation systems under Race-to-the-Top, they have not been very creative in designing their evaluation systems. They all look pretty much the same, with a few minor tweaks. They might count student achievement scores 25% instead of 40%. In fact, they are designed off of previous models with slight variations.
Another article that appeared in the New York Times, Grading Teachers by the Test, written by Eduardo Porter, suggests that according to Goodhart’ Law, an economic principle related to incentive design that sounds a lot like Heisenberg’s uncertainty principle in physics:
A performance metric is only useful as a performance metric as long as it isn’t used as a performance metric.
The idea in education being that if we rate teachers according to their students’ test scores do we run the risk of “fudging” the data to achieve what we want to achieve.
If we want to study organizations that are innovating their way to an evaluation system that meets the needs of their employees, then we have to go to the business world. We won’t find it in education. But I would argue that education has a lot to learn from the way some creative businesses approach giving constructive feedback to their employees.
In Harvard Business Review, Marcus Buckingham and Ashley Goodall write about the changes taking place at Deloitte Services LP in the article, Reinventing Performance Management. They describe how Deloitte is “rethinking peer feedback, and the annual review, and trying to design a system to fuel improvement.” It strikes me that if we speak with most educational administrators they would say their hope is that their school’s evaluation system would fuel improvement as well. Of course, the data shows that most teachers don’t believe their school’s evaluation system “fuels their improvement.” In a study done by Weisberg, Sexton, Mulhern, & Keeling (2009) called the Widget Effect, the authors write:
In districts that use binary evaluation ratings (generally “satisfactory” or “unsatisfactory”), more than 99 percent of teachers receive the satisfactory rating. (page 6)
If 99% of teachers are seen as satisfactory, then great teaching might go unrecognized while poor teaching does not get addressed. Another piece of data from the study shows that:
In fact, 73 percent of teachers surveyed said their most recent evaluation did not identify any development areas, and only 45 percent of teachers who did have development areas identified said they received useful support to improve. (page 6)
So the bottom line is that most of our evaluation systems do not “fuel improvement.” Not only are there flaws in the design of how we evaluate but there are also flaws in the way we go about implementation of the model. However, there is good data that suggests faculty believe that their principal’s feedback is important but it depends on whether principals are well-trained, understand the instruments they’re expected to use, understand their role in the process, and have confidence in differentiating for individual teachers’ needs. But some school systems, like New York State, are trying to deemphasize the principal’s role in the rating system. For that reason, and others, I think the Deloitte study is interesting for us to consider as a prototype for a new way of thinking about giving effective feedback to teachers.
Here is a high-level comparison of their old and new system.
|Old system||New system|
|Objectives||cascading||performance & strength oriented|
|360 degree tools||Yes||No|
In moving to their new system they used data from research, an understanding of their organizations needs, and a commitment to fuel the growth of their employees. The science of rating systems shows that “62% of the variance in the ratings could be accounted for by individual rater’s pecularities of perception” (page 43). What they concluded from looking at the research is that ratings do not measure the performance of the ratee as much as they reveal the biases of the individual rater. So they moved away from ratings. They also moved away from annual reviews to weekly and quarterly feedback based on team projects because their focus was on “spending more time helping their people use their strengths and we wanted a quick way to collect reliable and differentiated performance data” (page 44).
What Deloitte realized from a study done by the Gallop Organization on strengths-based leadership, as well as their own research using their high-performance design teams, is that if an evaluation systems focuses on strengths the person being evaluated invests more heavily in the process. Buckingham and Goodall write:
It found at the beginning of the study that almost all the variation between high- and lower-performing teams was explained by a very small group of items. The most powerful one proved to be “At work, I have the opportunity to do what I do best every day.” (page 44)
So if we work to align a person’s job responsibilities to their strengths, then we maximize opportunities for that person to be successful in their work. In the Deloitte study, here are the three items they found had high correlation with high-performing teams:
- Co-workers on the team were committed to doing quality work.
- The company’s mission inspired members of the team.
- Members of the team have a chance to use their strengths everyday.
When they designed their new system they had three objectives to fulfill. They were:
- The new system would allow them to recognize performance, particularly through variable compensation.
- The new systems had to facilitate ways in which they could CLEARLY SEE each person’s performance.
- The new system had to be able to fuel changes in performance.
I found it interesting that to achieve the second objective they redesigned the system and redefined the questions they asked of people being evaluated. First, they made their system highly relational, encouraging and creating expectations and time for each person to be in conversation with his or her immediate supervision or team-lead. To move away from rater reliability issues, they asked the team leader to use a set of four questions that focused more on the future relationship of the leader to the person being evaluated. The four questions were (page 46):
- Given what I know of this person’s performance, and if it were my money, I would award this person the highest possible compensation increase and bonus.
- Given what I know of this person’s performance, I would always want him or her on my team.
- This person is at risk for low performance.
- This person is ready for promotion today.
“In effect, we are aksing our team leaders what they would do with each team member rather than what they think of that individual.”
I find it interesting that they pivoted 180 degrees with their questions. So in education, what if the principal was required to answer the following questions:
- Would you recommend that your child be taught by this teacher for a full year?
- Would you pick this person to serve on your leadership team for building an ideal school?
- Would you pick this person to lead a new initiative in your school that requires an innovative leader?
Finally, in order to shift the responsibility from the team leader to a team member being evaluated, they set up a system where the person being evaluated identifies their strengths through a self-assessment tool and then shares those with other team members, the team lead and the organization. They have found:
that if you want people to talk about thow to do their best work in the near future, they need to talk often (page 48).
So their new system facilitates frequent conversation between team member and team lead about personal and professional strengths and progress towards goals. They designed for these conversations to be simple, frequent (weekly), quick and engaging.
As they have developed experience with their new system, there is a shift in the question that drives their work: from “what is the simplest view of you to what is the richest view of you?
So unlike evaluation systems being designed by state departments of education, or for that matter evaluation systems that exist in almost all public and private schools, we should be designing systems that provide for the richest view of our teachers. The richest view will not come from assigning 50% of the rating score to student achievement results. A teacher is a more complex professional than the results his or her students achieve on an imperfect standardized test that measures only a very small snapshot of what the student knows, understand and can do.
As educators, we have to be bold, creative, and thoughtful as we attempt to co-create the systems that will be used to evaluate our work. Our voice must be at the table in designing the process if it is going to succeed and fuel our improvement. Some answers to our questions are right before our eyes in the processes used by other organizations. Let’s learn from each other.
Check out this fascinating video about the properties of water and propylene glycol. The interaction of the two in a mixture. As one vaporizes in the mixture it influences the behavior of the drop and an adjacent drop. This could be a very interesting inquiry activity for physical science, chemistry, or physics students.
The Physics of a Water Droplet
There is an interesting article in the recent edition of Harvard Education Letter, STEAM Not STICKERS, by Edward P. Clapp and Raquel Jimenez. The authors discuss the importance of looking for authentic and intellectually justifiable ways to integrate art into the STEM disciplines. They looked at a variety of STEAM (A for Arts) projects in various sources that promote integration of arts into STEM. What they discovered was that many of these projects or initiatives do a disservice to the arts. They see two “prevailing trends” from their research:
- STEM activities or initiatives that call themselves STEAM but are no more than decorating STEM projects and calling it art. Hence the STEM with Stickers Effect.
- STEM activities or initiatives that call themselves STEAM but are no more than arts and crafts projects that appear to blend a S, T, E, or M with A. Little learning of art or of the other discipline. Hence the STEM Arts and Crafts Effect.
From my experience at the Center for Teaching, working with a variety of schools on STEM or STEAM project development, I would strongly concur with what Clapp and Jimenez have discovered. There is very little authentic work going on to integrated the theory and practice of art in the theory and practice of the STEM disciplines.
Here are a few essential questions on my mind:
- How does the theory and practice of one field inform or enrich the theory and practice of another field?
- How do we help teachers develop the understanding and mindset to design curricula that integrates these disciplines?
- How do we build into the curricula we design the knowledge, skills, and understanding that teaches students how to associatively think across disciplines?
- How do we keep STEM or STEAM from becoming the next bandwagon to jump on and then jump off in a few years?
With regard to the last question, it won’t happen if we only design curricula for students that is either the STEM with Stickers Effect or the STEM Arts and Crafts Effect. We have to be creative and invent new ways of teaching and learning that use substantive and intellectually demanding knowledge and skills from all these disciplines.
STEAM curriculum could be based primarily on science concepts with art woven in (purple graphic). It could be any of a number of the following permutations (cap letter being the dominate discipline in the curriculum or project). Maybe the most ideal scenario is one in which knowledge and skills from all five disciplines are used “equally;” STEAM.
Regardless of the disciplines used to build the curriculum, the designer (teacher) needs to do justice to the knowledge and skills that serve as the foundation for the discipline. What we should aim for is the development of STEAM experiences that are authentic and that require students to use the knowledge and skills that an artist, scientist, engineer, or technology expert, or mathematician would use in his or her work.
At The Westminster Schools (@WestminsterATL) in Atlanta, GA, we have been engaging in these conversations over the past two years. Our K-12 teachers have formed a professional learning community across the disciplines. We call it our STEAM PLC. This is a group of 18-22 dedicated teachers representing these disciplines who are committed to figuring out how to design and implement STEAM curriculum in their classrooms. They meet one a week for the entire school year to discuss, share, and plan curriculum. Some of their STEAM work is implemented solo in their classrooms, while other projects are cross-disciplinary and collaborative. Each year, their work has culminated in a K-12 STEAM exhibit showcasing the projects that have come from different classrooms. The work of this PLC has energized the school community, students, teachers, and administrators, to think beyond single disciplines. We are starting to envision curriculum that incorporates the knowledge and skills of different disciplines into a more holistic framework. I wouldn’t say we have arrived, but I would say that we are making great strides to design curricula that is not STEM with Stickers Effect or the STEM Arts and Crafts Effect. (the slide show below shows some of the work from the 2015 exhibit)
Here are some of the curricular ideas built into our STEAM exhibit (for more information contact firstname.lastname@example.org or email@example.com)
- 1st grade bee project…science of bees, science of bee collapse syndrome, stories about bees, art projects built around studying bees, design thinking or maker activities like building a 3-D scale model of a bee.
- Geometry and art
- Architecture, 3-D printing, and design
- 3-D printing and sculpture
- Designing a logo for iSummit (event held on Westminster’s campus)
- Masks, art, and interactive display
- Science (ecosystems), water, and art installation on STREAM of Conscious
- Building a City, a 3-D design project in Middle School to build a city
- Geometry installation in Middle School to build a 3 x 3 cube using large blocks according to the instructions provided
If we want students to learn important knowledge and skills from different disciplines, along with 21st Century skills like critical and creative thinking, communication and collaboration, and use their understandings to solve complex problems we will have to think seriously about the type of curricula we design. STEM with Stickers Effect or the STEM Arts and Crafts Effect curricula might be fun to do but it’s entirely possible it will not leave students any better prepared to design creative solutions to problems like global warming, scarce water supplies, pollution problems across the globe, or barbarous treatment of our fellow humans.
I recently visited one of the eleven schools that are part of Atlanta K12 Design Challenge (@ak12dc). Mountain Park Elementary, the Mustangs (@MPE_Mustangs), is a Fulton County School north of the I-285 loop. I met with Stacy Perlman, the school’s principal, and Wendy Kelly, the school’s project-based learning coach. After our meeting they took me on a tour of first grade classrooms and a number of TAG classrooms. It was a very inspiring meeting that provided a window into the school’s work since spring 2014. Both of them expressed great satisfaction with the AK12DC program and what the Design Team and faculty have been able to accomplish at Mountain Park.
In AK12DC, schools have learned and implemented a design thinking process modeled after work at the Stanford University’s Hasso Plattner Institute of Design to tackle a school challenge identified in Phase I of our project. After collecting empathy from their user, Mountain Park faculty, the school’s Design Team came up with the following point-of-view statement:
- we met…several hardworking Mountain Park teachers who hold themselves to
extremely high standards
- we were amazed to realize…how much they feel like their day-to-day schedules
and tasks prevent them from making the most of the time they have with their
students and colleagues.
- it would be game changing if we…staffers had clarity, confidence, and ownership
to implement school-wide enrichment to help students become real-world problem
One of their tasks in the spring of 2014 was to create a new schedule that would faculty to implement the school’s enrichment program with greater fidelity. In particular, they were interested in launching some project-based learning in their science and social studies classes at different grades. The following slide shows their first prototype of a new schedule which they tested in spring 2014.
They use the enrichment block built into the schedule for teachers and students to design, implement and showcase different PBL units that engage students in more open-ended, student-centered learning. The following slide illustrates a few examples of what they have been working on.
It was clear from my visit that the faculty has embraced using the enrichment block to experiment with some new ideas and curriculum. This year, they have been working on iterating their prototype and collecting more empathy data from their user to go a little deeper into their work. The following slide illustrates some of their project unknows and big questions that they are exploring. As Mountain Park’s Design Team goes deeper into addressing their point-of-view statement they are learning a great deal about their faculty culture and what it takes to promote innovation in their school. The following two slides illustrates their design-thinking action plan, with unknowns and questions to answer, as well as a glimpse at the concept map that guides their work.
Mountain Park’s work is just one example of what is happening in Atlanta K12 Design Challenge in Atlanta, GA. Ten other design teams from four more Fulton County schools and a group of six independent schools are working hard to use design thinking as a process for initiating and support innovation in their schools.
If you want more information about AK12DC follow us on Twitter @ak12dc or email me at firstname.lastname@example.org.
- More focus on vertical development
- horizontal development focuses on specific competencies of individuals whereas vertical development focuses on developmental stages that people need to move through.
- Transfer of greater developmental ownership to the individual
- “People develop fastest when they feel responsible for their own progress.”
- Greater focus on collective rather than individual leadership
- Petrie envisions us transitioning from a time when leadership resided in an individual or role, to a new phase where leadership resides in a “collective process that is spread throughout networks of people.” Reliance on a team instead of an individual!
- Much greater focus on innovation in leadership development methods
- Models of leadership that attend to developing a collective process or network don’t exist. Petrie believes we need to rapidly innovate ways of thinking about leadership development. “Organizations that embrace the change will do better than those that resist it.”