Saturday, October 19, 2013

What do AF accountability systems measure?

My boss and I were driving home after the football game Friday night discussing our AF grades (I said grades and not grade because as of Friday evening each school site had received 6 different versions of an AF grade).  We came to 3 major conclusions:  1) we need to verify that the data the SDE used for our calculation is correct because with 6 different grades we just can't trust that the final product is accurate, 2) the AF grade for our school tells us absolutely nothing we were not already aware of and working diligently to improve, and 3) AF in its current form isn't even close to measuring the quality of our school.  I don't want to waste your time with another diatribe of exposing the litany of errors made by the SDE in calculating AF grades (and using "transparency" as the reason as to why the site was left up is really weak).  I want to focus this blog on what AF actually measures and why it is not a very good system to provide the information it proposes to provide.

What does the AF grade of a school measure?  Proponents of AF accountability systems tell you AF systems are an important tool to tell community members, parents, and businesses the quality of the school and the quality of the job the people in the school are doing with the tax dollars they are given.  I say it doesn't do any of that.  Here is my logic.  Did you know that at any high school, AF grade is calculated on 7 tests (school performance) and student testing gains (growth) on Algebra I and English 2 (2 components: all student gains and bottom 25% of student performance gain). Here is why it is important for people to understand the components of our current AF system.  First, student performance is based on only 7 subjects.  On our HS accreditation report, (the one that was due last Tuesday) 119 courses are offered to students at our campus or at the career tech and that does not include the 49 students taking concurrent enrollment (so 49 students are taking 4 college courses this year: 2 each semester). So 50% of the high school's grade is based on how students perform in 7 of 119 or more subject offerings?   Half the grade is based on how students do in 6% of our classes!  Let's take this even further.  The other 50% of the grade is based on student testing gains (25% all student growth and 25% bottom quartile growth) in 2 subjects.  Does this sound like an accurate and all encompassing measure of the quality of the entire school?  This type of measurement of a school is analogous to a homeowner buying a house solely based on the quality of the doorknob on the front door!

The second problem with the current AF system is it doesn't measure the quality of teachers within a school, it measures the quality of the STUDENTS in a school.  And before any naysayers can say otherwise, I am not diminishing the efforts of our teachers. A teacher's role is extremely important in the academic success of students. But we are only talking about 6% of the teachers and not all or even a majority of the teachers.  To make matters worse, we are not even talking about all or even a majority of the students.  We ARE talking about how a very small portion of students account for a majority of the grade.  Here is an example of what I'm talking about:  There are 600 students in our HS.  Let us say that 100 students take Algebra I and English 2.  200 total tests meaning 50 total students account for the bottom 25%.  So 50 out of the 600 students or a little over 8% count 33% of the overall grade.  That is 75 out of 225 total tests (100 for the student performance, 100 for the all student growth, and 25 for the bottom quartile growth for each of the growth subjects) or 33% of a total schools grade. 

So with only 6% of the total classes making up 100% of the grade and 8% of the students making up 33% of the grade, AF in it's current format is a terrible way to measure the quality of the teachers or the overall accountability of a school.  Now let me explain why AF is a terrible way to compare schools. Students are not widgets.  If every school had the exact same kids with the exact same abilities, problems, parents, etc then the current AF system would be perfect to hold schools accountable.   If students were identical, then this system would be an excellent way of measuring the quality of the school and would be an excellent accountability measure.  But because we all have different students, it is impossible to calculate through a formula which school is performing better than another.  How do you compare students with different backgrounds and educational abilities to each other? I'm not saying students shouldn't be held to a standard, and I agree that schools should be measured by how well their students accomplish those goals. Shouldn't we let the parents, community, and businesses of each community decide if the school is doing a good job?