I am very concerned about the definition of an underperforming school district that could be written into the new education bill.
The new bill, specifically the section that will replace Chapter 71, Section 89 (i) (3), includes language that will trigger lifting the charter school cap for districts based on low Composite Performance Index (CPI) scores. The specific language is here:
Chapter 71, Section 89 (i) (3) In any fiscal year, if the board determines the combined Composite Performance Index scores on the English language arts and mathematics Massachusetts Comprehensive Assessment System exams for a school district place said district in the lowest 10 per cent of all statewide Massachusetts Comprehensive Assessment System exam performance scores released in the 2 consecutive school years prior to the date the Charter school application is submitted, the school district’s total Charter school tuition payment to commonwealth Charter schools may exceed 9 per cent of said district’s net school spending but shall not exceed 18 per cent.
The CPI is based on the average (mean) performance of all students in your district or school. Simply put, Proficiency Index points are awarded to each student based on their MCAS scaled scores:
200-208 (low Warning – Failing) = 0 Proficiency Index points
210-218 (high Warning – Failing) = 25 Proficiency Index points
220-228 (low Needs Improvement) = 50 Proficiency Index points
230-238 (high Needs Improvement) = 75 Proficiency Index points
240-280 (Proficient and Advanced) = 100 Proficiency Index points
The result is not surprising. Districts that educate a large number of students in poverty, students who come to school speaking languages other than English, are going to have low CPI scores. Districts where mothers with doctoral degrees drop kids at the door of the school in her late model Volvo will have high CPI scores.
If this provision survives to become law, it will codify a costly and punitive sanction that is tied to demographics and not the performance of the district. Districts with challenging populations, working successfully to raise achievement for their children, will be subjected to costly and highly punitive sanctions while poorly performing schools with privileged populations will fly under the radar.
It doesn’t make any sense. It’s also counterproductive if the goal is to raise standards and accountability in our schools.
It also doesn’t need to be that way. The Department of Elementary and Secondary Education has developed a new measure for student performance on the MCAS. It’s called the Student Growth Percentile (SGP). It is modeled on Colorado’s accountability system, and it appears to be a much more effective indicator for successful and struggling schools and districts.
Here’s how it works. The state looks for students with similar MCAS histories with the same subject and grade level. You then look at the distribution of these students in the most recent MCAS, and assign percentile scores based on this distribution. A SGP score of 80 means that child performed better than 80% of students with a similar academic history, a SGP score of 25 means that a child performed better than 25% of students with a similar academic history. The state categorizes student performance as follows:
1-19%ile VERY LOW GROWTH
20-39%ile LOW GROWTH
40-59%ile MODERATE GROWTH
60-79%ile HIGH GROWTH
80-99%ile VERY HIGH GROWTH
The state has a much more comprehensive explanation of SGP on their website:
http://www.doe.mass.edu/mcas/growth/
Comparing the CPI model to SGP, a student who has been advanced in the past two years (270-270) and drops to a low Proficient score (240) will get the highest possible score (100 points) but is likely to have a very low growth score (SGP below the 5%ile). Meanwhile, a student who has been in the Warning category (216-218) who moves into the middle of the Needs Improvement category (228) will have only 50 CPI points, but is likely to have a very high growth score (SGP above the 80%ile).
SGP AS AN ACCOUNTABILITY MEASURE
The state is now reporting the median SGP for schools and districts. The Median SGP is the SGP score for the student in the middle of a school or district’s distribution. Because the middle of the state’s distribution is the 50%ile (by definition), schools or districts with a Median SGP above 50 are performing better than the state as a whole. If you mirror the state distribution, 40% of students are in the Very Low and Low Growth categories. If your district’s Median SGP is at the 40%ile, than half of your students are in the Very Low and Low Growth categories. Because this is a measure of the middle child in your distribution, a median score in the 40th%ile is significantly below the rest of the state.
To avoid a major explanation of statistics, let’s just say that as the size of the distribution increases, the median score of the sample (subgroup, school, district) will tend to be closer to the median score of the entire distribution (state). A classroom with a Median SGP of 40 is less extreme than a school with Median SGP of 40, and is less extreme than a district with a Median SGP of 40.
To illustrate this point, you can look at the distribution of school and district Median SGP scores in Mathematics and English Language Arts that I posted here:
http://www.schlichtman.com/SGP_school_district_distribution.pdf
When you look at all the schools in the Commonwealth in English Language Arts, the bottom 5% of all schools have Median SGP scores below 32, and the top 5% of all schools have Median SGP scores above 69. In Mathematics, the bottom 5% of all schools have Median SGP scores below 31, and the top 5% of all schools have Median SGP scores above 71.
When you look at all the 379 districts in the Commonwealth in English Language Arts, the bottom 5% of all districts have Median SGP scores below 37, and the top 5% of all districts have Median SGP scores above 65. In Mathematics, the bottom 5% of all districts have Median SGP scores below 38, and the top 5% of all districts have Median SGP scores above 67.
DESE defines a district as a legal entity, and that includes some very small districts and charter schools. They can all have extreme scores (both high and low) due to their small size. If we restrict our analysis to districts reporting results for 300 or more students, the distribution tightens. In English Language Arts, the bottom 5% of the 251 largest districts have Median SGP scores below 39, and the top 5% of the 251 largest districts have Median SGP scores above 60. In Mathematics, the bottom 5% of the 251 largest districts have Median SGP scores below 40, and the top 5% of the 251 largest districts have Median SGP scores above 62.
WHY DOES THIS MATTER? WHAT DOES THIS MEAN?
If you look at Lowell’s math scores, you will find that the CPI scores for 2008 (62.4) and 2009 (64.9) are in the lowest 10% statewide, but the Median SGP (growth) scores for 2008 (56.0) and 2009 (55.0) are in the top 25% of Massachusetts districts. You have a district that has 31.5% of students Limited English Proficient (highest percentage in the state) and 67.6% low income with a high growth rate, but subject to the harsh fiscal sanctions of a doubling of the charter cap.
If you look at one of Lowell’s suburban neighbors, a district with only 1.2% of its students Limited English Proficient and 13.3% low income, the CPI scores for English Language Arts for 2008 (85.4) and 2009 (86.7), and the CPI scores for Mathematics in 2008 (75.8) and 2009 (75.7) were condierably higher than Lowell. The district has met AYP for ELA and Mathematics in the aggregate and all subgroups, and is in No Status under the state accountability system.
However, this suburban district has very low SGP scores. The ELA Median SGP in 2008 (38.0) and 2009 (39.0) are among the bottom 5% in the state, and the Mathematics ELA Median SGP in 2008 (42.0) and 2009 (42.0) are among the bottom 20% in the state.
The point is, the state legislature has a proposal before it that would penalize districts that are doing a good job of educating students under challenging circumstances, and ignores districts that perform poorly with an advantaged student population. I just think this is bad public policy.
Meanwhile, if you want to see how your district is performing, you can find out here:
http://profiles.doe.mass.edu/state_report/mcas.aspx
Thanks for this thoughtful analysis, Paul. I completely agree that a school district — like the one you serve in Lowell — might have low scores, but be doing a great job. You are suggesting that school districts that have low improvement should be targeted for charters, rather than those with low scores.
The theory of the reform is that charters might help even more and should be targeted to places with low scores, because they most urgently need to improve. I agree with your point that this might mean taking funds from a system that was doing a great job. I am prepared to support this bill because I hear the affected districts asking for it. If they wish to have a different formula that targets schools that are both low-scoring and low-improvement, that would be something I might be able to support if I were satisfied it would not make my communities more likely to be eligible — there is little demand for charters among parents in my district.
But I would not support switching the computation from a score-concept to an improvement-concept. That would mean wasting state attention on districts that (perhaps entirely due to high SES and despite a weak school system) are producing good outcomes.
There have been a number of conversations through the day and may be several flavors of amendment filed that are responsive to this.
I will be filing this language (which I know that Rep Garballey also supports).
by adding at line 573 after the phrase ’18 per cent’ the following sentence “Provided however that if a school district’s Student Growth Percentile (or other similar measure of improvement adopted by the Board) places the district in the upper half of districts based on improvement in the most recently computed school year, then said district’s total charter school tuition payment to commonwealth charter schools may exceed 9 percent of said district’s net school spending only with the approval of the district School Committee.”
This language should suffice to preserve local control in districts that are performing well. It will not increase the number of districts potentially subject to an increase in the cap (only those with test scores in the lowest 10 percent will be affected), but will exempt those that have good SGP performance.
/w.
I think that amendment would address my concern about targeting poor urban districts, and it is a solid improvement. I am thankful that you put forth an amendment to address this problem.
Paul,
Reporting mid-day on the progress of the bill, the sense is that this amendment would have too broad an effect and it is a non-starter which I am not going to press. My sense that any inclusion of additional narrowing factors will meet strong resistance, in part, because their effects may be less predictable than the basic score ranking. But I will continue to listen to other amendments on the floor that may make sense and pertain to this issue.
/w.