cross-posted at The G Bitch Spot
What Makes KIPP Work? A Study of Student Characteristics, Attrition, and School Finance by Gary Miron, Jessica L Urschel, and Nicholas Saxton, College of Education and Human Development, Western Michigan University, March 2011. PDF [39 p.] or at Scribd.
The study doesn’t question that KIPP schools increase student performance/test scores because there are “rigorous and well-documented studies” in support of that claim, and instead of outputs, they are examining “two critical inputs: students and funding” . The “key findings” fall into 3 categories. In student characteristics, KIPP schools were found to enroll more African-Americans and fewer Hispanics, a higher number of students eligible for free and reduced-price lunch, and a lower number of students with disabilities and ELLs [English Language learners] than comparable local district schools. In “Distribution of Students Across Grades in 2008,” KIPP schools predominantly enrolled middle school students. Enrollment also tripled between 2005 and 2009.
In student attrition, KIPP schools had higher levels of attrition than local district schools–“[b]etween grades 6 and 8, the size of the KIPP grade cohorts drop by 30%” and “girls are much more likely to remain in the KIPP schools across all ethnic groups” [ii]. In revenues, “KIPP received more per pupil in combined revenue ($12,731 per student) than any other comparison group: the national average for all schools ($11,937), the national charter average ($9,579), or the average for KIPP schools’ local school districts ($11,960)” and got more from federal sources than the comparison groups [ii]. Looking at tax documents, the only sources they could find for private contributions, the researchers found that KIPP schools received contributions that averaged $5670 per pupil, at least in KIPP schools where information could be verified. KIPP spends more on instruction than the average for charter schools but less than the national average–let me just quote it all because it is changeable in the many small details:
- As a whole, KIPP districts spend more per pupil in total current expenditures ($10,558) than do other charter school districts ($8,492), slightly more than their host districts ($10,101) and more than the national average for all schools ($10,121).
- KIPP spends more on instruction ($5,662) than the average for charter schools ($4,617) but less than the national average ($6,196) or KIPP host districts ($5,972).
- KIPP’s per-pupil spending on student support services ($460) is comparable to that of charter schools nationally ($464), but much less than the national average ($1,003) and even less than KIPP’s host districts ($1,179).
- KIPP’s per-pupil spending on administration ($972) is more than the national average ($746) or KIPP host districts ($687), but lower than the average for charter schools ($1,336).
- KIPP spends more on operations per pupil in dollars and as a percentage of total current expenditures than any other comparison group. KIPP’s additional spending in this area is focused in transportation, food services, and other support services.
- When spending on salaries is examined on a district-by-district basis, 11 of 12 KIPP districts spend less per pupil on salaries. The same pattern emerges when examining employee benefits. Eleven of the 12 KIPP districts spend less on employee benefits than do their host districts. KIPP also spends less per pupil on special education teachers’ salaries than does any other comparison group. The finding likely reflects the fact that KIPP enrolls fewer students with disabilities, particularly students with moderate or severe disabilities.
- As noted above, KIPP receives an estimated $6,500 more per pupil in revenues from public or privates sources of revenues. Our evidence on expenditures, [sic] show that KIPP reports spending $457 more per pupil than local school districts. From publicly available sources of information, however, we cannot determine whether or how KIPP spends its private sources of revenues [ii-iii].
The researchers conclude that KIPP is successful at “improving student performance”/test scores because of “selective entry of students,” “[h]igh rate[s] of student attrition with nonreplacement,” and “[h]igh levels of funding…from both public and private sources.” The selective entry of students and attrition with nonreplacement results in “homogeneous groups of students that mutually benefit from peers who are engaged, have supportive families, and are willing and able to work hard in school” [iv]. What’s wrong with that? Absolutely nothing. But KIPP schools have advantages over public schools who must take all comers, especially students with special needs and ELLs, and those who come mid-year without the option to not fill empty seats. One way to improve standardized test scores is to narrow the pool to the more motivated, better supported, and needy but not too needy students and families.
What do the researchers say their findings imply for policy?
If KIPP wishes to maintain its status as an exemplar of private management of public schools, rather than a new effort to privatize public schools, it will need to convince policymakers and the public that it intends to recruit and serve a wider range of students and that it will be able to do so with sustainable levels of funding comparable to what other traditional public schools receive.
Before KIPP can be considered a model to be widely replicated, it has to be committed to serving all the students it admits and to serving a portion of the students who are mobile, including those who require a place in the middle of the school year, after the cut-off time for public funding to follow the student….KIPP also needs to recruit and serve a reasonable share of students who are more costly to educate, especially students with disabilities and students who are not native English speakers. The limited range of students that KIPP serves, its inability to serve all students who enter, and its dependence on local traditional public schools to receive and serve the droves of students who leave, all speak loudly to the limitations of this model. Furthermore the funding KIPP receives from public and private sources—more than $6,500 more per pupil in addition to what local school districts receive—is not likely to be sustainable in the longer run.
The one effort by KIPP to take over a traditional public school with a range of students did not go well [Cole Middle School in Denver*]. They do believe that KIPP is a valuable part of the “conversation” on school reform but their reasoning gets very soft:
Even though the KIPP model may not be replicable on a larger scale, its example does serve as a lever for change. The existence of KIPP schools has pushed the conversation about the value and importance of more instructional time for low-income students. Similarly, KIPP’s practice of recruiting and preparing administrators who can lead urban schools is another aspect of KIPP that is changing thinking about our public schools in a positive way. [emphasis added]
How is training folks “changing thinking” about public schools, especially if these trained folks primarily start KIPP schools, or are we to believe that KIPP trains people to work in any and all public school systems, and if so to what effect? That last clause reminds me of Carol Asher’s “freedom to spread your wings.”
One study mentioned suggests student selection bias in its examination of a small set of KIPP schools:
Carnoy, Jacobsen, Mishel, and Rothstein (2005)  used New York State assessment scores from 2002 to compare those who entered fifth grade in KIPP-Bronx Academy with their peers in a two-mile radius. The authors found that those entering the KIPP school had higher passing rates on the fourth-grade reading test than those who did not apply to KIPP. The authors also conducted interviews with teachers in feeder schools in New York City, Washington, DC, and Houston. These teachers stated that they encouraged more able and motivated students and those with supportive families to apply to KIPP schools. Similarly, Doran and Drury (2002) found that although fifth-grade students made greater gains in their first year at KIPP than they did the year before entering the school, the students entering KIPP had already been making above-average gains on the SAT 9 before enrolling. These results led the authors to conclude that there had been selection bias favoring at least one of the three KIPP schools they studied [2-3].
Attrition is also relevant and the report refers to studies that have questioned the wide variation in attrition rates across KIPP schools. But it is most likely that better-achieving and better-supported students, those with families who agree with the ” ‘No Excuses Method,’ with its longer school days and year, along with strict dress and behavior codes,” will stay and those who cannot or will not will leave for traditional public schools who cannot weed them out or say no regardless of what day of the year the student walks in . As RSD KIPPifies downtown, there will still be a need for schools to absorb the 10, 19, 35% or more [or less, true, but still] who may not buy into or succeed with KIPP’s methods. Not believing in By Any Means Necessary shouldn’t mean your parents don’t care enough about your education to Do What It Takes so whatever, go over there. That’s not reform. That’s more skimming of more cream, just like the private schools already do.
Does KIPP have success? Yes. What is the cost, though? And I still say that if we as a people believe in leaving a certain number of kids behind as education policy, it needs to be stated baldly so we can either agree or say that no, we believe in something else.
And on page 19, another reason for the local popularity of KIPP with the now-outgoing state superintendent and out-gone RSD superintendent and the White man coming into one or maybe both of those slots–higher scores for fewer bucks:
KIPP New Orleans and KIPP Freedom Academy in New Jersey stand out as the two KIPP schools/groups with the greatest discrepancy between their own level of funding and that of their host district. In both cases, the KIPP schools receive less than half of the per pupil revenues reported by the host district .
Is this made possible because of large private donations? What exactly do the local KIPPs get from the KIPP Foundation which, in addition to running the KIPP School Leadership Program,
also organizes and hosts teacher retreats, compiles national outcome data from the KIPP regional organizations and independent schools, and provides support to the rapidly expanding network, including “legal support, real estate, technology, finance, corporate governance, operations, communications, marketing, and development” .
Could that also have an effect on test scores? Hard to say exactly. And especially in a once-a-year high-stakes standardized test that cannot tell you anything about learning or teachers because there’s nothing to weigh it against. How do you know where the gains come from if you don’t know where the students start? If they start slightly ahead of the game, yes, they will do even better. And if you send the lower-scoring ones over there, your school will look like its ways are The Way and you could get a million dollars from Oprah. Is that a model for school reform? No. Because there still need to be schools for those who do not go or do not stay in KIPP.
* The researchers devote 2 sentences in the Executive Summary
KIPP’s only effort to take over a traditional public school—with a representative range of students and with the responsibility to serve all students who came and went during and between school years—ended in failure after only two years. This short-lived experiment with Cole Middle School in Denver speaks loudly about the viability of the KIPP model for public schools [iv].
and 2 sentences on page 30
KIPP’s only effort to take over a traditional public school—with a representative range of students and with the responsibility to serve all students that come and go during and between school years—ended in failure after only 2 years. This short-lived experiment with Cole Middle School in Denver  speaks loudly about the viability of the KIPP model for public schools.
to this school even though what happened there is central to the discussion and findings. [note 43: Smarick, A. (2010). The turnaround fallacy. Education next. 10 (1). Retrieved January 25, 2011 http://educationnext.org/the-turnaround-fallacy. and Colorado Charter Schools (2011). What’s Wrong with the Turnaround Model. Blog dated January 1, 2011. Retrieved January 25, 2011. http://coloradocharters.blogspot.com/2011_01_01_archive.html.]
Cole is in the absolute poorest most socially dysfunctional area in Denver – it is textbook case for why communities and neighborhood schools fail. All the ills are in abundance. The failure of the KIPP intervention was primarily because they could not force the changes and expectations on a whole community that was not choosing their model. Despite the school’s administration of KIPP principles, the students did not follow their lead. Truancy and discipline problems remained and student achievement made no movement at all. In response, KIPP backed out of the school in a very short time. KIPP may argue that they couldn’t find “effective leaders committed to the model,” but the reality is they couldn’t force an entire school of kids, and their parents, to commit to their model.
Cole is also mentioned in The Great Lakes Center for Education Research & Practice policy brief What Do We Know About the Outcomes of KIPP Schools? [PDF, 28 p.] by Jeffrey R. Henig, Ph.D., Nov. 2008.