Incentives to improve the quality of care provided in Medicaid managed care plans are increasingly common and take many forms. One is a pay-for-performance program that automatically assigns new enrollees to better-performing Medicaid plans in California. Our qualitative and quantitative study of this program examined the expected and actual impacts of the performance incentive on all areas of care. We compared quality outcomes in plans included in the pay-for-performance, "auto-assignment" incentive and comparable outcomes in plans that were not included. We found that quality did not improve significantly faster in plans included in the incentive scheme. Combined with some evidence of negative impact on other areas of care, the findings raise questions about the usefulness of this program in California Medicaid, and about similar programs in other states.