How can screening threshold analyses inform improved intake decisions?
RESEARCH FROM THE FIELD
JOURNAL ARTICLE SUMMARY
How can screening threshold analyses inform improved intake decisions?
Kearney, A. D., Wilson, E. S., Hollinshead, D. M., Poletika, M., Kestian, H. H., Stigdon, T. J., Miller, E. A., & Fluke, J. D. (2023). Child welfare triage: Use of screening threshold analysis to evaluate intake decision-making. Children and Youth Services Review, 144
What can we learn from this study?
Child welfare agencies must constantly balance the need to keep children safe with the commitment to minimize unnecessary investigations, which are intrusive, cause trauma for children and families, and reduce the availability of agency resources for service provision. This study tracks the disposition of initial (index) hotline reports to determine the percent of reports that were accurately screened out or screened in for investigation.
Study details:
- Population: 252,161 unduplicated reports to the centralized hotline in Indiana (IN)
- Data source: IN’s child welfare information system
- Methodology: Screening threshold analysis, including receiver operating characteristic curve (ROC); regression analysis
- Dates: Initial (index) reports made between July 1, 2016, and June 30, 2018
What are the critical findings?
All adults in Indiana are mandated reporters. The screening accuracy of Indiana’s Department of Children’s Services (DCS) central intake unit was assessed by conducting a screening threshold analysis. This analysis compared the screening decision (that is, whether or not to investigate) with outcomes that occurred within the following 180 days to determine whether the screening decision was accurate. Outcomes within 180 days that were defined as warranting a screen-in (true positive) at baseline included: 1) substantiated allegations, 2) a subsequent report that was screened-in, 3) a subsequent assessment, or 4) a case opening.
- A very high proportion of reports (92.8%) were screened in for investigation, but only 41.7% were true positives (that is, they were appropriate decisions to screen in based on the presence of maltreatment or need for services).
- The false positive rate was 51.1%. About half of cases screened in for investigation were not substantiated (and did not have a subsequent screened in report, a subsequent assessment, or a case opening during the following 180 days).
- Decisions based on agency screening practices were only slightly better than chance.
- The false negative rate was 2.4% and the true negative rate was 4.8%, both considered low.
Why is this important for our work?
Indiana’s DCS has been operating a risk-averse screening system: investigating a very high percentage of reports to avoid unintentionally screening out a report when it should be investigated. To address the high rate of false positives, DCS is now examining factors involved in decision making, systematically making changes to policy and testing the results of those changes to reduce false positives without increasing false negatives. Other jurisdictions may benefit from examining their screening trends, making policy changes, and tracking the results to minimize harm and trauma to families, and reduce the burden to the system.
This summary synthesizes the findings from a single research study. To learn more about Indiana’s work to improve screening and intake procedures based on data, please review How can hotline data help child protection agencies better support families? Additional resources about hotlines are also available.
For additional information about this article, see the abstract or email KMResources@casey.org.