ITP List (Seminar page)

September 10, 2021
  • ITP Ed Sciences: Dr. Xiang Zhou

    September 10, 2021  12:00 pm - 1:30 pm
    Educational Sciences Building 259

    Dr. Xiang Zhou
    Harvard University

    See more details

September 17, 2021
September 24, 2021
  • ITP Ed Sciences: Dr. Karen Bogenschneider

    September 24, 2021  12:00 pm - 1:30 pm
    259 Educational Sciences

    Dr. Karen Bogenschneider

    Required Readings

    KarenBogenschneider is a Rothermel-Bascom Professor Emeritus of Human Ecology at theUniversity of Wisconsin-Madison. Since its inception in 1992 through 2016, Dr. Bogenschneiderserved as director of the Wisconsin Family Impact Seminars–a series ofpresentations, briefing reports, and discussion sessions that communicatehigh-quality, nonpartisan research to state policymakers. From 1999-2014,she served as Executive Director of the Family Impact Institute that providedtraining and technical assistance to about two dozen states that conducted over225 Family Impact Seminars. The second edition was just released of Prof.Bogenschneider’s and Tom Corbett’s book, Evidence-BasedPolicymaking: Envisioning a New Era of Theory, Research, and Practice, andher book, Family Policy Matters is inits third edition. She hasconducted five national trainings for researchers interested in working withpolicymakers and 16 trainings for faculty directors of the Family ImpactSeminars. She has received a number of awards for her research and outreach,including being honoredby the Wisconsin State Legislature for her service to the state. She recently servedon a National Academy of Science round table on the communication and applicationof social and behavioral science research.

    See more details

October 8, 2021
October 15, 2021
  • ITP Ed Sciences: Cancelled

    October 15, 2021  12:00 pm - 1:30 pm

    See more details

October 22, 2021
  • ITP Ed Sciences: TBA

    October 22, 2021  12:00 pm - 1:30 pm
    259 Educational Sciences

    See more details

November 5, 2021
November 12, 2021
December 3, 2021
  • ITP Ed Sciences: Dr. Daniel McCaffrey

    December 3, 2021  12:00 pm - 1:30 pm

    Dr. Daniel McCaffrey
    Educational Testing Service (ETS)

    "Are Your AI Scores Good Enough?"

    Use of computers to score performancesfrom standardized evaluations, such as using artificial intelligence (AI) andnatural language processing (NLP) to rate written or spoken responses tostandardized test items, is growing rapidly in popularity. Currently, thesemethods are used in many testing situations to produce scores. For example,tens of millions of responses from elementary and secondary students are scoredusing computer-based automated scoring and states like Ohio (Ohio Department ofEducation, 2018) are moving toward having all student responses from itselementary and secondary school testing program scored by such methods.Moreover, each year millions of responses from high stakes tests such as theGRER, TOEFLR, the Duolingo English Test, the Pearson Test of English and thePearson Test of English Academic are also scored by computer-based automatedmethods.

     Use of AI scoring for assessments invariablyleads to questions about the ability of the scores to support the claims of theitems and the tests and the fairness of the scores. Typically, evaluation ofscores involves statistical analyses of the agreement between AI scores andhuman ratings of the same constructed responses or the accuracy of scores aspredictors of the human ratings. In this talk I will discuss an alternativeframework for evaluating AI scores that focuses on building evidence to supportclaims about the scores. I will discuss how to use statistical analysis of AIscores in this framework and methods for assessing the fairness of scores.

    See more details