Understanding Systematic Miscalibration in Machine Learning Classifiers
Markelle Kelly and Padhraic Smyth
TREX Workshop, IEEE VIS, 2022
Variable-Based Calibration for Machine Learning Classifiers
Markelle Kelly and Padhraic Smyth
37th AAAI Conference on Artificial Intelligence (AAAI), 2023
Does the AI Know What I Know? A Framework for Modeling Human Perceptions of AI
Markelle Kelly, Aakriti Kumar, Padhraic Smyth, and Mark Steyvers
Accepted to ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2023
Understanding Mental Models of AI Agents
As part of the human-machine group at UCI, led by Dr. Padhraic Smyth and Dr. Mark Steyvers, I am researching how people develop mental models of AI agents. This work is based on crowdsourced experiments and Bayesian modeling techniques, rooted in principles from cognitive science and psychology.
UCI Machine Learning Repository
I am working on the NSF-funded reinvention of the UCI Machine Learning Repository, headed by Dr. Sameer Singh, Dr. Padhraic Smyth, and Dr. Philip Papadopoulos. In particular, I am developing automatically-generated performance baselines for the repository's datasets. I also contribute to general design and functionality decisions, and serve as a repository curator and librarian, curating and maintaining our datasets and their metadata.
As a member of the UCI Covid Awareness group, led by Dr. Vladimir Minin, I help maintain a dashboard that displays up-to-date information on COVID tests, cases, hospitalizations, and deaths in California, with a focus on Orange County. This dashboard combines data from several sources and aims to present it to California residents in an understandable manner.
Software Engineering at Project Jupyter
I worked for over two years as a software engineering intern for Project Jupyter, with a focus on JupyterLab, a web-based framework for interactive computing. Led by Dr. Brian Granger and Dr. Zach Sailer, I built two JupyterLab plugins that are now dependencies for multiple other projects, and developed methods for adding custom front ends to the Jupyter server. Along the way, I collaborated closely with other developers and UI/UX designers to conduct user testing and make design decisions. Cumulatively, my projects in JupyterLab development place me in the tool’s top 20 contributors of all time—work that both expanded existing computational software and enabled a range of scientific programming across disciplines.
Human Trafficking Detection with the Global Emancipation Network
I led a team with two other students, in collaboration with the Global Emancipation Network, to identify California businesses engaging in human trafficking. We scraped web data on illicit massage parlors and developed a novel model to predict their involvement in human trafficking. In April 2020, the model’s output was presented to the FBI, directly resulting in the investigation and subsequent raiding of multiple California human trafficking hubs. We also performed a case study regarding these parlors in San Luis Obispo County, culminating in a June 2020 report for the district attorney. This work set a precedent for similar endeavors in other states, including in Texas, Washington, and Michigan, thus enhancing our machine learning toolkit for the fight against human trafficking.
Understanding Co-Curricular Participation of Engineering Students
My undergraduate research included working with Dr. Chance Hoellwarth on an NSF-funded study on minority students in engineering. As a research assistant, I analyzed survey data, working to understand the factors that influence these students' participation in co-curricular activities. This work culminated in a proposal of action to increase diversity in these activities for Cal Poly’s College of Engineering, and an expanded study involving several other universities worldwide.