School of Computer Science Seminar with Professor Jeremy Bradbury Democratizing AI in Software Development: From Few-Shot Testing to Trustworthy Benchmarks and Accessible AI Tools

Professor Jeremy Bradbury from Ontario Tech will be visiting the School of Computer Science and will present Democratizing AI in Software Development: From Few-Shot Testing to Trustworthy Benchmarks and Accessible AI Tools.

Abstract: The rapid advancement of artificial intelligence, particularly Large Language Models (LLMs), is transforming software development. However, these benefits are not evenly distributed. Large technology companies have the resources to develop and deploy sophisticated AI models, while small and medium-sized organizations, as well as open-source development communities, often face significant barriers to adoption. Democratizing AI in software development therefore requires tools and methods that support domain-specific needs while reducing computational and data requirements.

This talk explores three major challenges to broader AI adoption in software development and recent research addressing them. First, we examine the challenge of limited computational and data resources, and present work applying few-shot learning in software testing to enable effective detection of problematic flaky tests with minimal training. Second, we address concerns about trust in third-party models with a method for evaluating data leakage in an existing LLM program-generation benchmark and developing improved benchmarks. Third, we explore onboarding challenges associated with learning to develop AI-enhanced software and present early research on using block-based programming to make development more accessible.

Bio: Jeremy Bradbury is a professor at Ontario Tech University in Oshawa, Ontario, Canada. He leads the Software & Education Research (SEER) Lab, where his team studies AI-assisted software development and AI-supported personalized learning in educational games. His research has been published in leading venues including the International Conference on Software Engineering (ICSE), the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), the IEEE International Conference on Software Testing, Verification and Validation (ICST), and the ACM Conference on International Computing Education Research (ICER). In addition to leading the SEER Lab, he is the co-lead for Methods for Responsible and Inclusive Development at the Mindful AI Research Institute and serves as Vice President of CS-Can | Info-Can, the national organization for computing research in Canada.