About
The Beyond the Model Meeting (BTMM) is an informal gathering focused on software engineering for building applications with machine-learning components. The meeting aims to foster collaboration and exchange ideas among researchers working on various aspects of integrating ML components into software systems. Topics of interest include, but are not limited to:
- Process and requirements for ML-powered systems
- User-interaction design for ML-powered systems
- Quality assurance and safety engineering for ML-powered systems
- Operations for ML-powered systems
- Regulatory oversight for ML-powered systems
The meeting is modeled after the Feature-Oriented Software Development (FOSD) meetings. It features a supportive environment where participants present their research, receive feedback, and engage in discussions. The primary objective is to bring together researchers at different career stages, including undergraduate and early-career graduate students, to share ideas and initiate collaborations.
Note: This meeting is not a publication venue. Participants can present previously published work or work in progress, with an emphasis on fostering discussions and collaborations rather than producing proceedings.
Scope
The BTMM focuses on software engineering activities relevant to building software applications with ML components (including LLMs). It adopts a system view, considering the entire system and its environment beyond the model. Both technical and empirical work is welcome. The following topics are in scope:
- Work considering ML components in the context of a system
- Activities across the software lifecycle related to ML component integration
The following are out of scope:
- Exclusively model-centric work (e.g., fairness testing of models without system consideration)
- Work using ML to support software engineering tasks (e.g., LLMs for code generation)
Important Dates
| Abstract Submission Deadline | Recommended by Feb 17, but accepted on a rolling basis until all slots are taken |
| Beyond-the-Model Meeting 2025 | May 4 - 5, 2025 |
Meeting Format
The meeting consists of short presentations (10-15 minutes) from each participant, followed by ample time for discussion. Presentations can cover:
- Planned research
- Ongoing projects
- Published work
Attending / Registration
We ask all interested participants to submit a talk abstract through the form above. We will accept at most 20 participants. We will usually confirm participation within 1 week of abstract submission, first come first serve. We reserve the right to decline talks that we consider as out of scope or if we run out of capacity to host more talks.
Once the abstract is accepted, no additional registration is required. If you would be willing to volunteer to sponsor the event by paying an optional registration fee, please contact the organizers.
If you cannot make it after all or want to talk about something else, please email us as soon as possible.
Participants
(In alphabetical order)
| Name | Title | University |
|---|---|---|
| Scott Barnett | Optimising the evaluation and architecture of LLM applications | Deakin University |
| Tayana Conte | Teaching Requirements Engineering for AI: A Goal-Oriented Approach in Software Engineering Courses | Federal University of Amazonas (UFAM) |
| Luís Cruz | Greening AI with Software Engineering - Next Steps | Delft University of Technology |
| Vincenzo De Martino | Teaching Non-Functional Requirements in MLOps in Higher Education through Project-Based Learning | University of Salerno |
| Daniel Feitosa | What Technical Debt do developers discuss in ML-enabled Software Products? | University of Groningen |
| Bruno Gadelha | Developing LLM-based systems: an user-centered design approach | Federal University of Amazonas (UFAM) |
| Jin Guo | Contestability during the AI software development | McGill University |
| Kimberly Hau | LLMs in Mobile Apps: Practices, Challenges, and Opportunities | University of Toronto |
| Yining Hong | Monitors for Managing Less-Critical Safety Issues in ML-Powered Applications | Carnegie Mellon University |
| Jennifer Horkoff | Requirements Engineering for Systems with AI/ML: NFRs, Data, and Scoping of Complex Systems | University of Gothenburg and Chalmers |
| Pooyan Jamshidi Dermani | Reconciling Accuracy, Cost, and Latency of Inference Serving Systems | University of South Carolina |
| Andreas Jedlitschka | Uncertainty Wrapper for GenAI | Fraunhofer IESE |
| Christian Kästner | Shaping Practices, Tools, and Norms for Explainable ML-Powered Applications | Carnegie Mellon University |
| Nadia Nahar | Assisting Interdisciplinary Negotiation of Model Requirements | Carnegie Mellon University |
| Shalaleh Rismani | System Safety for AI: Modeling Hazards in Machine Learning-Driven Software Systems | McGill University, Mila |
| Yining She | Safeguarding LLM Agent Action through Provenance Identification | Carnegie Mellon University |
| Veronica Xia | From Contestation to Contestability: Challenging AI Systems Throughout Development | McGill University |
| Chenyang Yang | What to Engineer in Prompts? An Analysis on Prompt Underspecification | Carnegie Mellon University |
| Shurui Zhou | Can We Do Better with What We Have Done? Unveiling the Potential of ML Pipeline in Notebooks | University of Toronto |
Schedule
Schedule Sunday, May 4
Welcome and introductions: 09:45 - 10:00
Session 1: Requirements Engineering for AI: 10:00 - 11:30
-
Jennifer Horkoff: Requirements Engineering for Systems with AI/ML: NFRs, Data, and Scoping of Complex Systems
-
Vincenzo De Martino: Teaching Non-Functional Requirements in MLOps in Higher Education through Project-Based Learning
-
Tayana Conte: Teaching Requirements Engineering for AI: A Goal-Oriented Approach in Software Engineering Courses
Lunch: 11:30 - 13:00
Session 2: LLM Applications & Architecture: 13:00 - 14:30
-
Bruno Gadelha: Developing LLM-based systems: an user-centered design approach
-
Chenyang Yang: What to Engineer in Prompts? An Analysis on Prompt Underspecification
-
Scott Barnett: Optimising the evaluation and architecture of LLM applications
Session 3: AI Safety: 15:00 - 16:00
-
Shalaleh Rismani: System Safety for AI: Modeling Hazards in Machine Learning-Driven Software Systems
-
Yining Hong: Monitors for Managing Less-Critical Safety Issues in ML-Powered Applications
Session 4: ML Applications: 16:15 - 17:15
-
Veronica Xia: From Contestation to Contestability: Challenging AI Systems Throughout Development
-
Shurui Zhou: Can We Do Better with What We Have Done? Unveiling the Potential of ML Pipeline in Notebooks
Dinner @ DZÔ VIET EATERY: 18:00 - 20:00
- Note: Participants need to cover their own cost with cash or paypal
Schedule Monday, May 5
Session 5: ML Software Engineering Practices: 09:30 - 10:30
-
Daniel Feitosa: What Technical Debt do developers discuss in ML-enabled Software Products?
-
Nadia Nahar: Assisting Interdisciplinary Negotiation of Model Requirements
Session 6: Explainability & Contestation: 11:00 - 12:00
-
Christian Kästner: Shaping Practices, Tools, and Norms for Explainable ML-Powered Applications
-
Jin Guo: Contestability during the AI software development
Lunch: 12:00 - 13:15
Session 7: Sustainable and Mobile ML: 13:15 - 14:35
-
Kimberly Hau: LLMs in Mobile Apps: Practices, Challenges, and Opportunities
-
Luís Cruz: Greening AI with Software Engineering - Next Steps
-
Pooyan Jamshidi Dermani: Reconciling Accuracy, Cost, and Latency of Inference Serving Systems
Session 8: AI Safety (II): 14:50 - 15:50
-
Andreas Jedlitschka: Uncertainty Wrapper for GenAI
-
Yining She: Safeguarding LLM Agent Action through Provenance Identification
Closing: 15:50 - 16:00
Social Activities: 16:00 - 18:00
Venue & Travel Information
BTMM 2025 will take place at University of Toronto, Canada.
Meeting Location
- Building: Bahen Centre for Information Technology (BA)
- Room: BA7180
- Address: 40 St George St, Toronto, ON M5S 2E4
Accommodation and Costs
- No registration fees.
- Participants cover their own transportation, accommodation, and meals.
- Sponsorships may cover group lunches or dinners, tbd.
(If you would like to volunteer to sponsor the event by paying an optional registration fee, please contact the organizers.)
Hotel Recommendations
- Delta Chelsea Inn
- Price per night: $290CAD+
- Address: 33 Gerrard St W, Toronto, ON M5G 1Z4 – 20min walk
- Boutique Toronto Hotel in Annex/Yorkville
- Price per night: $340CAD+
- Address: 280 Bloor Street West – 15min walk
- Holiday Inn Toronto Downtown Centre
- Price per night: $313CAD+
- Address: 30 Carlton Street, Toronto – 21min walk
- Airbnb around campus