Show threadArie van Deursen 🇪🇺🇳🇱Aug 7, 2025

GPT5’s “safe completion” was previously called “safe answering”, and is included in the benchmark we developed to assess the “Harmfulness of Applying Off-the-Shelf Large Language Models to Programming Tasks”.

https://dl.acm.org/doi/abs/10.1145/3729380

#gpt5 #safecompletion #harmfulness #fse2025

Andreas ZellerJun 25, 2025

Today at #FSE2025:

* 14:40 @ Cosmos 3C “Expressing and Checking Statistical Assumptions”: 84% of surveyed notebooks violate assumptions: https://conf.researchr.org/details/fse-2025/fse-2025-research-papers/93/Expressing-and-Checking-Statistical-Assumptions

OR

* 14:50 at Pirsenteret “jAST: Analyzing and Modifying Java ASTs with Python”: https://conf.researchr.org/details/fse-2025/fse-2025-demonstrations/25/jAST-Analyzing-and-Modifying-Java-ASTs-with-Python

Expressing and Checking Statistical Assumptions (FSE 2025 - Research Papers) - FSE 2025

The ACM International Conference on the Foundations of Software Engineering (FSE) is an internationally renowned forum for researchers, practitioners, and educators to present and discuss the most recent innovations, trends, experiences, and challenges in the field of software engineering. FSE brings together experts from academia and industry to exchange the latest research results and trends as well as their practical application in all areas of software engineering. The conference will be co-located with ISSTA 2025 (June 25th – 28th, 2025), Internetware 2025 (June 20th – 22nd, 2025), an ...

Andreas ZellerJun 24, 2025
My #Fandango team at #FSE2025 / #ISSTA2025: Alexi Turcotte, Marius Smytzek, me, Pepe Zamudio, and Laura Plein. What is #Fandango? Watch this space on Thursday for our big 1.0 release announcement and/or attend Pepe‘s presentation on Friday 16:00!
MattJun 24, 2025

Join me in Cosmos 3C at 15:00 at #fse2025 TODAY for my talk on TerzoN and the Composite Oracle, which combines implicit, example-based, and property-based test oracles in a thoughtful and developer-focused way.

Come for a cool NaNofuzz sticker & stay to hear the results of our randomized controlled human trail of professional developers using TerzoN & fast-check.

https://conf.researchr.org/details/fse-2025/fse-2025-research-papers/30/TerzoN-Human-in-the-Loop-Software-Testing-with-a-Composite-Oracle

TerzoN: Human-in-the-Loop Software Testing with a Composite Oracle (FSE 2025 - Research Papers) - FSE 2025

The ACM International Conference on the Foundations of Software Engineering (FSE) is an internationally renowned forum for researchers, practitioners, and educators to present and discuss the most recent innovations, trends, experiences, and challenges in the field of software engineering. FSE brings together experts from academia and industry to exchange the latest research results and trends as well as their practical application in all areas of software engineering. The conference will be co-located with ISSTA 2025 (June 25th – 28th, 2025), Internetware 2025 (June 20th – 22nd, 2025), a ...

Arie van Deursen 🇪🇺🇳🇱Jun 23, 2025

Also at #FSE2025: Our paper assessing LLM's capabilities to handle comments in languages other than English (we look at Chinese, Dutch, English, Greek, and Polish).

Today's metrics for assessing success "fail to reliably differentiate meaningful completions from random noise"

https://conf.researchr.org/details/promise-2025/promise-2025-papers/3/A-Qualitative-Investigation-into-LLM-Generated-Multilingual-Code-Comments-and-Automat

#llm #llm4code

A Qualitative Investigation into LLM-Generated Multilingual Code Comments and Automatic Evaluation Metrics (PROMISE 2025) - PROMISE 2025

The International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE) is an annual forum for researchers and practitioners to present, discuss and exchange ideas, results, expertise and experiences in construction and/or application of predictive models, artificial intelligence, and data analytics in software engineering. PROMISE encourages researchers to publicly share their data in order to provide interdisciplinary research between the software engineering and data mining communities, and seek for verifiable and repeatable experiments that are useful in ...

Arie van Deursen 🇪🇺🇳🇱Jun 23, 2025

Too bad I can't join FSE this year. If you're there, checkout our work analyzing how language models for code try to avoid giving 'harmful' responses (re intellectual property, malware, and biases).

https://conf.researchr.org/details/fse-2025/fse-2025-research-papers/68/Code-Red-On-the-Harmfulness-of-Applying-Off-the-shelf-Large-Language-Models-to-Progr

#llm #alignment #fse2025

Code Red! On the Harmfulness of Applying Off-the-shelf Large Language Models to Programming Tasks (FSE 2025 - Research Papers) - FSE 2025

The ACM International Conference on the Foundations of Software Engineering (FSE) is an internationally renowned forum for researchers, practitioners, and educators to present and discuss the most recent innovations, trends, experiences, and challenges in the field of software engineering. FSE brings together experts from academia and industry to exchange the latest research results and trends as well as their practical application in all areas of software engineering. The conference will be co-located with ISSTA 2025 (June 25th – 28th, 2025), Internetware 2025 (June 20th – 22nd, 2025), a ...

Andreas ZellerJun 23, 2025

Today at #FSE2025: Check out

* 14:20 @ Cosmos 3C: Bernd‘s journal-first work on Information Flow Fuzzing: https://conf.researchr.org/details/fse-2025/fse-2025-journal-first/31/Presentation-Proposal-for-Finding-Information-Leaks-with-Information-Flow-Fuzzing and
* 15:12 @ Vega: Laura‘s Student Research Competion talk on Predicting Software Changes from Desired Behavior Changes: https://conf.researchr.org/details/fse-2025/fse-2025-student-research-competition/4/Predicting-Software-Changes-from-Desired-Behavior-Changes

Presentation Proposal for: Finding Information Leaks with Information Flow Fuzzing (FSE 2025 - Journal First) - FSE 2025

The ACM International Conference on the Foundations of Software Engineering (FSE) is an internationally renowned forum for researchers, practitioners, and educators to present and discuss the most recent innovations, trends, experiences, and challenges in the field of software engineering. FSE brings together experts from academia and industry to exchange the latest research results and trends as well as their practical application in all areas of software engineering. The conference will be co-located with ISSTA 2025 (June 25th – 28th, 2025), Internetware 2025 (June 20th – 22nd, 2025), a ...

MattJun 22, 2025
Anybody on here at FSE or is it literally just me? #fse2025 #fse25
Show threadMattJun 21, 2025

can’t wait to travel* to #fse2025 tmrw!!

*=it’s down the street from this taco place