OnMar 30, 2024
Hugo van Kemenade

We got a human-submitted, AI-generated PR for @pillow that claimed to speedup an internal function by 54%.

It presented no benchmark script, so I benchmarked and found no noticeable improvement.

The PR claimed the "new optimized code was tested for correctness" and offered some "Generated Regression Tests" that it declared passed.

I ran them. After fixing an import error, the tests failed for both the PR and main.

No thank you, Codeflash AI.

https://github.com/python-pillow/Pillow/pull/7915 #Python #AI #CodeflashAI

Speed up Stat._getmedian() by 54% in src/PIL/ImageStat.py by ihitamandal 路 Pull Request #7915 路 python-pillow/Pillow

馃搫 Stat._getmedian() in src/PIL/ImageStat.py 馃搱 Performance improved by 54% (0.54x faster) 鈴憋笍 Runtime went down from 22998.37渭s to 14889.95渭s Explanation and details (click to show) The function can...

GitHub
Mar 30, 2024 at 1:38pmTusky
Show threadWill McGuganMar 30, 2024
@hugovk @pillow Feels like those automated linting services that make arbitrary changes or break things. But worse. This made you do work to reject it!
Show threadDanielMar 30, 2024

@hugovk @pillow Looks like Pillow is not the only repo this user is trying their luck

https://github.com/langchain-ai/langchain/pull/19779

鈿★笍 Speed up _hamming_distance() by 50% in libs/langchain/langchain/evaluation/embedding_distance/base.py by ihitamandal 路 Pull Request #19779 路 langchain-ai/langchain

馃搫 _hamming_distance() in libs/langchain/langchain/evaluation/embedding_distance/base.py 馃搱 Performance went up by 50% (0.50x faster) 鈴憋笍 Runtime went down from 749.61渭s to 500.81渭s Explanation and de...

GitHub
Show threadHugo van KemenadeMar 30, 2024

@djh @pillow Yup: https://github.com/search?q=Codeflash+ai&type=pullrequests

Plus some reverts of PRs that were accidentally merged.

Build software better, together

GitHub is where people build software. More than 150 million people use GitHub to discover, fork, and contribute to over 420 million projects.

GitHub
Show threadmike bayer  Mar 30, 2024
@hugovk @pillow if someone submitted an AI generated PR to my projects I'd ban them and report them for spam