Marines tricked an AI designed to detect humans by....not acting normal
This is an excerpt from Paul Scharre's upcoming book:
Marines tricked an AI designed to detect humans by....not acting normal
This is an excerpt from Paul Scharre's upcoming book:
What is this from? I must read more!
@cross figured you would enjoy this.
@ktp_programming Its an excerpt from an upcoming book, sources are no doubt included:
https://www.amazon.com/gp/product/0393866866/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1
Ok but why would anyone train an AI to detect humans instead of using motion detection? Perhaps because there are other moving objects they don't want to detect, like animals or missiles or cardboard bo... wait no we want to detect cardboard boxes. But you can't have *all* disguises in the training set, can you?
"Listen, and understand. That terminator is out there. It can’t be bargained with. It can’t be reasoned with. It doesn’t feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead."
>" What if I put this cardboard box on my head?"
"Mmm That might work"
I wonder if this AI would have spotted a man in a wheelchair.
@CommieGIR
Alt text:
human with a weapon looks different.” To improve their algorithm, the DARPA team spent a week with a group of Marines out at a test site. For six days, the Marines walked around and the engineers refined their algorithm for detecting people. Phil said, “On the seventh day I said, ‘We're done. We're flipping it.’ ”
Now the Marines would try to defeat the AI system. They parked the robot in the middle of a traffic circle and the Marines had to approach it undetected
1/4
@CommieGIR
starting from a long distance away. “If any Marines could get all the way in and touch this robot without being detected, they would win. I wanted to see, game on, what would happen.”
“Eight Marines—not a single one got detected,” Phil said. They defeated the AI system not with traditional camouflage, but with clever tricks that were outside of the AI system’s testing regime.
2/4
@CommieGIR
“Two somersaulted for 300 meters; never got detected. Two hid under a cardboard box. You could hear them giggling the whole time.” Like Bugs Bunny in a Looney Tunes cartoon, sneaking up on Elmer Fudd in a cardboard box. “One guy, my favorite,” Phil said, “he field stripped a fir tree and walked like a fir tree.
3/4
@CommieGIR
You can see his smile, and that’s about all you see.” The AI system had been trained to detect humans walking, not humans somersaulting, hiding in a cardboard box, or disguised as a tree. So these simple tricks, which a human would have easily seen through, were sufficient to break the algorithm.
4/4
i love this. so much.