Forgetful Bri Apr 28, 2024

Smartphone accelerometers have been shown to be effective keyloggers.

With enough data, I wonder if I could use my phone as a DDR mat just by placing it on the ground and having it pick up the vibrations of me stepping on the floor…

The answer to that is almost surely "yes". The bigger question is "do I have the mental energy to gather the data, create a dataset, clean the dataset, train a model, and tune the hyperparameters?". Probably not… Maybe I'll try gathering the sensor data at least. Labeling it won't be feasible, but I'll at least have an idea of what I would be working with.

Show threadSky

@forgetful_bri that would be really cool, could be a fun experiment but also hyperparameters are hell. I imagine it'd have to be very particular to one kind of surface tho too.

Btw accelerometer keylogger? fr?? Is there an article about that?

Apr 28, 2024 at 5:46pmWeb
Show threadForgetful Bri Apr 28, 2024

@skybox every single step of that process is its own special hell 

Gathering the sensor data *might* go smoothly.
Taking what will probably be inconsistent, event-based entries and converting them into a consistent time series would likely involve trying a few different windows to get things cleaned enough.
After that, there's labeling the dataset. Thankfully, I'd have full control over the labels, so I probably wouldn't need to worry about the classes being unbalanced or anything.
Then… yeah, hyperparameter hell. It would be tempting to just toss a random forest or really basic MLP at it just to see what comes out.

Not to mention the physical hell of me generating data. For a good model, I might need to "dance" for tens of hours.

Yup! A proof of concept has been around for over a decade: https://arstechnica.com/gadgets/2011/10/researchers-can-keylog-your-pc-using-your-iphones-accelerometer/

I'm not personally aware of it ever being used as an effective attack, but also wouldn't be too surprised to hear about one (especially with advancements since 2011). Accelerometers have truly massive potential for gathering information. From https://dl.acm.org/doi/abs/10.1145/3309074.3309076: "…accelerometer data alone may be sufficient to obtain information about a device holder's location, activities, health condition, body features, gender, age, personality traits, and emotional state."

Researchers can keylog your PC using your iPhone's accelerometer

An iPhone's accelerometer is good for more than just games, according to …

Ars Technica