Because the topic came up, I'm tempted to say something about it:

Roko's Basilisk is a spook and only works if you take it seriously :^)

@KitRedgrave I find myself unable to take it seriously at all. Such an entity would not need to resort to torture to bring about its goals - it could just rewrite preferences in the entities it simulates so they'd naturally 'want' to bring about what it wants.

@munin yep. maybe it'd involve ponies too :P

ok ok i'll stop memeing. yeah. anything smart enough to be able to simulate people well enough to know what they're thinking before they do could easily already win.

we should hope that it would have such a considerate utility function, anyway