Parker HigginsWorking on a project that would require a headless browser to screenshot web pages, and MAN do you folks know what the NYT home page looks like if you're not blocking ads? https://mastodon.xyz/media/2OPG6WGVY3nvu1lwzs8
Phil @xor What screenshot system are you using? This is an issue that comes up for me at work fairly often (with varying results).
@neuralgraffiti this is the new-ish headless chrome option (so just
chromium-browser --headless --disable-gpu --screenshot="nyt.png" https://www.nytimes.com/
for now.) New documentation for it inspired me: https://developers.google.com/web/updates/2017/04/headless-chrome
@xor Interesting. I've used a combination of curl + https://wkhtmltopdf.org/ with fairly good results (for capturing websites for litigation discovery).
@neuralgraffiti I use pdfkit (which outsources the rendering to wkhtmltopdf) for another project and it definitely has some quirks, but that's probably as good as it gets for that purpose
@xor Nice. Good to know. I've never tried pdfkit - just shell scripted wkhtmltopdf; I should check that out.
@neuralgraffiti oh wow it looks like every language calls their comparable blob "pdfkit" but I use the python one. Easy enough to set up but spits a lot of kind of useless errors (I think because browsers are used to reading EVERYTHING)
@xor Oh good - Python is what I'd be most comfortable with anyway. Considering trying to put together a more robust solution that would allow less-technical folks to run it.
@neuralgraffiti here's what I got, sorry I'm like this https://github.com/thisisparker/ftd/blob/master/getobits.py#L159-L172
@xor Thanks! Your code is more polite than mine.
@neuralgraffiti it's a sickly sort of politeness because I just made it talk to ME that way all the time :-/
Anyway the "OSError" happens like 50% of the time (and is genuinely harmless), other errors happen almost never.
@xor Could be worse. In a prior life, I wrote a script to check my code for inappropriate words before check-in, after some profane debug code made it in to production. Anyway, curious to hear how headless chrome ends up working for you.
patrickod@xor had this experience recently, had been years since I'd browsed without an adblocker. So much crap.