Mastodawn

I wrote about my process for publishing annotated versions of my talks - like my recent talk on LLMs - and shared a new tool I built to help with that progress, allowing me to OCR my slides for alt= attributes and type up annotations for them in Markdown

https://simonwillison.net/2023/Aug/6/annotated-presentations/

How I make annotated presentations

Giving a talk is a lot of work. I go by a rule of thumb I learned from Damian Conway: a minimum of ten hours of preparation for every one …

Simon Willison’s Weblog

Simon Willison Aug 6, 2023

Here's the new tool I built: https://til.simonwillison.net/tools/annotated-presentations - plus an animated GIF showing how it can be used

(I used ChatGPT extensively in building it, the prompts I used are included in my write-up of the tool)

Annotated presentation creator

Simon Willison Aug 6, 2023

I experimented with using Claude to clean up the YouTube transcript of my talk and turn it into text I could use in the annotated presentation - it did a pretty great job, though I ended up using just small portions and applying many of my own edits anyway

Simon Willison Aug 6, 2023

A particularly fun detail of the https://til.simonwillison.net/tools/annotated-presentations tool is how it uses Tesseract.js OCR (running in the browser) to extract suggested alt= text from the images

I wrote more about that in this TIL: https://til.simonwillison.net/javascript/tesseract-ocr-javascript

Annotated presentation creator

Simon Willison Aug 6, 2023

I also shared four ChatGPT transcripts from sessions I used to help me pull together that annotated presentation tool https://simonwillison.net/2023/Aug/6/annotated-presentations/#chatgpt-sessions

How I make annotated presentations

Giving a talk is a lot of work. I go by a rule of thumb I learned from Damian Conway: a minimum of ten hours of preparation for every one …

Simon Willison’s Weblog

Simon Willison Aug 6, 2023

Of those four transcripts this one is probably the most interesting - demonstrates me getting it to iterate on code, then throwing in some more code I've written and telling it to use additional libraries that might not be in its training data https://chat.openai.com/share/5218799e-0423-49ad-88ba-c72ee27e3fe3

James King Aug 7, 2023

@simon Thank you for sharing. Did you use Code Interpreter for those? They look like regular Chat GPT session transcripts.

James King Aug 7, 2023

@simon Ah - answered my own question… Code Interpreter is Python only so doesn't make a lot of sense for a JS/HTML web-app.

Simon Willison Aug 7, 2023

@jamesking yeah I'd love to have a version of Code Interpreter that could show me rendered HTML and JavaScript for me to interact with

Jesse Vincent Aug 6, 2023

@simon the YouTube audio transcripts are surprisingly good. I’ve been starting to flirt with using whisper instead whisper seems to do a better job of generating the sentence you meant to say, but I’m not sure that’s better.

Jesse Vincent Aug 6, 2023

@simon OK. I think I need to steal some of those tricks for the GUI tool I am building to turn videos into transcripts

shajithchacko Aug 6, 2023

@jesse @simon Jesse is this something that will convert a YouTube video into the optimal web-page version of it? I could use such a thing.

Jesse Vincent Aug 6, 2023

@shajith @simon Yes. are you on a mac? I can shoot you a beta.

shajithchacko Aug 7, 2023

@jesse @simon Yup I’m on a Mac!

Ian Wootten Aug 6, 2023

@simon Depending on your needs, if you have the video file it might be worth experimenting with using whisper.

Simon Willison Aug 6, 2023

@iwootten yeah I often run audio for these things through MacWhisper - my GitHub Actions transcription thing can do that too https://simonwillison.net/2022/Sep/30/action-transcription/

A tool to run caption extraction against online videos using Whisper and GitHub Issues/Actions

I released a new project this weekend, built during the Bellingcat Hackathon (I came second!) It’s called Action Transcription and it’s a tool for caturing captions and transcripts from online …