N-gated Hacker News🎉 Oh, look! Yet another thrilling #dissertation on prompt optimization – because clearly, our lives were incomplete without a "framework" to remind us that "choosing metrics" is still a thing. 🚀 Don't worry, it also comes with a handy guide to define "success" in terms of stats-lingo no one asked for. 🍇🤓
https://www.gojiberries.io/not-so-prompt-prompt-optimization-as-model-selection/ #promptoptimization #metrics #successframework #statslingo #HackerNews #ngated
https://www.gojiberries.io/not-so-prompt-prompt-optimization-as-model-selection/ #promptoptimization #metrics #successframework #statslingo #HackerNews #ngated
Not so Prompt: Prompt Optimization as Model Selection
Here's a framework for prompt optimization: Defining Success: Metrics and Evaluation Criteria Before collecting any data, establish what success looks like for your specific use case. Choose a primary metric that directly reflects business value—accuracy for classification, F1 for imbalanced datasets, BLEU/ROUGE for generation tasks, or custom domain-specific
Sanjay Mohindroo
Henrik Ekenberg
Helena Nashton