mmitchell_aiJul 19, 2023
Great reporting from @benjedwards on where we're at now with understanding GPT-4 perfomance "How are we meant to build dependable software on top of a platform that changes in completely undocumented and mysterious ways every few months?- simon willison (1/2)
https://arstechnica.com/information-technology/2023/07/is-chatgpt-getting-worse-over-time-study-claims-yes-but-others-arent-sure/
Study claims ChatGPT is losing capability, but some experts aren’t convinced

Either way, experts think OpenAI should be less opaque about its AI model architecture.

Ars Technica
Show threadmmitchell_ai
@benjedwards But I also want to note Jiahao Chen @jiahao and Anima's insightful thoughts here about RLHF destroying the calibration of base models and the potential degradation of performance being a function of the bias/variance tradeoff RLHF plays into.
Jul 19, 2023 at 11:50pmWeb