Hacker NewsStreamingLLM: tiny tweak to KV LRU improves long conversations
https://news.mit.edu/2024/new-way-let-ai-chatbots-converse-all-day-without-crashing-0213
#ycombinator #Song_Han #Guangxuan_Xiao #ChatGPT #large_language_models #ai_chatbots #Transformer_architecture
https://news.mit.edu/2024/new-way-let-ai-chatbots-converse-all-day-without-crashing-0213
#ycombinator #Song_Han #Guangxuan_Xiao #ChatGPT #large_language_models #ai_chatbots #Transformer_architecture
A new way to let AI chatbots converse all day without crashing
Researchers developed a technique that enables an AI chatbot like ChatGPT to conduct a day-long conversation with a human collaborator without slowing down or crashing, no matter how much text the conversation involves.