BorutzkiJun 2

If you ever ask why it’s worth updating #Python dependencies from time to time, #Pydantic serves as a nice example with the performance boost introduced. I actually could see it, given the complexity of models I deal with on a daily basis.

> Recent memory usage optimizations are most relevant for projects with lots of models, particularly those with nested/reused models. In these cases, you can expect a 2-5x reduction in memory usage.

#DailyPythonista
https://pydantic.dev/articles/pydantic-v2-11-release

Pydantic v2.11 | Pydantic

Pydantic v2.11 release highlights

Julien MaupetitMay 28

🐍 Loading #pydantic models from #JSON without running out of memory — by @itamarst

TIL #python slots usage 🙏

https://pythonspeed.com/articles/pydantic-json-memory/

Loading Pydantic models from JSON without running out of memory

Pydantic’s JSON loading uses a huge amount of memory; here’s how to reduce it.

Python⇒Speed
 Hacker NewsMay 25
Design Pressure: The Invisible Hand That Shapes Your Code
https://hynek.me/talks/design-pressure/
#ycombinator #Python #OOP #object_oriented_code #software_design #design_pressure #ORM #Pydantic
Design Pressure

Ever had this weird gut feeling that something is off in your code, but couldn’t put the finger on why? Are you starting your projects with the best intentions, following all best practices, and still feel like your architecture turns weird eventually?

Hynek Schlawack
Gonçalo ValérioMay 23

"Loading Pydantic models from JSON without running out of memory"

https://pythonspeed.com/articles/pydantic-json-memory/

#python #pydantic

Loading Pydantic models from JSON without running out of memory

Pydantic’s JSON loading uses a huge amount of memory; here’s how to reduce it.

Python⇒Speed
Python RennesMay 23
Si vous utilisez #Pydantic, et que vous êtes confronté·es à des problèmes d'allocation mémoire lors de chargement de #JSON volumineux, voici une exemple de comment contourner le problème, en utilisant la bibliothèque #Python ijson.
https://pythonspeed.com/articles/pydantic-json-memory/
Loading Pydantic models from JSON without running out of memory

Pydantic’s JSON loading uses a huge amount of memory; here’s how to reduce it.

Python⇒Speed
GripNewsMay 23
🌖 從 JSON 加載 Pydantic 模型，避免記憶體不足
➤ 解決 Pydantic 解析大型 JSON 檔案的記憶體瓶頸
✤ https://pythonspeed.com/articles/pydantic-json-memory/
本文探討了使用 Pydantic 從大型 JSON 文件中載入數據時，記憶體使用量過高的問題。作者介紹了兩種降低記憶體佔用的方法：一是使用增量 JSON 解析器 ijson，二是利用 `dataclass` 搭配 `slots`，以更有效率的方式儲存物件。透過實測，作者展示了不同方法的記憶體使用量差異，並建議 Pydantic 開發者考慮在未來版本中優化記憶體管理。
+ 這篇文章很有幫助，我一直遇到 Pydantic 解析大型 JSON 的問題，現在知道可以嘗試 ijson 和 dataclass 了！
+ 作者提出的解決方案很實用，而且提供了具體的程式碼範例，方便直接應用。希望 Pydantic 未來能內建這些優化功能。
#Python效能 #Pydantic #JSON解析 #記憶體管理
Loading Pydantic models from JSON without running out of memory

Pydantic’s JSON loading uses a huge amount of memory; here’s how to reduce it.

Python⇒Speed
N-gated Hacker NewsMay 22
🚀 Ah, the classic dilemma: how to load massive #JSON files into #Pydantic without summoning the out-of-memory demon. 😅 Clearly, the solution is to write yet another verbose article pretending it's rocket science, because, you know, common sense isn't enough. 🧠✨
https://pythonspeed.com/articles/pydantic-json-memory/ #Loading #OutOfMemory #JSONFiles #TechHumor #DataProcessing #HackerNews #ngated
Loading Pydantic models from JSON without running out of memory

Pydantic’s JSON loading uses a huge amount of memory; here’s how to reduce it.

Python⇒Speed
Hacker NewsMay 22

Loading Pydantic models from JSON without running out of memory

https://pythonspeed.com/articles/pydantic-json-memory/

#HackerNews #Pydantic #JSON #Memory #Optimization #Memory #Management #Python #Programming

Loading Pydantic models from JSON without running out of memory

Pydantic’s JSON loading uses a huge amount of memory; here’s how to reduce it.

Python⇒Speed
Mathias PanzenböckMay 20
What does #pydantic have to do with #AI now!? And which version to use anyway? pydantic, pydantic2, or pydantic3? My distribution only has python3-pydantic. #Python
scyMay 14

Did some benchmarking of how fast #Python (3.13) w/ async #httpx and #Pydantic could query a JSON API vs. #GoLang (both including unmarshalling to native types and some regex validation).

While Python could do about 2.5k requests per second against localhost when running singlecore, Go did 10k.

However, not only is Go "only" 4× as fast, these differences quickly shrink to 30 % when you introduce an actual network between the hosts, and become irrelevant with rate limiting & DDoS protection.