Is there something like mongodb/couchdb that does the sqlite philiosophy of an in-process database that's just saved out to a file?

I need to convert a python program to store some data in a database and I'd rather not deal with SQL.

I mean I could always build a small wrapper around an SQLite database I stuff JSON into

but I'd rather not have to

I just stared at my database schema and went "I don't really want to write another INSERT (...) INTO TABLE_BLAH VALUES(?,?,?); statement. I've done my time

@foone

It could be worse, you could be working with mnesia and erlang. (Which I love, don't get me wrong, it's just a really weird db.)

@foone That's why perl 3 gave us dbmopen()! https://perldoc.perl.org/functions/dbmopen
dbmopen - Perldoc Browser

@foone This is the thinking that leads to the eldritch horrors of ORM frameworks.
@darkling yeah I messed with those once in college and my reaction was "why would I do this, I can just write SQL?"

@foone I keep getting ads for some AI BS with a tagline like "write SQL you'll love" and just, what? No. Fuck off.

I write my own SQL tysvm. And it's good SQL that does the job as efficiently as the gods-awful database I work with allows.

I reduced overhead by 99.6% using SQL that I wrote.

I just detest this trend. It's mind numbing.

... got a bit ranty there, sorry.

@darkling @foone I don't think so, in better languages it leads to things like these. The sheer instability of Python is probably why no one has bothered.

Persistent objects predate SQL as a concept.

"why would I do this, I can just write SQL?"

As for why? It can outperform SQL<->object conversion (including when you do it manually with optimizations an ORM couldn't safely do) by magnitudes when done well.

GitHub - CodyReichert/awesome-cl: A curated list of awesome Common Lisp frameworks, libraries and other shiny stuff.

A curated list of awesome Common Lisp frameworks, libraries and other shiny stuff. - CodyReichert/awesome-cl

GitHub

@foone Would rocksDB work for what you want?

https://rocksdb.org/

RocksDB | A persistent key-value store

RocksDB is an embeddable persistent key-value store for fast storage.

RocksDB
@foone just write JSON to disk and be done with it then? Hard to know what your constraints are.
@foone https://github.com/Jwink3101/jsonlitedb is one example of roughly that wrapper, although I’ve not used it (I have my own based on some of the ideas in my post at https://dgl.cx/2020/06/sqlite-json-support). I actually mostly use key/value DBs where I can (often leveldb, as it’s simple and has cross language implementations, but the other ones people have pointed out are reasonable choices too).
GitHub - Jwink3101/jsonlitedb: SQLite3-backed JSON document database with support for indices and advanced queries

SQLite3-backed JSON document database with support for indices and advanced queries - Jwink3101/jsonlitedb

GitHub
@dgl ooh I think this might be exactly what I need, thanks

@foone

LMDB : the key value store backing OpenLDAP. Stood out for me as one of the better choices when looking for something to manage SNOMED CT (a big medical code set).

https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database

Lightning Memory-Mapped Database - Wikipedia

@foone

Even better, it does have a Python binding

https://github.com/jnwatson/py-lmdb

GitHub - jnwatson/py-lmdb: Universal Python binding for the LMDB 'Lightning' Database

Universal Python binding for the LMDB 'Lightning' Database - jnwatson/py-lmdb

GitHub

@foone berkeley db / gdbm / tokyocabinet / LDBM / etc.

Python's "dbm" module actually has a sqlite backend!

@foone duckdb does this, but it's basically a client for json and parquet and the like that is data warehouse/lake oriented.
@foone DuckDB ? Or just Sqlite with the JSON1 extension might be enough ?
@foone there is couchdb-embedded

@foone none that we've ever seen. it's a lot of work to provide that abstraction (the sqlite source is a good read, it goes into some depth)

well, notionally you could go all the way back to berkeleydb, but other than that

@foone Just pickle a dict. You will surely not regret pickling your dict. /s
shelve — Python object persistence

Source code: Lib/shelve.py A “shelf” is a persistent, dictionary-like object. The difference with “dbm” databases is that the values (not the keys!) in a shelf can be essentially arbitrary Python o...

Python documentation

@foone It may be over engineered for your purposes, but maybe automerge could work. Automerge can work like a document database and it has a file storage adapter. It’s meant for sharing data between multiple clients, so it may have a lot of stuff you don’t need. It’s written in rust though the first class support seems to target javascript.

https://automerge.org/

Automerge

Automerge is a library for building collaborative, local-first applications.

@foone The question is: what operations do you need on this database.
@foone I think I’ve written several iterations of exactly this