Even Rouault2d ago
Your yearly remainder that @sozip exists (even if I've lost the credentials for the account. oops) and you can use it to offer a cloud native experience (and even a local one) to consume huuuge files compressed as ZIP. More at https://github.com/sozip/sozip-spec
Show threadChris Toney2d ago

@EvenRouault @sozip I use it all the time and share with colleagues, mainly from gdalraster::addFilesInZip(). We used it yesterday on a set of huge CSVs of x, y, model value (I wasn't involved in deciding to create those CSVs but they're GDAL-readable and compressed now :)

https://firelab.github.io/gdalraster/reference/addFilesInZip.html

Create/append to a potentially Seek-Optimized ZIP file (SOZip) — addFilesInZip

addFilesInZip() will create new or open existing ZIP file, and add one or more compressed files potentially using the seek optimization extension. This function is basically a wrapper for CPLAddFileInZip() in the GDAL Common Portability Library, but optionally creates a new ZIP file first (with CPLCreateZip()). It provides a subset of functionality in the GDAL sozip command-line utility (https://gdal.org/en/stable/programs/sozip.html). Requires GDAL >= 3.7.

Show threadEven Rouault2d ago
@ctoney @sozip The SOZIP optimization will not bring anything to CSV-in-ZIP, since a CSV is consumed only by sequential reading, which regular ZIP is good at.
Quite funny that half of my posts about SOZIP are to explain how useless it is for various mentionned used cases.
Huge GeoPackage in ZIP is mostly the canonical use case.
Show threadautra2d ago
@EvenRouault @ctoney @sozip can I consume it with http-range requests? 😍
Show threadEven Rouault
@autra @ctoney @sozip Drop (the URL to) https://data.3dbag.nl/v20250903/3dbag_nl.gpkg.zip in QGIS and enjoy (https://3dbag.nl/en/download)
Apr 4 at 10:10pmWeb