If you are a #dplyr user and need to work with large data systems that are not supported in #rstats, you may want to give #IbisProject a try.
We wrote a guide: "Ibis for dplyr users" to help get you started:
https://ibis-project.org/ibis-for-dplyr-users/
If you have suggestions to improve this guide, please reach out on GitHub (https://github.com/ibis-project/ibis)
In the afternoon we will also show how to use #IbisProject & #DuckDB to efficiently "sessionize" an event log to extract time-to-failure training data implicitly encoded in heartbeat-style events.
This tutorial will be very hands-on but also packed with methodological insights.
Hoping to meet you at #JupyterCon!
Congratulations to the Ibis Project, which hit 280K downloads in October 2022!
Looking forward to the new features coming with Ibis 4.0 early next year - "Version 4.0 will also bring the read function, allowing users to read files using the default backend (currently DuckDB) without spinning up a connection."
@damoncroberts you might want to have a look at https://ibis-project.org/ which offers a lazy dataframe Python API with duckdb as the default backend but also implements backends for many SQL databases (clickhouse, postgresql, sqlite, bigquery...) or dataframe libraries (pandas, dask, PySpark, polars...)
François Michonneau
Olivier Grisel
earle
Olivier Grisel