Another post on #Quansight PBC blog: "BLAS/LAPACK #packaging"

https://labs.quansight.org/blog/blas-lapack-packaging

"""
#BLAS and #LAPACK are the standard libraries for linear algebra. The original implementation, often called Netlib LAPACK, developed since the 1980s, nowadays serves primarily as the origin of the standard interface, the reference implementation and a conformance test suite. The end users usually use optimized implementations of the same interfaces. The choice ranges from generically tuned libraries such as OpenBLAS and BLIS, through libraries focused on specific hardware such as Intel® oneMKL, Arm Performance Libraries or the Accelerate framework on macOS, to ATLAS that aims to automatically optimize for a specific system.

The diversity of available libraries, developed in parallel with the standard interfaces, along with vendor-specific extensions and further downstream changes, adds quite a bit of complexity around using these libraries in software, and distributing such software afterwards. This problem entangles implementation authors, consumer software authors, build system maintainers and distribution maintainers. Software authors generally wish to distribute their packages built against a generically optimized BLAS/LAPACK implementation. Advanced users often wish to be able to use a different implementation, more suited to their particular needs. Distributions wish to be able to consistently build software against their system libraries, and ideally provide users the ability to switch between different implementations. Then, build systems need to provide the scaffolding for all of that.

I have recently taken up the work to provide such a scaffolding for the Meson build system; to add support for BLAS and LAPACK dependencies to Meson. While working on it, I had to learn a lot about BLAS/LAPACK packaging: not only how the different implementations differ from one another, but also what is changed by their respective downstream packaging. In this blog post, I would like to organize and share what I have learned.
"""

#CondaForge #Debian #Fedora #Gentoo

New on #Quansight PBC blog: Python Wheels: from Tags to Variants

https://labs.quansight.org/blog/python-wheels-from-tags-to-variants

"""
Many #Python distributions are uniform across different Python versions and platforms. For these distributions, it is sufficient to publish a single wheel that can be installed everywhere. However, some packages are more complex than that; they include compiled Python extensions or binaries. In order to robustly deploy these software on different platforms, you need to publish multiple binary packages, and the installers need to select the one that fits the platform used best.

For a long time, Python wheels made do with a relatively simple mechanism to describe the needed variance: Platform compatibility tags. These tags identified different Python implementations and versions, operating systems, and CPU architectures. Over time, they were extended to facilitate new use cases. To list a couple: PEP 513 added manylinux tags to standardize the core library dependencies on GNU/Linux systems, and PEP 656 added musllinux tags to facilitate Linux systems with musl libc.

However, not all new use cases can be handled effectively within the framework of tags. To list a few:

• The advent of GPU-backed computing made distinguishing different acceleration frameworks such as NVIDIA CUDA or AMD ROCm important.
• As the compatibility with older CPUs became less desirable, many distributions have set baselines for their binary packages to x86-64-v2 microarchitecture level, and Python packages need to be able to express the same requirement.
• Numerical libraries support different BLAS/LAPACK, MPI, OpenMP providers, and wish to enable the users to choose the build matching their desired provider.

While tags could technically be bent to facilitate all these use cases, they would grow quite baroque, and, critically, every change to tags needs to be implemented in all installers and package-related tooling separately, making the adoption difficult.

Facing these limitations, software vendors have employed different solutions to work around the lack of an appropriate mechanism. Eventually, the #WheelNext initiative took up the challenge to design a more robust solution.
"""

#packaging

Czas na ważne ogłoszenie parafialne: dziś zacząłem pracę w #Quansight Labs.

Chciałbym przy tej okazji podziękować wszystkim osobom, które mnie wspierały w ostatnich latach. Wasza pomoc wiele dla mnie znaczyła!

Oznacza to również, że nie będę miał już tyle samo czasu dla #Gentoo. Myślę jednak, że udało mi się nie zostawić żadnych dużych projektów niedokończonych.

Time for an important announcement: today I've started working for #Quansight Labs.

I would like to take this opportunity to thank all of you who supported me over the past years. Your help was greatly appreciated!

That said, this also means that I'm going to spend less time on #Gentoo than I used to. I think I've managed not to leave any loose ends, and finish all the big efforts in time.

I am excited to read about numpy_quaddtype, a project to include quad precision in numpy. The standard precision in numpy (and most other places) is double precision: numbers are stored in 64 bits and the precision is about 16 decimal digits. This is usually enough but not always.

Numpy does have longdouble, which may or may not increase precision, depending on your platform, but even if it does, the increase is very modest. If I need more precision, I typically use FLINT, but that is meant for super high precision and rigorous computations. It will be very good to have another tool.

More details in this blogpost: https://labs.quansight.org/blog/numpy-quaddtype-blog

#FloatingPoint #numpy #quansight #NumericalAnalysis

I'm overdue for an #introduction. My name is Dhavide; I'm a Canadian mathematician & computer scientist. Once upon a time, I earned my living as a professor. I currently work as the Director of Education & a Principal Solution Architect for #Quansight LLC.

My main social media outlet was Twitter until recently. I'm excited by the opportunity the Fediverse provides to reassess ways for technologies to serve people—and democracy—rather than advertisers.

(1/4)