Rackslab🚀 Slurm-web v6.1.0 is out!
New core allocation visualization, optional RacksDB lazy loading, expanded SLES/openSUSE docs, and dependency/security fixes.
Warm thanks to NVIDIA and Zuse Institute Berlin (ZIB) for their contributions ❤️
Andreas Skau
HabrОт майнинга на попутном газе к AI-фабрикам: история Crusoe
У AI-индустрии есть серьезная проблема: как развернуть вычислительную инфраструктуру раньше и быстрее (да еще и дешевле) конкурентов? Основной дефицитный ресурс сейчас — электричество, а не чипы или их компоненты, как вы могли предположить. Техногиганты думают, где поставить стойки, чем их охлаждать, но главное, где взять энергию, чтобы питать всю AI-систему. И у одного стартапа из Денвера есть нестандартное решение — портативные модульные AI-дата-центры, которые можно размещать в самых нестандартных условиях. Компания пришла в ИТ из мира крипты: изначально она вела деятельность установкой майнинг-машин, которые брали энергию от попутного газа на нефтяных вышках. Сегодня я расскажу вам о компании Crusoe — которая крайне нестандартно превращает энергию в вычислительную мощность. Разберем их бизнес-модель и поймем, что такое вертикально интегрированная AI-инфраструктура.
https://habr.com/ru/companies/ru_mts/articles/1022116/
#Crusoe #AIинфраструктура #датацентры #GPUоблако #облачные_вычисления #inference #Kubernetes #Slurm #edge_computing #энергетика
Christian MeestersRE: https://fediscience.org/@snakemake/116295568336688286
This is a big step forward: The SLURM plugin for Snakemake now supports so-called job arrays. These are cluster jobs, with ~ equal resource requirements in terms of memory and compute resources.
The change in itself was big: The purpose of a workflow system is to make use of the vast resources of an HPC cluster. Hence, jobs are submitted to run concurrently. However, for a job array, we have to "wait" for all eligible jobs to be ready. And then we submit.
To preserve concurrent execution of other jobs which are ready to be executed, a thread pool has been introduced. In itself, I do not see job arrays as such a big feature: The LSF system profited much more from arrays than the rather lean SLURM implementation does.
BUT: the new code base will ease further development to pooling many shared memory tasks (applications which support no parallel execution or are confined to one computer by "only" supporting threading). Until then, there is more work to do.
#HPC #SLURM #Snakemake #SnakemakeHackathon2026 #ReproducibleComputing #OpenScience
Tyler SmithA few #slurm tidbits:
Total submitted jobs per user, sorted:
```
squeue | sed 's/ \+/\t/g' | cut -f5 \
| sort | uniq -c | sort -hr
```
Running jobs per user:
```
squeue | grep ' R ' | sed 's/ \+/\t/g' \
| cut -f5 | sort | uniq -c | sort -hr
```
Pending jobs per user:
```
squeue | grep ' PD ' | sed 's/ \+/\t/g' \
| cut -f5 | sort | uniq -c | sort -hr
```
As for the little executor plugin for the #SLURM batch system (for which I promised a release supporting array job support) ... Well, only a little bug fix release could be accomplished: https://github.com/snakemake/snakemake-executor-plugin-slurm/releases/tag/v2.5.4
Unfortunately, I wanted to use the common #Snakemake logo without the letters "#HPC" and missed one entry. So our announcement bot did not work.
Anyway, a faulty file system connection kept me from debugging the new feature. Stay tuned. It is almost ready.
Finally, some personal progress: Thanks to @fbartusch a bug of the #SLURM executor plugin for Snakemake was fixed (dealing with nested quoting). A release is upcoming.
And: I generated my first (still faulty) test #nanopub from Snakemake 🥳
This cannot be:
I am trying to compile a few stats for the #Snakemake executor plugin for #SLURM on #HPC systems. Preparing for a lighting talk at the #SnakemakeHackathon2026
PyPi: 20,000 downloads last month
BioConda: > 60,000 total (aggregated over all versions)
Impressive as it might be, this is contradictory. PyPi would exceed BioConda by a huge margin.
Does anyone know how to get all-time statistics from either platform? #BioConda or #PyPi?
The #Snakemake plugin for #SLURM on #HPC clusters will support JobArrays, soon:
1057691_1 2dcf44cc-+ rule_map_reads_wild+ 32 COMPLETED 0:0
1057691_2 2dcf44cc-+ 32 RUNNING 0:0
1057691_3 2dcf44cc-+ 32 RUNNING 0:0
1057691_4 2dcf44cc-+ 32 RUNNING 0:0
1057691_5 2dcf44cc-+ 32 RUNNING 0:0
1057691_6 2dcf44cc-+ 32 RUNNING 0:0
Hope to do more during next week's #SnakemakeHackathon2026 / #SnakemakeHackathon
Felix BartuschI had the chance to present a poster about our HPC cluster BinAC 2 at #deRSE26
We're providing computational resources for researchers in Baden-Württemberg working in the fields of Bioinformatics, Astrophysics, Geosciences, Pharmacy and Medical Informatics:
https://wiki.bwhpc.de/e/BinAC2
If you're a researcher at an university in Baden-Württemberg from another field, check out the other HPC clusters bwHPC provides:
https://www.bwhpc.de/cluster.php
Poster on Zenodo:
https://zenodo.org/records/18860391
#hpc #bioinformatics #astrophysics #geosciences #bwhpc #lustre #Slurm
