The #SnakemakeHackathon2026 has ended, we are still preparing our preprint release. But, our host has prepared a note on their homepage: https://go.tum.de/946236 🥳

#Snakemake #ReproducibleComputing

RE: https://fediscience.org/@snakemake/116295568336688286

This is a big step forward: The SLURM plugin for Snakemake now supports so-called job arrays. These are cluster jobs, with ~ equal resource requirements in terms of memory and compute resources.

The change in itself was big: The purpose of a workflow system is to make use of the vast resources of an HPC cluster. Hence, jobs are submitted to run concurrently. However, for a job array, we have to "wait" for all eligible jobs to be ready. And then we submit.

To preserve concurrent execution of other jobs which are ready to be executed, a thread pool has been introduced. In itself, I do not see job arrays as such a big feature: The LSF system profited much more from arrays than the rather lean SLURM implementation does.

BUT: the new code base will ease further development to pooling many shared memory tasks (applications which support no parallel execution or are confined to one computer by "only" supporting threading). Until then, there is more work to do.

#HPC #SLURM #Snakemake #SnakemakeHackathon2026 #ReproducibleComputing #OpenScience

Did you know? We started to gather cluster profiles for Snakemake users in a little repo: https://github.com/snakemake/snakemake-cluster-profiles

They should serve as a template for others.

#HPC #Snakemake #ReproducibleComputing #SnakemakeHackathon2026

RE: https://fediscience.org/@snakemake/116222696140712833

What a week at the #SnakemakeHackathon2026 !

What a wonderful week with wonderful people!

We were pretty productive and this #Snakemake release is just the peak of it. The list of features, bug fixes, performance improvement and additional documentation is so long — our little announcement robot cannot display it all. Even here on FediiScience with its 1500-character limit!

#ReproducibleComputing #OpenScience

The #SnakemakeHackathon2026 just started! Lots of interesting topics. If you are curious: #SnakemakHackathon or #SnakemakeHackathon2026 are your hashtags to follow. 😉

#reproducibleComputing #OpenScience #Datananalysis

What do you see here? This is an example knowledge graph describing a #Snakemake analysis workflow. You see the workflow description, a linked data set and a linked report.

All work done to boost #HPC user support for those conducting their workflows on HPC systems (you can run Snakemake on other platforms, too).

My to-do list:
- an assertion template for workflows: ✅
- another for reports: ✅ (simple datasets are already in the #nanopub verse)
- a plugin to gather software metadata and publish as a nanopub ❌ (half done: #SnakemakeHackathon2026 )

Kudos to @nanopub / @tkuhn and @johanneskoester - without them this pursuit would (have been) futile! And my feeling is that @fbartusch will play an important role in any further development ...

#OpenScience #ReproducibleComputing

I want to reach out: I have this pending release for the SLURM executor (https://github.com/snakemake/snakemake-executor-plugin-slurm/pull/412 ). It implements better error feedback (in case of hardware failures and otherwise). It would need some thorough checking, and I cannot provoke too many hardware failures. Is anyone working on an older cluster? 😉

Feedback (as issues) would be appreciated. Also nice, if you tell me it is working, here.

#Snakemake #HPC #SLURM #ReproducibleComputing

The last blog post I wrote was about Life Science Support on HPC clusters. Honestly? It was more of a rant. Not a good blog post.

So, someone suggested I delete it, which I did. It took me a long time to recover. Now, I have re-written this blog post. I think it is better. I weighted every phrase. It still has some rant flavour.

Here it is: https://blogs.fediscience.org/rupture-de-catenaire/2025/12/15/empowering-life-scientists-what-hpc-support-should-look-like/

The upcoming articles will have more of an example character. But I still will not be able to update on a regular basis.

#Bioinformatics #HPC #ReproducibleComputing #Snakemake #Nextflow

Just a minute ago, I accidentally discovered Workflowhub (https://about.workflowhub.eu/) when browsing through a few Nanopubs. I was not aware of this repository.

A first thought: "Yet another repository to register #FAIR materials? Seriously?" Then again, I am all in favour of #reproducibleComputing and want to see more of it.

On the other hand: transparency in our work is central to the #OpenScience idea. NOT cluttering a file space so that we can be transparent in our work is paramount.

Lately, we have seen #nanopub, #ROcrates, and apparently WorkflowHub, too. (This is by no means a comprehensive list.) I wonder: Are we getting too many solutions to keep up-to-date? Are we risking frightening young researchers with the high standards we are setting? We are by no means where we want to be in terms of #transparency and #reproducibility in research, or are we?

Note, that I understand how competing systems and tool sets arise. Still, my nonrhetorical questions (1 & 2) stand.