Hanno ReinI've been thinking about using lustre for my cluster to improve the IO performance. But the documentation is really not that great... For example, I cannot seem to find out if I can install it on debian or if it needs to be some sort of RHEL. Does anyone know of a good up-to-date tutorial or howto? #HPC
Dr. Juande Santander-Vela@hannorein Using also #Lustre #LustreFS #IO #IOOptimization #HPC to raise the amount of eyes on your request.
Janne Moren@hannorein
You can definitely install and run it on Debian and Ubuntu. "AI" clusters tend to use Ubuntu, and Lustre is a common file system on such machines. DDN (who maintain Lustre) also uses it in their products.
Edit: What about this link? https://wiki.lustre.org/Debian_Install
@jannem Debian Lenny came out 16 years ago….
@hannorein
Has the process meaningfully changed though?
Has the process meaningfully changed though?
@jannem maybe not! I guess I just need to give it a try.
loStronzoRocco@hannorein Do you happen to know what applications are primarily run on your cluster? Lustre can be incredibly useful, but a lot of “modern” applications actually cripple Lustre’s performance and subsequently will cripple the cluster.
@desantis I’m not sure what you are referring to. What is a modern application? And what is the alternative to lustre? My main goal is to do the IO over infiniband rather than Ethernet.
@hannorein Apologies for the vagueness.! Applications such as Quantum Espresso (and its various suite of tools), PyTorch workflows that aren’t built with MPI-IO in mind, Berkeley GW, etc.; just a few called from recent memory. If your main goal is to use Infiniband as the transport medium instead of Ethernet, you have simple options such as exporting NFS over the Infiniband network or BeeGFS (think NFS and Lustre). It just depends on the workflow.
@desantis Not so keen on the license for BeeGFS. By NFS over Infiniband you mean via IPoIB?
@hannorein BeeGFS is free, so no need to pay unless you want their (ThinkParq) support. And, yes - NFS using IPoIB!
@hannorein Well, crap. I didn’t fully read the license comment. From a sysadmin point of view - BeeGFS is simple to deploy and works, 100% I didn’t read the license!
@desantis I haven't tried NFS over IPoIB, but I suspect lustre might perform better.