ArneBabToday I learned that #CERN has been using #HTCondor for data processing since 2016.
https://htcondor.org/featured-users/2025-12-03-cern-stresstests-htcondor.html
Michał SzczepanikA new post on DataLad blog: sharing my experiences from implementing a DataLad workflow, inspired by the existing "FAIRly big" paper, to cut and publish conference videos, on a cluster.
Looking back, everything seems streamlined and logical, but getting there involved discovering the fine details of DataLad, git-annex, HTCondor, bash (and also Matroska metadata and video codecs). Hope it's an useful take.
https://blog.datalad.org/posts/fairly-big-video/
#datalad #gitAnnex #HTCondor #metadata #workflow #distribits
Ben Cotton (he/him)Suddenly remembering the time I tried to write an #HTCondor file transfer plugin for S3. It didn't work because the underlying code did not expect anything except [a-z] in the protocol.
Generic white guy SysAdmin@drmorr just learning. My SysAdmin work regularly sits at the intersection of middleware tech-stacks, so trying to understand what's going on. Most of my time is in batch compute ( #Slurm , #HTCondor ). But I keep hearing #CloudNative will eat the world, so am curious if it can properly what the tools I already use can do. 🤨
Generic white guy SysAdmin@vsoch Now to figure out specify a desired container programmatically during #HTCondor job submission. https://htcondor.readthedocs.io/en/latest/users-manual/container-universe-jobs.html
Bouncing back to #HTCondor Workshop material, here is my talk on #EDIA issues in Research Computing. Response was... mixed, but given the scale of the problem, this was expected. 😣
https://indico.cern.ch/event/1274213/sessions/510851/#20230921
https://indico.cern.ch/event/1274213/sessions/510851/#20230921
HTCondor Workshop Autumn 2023
Dear HTCondor Community,See last update about the whereabouts here: https://indico.cern.ch/event/1274213/page/29408-accommodation-local-transportWe are very pleased to announce that the 2023 European HTCondor Workshop will be held from Tuesday 19th September to Friday 22nd September, at IJCLab in Orsay, France.To allow ease of travel to and from the venue the meeting will start at on Tuesday morning and run until lunchtime on the Friday.Registration is now open.There will be a small workshop...
Brian B is back again to talk about the wonders of managing data transfers with #HTCondor tools and being wary of destroying #NFS servers.
https://indico.cern.ch/event/1274213/contributions/5571140/
https://indico.cern.ch/event/1274213/contributions/5571140/
HTCondor Workshop Autumn 2023
Dear HTCondor Community,See last update about the whereabouts here: https://indico.cern.ch/event/1274213/page/29408-accommodation-local-transportWe are very pleased to announce that the 2023 European HTCondor Workshop will be held from Tuesday 19th September to Friday 22nd September, at IJCLab in Orsay, France.To allow ease of travel to and from the venue the meeting will start at on Tuesday morning and run until lunchtime on the Friday.Registration is now open.There will be a small workshop...
And to wrap up the session before lunch, Brian Bockelman gives us a Token Taxonomy, with respect to #HTCondor. I appreciate it because #authentication & #authorization still confuse 😖 the heck out of me.
https://indico.cern.ch/event/1274213/contributions/5571155/
https://indico.cern.ch/event/1274213/contributions/5571155/
HTCondor Workshop Autumn 2023
Dear HTCondor Community,See last update about the whereabouts here: https://indico.cern.ch/event/1274213/page/29408-accommodation-local-transportWe are very pleased to announce that the 2023 European HTCondor Workshop will be held from Tuesday 19th September to Friday 22nd September, at IJCLab in Orsay, France.To allow ease of travel to and from the venue the meeting will start at on Tuesday morning and run until lunchtime on the Friday.Registration is now open.There will be a small workshop...
Very engaging talk by Peter Couvares about #IGWN data management issues, particularly the inscrutable errors with data transfer problems on #HTCondor systems. #CVMFS #OSDF
https://computing.docs.ligo.org/guide/
https://indico.cern.ch/event/1274213/contributions/5571138/
https://computing.docs.ligo.org/guide/
https://indico.cern.ch/event/1274213/contributions/5571138/
LOTS of discussion about the job life cycle and what are the reponsibilities of the main #HTCondor daemons. This can get quite complicated with multi-pool & ephemeral clusters.