PonzianiMay 13, 2024

Question on chown in parallel

https://sh.itjust.works/post/19208347

Question on chown in parallel - sh.itjust.works

Hello I am seeking a simple solution to running a list of “chown -R” <mydir>" commands in script.sh [http://script.sh] It takes a long time to sequentially execute all of these chown commands recursively because the directories have so many files. I want to be able to tackle the root level directories in parallel to speed things up. I imagine there must be a simple way to do this while keeping the list of commands in a single file. xargs and some of the other things I saw online looked like bad fits or would be over engineering this problem.

Show threadricecakeMay 13, 2024

find <directory> -type f -print0 | xargs -0 -P 4 -n 500 chown

That should find every file in your directory recursively, pass it to xargs, which will then spawn up to four processes which will each call chown on up to 500 files, and it’ll make additional processes as they finish.

In general though, if you regularly need to chown that many files, it’s better to find a way to make sure they have the right ownership from the start.

Show threadPonzianiMay 13, 2024
Thanks for adding that tidbit at the end. The reason that permissions get out alignment is due to different non-privledged accounts (for saftey) will write or copy files somewhat regularly from outside of the main system. I am the furthest thing from a linux expert so maybe you would have a recommendation or better insight after explaining that? This necessitates changing the owner and permissions regularly, especially when I need to interact with the files adhoc and have to wait for my script to run and complete.
Show threadricecakeMay 13, 2024

Hrm, you might look into file ACLS.

serverfault.com/…/linux-setfacl-set-all-current-f…

serfacls is a command that lets you make user (or other) level permissions changes outside of the usual ownership semantics.
So you could for example do something like setfacl -d -R u:<your username>:rwx /the/very/top/directory/ That should make it so that newly created files and folders have a default acl allowing you access. Run it again with the m flag to modify existing files.
It’ll take a minute to loop through everything, but you should only have to run it once so it’s not a recurring issue.

I hope that gets you what you need. :)

linux/setfacl - Set all current/future files/directories in parent directory to 775 with specified owner/group

I have a directory called "members" and under it there are folders/files. How can I recursively set all the current folders/files and any future ones created there to by default have 775 permission...

Server Fault
Show thread𝕽𝖚𝖆𝖎𝖉𝖍𝖗𝖎𝖌𝖍

facls are the shizzle. Seriously. I’m really not sure why people use chmod at all anymore. It’s fewer characters, maybe?

For OP, a tool like fd can turn a script into a very short one-liner; and unlike find, it runs execs in parallel by default:

sudo -E me=$(id -un) fd . \<path> -t f -x setfacl -m u:${me}:rw '{}' sudo -E me=$(id -un) fd . \<path> -t d -x setfacl -m u:${me}:rwx '{}'

will do the thing in parallel; the first line, for all the files; the second, for all the directories.

As others have said, if you’re needing to do this a lot, it’s best to fix whatever is setting the perms in the first place, or as @ricecake and others have said, set the perms/facls to be sticky so they get inherited.

facls are far more expressive than base perms, and are supported by every major, current, Linux filesystem. Not FAT, but ACLs on FAT FSes are all f’ed up anyway.

May 13, 2024 at 2:47pmWeb
Show threadricecakeMay 13, 2024

My guess is that it’s not “the standard” for managing file ownership stuff, since it doesn’t manage ownership. As a result, they’re shown less often in tutorials and tool output.
The ownership semantics still needs to exist and get managed, and so a lot of less sophisticated software will just check ownership, not it’s actual ability to access.

Tools and capabilities come quick, but the ecosystem as a whole moves glacially slow. Often that’s good, because it means user land APIs and programs don’t often just fail for no good reason, which creates the stability that makes it popular and useful. It also makes it painful to get “new stuff” into widespread use, where “new” means less than 30 years old.
You see the same thing with selinux. It’s fine now! But it’s still scary. And we’ll finally have btrfs as the standard in 2040 I’ll wager.