Tools/mechanisms For The Management Of Access Permissions In Big Filebased Datasets
Hello,
we are currently managing access permissions through classical user-group-others permissions on a multi-petabyte directory tree with partially very deep and broad directories. Projects are represented by directory trees and mapped through GIDs. Lately we had lots of
“singular” permission request (one single user needs access to a single dataset but should not be able to see all other datasets belonging to the same project). We realized, that the UGO model doesn’t scale and is becoming more and more unmanageable.
Can you recommend tools/mechanisms/technologies to overcome the drawbacks of the UGO model? We are thinking about some purely ACL based mechanism (but are open to other ideas). All filesystems in question are mounted via NFSv4 and the clients are (almost) completely CentOS 7.x hsots. Ideally the tool would have some web UI and some kind of
(REST)API which allows us to modify permissions from our inhouse data management application (which does /not/ manage permissions, just the structure of the data). Additionally it should be able to visualize/report permissions in directory.
I wasn’t very successful in googling possible candidates, hence the question to the list.
Cheers frank
4 thoughts on - Tools/mechanisms For The Management Of Access Permissions In Big Filebased Datasets
Well, there are extended ACLs if they’re available in CentOS, when I first worked with them (long ago) they were new (and on a different Distro). I hope support for them has improved. They allow multiple users/groups to be assigned permissions to a file/directory. The problem then was that chmod (and other programs) were not extended-ACL-aware and could over-ride extended ACLs. There was a mechanism to recover from the situation but what it basically came down to was eternal vigilance – the system administrators had to understand (and agree about) extended ACLs and be careful/diligent in applying them. There are hacks which could possibly help (rename chmod and replace it with a script warning about extended ACLs) but, in the final analysis, it’s not a decision to be undertaken lightly (unless the situation has changed dramatically).
Leroy Tennison Network Information/Cyber Security Specialist E: leroy@datavoiceint.com
2220 Bush Dr McKinney, Texas
75070
http://www.datavoiceint.com TThis message has been sent on behalf of a company that is part of the Harris Operating Group of Constellation Software Inc. These companies are listed here
. If you prefer not to be contacted by Harris Operating Group please notify us
. This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged or confidential or otherwise legally exempt from disclosure. If you are not the named addressee, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message.
Thank you. Basically our problem are not the ACLs or their support per se, but that we have to manage a huge number of individual ACLS (several hundred users in more than hundred projects) in multi-petabyte filesystem and still have to keep overview and control. Our problem is more the management side. Effectively we are looking for a tool that helps us manage these permissions and we would accept whatever permissions mechanism this tool uses (UGO/ACLs).
Cheers frank
I want ACLs to work. There’s a real problem to solve, which is that the old user:group rwx Unix permission system doesn’t let you express common wishes like “Angel & Bobby own this file, and groups Cookie and Danish can read and write it, and user Egbert can write it.”
The problem is, ACLs are hidden by default with respect to “ls -l”, and when you do make them visible with getfacl, you now have a complex mental parsing problem to solve before you understand the meaning of the ACL. Add in ACL inheritance and you’ve got a real mess.
Make a facility hidden and complex, and you pretty much guarantee that few will use that facility, and those who do will at times create messes they can’t properly understand. A security mechanism that’s most often underused, misapplied, or both is a bad system.
FOSS is good at solving such problems, so the only way I can see that tools to solve this problem don’t exist is that few actually use ACLs, perhaps because of the reasons above.
Who here uses ACLs to good effect? Are you using more than just getfacl/setfacl to do it?
We use NFSv4 ACLs on Lustre and Isilon filesystems, so we employ nfs4_getfacl and nfs4_setfacl — but all of our work is done on the command line, not via a GUI and larger management tool.
Our best practice is to script up the ACLs so they can be reapplied in case they get deleted or inappropriately changed. My current scripting logic usually writes the desired ACLs to temp files and deploys them in one swoop.
Take the following case:
owner: bob read-write group: boblab read-only group: alicelab target directory: /srv/group/boblab
A skeleton version of the script would look something like this
# define directory-level ACL and write to temp file cat <<__diracl__> /tmp/diracl A::OWNER@:rwaDdxtTnNcCoy A::GROUP@:rwaDxtTnNcy A::EVERYONE@:tncy A:fdg:boblab@domain.com:RWX
A:fdg:alicelab@domain.com:RX
__DIRACL__
# define file-level ACL and write to temp file cat <<__fileacl__> /tmp/fileacl A::OWNER@:rwaDdxtTnNcCoy A::GROUP@:rwaDxtTnNcy A::EVERYONE@:tncy A:g:boblab@domain.com:RWX
A:g:alicelab@domain.com:RX
__FILEACL__
# apply ownership, perms, and ACLs. chown -R bob:boblab /srv/group/boblab chmod -R ug+rw,o-rwx /srv/group/boblab find /srv/group/boblab -type d \
-exec nfs4_setfacl -S /tmp/diracl {} \;
find /srv/group/boblab -type f \
-exec nfs4_setfacl -S /tmp/fileacl {} \;
Once the directory ACLs are applied, any new files created within those directories should inherit the proper ACLs.
—
Paul Heinlein heinlein@madboa.com
45°38′ N, 122°6′ W