With quite a lot of both physical machines and even more virtual machines it can be tricky to keep file permissions on shared storage like Ceph in sync. Adding two users in the wrong order on a node can give them the wrong UIDs. This was less of a problem when I accessed files via CIFS/Samba but now I only do that for Windows-machines. Linux-machines use CephFS so UIDs and GIDs have to be correct on all nodes. Therefor I use FreeIPA to keep this stuff in sync. I spent what felt like months trying to get a multi-master LDAP cluster to work – though it might have been a weekend – but gave up.
Turns out FreeIPA has built-in support for replication of data. So I have freeipa01 running on a Ceph RBD disk allowing me to keep it running even as physical nodes are rebooted. freeipa02 is run on a local disk on one of my servers and it’s there in case Ceph stops working. FreeIPA also handles DNS which it replicates down to Bind daemons on three virtual machines that are also running on local storage. The DNS function NOTIFY is used to keep replication lag down when syncing down to the bind slaves.
What else does FreeIPA do? Kerberos and single-sign-on, currently only used for SSH. Certificate issuing, standard centralized PAM log on against the central user directory. I’ve come to use it quite a lot.