LDAP: backup with slapcat vs ldapsearch

Used: openldap-servers-2.4.23-34.el6_5.1.x86_64

Task: create script for crontab to create scheduled database full backup.

1) slapcat - create file in in the default format, Berkeley DB.

2) slapcat can be done while slapd running (if bdb/hdb database used).

3) To restore file after slapcat - must be used slapadd (not ldapadd).

4) slapcat/add doesn't require password.

5) slapadd can be done only when slapd stopped.

Example:

 $ slapcat -f /etc/openldap/slapd.conf -b "dc=db_1" -l db_1_backup.ldif
 $ slapadd -l db_1_backup.ldif

Instead of slapcat/add - let's took a look at ldapsearch/add:

1) ldapsearch - creates file with almost same information as slapcat;

2) ldapadd - can use file from ldapsearch, doesn't require slapd to be stopped;

3) ldapadd/search - requires password.

Example:

 $ ldapsearch -D "cn=root,dc=db_1" -W -b "dc=db_1" "dc=db_1" -LLL > db_1_backup2.ldif
 $ ldapadd -x -D "cn=root,dc=db_1" -W -f db_1_backup2.ldif

So - question is:

1) Am I missing anything in this tools description?

2) What else is difference between ldapadd/slapadd and ladpsearch/slapcat?

Solution 1:

Good summary, some additional points:

• slapcat dumps from whatever the (local) direct storage backend is, it need not be Berkeley (hdb or bdb), it also works with OLC (cn=config). It dumps to LDIF format. (By direct I mean directly managed by OpenLDAP, not for example an SQL backend, even if its stored locally.)
• ldapadd requires that slapd is running, slapadd requires that it is not running
• ldapsearch requires that slapd is running, slapcat doesn't care if it's running with a BDB backend, as you noted

In short:

• slapcat is the way to get a good backup that you can restore quickly, albeit with downtime on the master (you can work around this with various types of replication set up). This is what you should use for a general backup, and pre-upgrade backups.
• ldapsearch (without +) will get you a portable backup you can probably load with little difficulty into any other directory server, but it will only be a viable restore in a simple OpenLDAP set up (no replication, no special overlays, no rewriting), and if you don't care about preserving UUID/create/modify meta-data. You'll need any extra schema files that go along with your data too.
• ldapadd (using its other identity ldapmodify) can be used to easily apply LDAP modifications (object modify, delete and rename) which are not feasible or possible with slapadd/slapcat alone

For most admins the main considerations arise from the slightly different contents of the LDIF in each case, and the requirement for slapd to be running (or not). The more important differences are:

1. slapcat is faster because it simply dumps the database, skipping the LDAP protocol overheads, authentication, access control, object and time limits, overlays; and it does not search according to the LDAP hierarchy.
2. slapadd is faster (again, no LDAP protocol overheads), and in the case that you are restoring a known-good backup you can run in quick mode (-q) to speed up large imports. You can also disable schema checking (-s), though note than small changes in schema or data validation between OpenLDAP versions are not unheard of.
3. slapcat is limited to local databases, it will not cross over to other directories (e.g. with back-ldap, back-meta) the way ldapsearch will. The same applies to slapadd/ldapadd.
4. ldapsearch will return dynamic attributes which are not stored in a backend, e.g. hasSubordinates or those maintained by overlays (e.g. slapo-memberof). You will have problems loading these with ldapadd (e.g. operational attributes with no-user-modification). Rewriting (slapo-rwm) may also distort ldapsearch's view of the directory contents.
5. slapcat includes internal (operational) attributes, if you are using replication these attributes are critical. With replication then you are somewhat less dependent on backups, but if you use ldapadd to reload your master, every object will be recreated by replication (changed entryUUID entryCSN) Although you can include operational attributes by using the special "+" attribute with ldapsearch (or allop overlay), this is not the same thing as slapcat, see the previous point for why this is so. These attributes also include create/modify DN and timestamps, which may be important to some applications.
6. because slapcat does not observe the LDAP hierarchy (implied ordering), there's no guarantee that its data ordering is going to be viable with ldapadd - i.e. even if you strip out the operational attributes, ldapadd can complain because sub-ordinates may appear before their superiors (parents). The LDAP specs require a parent to exist, but also leave the search result ordering undefined in this respect. See Howard's comment below, OpenLDAP's slapadd silently supports unordered data for some backends. In a pinch you may be able to repeatedly use slapadd with the continue on error option (-c) until all the "out of order" parents are created, stopping when you no longer receive any error code 32 (no such object, meaning missing parent) and receive only code 68 (already exists) for every object.
7. ldapadd is subject to the LDAP rules and overlays, e.g. referential integrity, ppolicy (password policy)
8. slapcat prefers to use base-64 encoded attribute values for at least userPassword (indicated with :: after the attribute name)
9. ldapsearch has more options for LDIF formatting, and writing large attributes to separate files. It may also handle referrals and aliases.

Solution 2:

slapcat does not work if you have overlays, e.g. memberOf. So if you do a slapcat / slapadd cycle membership overlays will no longer work.