isc-dhcp leases file never cleaned
As MadHatter mentioned in a comment, the leases file is periodically re-created to avoid this problem. While the period isn't mentioned in the documentation, discussions on the dhcp-users mailinglist indicates that it should be done once an hour, and I've checked the source code and found that this is correct.
Unfortunately this isn't a configurable option. In order to change it, you'd need to compile the dhcp server from source. In the file server/db.c you'd need to change the line
#define LEASE_REWRITE_PERIOD 3600
to the number of seconds you'd prefer.