ZFS - is it safe to remove files served by ZFS during a resilver/rebuild?

Any userland storage operations are fine to perform, but they do effectively halt the resilvering while they're being performed. ZFS prioritizes normal operations (using the file system) over resilvering operations. Resilvering reads all the data in the pool in a nonsequential order and writes only the required data to the new disk(s). Deletes aren't much different than writes, especially since I imagine you probably have snapshots which mean that the deletes are basically non-operations in terms of disk hits.

Destroying snapshots during a resilver is something I wouldn't recommend. But it's not out of safety or a concern for data loss, it's because it makes the pool useless. A several year old memory tells me that performance hit on a small (25TB 12x2TB, ZIL and SLOG on ~fast SSDs) pool destroying ~1TB on a single disk resilver made the system unusable for around a day.

In short, ZFS does everything it can to provide you with reliable access to your data, and let you use it normally, while it does all its housekeeping in the background. Resilvers are critical, but they're also interruptible background tasks.

More info about how resilvering works can be found here, and in the Oracle docs.