How to remove ipset entry with timeout from firewalld using firewall-cmd?
Well as it says in the official firewalld docs,
Adding an entry to an ipset with option timeout is permitted, but these entries are not tracked by firewalld.
I guess that answers my question.