Does systemd allow for functionality like authbind? Where you can allow a non-root user to bind to a priv port?
You could use systemd's socket activation functionality to achieve what you want (if the program you're running supports it). Systemd binds to the port as root then passes the socket to the unprivileged program (ex. Apache).
Also look into the AmbientCapabilities directive and CAP_NET_BIND_SERVICE.