Let's Encrypt certificate renewed, but the site delivers the old one

The web server deamons (apache, nginx, ...) only load the certificates when the configuration is loaded and keep them in memory while they are running. certbot provides some hook parameters that you can use to reload the daemon after a certificate is renewed.

Example from the documentation:

certbot renew --pre-hook "service nginx stop" --post-hook "service nginx start"

Reloading the service with the post hook should suffice though. This is what I use:

certbot renew --deploy-hook "service nginx reload"

Of course you could just run service nginx reload after every run of certbot, but using the renew hook has the benefit that the daemon is only reloaded if certificates have actually been renewed.