How does a webserver know which files to send in an HTTP/2 request?

This is entirely up the server and how it has been configured.

Most servers are not intelligent enough to know what to push and depend on being configured. So you can set up config to say if any index.html file is requested then push common.css and common.js. It’s important then to consider the next page that’s visited - there is no need to push those files again, as user should already have them. You can use a cookie-based approach to track this. See my post here for how to configure this in Apache. Some servers (e.g. Apache) also maintain a list of known pushed assets for that connection to avoid over pushing, though that only works on same connection so cookie-based approach is better.

Many servers and CDNs can use HTTP link headers to inform the web server which assets to push. This allows the control to rest with backend application servers, but the push to happen from the edge web servers so it doesn’t all need to be explicitly configured on the web server.

Other servers try to be more intelligent with this and try to guess what resources to push based on observing requests. Jetty has a facility to do this for example. How accurate or useful this is I can’t attest to.

Or is it the case that while possible in theory, this doesn't happen in practice, and the browser just makes a request for the assets later?

While it is certainly possible to push (my blog does it as can be seen in above post), there are real problems with push. Over-pushing is a real risk and even without that the benefits have never really been proven. Plus there are implementation issues and complications to consider too. So the reality is it is not used much. From a study I completed last year about half a percent (0.5%) of sites used HTTP/2 push. Chrome has stated for a while now that it is considering turning off support for HTTP/2 push. Use it with caution.