How can I limit the disk usage of the Rosetta 2 oah cache?

Unfortunately I don't think Apple ships any configuration options or tooling to allow the user to bound the size of the Rosetta2 cache - nor tools for pruning it or similar.

For this particular usecase with a CI server, I would probably suggest making a small addition to your CI build script that removes its additions to the Rosetta2 cache.

In particular, every time you build a new executable and run it for the first time using Rosetta2, a new .aot file will appear in /var/db/oah. After you have finished using the build product, you could simply have the build script remove that file.

You can find the name of the file by tracing the program the first time you run it, which will reveal the name of the cache file. Another alternative is to set the environment variable ROSETTA_PRINT_SEGMENTS before running the program. This will output the name of the .aot file to stderr when running the program (usually you'll see multiple .aot files because dependencies are loaded in, but the principle is the same).