How do I cache steps in GitHub actions?
Say I have a GitHub actions workflow with 2 steps.
- Download and compile my application's dependencies.
- Compile and test my application
My dependencies rarely change and the compiled dependencies can be safely cached until I next change the lock-file that specifies their versions.
Is a way to save the result of the first step so that in future workflow can skip over that step?
Caching is now natively supported via the cache action. It works across both jobs and workflows within a repository. See also: https://help.github.com/en/actions/automating-your-workflow-with-github-actions/caching-dependencies-to-speed-up-workflows.
Consider the following example:
name: GitHub Actions Workflow with NPM cache
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Cache NPM dependencies
uses: actions/cache@v1
with:
path: ~/.npm
key: ${{ runner.OS }}-npm-cache-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-npm-cache-
- name: Install NPM dependencies
run: npm install
Where the path
and key
parameters of the cache
action is used to identify the cache.
The optional restore-keys
is used for a possible fallback to a partial match (i.e. if package-lock.json
changes the previous cache will be used).
Prefixing the keys with some id (npm-cache
in this example) is useful when the restore-keys
fallback is used and there're multiple different caches (e.g. for JS packages and for system packages). Otherwise, one cache could fall back to the other unrelated cache. Similarly, an OS prefix useful when using matrix builds so caches of different systems don't get mixed up.
You can also build your own reusable caching logic with @actions/cache such as:
- 1-liner NPM cache
- 1-liner Yarn cache
Old answer:
Native caching is not currently possible, expected to be implemented by mid-November 2019.
You can use artifacts (1, 2) to move directories between jobs (within 1 workflow) as proposed on the GH Community board. This, however, doesn't work across workflows.
The cache
action can only cache the contents of a folder. So if there is such a folder, you may win some time by caching it.
For instance, if you use some imaginary package-installer
(like Python's pip
or virtualenv
, or NodeJS' npm
, or anything else that puts its files into a folder), you can win some time by doing it like this:
- uses: actions/cache@v2
id: cache-packages # give it a name for checking the cache hit-or-not
with:
path: ./packages/ # what we cache: the folder
key: ${{ runner.os }}-packages-${{ hashFiles('**/packages*.txt') }}
restore-keys: |
${{ runner.os }}-packages-
- run: package-installer packages.txt
if: steps.cache-packages.outputs.cache-hit != 'true'
So what's important here:
- We give this step a name,
cache-packages
- Later, we use this name for conditional execution:
if
,steps.cache-packages.outputs.cache-hit != 'true'
- Give the cache action a path to the folder you want to cache:
./packages/
- Cache key: something that depends on the hash of your input files. That is, if any
packages.txt
file changes, the cache will be rebuilt. - The second step, package installer, will only be run if there was no cache
For users of virtualenv
: if you need to activate some shell environment, you have to do it in every step. Like this:
- run: . ./environment/activate && command
My dependencies rarely change and the compiled dependencies can be safely cached until I next change the lock-file that specifies their versions. Is a way to save the result of the first step so that in future workflow can skip over that step?
The first step being:
Download and compile my application's dependencies.
GitHub Actions themselves will not do this for you. The only advice I can give you is that you adhere to Docker best practices in order to ensure that if Actions do make use of docker caching, your image could be re-used instead of rebuilt. See: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache
When building an image, Docker steps through the instructions in your Dockerfile, executing each in the order specified. As each instruction is examined, Docker looks for an existing image in its cache that it can reuse, rather than creating a new (duplicate) image.
This also implies that the underlying system of GitHub Actions can/will leverage the Docker caching.
However things like compilation, Docker won't be able to use the cache mechanism, so I suggest you think very well if this is something you desperately need. The alternative is to download the compiled/processed files from an artifact store (Nexus, NPM, MavenCentral) to skip that step. You do have to weight the benefits vs the complexity you are adding to your build on this.