How to fix a permission denied (publickey) error for a git submodule update in the Github Travis CI build?

I cannot update the git submodule, with the error:

$ git submodule init
Submodule 'build/html' ([email protected]:quadroid/clonejs.git) registered for path 'build/html'
...
$ git submodule update
Cloning into 'build/html'...
Warning: Permanently added 'github.com,207.97.227.239' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

But when I do the same tasks locally, everything is OK.

How do I fix this so that the Travis CI build passes and I can still click on the submodule in the repo to direct to it?

Solution 1:

This can (thankfully) be easily solved by modifying the .gitmodules file on-the-fly on Travis, so that the SSH URL is replaced with the public URL, before initializing submodules. To accomplish this, add the following to .travis.yml:

# Handle git submodules yourself
git:
    submodules: false
# Use sed to replace the SSH URL with the public URL, then initialize submodules
before_install:
    - sed -i 's/[email protected]:/https:\/\/github.com\//' .gitmodules
    - git submodule update --init --recursive

Thanks to Michael Iedema for his gist from which I derived this solution.

If your submodules are private repositories, it should work to include credentials in the https URLs, I recommend making a GitHub access token with restricted permissions for this purpose:

# Replace <user> and <token> with your GitHub username and access token respectively
- sed -i 's/[email protected]:/https:\/\/<user>:<token>@github.com\//' .gitmodules

Solution 2:

I'd recommend using the https scheme for submodules, as that'll allow you to pull on Travis and push locally: https://github.com/quadroid/clonejs.git.

Solution 3:

Travis now supports accessing submodule using ssh, which is by far the easiest solution. You only need to associate your ssh key (or the ssh key of a dedicated CI user) with the Github project you are building, as described in the documentation for private dependencies.

$ travis sshkey --upload ~/.ssh/id_rsa -r myorg/main

Note that Travis recommends creating a dedicated user so that you do not have to use your own ssh key.

Solution 4:

You get this error because you specified your submodules via ssh-urls. For ssh access from the travis-ci environment you need to configure a key.

Alternatively, you can just use relative URLs for your git submodules since you project and your submodules are all available on Github.

Git resolves relative urls against the ORIGIN.

Example:

Using the first 2 entries from your .gitmodules:

[submodule "lib/es5-shim"]
        path = lib/es5-shim
        url = [email protected]:kriskowal/es5-shim.git
[submodule "build/html"]
        path = build/html
        url = [email protected]:quadroid/clonejs.git

Replaced with relative URLs:

[submodule "lib/es5-shim"]
        path = lib/es5-shim
        url = ../../kriskowal/es5-shim.git
[submodule "build/html"]
        path = build/html
        url = ../clonejs.git

Then when cloning - say - via https the origin is set like this:

$ git clone https://github.com/quadroid/clonejs.git
$ cd clonejs
$ git remote -v
origin  https://github.com/quadroid/clonejs.git (fetch)
origin  https://github.com/quadroid/clonejs.git (push)

When cloning via ssh:

$ git clone [email protected]:quadroid/clonejs.git
$ cd clonejs
$ git remote -v                                
origin  [email protected]:quadroid/clonejs.git (fetch)
origin  [email protected]:quadroid/clonejs.git (push)

With relative urls, the usual submodule sequence works independently of the origin:

$ git submodule init
$ git submodule update

Solution 5:

You can also just directly manipulate your .gitmodules file via git. (Inspired by this answer).

git config --file=.gitmodules submodule.SUBMODULE_PATH.url https://github.com/ORG/REPO.git