How do I use a Git submodule with a Composer loaded library?

composer-php

I have a Zend Framework 2 application. It contains some library code containing business logic and some other utilities that will be common to other applications that will be created later.

My intention is to share it across projects using Composer. The question is, how do I do this properly and streamline the development? I will almost certainly need to make changes and additions to the library, from within the other project.

I tried setting up vendor/stuff as a git submodule containing the package needed, and referencing it in the primary composer.json like this (ref):

"repositories": [
    {
        "type": "git",
        "url": "vendor/stuff"
    }
],
"require": {
    "stuff/library": "master"
},

Composer isn't able to load it in this way. It complains that the package could not be found, presumably because it's ignoring the fact that the URL is both local and relative. Technically, it doesn't need to; the vendor/stuff folder was initialised separately through git submodule commands.

Solution 1:

Unfortunately* Composer doesn't support Git submodules, as the main aim of Composer is to provide a similar inter-project dependency functionality and it would be pointless to try to replicate submodules in Composer.

I have the same problem that you are trying to solve, of developing a library while simultaneously developing the application that uses that library. There are a couple of ways to solve that problem just using composer.

Make a symbolic link for the library directory

This is the quickest and dirtiest way of doing it. Just do a composer update to create the appropriate directory for the library in your vendors directory, then replace it with symbolic link from your the directory that contains your library.

Obviously this is not great as you can accidentally overwrite code that you may have edited by running composer update.

Use Composer prefer source option

Composer has the option to download the source via a Git clone (--prefer-src) rather than downloading a zipball (--prefer-dist) which is the default. This allows you to edit the source code inside the vendors directory and then commit it through Git.

e.g. Say you have project that requires amongst other libraries symfony/yaml which you want to fix a bug in. What you could do is:

composer update - This will download all of dependencies of the project.
composer update symfony/yaml --prefer-source - This will now update just the symfony/yaml directory in the vendors directory.
Fix the bug and then commit it through git.

Use Composer local repository

The way I ---actually--- used to work when I'm developing a project and it's requirement in tandem, is to use Composers ability to set explicitly set a repository to use to resolve dependencies. e.g. if your code is in:

/projects/library/
/projects/project/

In the composer file for your project add the repository entry:

"repositories": [
    {
        "type": "vcs",
        "url": "/projects/library/"
    }
]

Running composer update will now look at the Git entries in /projects/library/ to resolve any dependencies on the library in preference to those listed in Packagist or other repository.

This does mean that when you want to test a change in the library code you need to:

Commit it, so that it has a Git entry.
Run Composer update in the project directory to get the latest version.

But you avoid having to push the commit to an external repository, which is good as it means you aren't pushing code that may not work, and it means you can work offline as Git commits don't require an internet connection.

Although that is apparently the best way of working, it's still kind of dangerous as it is too easy to accidentally check in a version of your composer.json that has references to local directories, which obviously will break the project for everyone else.

To avoid this I made a couple of small scripts that i) backup my real composer.json file, ii) Add in some local repositories, iii) run composer update iv) Restore the real composer.json file.

localupdate.sh

cp -f composer.json composer.json.bak
php composerLocal.php
composer update
cp -f composer.json.bak composer.json

composerLocal.php

<?php

$srcFile = file_get_contents("composer.json");
$hackFile = file_get_contents("composer.local");
$finalString = str_replace('"LOCALHACK",', $hackFile, $srcFile);
file_put_contents("composer.json", $finalString);

?>

composer.local

"LOCALHACK",

"repositories": [
    {
        "type": "vcs",
        "url": "/projects/library1"
    },
    {
        "type": "vcs",
        "url": "/projects/library2"
    }   
],

And then place "//": "LOCALHACK", somewhere in your projects composer.json file. Running localupdate.sh now safely does a composer update against local repositories without any chance of committing a bad version of composer.json.

Just clone it yourself with Git

This is how I tend to work now:

i) Composer update in the project ii) Go into the vendors directory and delete the library that I want to be developing at the same time. iii) Git clone from whichever repo you are developing the library in, into the appropriate vendors directory.

Composer understands git repos, so won't overwrite a git cloned directory(, though it does seem to get a little confused about editing the composer.json of the library).

Doing the git clone yourself, gives you complete control over what gets installed, and allows you to install from a repo that composer doesn't know about, or an untagged version, without having to edit the composer.json in the project.

That's the key feature of doing the git clone yourself; by not touching the composer.json of the project, it's completely safe, with no possibility of checking in a composer.json that has been modified to use local/custom repos.

Edit - 6th Sept 2014

The validation for composer.json files has been tightened and it is no longer possible to have a "//": "LOCALHACK" entry in the file. Which is another reason why the Composer guys not having versioning for the Composer project is nuts.

* I actually think Git Submodules are a dumb, dumb, dumb implementation to 'solve' a difficult problem in a way that only causes more problems down the road, and so Composer not supporting them is way more 'fortunate' than 'unfortunate'. Obviously other people do use them, and are happy with them, so that is just my opinion, man, but if you're using Composer you shouldn't have a need for submodules.

Solution 2:

Desired behavior:

You have 2 Zend projects: project_A and project_B
both project use the same common library named library_C
library_C is required from project_A and project_B trough the composer
library_C should be editable in vendor folder in project_A and project_B and you should be able to push those changes
your git URL of library is https://github.com/stuff/library_C.git

How to do it:

Put you library inside composer.json of project_A and project_B

"require": {
    "stuff/library_C": "master"
},

2.run composer install. Now your library is in folder vendor/stuff/library_C

3.Now when you want to edit something inside library_C in project_A or project_B, delete folder vendor/stuff/library_C and run this command

git submodule add -f https://github.com/stuff/library_C.git vendor/stuff/library_C

this command will recreate this folder vendor/stuff/library_C, but this folder is now another git repo (submodule)

4.Go to folder vendor/stuff/library_C change anything commit it and push, AND THAT IS ALL!!!

After that when you run composer update stuff/library_C in other project the code will be updated.

Note: after you run git submodule add you will be on a branch master so you must checkout to proper branch or tag and do a change there.