Git bisect with merged commits

Solution 1:

This is a very old but unanswered question. I decided to investigate, and found that I could show that the behavior of Git is different to what the question says it is. One explanation is that Git improved the algorithm for bisect, or that the questioner made a mistake in marking commits.

I am trying to learn more and git bisect, but am having trouble with this history. I know that 107ca95 is good and 3830e61 is bad. When I run a git bisect, commits 107ca95..3e667f8 are ignored. I happen to know that 43a07b1 is the commit that introduced a regression, but it is never evaluated.

I wrote some code to check whether it is evaluated or not. My test shows that it is evaluated. Run the code below and verify that a commit with message Add menu styles. appears.

Further comments:

• "commits 107ca95..3e667f8 are ignored": Please note, that the commit you marked as "good" will not be evaluated because git already knows it to be good.
• Please read the section "Bisection Algorithm" in this article by Christian Couder. Also the section "Checking merge bases" might be relevant.
• As mentioned above, the question was certainly using a different version then the one I used (Question is from 2013, Git 2.11 is from 2016).

Bisect run output

• Notice that first 'Add Admin notice' is checked (line 4) because that provides the most information. (Read "Checking merge bases" from article mentioned above.)
• From then on, it bisects the linear history as would be expected.

# bad: [d7761d6f146eaca1d886f793ced4315539326866] Add data escaping. (Bad)
# good: [f555d9063a25a20a6ec7c3b0c0504ffe0a997e98] Add Responsive Nav. (Good)
git bisect start 'd7761d6f146eaca1d886f793ced4315539326866' 'f555d9063a25a20a6ec7c3b0c0504ffe0a997e98'
# good: [1b3b7f4952732fec0c68a37d5f313d6f4219e4ae] Add ‘Admin’ notice. (Good)
git bisect good 1b3b7f4952732fec0c68a37d5f313d6f4219e4ae
# bad: [f9a65fe9e6cde4358e5b8ef7569332abfb07675e] Add icons. (Bad)
git bisect bad f9a65fe9e6cde4358e5b8ef7569332abfb07675e
# bad: [165b8a6e5137c40ce8b90911e59d7ec8eec30f46] Add menu styles. (Bad)
git bisect bad 165b8a6e5137c40ce8b90911e59d7ec8eec30f46
# first bad commit: [165b8a6e5137c40ce8b90911e59d7ec8eec30f46] Add menu styles. (Bad)

Code

Run in Python 3, with Git 2.11.0. Command to run: python3 script.py

""" The following code creates a git repository in '/tmp/git-repo' and populates
it with the following commit graph. Each commit has a test.sh which can be used
as input to a git-bisect-run.

The code then tries to find the breaking change automatically.
And prints out the git bisect log.

Written in response to http://stackoverflow.com/questions/17267816/git-bisect-with-merged-commits
to test the claim that '107ca95..3e667f8 are never checked out'.

Needs Python 3!
"""


from itertools import chain
import os.path
import os
import sh

repo = {
0x3830e61:  {'message': "Add data escaping.", 'parents': [    0x0f5e148    ], 'test': False} , # Last:    (Bad)
0x0f5e148: {'message': "Improve function for getting page template.", 'parents': [ 0xaaf8dc5], 'test': False},
0xaaf8dc5: {'message': "Merge branch 'navigation'", 'parents': [ 0x3e667f8, 0xea3d736], 'test': False},
    0x3e667f8: {'message': "Add icons.", 'parents': [  0x43a07b1], 'test': False},
    0x43a07b1: {'message': "Add menu styles.", 'parents': [    0x107ca95], 'test': False}  , # First:       (Breaks)
    0x107ca95: {'message': "Add Responsive Nav.", 'parents': [   0xf52cc34], 'test': True}, # First:        (Good)
  0xea3d736: {'message': "Add ‘Admin’ notice.", 'parents': [ 0x17ca0bb], 'test': True},
  0x17ca0bb: {'message': "Update placeholder text.", 'parents': [  0xf52cc34], 'test': True},
0xf52cc34: {'message': "Add featured image.", 'parents': [  0x2abd954], 'test': True},
0x2abd954: {'message': "Style placeholders.", 'parents': [], 'test': True},
}

bad = 0x3830e61
good = 0x107ca95


def generate_queue(_dag, parents):
    for prev in parents:
        yield prev
        yield from generate_queue(_dag, _dag[prev]['parents'])

def make_queue(_dag, inits):
    """ Converts repo (a DAG) into a queue """
    q = list(generate_queue(_dag, inits))
    q.reverse()
    seen = set()
    r = [x for x in q if not (x in seen or seen.add(x))]
    return r

if __name__ == '__main__':
    pwd = '/tmp/git-repo'
    sh.rm('-r', pwd)
    sh.mkdir('-p', pwd)
    g = sh.git.bake(_cwd=pwd)
    g.init()

    parents = set(chain.from_iterable((repo[c]['parents'] for c in repo)))

    commits = set(repo)
    inits = list(commits - parents)
    queue = make_queue(repo, inits)

    assert len(queue) == len(repo), "queue {} vs repo {}".format(len(queue), len(repo))

    commit_ids = {}
    # Create commits
    for c in queue:
        # Set up repo
        parents = repo[c]['parents']
        if len(parents) > 0:
            g.checkout(commit_ids[parents[0]])
        if len(parents) > 1:
            if len(parents) > 2: raise NotImplementedError('Octopus merges not support yet.')
            g.merge('--no-commit', '-s', 'ours', commit_ids[parents[1]])  # just force to use 'ours' strategy.

        # Make changes
        with open(os.path.join(pwd, 'test.sh'), 'w') as f:
            f.write('exit {:d}\n'.format(0 if repo[c]['test'] else 1))
        os.chmod(os.path.join(pwd, 'test.sh'), 0o0755)
        with open(os.path.join(pwd, 'message'), 'w') as f:
            f.write(repo[c]['message'])
        g.add('test.sh', 'message')
        g.commit('-m', '{msg} ({test})'.format(msg=repo[c]['message'], test='Good' if repo[c]['test'] else 'Bad'))
        commit_ids[c] = g('rev-parse', 'HEAD').strip()

    # Run git-bisect
    g.bisect('start', commit_ids[bad], commit_ids[good])
    g.bisect('run', './test.sh')
    print(g.bisect('log'))

Solution 2:

This is already answered

Basic idea - to find which commit from feature-branch breaks your master, you will have to reapply it on top of ea3d736 - relevant master HEAD.

Following is an example (from git doc)of test script which does that for you:

$ cat ~/test.sh
#!/bin/sh

# tweak the working tree by merging the hot-fix branch
# and then attempt a build
if  git merge --no-commit ea3d736 &&
    make
then
    # run project specific test and report its status
    ~/check_test_case.sh
    status=$?
else
    # tell the caller this is untestable
    status=125
fi

# undo the tweak to allow clean flipping to the next commit
git reset --hard

# return control
exit $status

Run it:

git bisect start 3830e61 f52cc34 
git bisect good ea3d736 17ca0bb #If you want to test feature branch only
git bisect run ~/test.sh

Solution 3:

Warning: the git bisect section regarding "Automatically bisect with temporary modifications" has been updated with Git 2.25 (Q1 2020).

(And git bisect --first-parent is available with Git 2.29+ -- Q4 2020)

It involves the step where you reapply the commit you are testing on top of your relevant master commit (which was ea3d736 in the OP's case)

The "git merge --no-commit" needs "--no-ff" if you do not want to move HEAD, which has been corrected in the manual page for "git bisect".

See commit 8dd327b (28 Oct 2019) by Mihail Atanassov (matana).
^{(Merged by Junio C Hamano -- gitster -- in commit fac9ab1, 01 Dec 2019)}

Documentation/git-bisect.txt: add --no-ff to merge command

^{Signed-off-by: Mihail Atanassov
Reviewed-by: Jonathan Nieder}

The hotfix application example uses git merge --no-commit to apply temporary changes to the working tree during a bisect operation.

In some situations this can be a fast-forward and merge will apply the hotfix branch's commits regardless of --no-commit (as documented in the git merge manual).

In the pathological case this will make a [git bisect](https://git-scm.com/docs/git-bisect) run invocation loop indefinitely between the first bisect step and the fast-forwarded post-merge HEAD.

Add --no-ff to the merge command to avoid this issue.

git merge mentions indeed:

Note that fast-forward updates do not create a merge commit and therefore there is no way to stop those merges with --no-commit.

Thus, if you want to ensure your branch is not changed or updated by the merge command, use --no-ff with --no-commit.