What's the difference between "groups" and "captures" in .NET regular expressions?

I'm a little fuzzy on what the difference between a "group" and a "capture" are when it comes to .NET's regular expression language. Consider the following C# code:

MatchCollection matches = Regex.Matches("{Q}", @"^\{([A-Z])\}$");

I expect this to result in a single capture for the letter 'Q', but if I print the properties of the returned MatchCollection, I see:

matches.Count: 1
matches[0].Value: {Q}
        matches[0].Captures.Count: 1
                matches[0].Captures[0].Value: {Q}
        matches[0].Groups.Count: 2
                matches[0].Groups[0].Value: {Q}
                matches[0].Groups[0].Captures.Count: 1
                        matches[0].Groups[0].Captures[0].Value: {Q}
                matches[0].Groups[1].Value: Q
                matches[0].Groups[1].Captures.Count: 1
                        matches[0].Groups[1].Captures[0].Value: Q

What exactly is going on here? I understand that there's also a capture for the entire match, but how do the groups come in? And why doesn't matches[0].Captures include the capture for the letter 'Q'?

Solution 1:

You won't be the first who's fuzzy about it. Here's what the famous Jeffrey Friedl has to say about it (pages 437+):

Depending on your view, it either adds an interesting new dimension to the match results, or adds confusion and bloat.

And further on:

The main difference between a Group object and a Capture object is that each Group object contains a collection of Captures representing all the intermediary matches by the group during the match, as well as the final text matched by the group.

And a few pages later, this is his conclusion:

After getting past the .NET documentation and actually understanding what these objects add, I've got mixed feelings about them. On one hand, it's an interesting innovation [..] on the other hand, it seems to add an efficiency burden [..] of a functionality that won't be used in the majority of cases

In other words: they are very similar, but occasionally and as it happens, you'll find a use for them. Before you grow another grey beard, you may even get fond of the Captures...

Since neither the above, nor what's said in the other post really seems to answer your question, consider the following. Think of Captures as a kind of history tracker. When the regex makes his match, it goes through the string from left to right (ignoring backtracking for a moment) and when it encounters a matching capturing parentheses, it will store that in $x (x being any digit), let's say $1.

Normal regex engines, when the capturing parentheses are to be repeated, will throw away the current $1 and will replace it with the new value. Not .NET, which will keep this history and places it in Captures[0].

If we change your regex to look as follows:

MatchCollection matches = Regex.Matches("{Q}{R}{S}", @"(\{[A-Z]\})+");

you will notice that the first Group will have one Captures (the first group always being the whole match, i.e., equal to $0) and the second group will hold {S}, i.e. only the last matching group. However, and here's the catch, if you want to find the other two catches, they're in Captures, which contains all intermediary captures for {Q} {R} and {S}.

If you ever wondered how you could get from the multiple-capture, which only shows last match to the individual captures that are clearly there in the string, you must use Captures.

A final word on your final question: the total match always has one total Capture, don't mix that with the individual Groups. Captures are only interesting inside groups.

Solution 2:

This can be explained with a simple example (and pictures).

Matching 3:10pm with the regular expression ((\d)+):((\d)+)(am|pm), and using Mono interactive csharp:

csharp> Regex.Match("3:10pm", @"((\d)+):((\d)+)(am|pm)").
      > Groups.Cast<Group>().
      > Zip(Enumerable.Range(0, int.MaxValue), (g, n) => "[" + n + "] " + g);
{ "[0] 3:10pm", "[1] 3", "[2] 3", "[3] 10", "[4] 0", "[5] pm" }

So where's the 1?

Since there are multiple digits that match on the fourth group, we only "get at" the last match if we reference the group (with an implicit ToString(), that is). In order to expose the intermediate matches, we need to go deeper and reference the Captures property on the group in question:

csharp> Regex.Match("3:10pm", @"((\d)+):((\d)+)(am|pm)").
      > Groups.Cast<Group>().
      > Skip(4).First().Captures.Cast<Capture>().
      > Zip(Enumerable.Range(0, int.MaxValue), (c, n) => "["+n+"] " + c);
{ "[0] 1", "[1] 0" }

Courtesy of this article.

Solution 3:

A Group is what we have associated with groups in regular expressions

"(a[zx](b?))"

Applied to "axb" returns an array of 3 groups:

group 0: axb, the entire match.
group 1: axb, the first group matched.
group 2: b, the second group matched.

except that these are only 'captured' groups. Non capturing groups (using the '(?: ' syntax are not represented here.

"(a[zx](?:b?))"

Applied to "axb" returns an array of 2 groups:

group 0: axb, the entire match.
group 1: axb, the first group matched.

A Capture is also what we have associated with 'captured groups'. But when the group is applied with a quantifier multiple times, only the last match is kept as the group's match. The captures array stores all of these matches.

"(a[zx]\s+)+"

Applied to "ax az ax" returns an array of 2 captures of the second group.

group 1, capture 0 "ax "
group 1, capture 1 "az "

As for your last question -- I would have thought before looking into this that Captures would be an array of the captures ordered by the group they belong to. Rather it is just an alias to the groups[0].Captures. Pretty useless..

Solution 4:

From the MSDN documentation:

The real utility of the Captures property occurs when a quantifier is applied to a capturing group so that the group captures multiple substrings in a single regular expression. In this case, the Group object contains information about the last captured substring, whereas the Captures property contains information about all the substrings captured by the group. In the following example, the regular expression \b(\w+\s*)+. matches an entire sentence that ends in a period. The group (\w+\s*)+ captures the individual words in the collection. Because the Group collection contains information only about the last captured substring, it captures the last word in the sentence, "sentence". However, each word captured by the group is available from the collection returned by the Captures property.

Solution 5:

Imagine you have the following text input dogcatcatcat and a pattern like dog(cat(catcat))

In this case, you have 3 groups, the first one (major group) corresponds to the match.

Match == dogcatcatcat and Group0 == dogcatcatcat

Group1 == catcatcat

Group2 == catcat

So what it's all about?

Let's consider a little example written in C# (.NET) using Regex class.

int matchIndex = 0;
int groupIndex = 0;
int captureIndex = 0;

foreach (Match match in Regex.Matches(
        "dogcatabcdefghidogcatkjlmnopqr", // input
        @"(dog(cat(...)(...)(...)))") // pattern
)
{
    Console.Out.WriteLine($"match{matchIndex++} = {match}");

    foreach (Group @group in match.Groups)
    {
        Console.Out.WriteLine($"\tgroup{groupIndex++} = {@group}");

        foreach (Capture capture in @group.Captures)
        {
            Console.Out.WriteLine($"\t\tcapture{captureIndex++} = {capture}");
        }

        captureIndex = 0;
    }

    groupIndex = 0;
    Console.Out.WriteLine();
        }

Output:

match0 = dogcatabcdefghi
    group0 = dogcatabcdefghi
        capture0 = dogcatabcdefghi
    group1 = dogcatabcdefghi
        capture0 = dogcatabcdefghi
    group2 = catabcdefghi
        capture0 = catabcdefghi
    group3 = abc
        capture0 = abc
    group4 = def
        capture0 = def
    group5 = ghi
        capture0 = ghi

match1 = dogcatkjlmnopqr
    group0 = dogcatkjlmnopqr
        capture0 = dogcatkjlmnopqr
    group1 = dogcatkjlmnopqr
        capture0 = dogcatkjlmnopqr
    group2 = catkjlmnopqr
        capture0 = catkjlmnopqr
    group3 = kjl
        capture0 = kjl
    group4 = mno
        capture0 = mno
    group5 = pqr
        capture0 = pqr

Let's analyze just the first match (match0).

As you can see there are three minor groups: group3, group4 and group5

    group3 = kjl
        capture0 = kjl
    group4 = mno
        capture0 = mno
    group5 = pqr
        capture0 = pqr

Those groups (3-5) were created because of the 'subpattern' (...)(...)(...) of the main pattern (dog(cat(...)(...)(...)))

Value of group3 corresponds to it's capture (capture0). (As in the case of group4 and group5). That's because there are no group repetition like (...){3}.

Ok, let's consider another example where there is a group repetition.

If we modify the regular expression pattern to be matched (for code shown above) from (dog(cat(...)(...)(...))) to (dog(cat(...){3})), you'll notice that there is the following group repetition: (...){3}.

Now the Output has changed:

match0 = dogcatabcdefghi
    group0 = dogcatabcdefghi
        capture0 = dogcatabcdefghi
    group1 = dogcatabcdefghi
        capture0 = dogcatabcdefghi
    group2 = catabcdefghi
        capture0 = catabcdefghi
    group3 = ghi
        capture0 = abc
        capture1 = def
        capture2 = ghi

match1 = dogcatkjlmnopqr
    group0 = dogcatkjlmnopqr
        capture0 = dogcatkjlmnopqr
    group1 = dogcatkjlmnopqr
        capture0 = dogcatkjlmnopqr
    group2 = catkjlmnopqr
        capture0 = catkjlmnopqr
    group3 = pqr
        capture0 = kjl
        capture1 = mno
        capture2 = pqr

Again, let's analyze just the first match (match0).

There are no more minor groups group4 and group5 because of (...){3} repetition ({n} wherein n>=2) they've been merged into one single group group3.

In this case, the group3 value corresponds to it's capture2 (the last capture, in other words).

Thus if you need all the 3 inner captures (capture0, capture1, capture2) you'll have to cycle through the group's Captures collection.

Сonclusion is: pay attention to the way you design your pattern's groups. You should think upfront what behavior causes group's specification, like (...)(...), (...){2} or (.{3}){2} etc.

Hopefully it will help shed some light on the differences between Captures, Groups and Matches as well.