Rationale for Matcher throwing IllegalStateException when no 'matching' method is called
TL;DR
What are the design decisions behind Matcher
's API?
Background
Matcher
has a behaviour that I didn't expect and for which I can't find a good reason. The API documentation says:
Once created, a matcher can be used to perform three different kinds of match operations: [...] Each of these methods returns a boolean indicating success or failure. More information about a successful match can be obtained by querying the state of the matcher.
What the API documentation further says is:
The explicit state of a matcher is initially undefined; attempting to query any part of it before a successful match will cause an IllegalStateException to be thrown.
Example
String s = "foo=23,bar=42";
Pattern p = Pattern.compile("foo=(?<foo>[0-9]*),bar=(?<bar>[0-9]*)");
Matcher matcher = p.matcher(s);
System.out.println(matcher.group("foo")); // (1)
System.out.println(matcher.group("bar"));
This code throws a
java.lang.IllegalStateException: No match found
at (1)
. To get around this, it is necessary to call matches()
or other methods that bring the Matcher
into a state that allows group()
. The following works:
String s = "foo=23,bar=42";
Pattern p = Pattern.compile("foo=(?<foo>[0-9]*),bar=(?<bar>[0-9]*)");
Matcher matcher = p.matcher(s);
matcher.matches(); // (2)
System.out.println(matcher.group("foo"));
System.out.println(matcher.group("bar"));
Adding the call to matches()
at (2)
sets the Matcher
into the proper state to call group()
.
Question, probably not constructive
Why is this API designed like this? Why not automatically match when the Matcher
is build with Patter.matcher(String)
?
Solution 1:
Actually, you misunderstood the documentation. Take a 2nd look at the statement you quoted: -
attempting to query any part of it before a successful match will cause an IllegalStateException to be thrown.
A matcher may throw IllegalStateException
on accessing matcher.group()
if no match was found.
So, you need to use following test, to actually initiate the matching process: -
- matcher.matches() //Or
- matcher.find()
The below code: -
Matcher matcher = pattern.matcher();
Just creates a matcher
instance. This will not actually match a string. Even if there was a successful match.
So, you need to check the following condition, to check for successful matches: -
if (matcher.matches()) {
// Then use `matcher.group()`
}
And if the condition in the if
returns false
, that means nothing was matched. So, if you use matcher.group()
without checking this condition, you will get IllegalStateException
if the match was not found.
Suppose, if Matcher
was designed the way you are saying, then you would have to do a null
check to check whether a match was found or not, to call matcher.group()
, like this: -
The way you think should have been done:-
// Suppose this returned the matched string
Matcher matcher = pattern.matcher(s);
// Need to check whether there was actually a match
if (matcher != null) { // Prints only the first match
System.out.println(matcher.group());
}
But, what if, you want to print any further matches, since a pattern can be matched multiple times in a String, for that, there should be a way to tell the matcher to find the next match. But the null
check would not be able to do that. For that you would have to move your matcher forward to match the next String. So, there are various methods defined in Matcher
class to serve the purpose. The matcher.find()
method matches the String till all the matches is found.
There are other methods also, that match
the string in a different way, that depends on you how you want to match. So its ultimately on Matcher
class to do the matching
against the string. Pattern
class just creates a pattern
to match against. If the Pattern.matcher()
were to match
the pattern, then there has to be some way to define various ways to match
, as matching
can be in different ways. So, there comes the need of Matcher
class.
So, the way it actually is: -
Matcher matcher = pattern.matcher(s);
// Finds all the matches until found by moving the `matcher` forward
while(matcher.find()) {
System.out.println(matcher.group());
}
So, if there are 4 matches found in the string, your first way, would print only the first one, while the 2nd way will print all the matches, by moving the matcher
forward to match the next pattern.
I Hope that makes it clear.
The documentation of Matcher
class describes the use of the three methods it provides, which says: -
A matcher is created from a pattern by invoking the pattern's matcher method. Once created, a matcher can be used to perform three different kinds of match operations:
The matches method attempts to match the entire input sequence against the pattern.
The lookingAt method attempts to match the input sequence, starting at the beginning, against the pattern.
The find method scans the input sequence looking for the next subsequence that matches the pattern.
Unfortunately, I have not been able find any other official sources, saying explicitly Why and How of this issue.
Solution 2:
My answer is very similar to Rohit Jain's but includes some reasons why the 'extra' step is necessary.
java.util.regex implementation
The line:
Pattern p = Pattern.compile("foo=(?<foo>[0-9]*),bar=(?<bar>[0-9]*)");
causes a new Pattern object to be allocated, and it internally stores a structure representing the RE - information such as a choice of characters, groups, sequences, greedy vs. non-greedy, repeats and so on.
This pattern is stateless and immutable, so it can be reused, is multi-theadable and optimizes well.
The lines:
String s = "foo=23,bar=42";
Matcher matcher = p.matcher(s);
returns a new Matcher
object for the Pattern
and String
- one that has not yet read the String. Matcher
is really just a state machine's state, where the state machine is the Pattern
.
The matching can be run by stepping the state machine through the matching process using the following API:
-
lookingAt()
: Attempts to match the input sequence, starting at the beginning, against the pattern -
find()
: Scans the input sequence looking for the next subsequence that matches the pattern.
In both cases, the intermediate state can be read using the start()
, end()
, and group()
methods.
Benefits of this approach
Why would anyone want to do step through the parsing?
-
Get values from groups that have quantification greater than 1 (i.e. groups that repeat and end up matching more than once). For example in the trivial RE below that parses variable assignments:
Pattern p = new Pattern("([a-z]=([0-9]+);)+"); Matcher m = p.matcher("a=1;b=2;x=3;"); m.matches(); System.out.println(m.group(2)); // Only matches value for x ('3') - not the other values
See the section on "Group name" in "Groups and capturing" the JavaDoc on Pattern
- The developer can use the RE as a lexer and the developer can bind the lexed tokens to a parser. In practice, this would work for simple domain languages, but regular expressions are probably not the way to go for a full-blown computer language. EDIT This is partly related to the previous reason, but it can frequently be easier and more efficient to create the parse tree processing the text than lexing all the input first.
- (For the brave-hearted) you can debug REs and find out which subsequence is failing to match (or incorrectly matching).
However, on most occasions you do not need to step the state machine through the matching, so there is a convenience method (matches
) which runs the pattern matching to completion.