Does C# have a String Tokenizer like Java's?

I'm doing simple string input parsing and I am in need of a string tokenizer. I am new to C# but have programmed Java, and it seems natural that C# should have a string tokenizer. Does it? Where is it? How do I use it?

Solution 1:

You could use String.Split method.

class ExampleClass
{
    public ExampleClass()
    {
        string exampleString = "there is a cat";
        // Split string on spaces. This will separate all the words in a string
        string[] words = exampleString.Split(' ');
        foreach (string word in words)
        {
            Console.WriteLine(word);
            // there
            // is
            // a
            // cat
        }
    }
}

For more information see Sam Allen's article about splitting strings in c# (Performance, Regex)

Solution 2:

I just want to highlight the power of C#'s Split method and give a more detailed comparison, particularly from someone who comes from a Java background.

Whereas StringTokenizer in Java only allows a single delimiter, we can actually split on multiple delimiters making regular expressions less necessary (although if one needs regex, use regex by all means!) Take for example this:

str.Split(new char[] { ' ', '.', '?' })

This splits on three different delimiters returning an array of tokens. We can also remove empty arrays with what would be a second parameter for the above example:

str.Split(new char[] { ' ', '.', '?' }, StringSplitOptions.RemoveEmptyEntries)

One thing Java's String tokenizer does have that I believe C# is lacking (at least Java 7 has this feature) is the ability to keep the delimiter(s) as tokens. C#'s Split will discard the tokens. This could be important in say some NLP applications, but for more general purpose applications this might not be a problem.

Solution 3:

The split method of a string is what you need. In fact the tokenizer class in Java is deprecated in favor of Java's string split method.

Solution 4:

I think the nearest in the .NET Framework is

string.Split()

Solution 5:

For complex splitting you could use a regex creating a match collection.