How do I verify that a string only contains letters, numbers, underscores and dashes?

Solution 1:

A regular expression will do the trick with very little code:

import re

...

if re.match("^[A-Za-z0-9_-]*$", my_little_string):
    # do something here

Solution 2:

[Edit] There's another solution not mentioned yet, and it seems to outperform the others given so far in most cases.

Use string.translate to replace all valid characters in the string, and see if we have any invalid ones left over. This is pretty fast as it uses the underlying C function to do the work, with very little python bytecode involved.

Obviously performance isn't everything - going for the most readable solutions is probably the best approach when not in a performance critical codepath, but just to see how the solutions stack up, here's a performance comparison of all the methods proposed so far. check_trans is the one using the string.translate method.

Test code:

import string, re, timeit

pat = re.compile('[\w-]*$')
pat_inv = re.compile ('[^\w-]')
allowed_chars=string.ascii_letters + string.digits + '_-'
allowed_set = set(allowed_chars)
trans_table = string.maketrans('','')

def check_set_diff(s):
    return not set(s) - allowed_set

def check_set_all(s):
    return all(x in allowed_set for x in s)

def check_set_subset(s):
    return set(s).issubset(allowed_set)

def check_re_match(s):
    return pat.match(s)

def check_re_inverse(s): # Search for non-matching character.
    return not pat_inv.search(s)

def check_trans(s):
    return not s.translate(trans_table,allowed_chars)

test_long_almost_valid='a_very_long_string_that_is_mostly_valid_except_for_last_char'*99 + '!'
test_long_valid='a_very_long_string_that_is_completely_valid_' * 99
test_short_valid='short_valid_string'
test_short_invalid='/$%$%&'
test_long_invalid='/$%$%&' * 99
test_empty=''

def main():
    funcs = sorted(f for f in globals() if f.startswith('check_'))
    tests = sorted(f for f in globals() if f.startswith('test_'))
    for test in tests:
        print "Test %-15s (length = %d):" % (test, len(globals()[test]))
        for func in funcs:
            print "  %-20s : %.3f" % (func, 
                   timeit.Timer('%s(%s)' % (func, test), 'from __main__ import pat,allowed_set,%s' % ','.join(funcs+tests)).timeit(10000))
        print

if __name__=='__main__': main()

The results on my system are:

Test test_empty      (length = 0):
  check_re_inverse     : 0.042
  check_re_match       : 0.030
  check_set_all        : 0.027
  check_set_diff       : 0.029
  check_set_subset     : 0.029
  check_trans          : 0.014

Test test_long_almost_valid (length = 5941):
  check_re_inverse     : 2.690
  check_re_match       : 3.037
  check_set_all        : 18.860
  check_set_diff       : 2.905
  check_set_subset     : 2.903
  check_trans          : 0.182

Test test_long_invalid (length = 594):
  check_re_inverse     : 0.017
  check_re_match       : 0.015
  check_set_all        : 0.044
  check_set_diff       : 0.311
  check_set_subset     : 0.308
  check_trans          : 0.034

Test test_long_valid (length = 4356):
  check_re_inverse     : 1.890
  check_re_match       : 1.010
  check_set_all        : 14.411
  check_set_diff       : 2.101
  check_set_subset     : 2.333
  check_trans          : 0.140

Test test_short_invalid (length = 6):
  check_re_inverse     : 0.017
  check_re_match       : 0.019
  check_set_all        : 0.044
  check_set_diff       : 0.032
  check_set_subset     : 0.037
  check_trans          : 0.015

Test test_short_valid (length = 18):
  check_re_inverse     : 0.125
  check_re_match       : 0.066
  check_set_all        : 0.104
  check_set_diff       : 0.051
  check_set_subset     : 0.046
  check_trans          : 0.017

The translate approach seems best in most cases, dramatically so with long valid strings, but is beaten out by regexes in test_long_invalid (Presumably because the regex can bail out immediately, but translate always has to scan the whole string). The set approaches are usually worst, beating regexes only for the empty string case.

Using all(x in allowed_set for x in s) performs well if it bails out early, but can be bad if it has to iterate through every character. isSubSet and set difference are comparable, and are consistently proportional to the length of the string regardless of the data.

There's a similar difference between the regex methods matching all valid characters and searching for invalid characters. Matching performs a little better when checking for a long, but fully valid string, but worse for invalid characters near the end of the string.

Solution 3:

There are a variety of ways of achieving this goal, some are clearer than others. For each of my examples, 'True' means that the string passed is valid, 'False' means it contains invalid characters.

First of all, there's the naive approach:

import string
allowed = string.letters + string.digits + '_' + '-'

def check_naive(mystring):
    return all(c in allowed for c in mystring)

Then there's use of a regular expression, you can do this with re.match(). Note that '-' has to be at the end of the [] otherwise it will be used as a 'range' delimiter. Also note the $ which means 'end of string'. Other answers noted in this question use a special character class, '\w', I always prefer using an explicit character class range using [] because it is easier to understand without having to look up a quick reference guide, and easier to special-case.

import re
CHECK_RE = re.compile('[a-zA-Z0-9_-]+$')
def check_re(mystring):
    return CHECK_RE.match(mystring)

Another solution noted that you can do an inverse match with regular expressions, I've included that here now. Note that [^...] inverts the character class because the ^ is used:

CHECK_INV_RE = re.compile('[^a-zA-Z0-9_-]')
def check_inv_re(mystring):
   return not CHECK_INV_RE.search(mystring)

You can also do something tricky with the 'set' object. Have a look at this example, which removes from the original string all the characters that are allowed, leaving us with a set containing either a) nothing, or b) the offending characters from the string:

def check_set(mystring):
    return not set(mystring) - set(allowed)

Solution 4:

If it were not for the dashes and underscores, the easiest solution would be

my_little_string.isalnum()

(Section 3.6.1 of the Python Library Reference)