How do I use testfor to testfor players nearby?
You have a space in the selector.
testfor @a[r=3]
No space between @a and [r=3].
Just to note it, the comparator output signal strength will be equal to the amount of players it finds in that radius.