I found a headless horseman! Now what?

Solution 1:

He is meant to lead you to a graveyard filled with a bunch of skeletons and a locked master chest filled with a few valuable items.

Not sure why he didn't for you though.

Source.