Why PHP variables start with a $ sign symbol?

Because PHP was based on Perl which used $, though the symbols Perl used were meaningful and plenty used to indicate the data type, ( such as @ used to indicate an array ) PHP just has $.

PHP in its early stages was a simplistic version of Perl but over time incorporated more of Perl's features, though one may argue PHP was for a long time a simplistic primitive version of Perl since before PHP 5.3 it did not include features that have been around in other languages such as closures/namespacing.

http://en.wikipedia.org/wiki/PHP

Larry Wall, the creator of Perl, was inspired to use $ from shell scripting: http://en.wikipedia.org/wiki/Sigil_%28computer_programming%29

This has been common in computer languages for a long time, that's all. Long before Perl, too! For instance, check out Commodore 64 BASIC

10 PRINT "WHAT IS YOUR NAME?"
20 INPUT A$
30 IF A$="BAHKTIYOR" THEN PRINT "HEY CHECK OUT THAT DOLLAR SIGN"

In BASIC the $ was after the variable name, however.