What is the difference between American and British pronunciations of "world" and "girl"?
I can definitely hear a distinct difference but I am not sure if it is from the long vowel or from the "r".
There is a definite difference in the /r/; see this on rhotic and non-rhotic accents where rhotic accents are typical in most of America but not in England outside the south-west. So for girl (/ɡɝl/ and /ɡɜːl/) and world (/wɝld/ and /wɜː(ɹ)ld/) the /r/ is almost not pronounced in non-rhotic accents, though the vowels are similar in both accents.