Is there a difference between "Joe said" and "said Joe"?

Solution 1:

OP's second version is standard usage, which is why @Barrie calls it the "uninverted form". But note he's only referring to inversion of subject - verb (the object here being the quoted speech).

The most common structure for English sentences is subject - verb - object...

Joe said "The sky is blue".

...and the most common "sentence inversion" is object - verb subject...

"The sky is blue," said Joe.

There's nothing "ungrammatical" about object - subject - verb as in OP's first version, which may be prefered for stylistic reasons in some contexts. And least common, verb - subject - object is still perfectly valid English too...

Said Joe, "The sky is blue".

...but you'd normally only see this in poetic or other stylised writing, rather than in speech.

Solution 2:

Both may be used after what is said, but only the uninverted form is normally used before what is said.

Solution 3:

Here is what Browne and King have to say in Self-Editing for Fiction Writers:

Place the character’s name or pronoun first in a speaker attribution (“Dave said”). Reversing the two (“said Dave”), though often done, is less professional. It has a slightly old-fashioned, first-grade-reader flavor (“Run spot, run” said Jane). After all, “said he” fell out of favor sometime during the Taft administration.

King, Dave; Browne, Renni (2010-06-03). Self-Editing for Fiction Writers, Second Edition: How to Edit Yourself Into Print (Kindle Locations 1120-1122). Harper Collins, Inc.. Kindle Edition.