Comma usage with "of course"
Solution 1:
When you use the phrase 'of course' you have to use a comma. It doesn't matter whether it is in the middle or at the beginning of a sentence. I'm completely sure that the first two sentences are right, but the last one seems wrong to me.
Solution 2:
Using 'of course' without commas can imply a refutation of prior skepticism. In these cases, the speaker emphasizes 'course.'
Compare these two separate situations:
Frazzled mother: "Are you going to remember your lunch?" Indignant child: "Of course I am going to remember my lunch!"
Calm child to happy mother: "I'm off to school! Of course, I am going to remember my lunch!"
In the first case, the child would emphasize 'course' and not pause anywhere.
So, then, presume someone asked "Weren't your thoughts on that rather goofy?" It could be perfectly correct to answer "Considering the matter in hindsight, those thoughts of mine were of course ridiculous." Here, 'course' is emphasized in speech, which leads to no pauses, and hence, no commas.
If you don't want to imply prior skepticism, you should probably be using a comma.
An example and explanation of this is given at https://www.dailywritingtips.com/how-to-punctuate-introductory-phrases/, which explains comma usage for introductory phrases. This particular quote uses 'of course' as an example of the Emphasis class of adverbial conjunctions, which should generally be followed by a comma:
(An exception can be made for this particular phrase: There’s a subtle but distinct difference between “Of course, you’ll want to do it your way” and “Of course you’ll want to do it your way.” In the first sentence, your is stressed; in the second, course, perhaps accompanied by a sneer, is emphasized, with a secondary stress on your — and likely an exclamation point to signal emotion.)