Why doesn't vim always give me the option to delete a swap file?

Solution 1:

From :help swap-exists:

D  Delete the swap file.  Use this when you are sure you no longer need it.
   For example, when it doesn't contain changes, or when the file itself is
   newer than the swap file.
      On Unix this choice is only offered when the process that created the
   swap file does not appear to be running.

So it sounds like Vim somehow believes another Vim process is still actively running on that file.

Solution 2:

Your second question: "how can I get vim to allow me to delete swap files from within vim"

is answered here: What's the easiest way to delete Vim swapfiles I've already recovered from?