How to delete all the recordings from the Voice Memos app on iOS?

Open the app and you'll find a little symbol with 3 horizontal stripes in the right bottom corner. You'll get a list and you can tap the memo and click "delete" right afterwards.

Or in iTunes you can select all voice memo's after you've synced them to your PC and delete them from there.

I just found another easy way to keep them on your computer and get them off your iPhone!

Just go to the playlist with all your voice memos in iTunes, Select All, press Cmd + i or right click and select information from contextual menu, go to options and change media type to music from voice memos. It will also be transferred within your finder structure to a new artist folder that's called "alan smithees iPhone 5" or whatever and will not be included in your synched voicemail anymore.