How can I remove songs from the Music app on my iPhone?
iOS 7 has changed the face of the Music app somewhat. The biggest thing that was introduced is that now, all of you iTunes store music purchases show up automatically in the Music.app. This allows you to stream those songs without actually storing them on your device (an excellent option).
As a result you have two options:
You can manage music the old way by opening Settings.app -> Music -> and disabling "Show All Music". This will prevent your purchases from displaying in the music app. To delete any songs that you have on there, you can swipe (from right to left) and select Delete (on the actual songs themselves - this doesn't work on albums).
When "Show All Music" is enabled, I don't believe there is any way to remove songs that were purchased through iTunes. If you are not on wifi, you can go to Settings.app -> Cellular -> disable "Music", which will prevent any songs that haven't been downloaded from streaming over cellular data (and which would prevent those songs from playing if that's your desire).
The following is is for iOS 6
Yes, you can.
To delete single songs in Music.app slide from one direction towards the other on a particular song and select "Delete". You can also use this approach with Playlists, Albums, Artists, etc.
If you seeking to delete or edit playlists you can see those buttons at the top of each playlist.
Keep in mind, unless you make the needed changes on your computer, these deleted songs/artists/albums/playlists with sync back to your iPhone.