How can I break a symbolic link loop?
It's not a problem to delete symbolic links. I'm not sure why you think that you need to delete the file the link points to.
Just delete them. Try this:
ln -s thing1 thing2 # thing1 does not exist
ln -s thing2 thing1 # circular reference
rm thing1 thing2 # no problem
Nevermind, I can delete both links simultaneously with rm. Why did someone say I had to delete the target file...
When you delete a symbolic link in Linux, the link is deleted and not the target file.