Saved passwords in Safari keep coming back

Solution 1:

Found the solution. I searched for keychain in spotlight and found something called Keychain Access. All the passwords were here under Login. When I deleted them from here, it removed them from Safari as well.

Safari's preference to Remove All just removes the locally stored passwords in Safari and not the ones that are in the user keychain.

Additionally, go to System Preferences and make sure that keychain syncing is disabled if you don't want any data stored in iCloud. You can empty that keychain if desired using Keychain Access before disabling the sync for extra security if you are sure no other devices depend on keychain syncing.