Time Machine drive takes long time (~5 minutes) to mount on boot

On macOS you don’t need to install any drivers for external drives that comply with the USB Mass Storage class, which the Samsung does (unless you’re doing something funny like using their proprietary encryption scheme). Any external USB SSD will ‘just work’ when you plug it in. I would uninstall the driver as it can do more harm than good, particularly with the major rewrites to the driver architecture in Catalina and Big Sur which not all third-party developers have caught up with yet.

You’re definitely right to worry about the slow mounts on boot, though. That can be due to a variety of things. One of these is a heavily worn SSD, which could be the case if you’re using it for Time Machine, but two years isn’t a very long time. It may still be valuable for you to run the Smartmontools package (which will also require you to install USB SMART support) in order to determine the wear & lifespan on your SSD. Heavily worn SSDs will have increased latencies which get worse over time. If you’re only seeing this issue on first mount and you’re not seeing a reduction in read or write speed, though, then that’s unlikely to be the issue.

It sounds like you’re more concerned with your kernel panics. Without seeing the panic reports, we can’t determine if they’re something you should worry about or if they’re even related to your SSD. Can you please provide your panic logs? They’ll be found in /Library/Logs/DiagnosticReports and have the file extension .panic. Put those up on PasteBin and we can review them.

To your question of preserving your backups: I don’t know of a straightforward way to cherry-pick specific checkpoints (although there may exist tools out there that I just haven’t looked for, so feel free to DuckDuckGo it). Time Machine is persnickety about the contents of its backup bundle and any omissions could result in an unreadable backup. I would instead try cloning your entire SSD to another drive wholesale and pointing TM to that. (But I have never done this and it’s entirely possible that TM will detect the different device UUID and force you to recreate the backup anyway.) You have to be ready to accept that you might need to start from scratch.