Remove non-existent AFP server from Finder sidebar?

You probably configured Avahi (a free zero-configuration networking implementation for Linux including multicast DNS/DNS-SD service discovery AKA Bonjour) manually to propagate the AFP-service running on your Linux server in the .local network.

Even after disabling the AFP service on the Linux server, the Linux server's Bonjour name will still be registered then and thus shown in the Mac's sidebar.

You can check this by entering dns-sd -B _afpovertcp._tcp in Terminal.app on the Mac.

So remove the manually created config file (here: /etc/avahi/services/afpd.service) and restart the avahi service (probably sudo service avahi-daemon restart) to get rid of the non-existing AFP server name in the Mac's sidebar.