Can I put mods into an existing Minecraft server?
Yes, (usually) you can do that.
Not being able to would be pretty pointless, since no one sets up a whole server with all plugins and mods without ever joining it to test things before it's set up completely.
Some mods might require a new map (usually their own one), though. But I've never seen a mod requiring a brand new server.