Cross-Browser Extensions API?
There are tools for developing cross-platform browser plugins.
Are there any similar tools or APIs for browser extensions (i.e. toolbars, or filter systems like AdBlock)?
Yes, but unfortunately each one has significant drawbacks.
Kynetx
supports IE, Firefox, Safari, and Chrome, but extensions are dependent on the Kynetx extension to work. In addition, Kynetx apps are built using an odd, proprietary Kynetx Rules Language. There is no cost to use the Kynetx platform. UPDATE: Kynetx has shut downWebMynd
supports IE, Firefox, Safari, Chrome, and is in private beta. From what I have been able to determine, WebMynd is a very small San Francisco-based startup that is still getting off the ground. Their library is standalone and provides a Javascript API, and they use a monthly billing model. UPDATE: company pivoted, now for mobile, trigger.ioCrossrider
supports IE, Firefox, Chrome and Safari. Recently out of beta (as of June 2012), Crossrider is free and offers a jQuery-based development experience. Publishing methods for your extensions include an embeddable website widget, download links, and a customized windows installer. However, CrossRider does not allow you to self-host your plugin files. UPDATE: As of 27.09.2016, CrossRider is terminated and it will be available only until 27.10.2016.KangoExtensions
supports IE, Firefox, Chrome, Safari, Opera. Seems very new and bleeding edge, like Crossrider, also looks promising.UPDATE: No updates in over two years.
IE and Firefox seem to be the hardest to develop for, although Mozilla JetPack aims to greatly simplify Firefox extension development.
There is also very young framework for cross-browser development called BabelExt, from the author of Reddit Enhancement Suite. I like that it builds restartless Jetpacks for Firefox.
See more on github, https://github.com/honestbleeps/BabelExt
Mozilla has recently introduced WebExtensions API. For now it's actually almost a copy of Chrome Extension API (but some APIs aren't implemented yet). It has been introduced just a few days ago so it still has a lot of problems and unsupported APIs. But if you're interested in writing extensions for Chrome, Opera and Firefox you should pay attention to the state of WebExtensions API.
UPD 27.10.16. There is a page Are-we-WebExtensions-Yet which summarises the current state of WebExtensions. By their current estimate 44.99% extensions from Chrome store should work in Firefox.