Why notepad++ plugins don't work in macro?
At least in version 5.9.2 of Notepad++, Settings-> Shortcut Mapper has a button for "Plugin commands."
You can assign a plugin command directly to any shortcut of your choice, without needing a macro or script. "TextFx -> Convert -> Decimal To Hex" appears on line 144 as "C:Convert Decimal Number to Hex".
Or, add the plugin command to your right-click menu. Use Settings-> Edit Popup ContextMenu. Simpler than I expected!
You could try to do it with the pythonscript plugin (you can get it with the plugin manager): http://sourceforge.net/projects/npppythonscript/
Once the plugin is installed create a new script and put :
notepad.runMenuCommand('TextFX Convert', 'Decimal To Hex')
docs: http://npppythonscript.sourceforge.net/docs/latest/notepad.html#Notepad.runMenuCommand
You can then add it to the pythonscript menu, and assign it a shortcut from the "shortcut mapper" window.
Option 2
You can also try the more troublesome way by using the Universal macro patch, which is located over here:
http://sourceforge.net/tracker/?func=detail&aid=3132767&group_id=95717&atid=612384
You would need to compile n++ though, and I'm not sure on how to do that.
Good luck.