Detect if an application uses gtk or qt
Solution 1:
An application can use Qt and Tk and Gtk (despite this is generally -but not always- a very bad practice).
So the only way is to start the app and examine pmap output using app's pid, as Lekensteyn proposed in the comment.