Are "bots" that only observe the screen allowed?
Alt1 toolkit works exactly like this - it attaches to your Runescape process and observes images. One of the functions is to alert the user before they are lobbied due to the AFK timer.
And it has been approved by an official Jagex staff over 5 years ago, stating that since it does not interact with the game directly, it is fine to use. His exact words are:
"It doesn't appear you can use it to control or automate game play, so it's unlikely it could cause any problems with our anti-cheating detection methods, ie; players shouldn't be banned for using it.
However, with using any 3rd party software, you do so at your own risk, and we'd never condone the use of such software whilst playing RuneScape."
So the TL;DR - it's fine to use, but as with all third party software you need to be careful, and make sure it isn't breaking any of our Rules.
Thus, since your program does not even attach itself to the runescape process, I would say that it is just as safe to use since it does not break Runescape's Terms of Service.
I would say no it is not forbidden.
If it really just takes a screenshot and compares an area of the screenshot with a base image it should not even be traceable by Runescape. Taking a Screenshot is usually nothing they should be able to ban you for. Since you say it would only notify you, you still have to put the work into the game yourself.
But obviously botting and scripting is against games term and so you would be better off asking in the Runescape forum or asking a mod about it.
I would not risk my account before I found an answer in the official game forum.