How can these games be removed from this list I have on Steam?
I guess you mean the 'Show hidden icons' button ('^') on your Windows taskbar, and the Steam "quick menu":
It shows the games you have recently played or acquired (I've not played RobocraftX, for example), so, if you want to get rid of those specific items, try playing or downloading other games.