In Reflected XSS, why do we need to sanitize single quote, double quote, ampersand, and backslash

Based on this article

https://resources.infosecinstitute.com/topic/how-to-prevent-cross-site-scripting-attacks/

Reflected XXS happens when data injected is reflected in the response. I get the idea that if I, for example, have a search box in my page and the search term inputted by a user is displayed in the page, someone could write as a search term:

<script>alert('x');</script>

and that would be read as regular HTML element in the page that displays the response.

But lets say greater than and less than are already blocked in input (meaning they wouldn't be able to put in script tags or any tag), what's the issue if I allow single quote, double quote, ampersand, and backslash reflected in the response. I'm trying to make sense of it but I am not sure if I am understanding correctly.

Today the web stack is big and complex with many languages. We have HTML, CSS, JavaScript, VB-Script, SVG, URLs…

Each with its own rules for:

• Encoding
• Quoting
• Commenting
• Escaping

Also, each one can be nested inside each other:

And just replacing <> fixes some issues, but not all of them as you don't know where you data will end up, is it in HTML? as a HTML Attribute? inside a JavaScript string? Each one needs different encoding to become safe.

So, the world is a bit more complicated.....