Asus Zenscreen Display Resolution Configuration (macOS SwitchResX)
The display is only capable of a maximum res of 1920x1080 [Full HD]. Nothing higher than that is going to work.
Your choices are tight & small or fuzzy & larger. That's it. You need a retina-type screen to sub-divide from higher resolutions.