Word for "moving on a ledge"
Solution 1:
He sidled along the ledge.
moving sideways or obliquely.
Solution 2:
One could edge along the ledge. This implies the sideways movement, as well as the cautiousness.
17. to move sideways: to edge through a crowd.
18. to advance gradually or cautiously: a car edging up to a curb.
Although inching doesn't imply one's back is against the wall, it does imply the careful movement one would exhibit.
Perhaps use both!
He edged along, inching his way back to the window.