How should I update Xcode when I installed it from the app store but it hasn't updated to most recent version?

Xcode is not automatically updated as a consequence of updating to Big Sur.

Just installing Xcode from the App Store itself won't automatically ensure it is updated. This only happens if you have enabled automatic updates in the Mac App Store settings.

In any case, there can be a lag from the time the new version is released and until the automatic update feature on your Mac notices it and upgrades.

You can always just open the App Store, go to the page for Xcode and hit the "Update" button there.

If you download and install Xcode manually from a link, then no, it will not automatically update from that point on.