SwiftUI changing the position of a view programmatically
To me, your wild guess of using the .offset(x:y:) modifier seems to put you on the right track.
Grab the position from the GeometryReader proxy of the view in question in .onAppear{} and restore it using the .offset.
This earlier answer of mine does this using the x-axis only, should be easy enough to apply it to the y-axis as well : https://stackoverflow.com/a/60854527/301790
If you need more guidance consider posting code ready to copy-and-paste into Xcode.