The last moment of something but not sure when to use

Solution 1:

My hands were trembling until the last moment of checking the results.

There is nothing wrong with this.

Until the last moment, my hands were trembling by checking the results.

This isn't quite right. You could change the second part to my hands were trembling as I checked the results, although I think (personally, it's a bit subjective) that it would be better the other way around, i.e.

As I checked the results, my hands were trembling until the last moment.

but as I suggested in the comments, something like

I found my hands were shaking, which continued until I had finished checking the results.

would seem more natural, because from the sounds of your example sentence, it seems you're trying to say that your hands stopped shaking when you had verified your results, whereas the last moment means just before you finished.

For example, if you did find a mistake, that would be "at the last moment".