How do I get to the Windmill?

You need to go to Zora's Domain, and you can get there without swimming. Merge into the wall near the Zora who says he can't swim, and that will get you around the deep water and from there, Zora's Domain is nearby. Once you get there, go in and talk to the Zoras, who will ask you to do something for the Zora Queen:

You will have to get the Smooth Stone that the man you meet outside Zora's Domain stole (he will be in Kakariko Village, after you talk to the Zoras)

Once you accomplish that task, you will be rewarded with the flippers that allow you to swim.

As you presumed you will have to learn to swim which to link means,get Zora's Flippers.

She can be found in Zora's domain.

Here is more detailed guide