Brokering peace between quarian and geth options don't appear

Solution 1:

i think you need either very high renagade or very high paragon score to actually pull it off, this is what i did to make peace

saved the two quarians in ME2 (in the first and second mission you do with tali before she joins your team), rewrote heretics, tali not exiled, saved admiral, did the reaper hack thing, and had a very high paragon score

Solution 2:

Near as I can tell, no one has a good or right answer to this. The clearest indication and closest thing I think to be right is that during ME2, there's a conversation in which you speak to one of the admirals and they thank you for helping Tali. One of the options is saying that you did it for Tali. Another is encouraging them to go to war. The third is telling them something along the lines of seeking peace. This is the closest thing to a "right" answer that I've heard. No one really has any idea though.