SPF fail vs. soft-fail pros and cons

Question

What are the advantages and disadvantages of using Fail vs. a SoftFail in my SPF record?

What I found on the topic

Back in 2007, knowledgeable-seeming folks seem to have said SoftFail was just for testing and encouraged changing it to reject once you have everything setup properly (here and here)

This forum post calls SoftFail "wrongly configured", but then says that Google uses it. I trust Google for best practices more than a random forum poster! I checked and indeed, Gmail uses ~all (a SoftFail) in their SPF record.

On the sender end of things, email deliverability experts seem to encourage using SoftFail:

Fail "is more aggressive [than SoftFail] and is known to create more issues than it solves (we don’t recommend it)."

—Postmark SPF Guide

That's rather vague.

"I generally recommend publishing ~all records for my clients. There’s not a huge benefit to publishing -all and sometimes mail gets forwarded around. The one time I recommend a -all record is when a domain is getting forged into spam. Domain forgery can cause a lot of bounces. The amount of bounces can be bad enough to take down a mail server, particularly those with a small userbase. Many ISPs will check SPF before sending back a bounce and so a -all record can decrease the amount of blowback the domain owner has to deal with."

—Email deliverability consultants Word to the Wise

Yet how will a webmaster know if there is a substantial amount of domain forgery going on? Isn't a best practice to prepare for the worst and anticipate forgery in advance?

On the receiving end, Terry Zink, who works in enterprise spam filtering, offers a strong case for hard Fail to prevent phishing emails from going through, and says most people use SoftFail because organizations are more afraid of emails being lost than about forged emails. What is the likelihood that a forged phishing email which SPF SoftFails actually gets to someone's inbox?

Sondra, you already found a related question, but the highest scoring answer doesn't do justice to your questions, in my opinion.

Let me start with your last question: What is the likelihood that a forged phishing email which SPF SoftFails actually gets to someone's inbox? Huge! Combined with DMARC quarantine/reject policy and the receiving mailbox on Office 365, Outlook.com, Gmail, Yahoo or other major hoster, very unlikely.

Your first question: What are the advantages of a Fail over a Soft Fail SPF record. As mentioned in your own research, a disadvantage will be that forwarded emails will be rejected unless the bounce address (return-path) is rewritten by the forwarder. An advantage is the domain being better protected against spoofing, but only for the simplest of attempts.

As mentioned above, the caveat with SPF is that it checked against SMTP envelope sender, saved in the message header field Return-Path. An actual recipient will have no knowledge of this field, because most email clients will only present them with an other field, the header From. For example: I send an email with a header From: [email protected], but I use [email protected] as the envelope sender. Even though microsoft.com publishes a Fail SPF policy, it will not fail SPF because example.com does not publish an SPF record. The recipient will just see an email from [email protected].

This is where DMARC comes in. DMARC requires authentication alignment with the domain used in the From header, either for SPF or DKIM. Meaning the domain used in the envelope sender (Return-Path) and the header From: should share an organizational domain. DMARC only cares about PASS results for underlying authentication mechanisms (SPF or DKIM), so, for SPF in that respect ~all and -all are treated exactly the same.

Terry Zink has published multiple articles on DMARC, one of which: Enhanced email protection with DKIM and DMARC in Office 365

There is a lot one could learn about SPF, DKIM and DMARC, which is beyond the scope of this answer. DMARC is not easy to implement nor flawless, but it does protect your domain against spoofing, much better than only SPF. Also, all depends on the receiving party and how they deal with SPF and DMARC (if at all).