What's the difference between MemoryCache.Add and MemoryCache.Set?

I read the MSDN documentation but didn't really understand it.

I believe that the behavior of Set is "replace existing, or add" (atomically).

Is that correct?

Add does nothing (returns false) if there is already a value for that key. Set does an insert or update, as necessary.

Remove + Add would leave a gap in the middle when another thread querying that key would get no clue (Set does not; the swap is typically atomic); as such, while Set has the same end result as Remove + Add, the mechanism difference is important since it could impact other callers.

For example of Add:

Return Value

Type: System.Boolean true if insertion succeeded, or false if there is an already an entry in the cache that has the same key as key.