Should I use Java8/Guava Optional for every method that may return null?

Optional is used to represent nullable object, Some uses of this class include

1. As a method return type, as an alternative to returning null to
   indicate that no value was available
2. To distinguish between "unknown" (for example, not present in a map) and "known to have no value" (present in the map, with value
   Optional.absent())
3. To wrap nullable references for storage in a collection that does not support null (though there are several other approaches to this that should be considered first)

For the first case, do I need to return Optional in all nullable return method?

So What’s Wrong with Optional?

The question we face is: will JDK 8 Optional objects get rid of null references? And the answer is an emphatic no! So, detractors immediately question its value asking: then what is it good for that we couldn't already do by other means?

Unlike functional languages like SML or Haskell which never had the concept of null references, in Java we cannot simply get rid of the null references that have historically existed. This will continue to exist, and they arguably have their proper uses (just to mention an example: three-valued logic).

I doubt that the intention with the Optional class is to replace every single nullable reference, but to help in the creation of more robust APIs in which just by reading the signature of a method we could tell if we can expect an optional value or not and force the programmer to use this value accordingly. But ultimately, Optional will be just another reference and subject to the same weaknesses of every other reference in the language (e.g. you could return a null Optional). It is quite evident that Optional is not going to save the day.

How these optional objects are supposed to be used or whether they are valuable or not in Java has been the matter of a heated debate in the project lambda mailing list. From the detractors we hear interesting arguments like:

• The fact that other alternatives exist ( e.g. IDES like IntelliJ and Eclipse IDE support a set of proprietary annotations for static analysis of nullability, the JSR-305 with annotations like @Nullable and @NonNull).
• Some would like it to be usable as in the functional world, which is not entirely possible in Java since the language lacks many features existing in functional programming languages like SML or Haskell (e.g. pattern matching).
• Others argue about how it is impossible to retrofit preexisting code to use this idiom (e.g. List.get(Object) which will continue to return null).
• And some complain about the fact that the lack of language support for optional values creates a potential scenario in which Optional could be used inconsistently in the APIs, by this creating incompatibilities, pretty much like the ones we will have with the rest of the Java API which cannot be retrofitted to use the new Optional class. This is pretty much your question here. In languages with support for optional types like in Ceylon or like in Kotlin, you would not even question this.
• A compelling argument is that if the programmer invokes the get method in an optional object, if it is empty, it will raise a NoSuchElementException, which is pretty much the same problem that we have with nulls, just with a different exception.

So, it would appear that the benefits of Optional are really questionable and are probably constrained to improving readability and enforcing public interface contracts.

I do believe that the adoption of this Optional functional idiom is likely to make our code safer, less prompt to null dereferencing problems and as a result more robust and less error prone. Of course, it is not a perfect solution because, after all, Optional references can also be erroneously set to null references, but I would expect that programmers stick to the convention of not passing null references where an optional object is expected, pretty much as we today consider a good practice not to pass a null reference where a collection or an array is expected, in these cases the correct is to pass an empty array or collection. The point here is that now we have a mechanism in the API that we can use to make explicit that for a given reference we may not have a value to assign it and the user is forced, by the API, to verify that.

Quoting Google Guava's article about the use of optional objects:

“Besides the increase in readability that comes from giving null a name, the biggest advantage of Optional is its idiot-proof-ness. It forces you to actively think about the absent case if you want your program to compile at all, since you have to actively unwrap the Optional and address that case”.

So, I guess it's up to every API designer to choose how far they want to go in the use of Optional.

Some influential developers like Stephen Colebourne and Brian Goetz have published a few interesting articles more recently on the proper use of optional. I particularly found useful the following:

• Should Java Getters Return Optional
• Java SE 8 Optional, a pragmatic approach

Compared with Guava, one annoying problem with java.util.Optional is that it does not provide a method like

orElse(Optional<T>): Optional<T>

which is, on the other hand, defined in com.google.common.base.Optional as

or(Optional<T>): Optional<T>

The lack of this specific feature limits monadic applications of Java 8's Optional.

UPDATE:

Guava's or(Optional<T>) can be replicated as in with Java 8 Optional as

optionalA.map(Optional::of).orElse(optionalB)

or

optionalA.map(Optional::of).orElseGet(() -> computeOptionalB)

UPDATE:

In Java 9 (finally!) you'll be able to use Optional.or(Supplier<Optional<T>>):

optionalA.or(() -> computeOptionalB)

That's neat!