Why does the Java compiler 11 use invokevirtual to call private methods?

This was done as part of https://openjdk.java.net/jeps/181: Nest-Based Access Control, so that the JVM can allow access to private methods from nested classes.

Before that change, the compiler would have to generate a package-protected synthetic method in the Base class, which the nested class invokes. That synthetic method would in turn call the private method in the Base class. The feature in Java 11 enhances the JVM to allow that without the compiler having to generate a synthetic method.

Concerning the point on whether invokevirtual will call the method in the Derived class, the answer is no. The private method is still not subject to method selection of the runtime class (this has never changed):

During execution of an invokeinterface or invokevirtual instruction, a method is selected with respect to (i) the run-time type of the object on the stack, and (ii) a method that was previously resolved by the instruction. The rules to select a method with respect to a class or interface C and a method m_R are as follows:

1. If m_R is marked ACC_PRIVATE, then it is the selected method.

EDIT:

Based on the comment "Would it be valid to still use invokespecial if the private method is called from the method owner class and use invokevirtual if it is called from a nested class?"

As Holger mentioned, yes it is valid, but based on the JEP, I'm guessing a decision was made to switch to invokevirtual for simplicity (I cannot confirm this though, it's just a guess):

With the change to the access rules, and with suitable adjustments to byte code rules, we can allow simplified rules for generating invocation bytecodes:

• invokespecial for private nestmate constructors,
• invokevirtual for private non-interface, nestmate instance methods,
• invokeinterface for private interface, nestmate instance methods; and
• invokestatic for private nestmate, static methods

Another interesting note from JDK-8197445 : Implementation of JEP 181: Nest-Based Access Control:

Traditionally, invokespecial is used to invoke private members, though invokevirtual also has this capability. Rather than perturb the complex rules about supertypes enforced by invokespecial, we require invocations of private methods in a different class to use invokevirtual.

Top-Up To an already good answer

Thought it would be proper to add some more info to the answer already provided and accepted, although it isn't strictly necessary, it may help broaden the understanding, hence it's within SO users best interest.

Nest-Based Access Control

In earlier versions, before Java 11, as already pointed out by @m-a in the accepted answer, the compiler would need to create bridge methods to allow classes to access each other's private members in such conditions. These accessibility-broadening bridge methods gets called in the execution context, where the compiler inserts code into a running program.

Doing this increases both the size of the deployed applications and increases the complexity, plus it makes it harder to understand what is going on behind the scenes.

Java 11 introduced the concept of nest-based access control. Along with the notion of nestmates and the associated access rules within the JVM, this allows classes and interfaces to be nested within each other.

Nested types can be private fields, methods, and constructors.

Using the updated reflection API you can now query for information about the nest-based access control functionality.

Some New Goodness in Java 11

The getNestHost() method is used to get the name of nest host and the isNestmateOf() method can be used to check whether a class is a nestmate. Also, getNestMembers() method returns an array of nest members.

Here is a link to a generic example, courtesy of Baeldung.com, Nest Based Access Control that makes the benefits stand out pretty well IMHO.

Notice that there isn't a compiler generated bridging method in the disassembled code. Also, the Inner class can now make a direct call to the outerPrivate() method in the example linked to above.