Why is it allowed to access a private field of another object?

Private fields protect a class, not an instance. The main purpose is to allow a class to be implemented independently of its API. Isolating instances between themselves, or protecting the instance's code from the static code of the same class would bring nothing.

This is because they are of the same class. This is allowed in Java.

You will need this access for many purposes. For example, in an implementation of equals:

public class A {
  private int i;

  @override public boolean equals(Object obj){
     if(obj instanceof A){
        A a = (A) obj;
        return a.i == this.i; // Accessing the private field
     }else{
       return false
     }
  }
}