Note: Read the edits at the bottom


OK, I've found a way to do it, but I had to change the code of multiple classes, so that the bottom sheet would know of the state of the appBarLayout (expanded or not), and ignore scroll-up in case it's not expanded:

BottomSheetLayout.java

Added fields:

private AppBarLayout mAppBarLayout;
private OnOffsetChangedListener mOnOffsetChangedListener;
private int mAppBarLayoutOffset;

init() - added this:

    mOnOffsetChangedListener = new OnOffsetChangedListener() {
        @Override
        public void onOffsetChanged(final AppBarLayout appBarLayout, final int verticalOffset) {
            mAppBarLayoutOffset = verticalOffset;
        }
    };

Added function to set the appBarLayout:

public void setAppBarLayout(final AppBarLayout appBarLayout) {
    if (mAppBarLayout == appBarLayout)
        return;
    if (mAppBarLayout != null)
        mAppBarLayout.removeOnOffsetChangedListener(mOnOffsetChangedListener);
    mAppBarLayout = appBarLayout;
    mAppBarLayout.addOnOffsetChangedListener(mOnOffsetChangedListener);
}

onDetachedFromWindow() - added this:

    if (mAppBarLayout != null)
        mAppBarLayout.removeOnOffsetChangedListener(mOnOffsetChangedListener);

onTouchEvent() - added this:

      ...
      if (bottomSheetOwnsTouch) {
        if (state == State.EXPANDED && scrollingDown && mAppBarLayout != null && mAppBarLayoutOffset != 0) {
            event.offsetLocation(0, sheetTranslation - getHeight());
            getSheetView().dispatchTouchEvent(event);
            return true;
        }
      ...

Those were the main changes. Now for what sets them:

MyFragment.java

onCreateView() - added this:

    mBottomSheetLayout.setAppBarLayout((AppBarLayout) view.findViewById(R.id.appbar));

I also added this function:

 public void setBottomSheetLayout(final BottomSheetLayout bottomSheetLayout) {
    mBottomSheetLayout = bottomSheetLayout;
}

Now this is how the activity tells the fragment about the appBarLayout:

            final MyFragment myFragment = new MyFragment();
            myFragment.setBottomSheetLayout(bottomSheetLayout);
            myFragment.show(getSupportFragmentManager(), R.id.bottomsheet);

The project is now available on GitHub:

https://github.com/AndroidDeveloperLB/ThreePhasesBottomSheet

I hope it doesn't have any bugs.


The solution has bugs, sadly, so I won't mark this answer as the correct one:

  1. It only works well on Android 6 and above. Others have a weird behavior of showing the bottom sheet expanded for a tiny fraction of a time, each time when showing it.
  2. Orientation changes do not save the state of the scrolling at all, so I've disabled it.
  3. Rare issue of being able to scroll inside the bottom sheet's content while it's still collapsed (at the bottom)
  4. If a keyboard was shown before, the bottom sheet might get to be full screen when trying to peek.

If anyone can help with it, please do.


For issue #1, I've tried adding a fix by setting the visibility to INVISIBLE when the bottom sheet doesn't peek yet, but it doesn't always work, especially if a keyboard is shown.


For issue #1, I've found how to fix it, by just wrapping (in "fragment_my.xml") the CoordinatorLayout with any view that you wish to use (I used FrameLayout), and also put a full-sized view in it (I just put "View") , as such:

<FrameLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">
    <!--This full sized view, together with the FrameLayout above, are used to handle some weird UI issues on pre-Android-6 -->
    <View
        android:layout_width="match_parent"
        android:layout_height="match_parent"/>

    <...CollapsingToolbarLayout 
    ...

It probably confused the bottomSheet when I had the CoordinatorLayout being its view. I've updated the project, but still, if there is any way to have a nicer solution, I'd like to know about it.


In recent months, Google has published its own bottomSheet class, but as I've found it has a lot of issues, so I can't even try it out.


BIG UPDATE

Because there were like 4 or 5 questions about the same topic, BUT with DIFFERENT requirements, and I tried to answer all of them, but a non-polite admin deleted/closed them, making me create a ticket for each one and changing them to avoid "copy-paste" I will let you a link to the full answer in where you can find all the explanation about how to get full behavior like Google Maps.


Answering your question

How to mimic Google Maps' bottom-sheet 3 phases behavior?

With support library 23.x.x+ you can do it by modifying the default BottomSheetBehavior, adding one more stat with the following steps:

  1. Create a Java class and extend it from CoordinatorLayout.Behavior<V>

  2. Copy paste code from the default BottomSheetBehavior file to your new one.

  3. Modify the method clampViewPositionVertical with the following code:

    @Override
    public int clampViewPositionVertical(View child, int top, int dy) {
        return constrain(top, mMinOffset, mHideable ? mParentHeight : mMaxOffset);
    }
    int constrain(int amount, int low, int high) {
        return amount < low ? low : (amount > high ? high : amount);
    }
    
  4. Add a new state:

    public static final int STATE_ANCHOR_POINT = X;
    
  5. Modify the next methods: onLayoutChild, onStopNestedScroll, BottomSheetBehavior<V> from(V view) and setState (optional)

I'm going to add those modified methods and a link to the example project.

And here is how its looks like
CustomBottomSheetBehavior