Which file system to use in between OSX and Linux

Linux can write to HFS+ without problems. If you really need read/write support from both OSes, then you only have one choice, namely to format as HFS+, since Time Machine won't work with others and macOS can't natively write to ext4 (see How can I mount an ext4 file system on OS X?)

Since Linux allows writing to journaled HFS+ volumes, use Disk Utility.app on your Mac to format the partition with HFS+ (journaled).

Newer macOS versions will by default use APFS instead of HFS+, but Linux support for APFS is still quite limited. There's a read-only FUSE driver, but in the future write support may be added.

Just to add to slhck's answer, formatting the partition as non-journaled was a little tricky using Disk Utility on OSX 10.7 as the option for HFS+ nonjournaled is not apparent in the formatting (erase) options.

The solution is to format it as HFS+ journaled then turn off journaling by selecting the partition then, while holding option key, open the "File" menu and select "Disable Journaling".

Weird.

See https://discussions.apple.com/thread/3232454?start=0&tstart=0

There is a modern possibility now for macOS and Linux compatibility at the filesystem level: ZFS. It works well and it's supported in both environments (see https://zfsonlinux.org/ and https://openzfsonosx.org/).

It's a possibility I haven't seen mentioned until now in this context, here, or in answers to similar questions.

Pros:

• Modern, actively developed, and future proof open-source filesystem.
• It plays well with permissions and UNIX in general (FAT-like filesystems don't do that in any of their incarnations, as far as I know).
• Extremely well suited for a shared partition in a dual-boot system.
• As a bonus, ZFS offers a lot of unique features, like data integrity, pool management, copy on write, snapshots (https://itsfoss.com/what-is-zfs/). Only recently Btrfs (Linux) and APFS (macOS) have been trying to catch up with similar features.

Cons:

• It might require a bit of knowledge and management, given that it's both a file system and a volume manager rolled into one.
• It requires the installation of third-party system software on both macOS and Linux (on macOS it can be installed using brew cask and on Linux, it might be supported by distribution, as Ubuntu does for example).
• It could be not that well suited to portable disks depending on the use case. Chances are most other people's macOS and Linux systems won't be able to read your ZFS formatted disks "as is" (see point above), and a connection failure, like detaching the cable inadvertently, might have annoying consequences.

Personally, I've used ZFS with a dual-boot (macOS/Linux) Macbook Pro for several years now, both with USB3 disks and internal SATA disks. Depending on the situation, it's a very good solution.

There is also an actively developed, albeit very recent, ZFS Windows port. It might be several years before it's usable in production though (https://github.com/openzfsonwindows/ZFSin).