Shrinking a Core Storage volume without the booter partition
Solution 1:
I followed David Anderson's advice and discovered that the partition (as described by GPT) is 3584 bytes (7 LBA blocks) larger than the Core Storage Physical Volume. Using this information, I did manage to shrink the volume to add a second partition.
The process is, however, quite scary: it involves a kernel panic, then some more cryptic, unhelpful, unfixable diskutil
errors, and then yet some more frightening failures.
And so let this be an anti-answer:
- if you are dealing with an Apple-designed filesystem (such as HFS and APFS),
- and you don't have professional knowledge of the inner workings of them,
- and you find your volume in an inconsistent state — it should be considered inconsistent as soon as Disk Utility starts complaining —
- and yet the volume is somehow still usable (you can see your files and such):
Then, for the love of your data and your well-being, instead of trying to fix the problem by meddling with the volume header or partition table (like I did),
Just move everything out and reformat.
I eventually lost no data. Here is my process. I hope that people more knowledgeable than me could point out all the mistakes I made.
Note: 1 block is 512 bytes.
-
(First, I made a copy of the first 512 bytes of the partition, which contains the Volume Header, using
dd
). -
Using the
gpt
command, I learned that my second GPT partition (the main partition) has a size of27344355255
blocks; following this specification of Core Storage (which is the abstraction layer that allows HFS encryption), I examined the partition volume header and learned that the Core Storage Physical Volume only occupies27344355248
blocks. This means there was an extra space of 7 blocks. -
I then decided to free these 7 blocks by recreating the GPT table (using the
gpt
command), also marking the type of the new partition as Apple Boot. -
I attempted to create an HFS+ volume on the new partition using
diskutil
. This is not possible because an HFS+ volume requires a minimum of 512 KB. -
With the unformatted Apple Boot partition, I made the risky move of asking Disk Utility to resize the 14 TB partition. This time, Disk Utility did not complain about missing booters. However, before the resizing had finished, a kernel panic struck.
-
After restarting,
diskutil
reported that the original 14 TB partition is now 12 TB, still intact. The 7-block Apple Boot partition now occupied the remaining 2 TB free space, still unformatted. -
Then, I modified the GPT table again, this time adding a
262144
-block Apple Boot partition after the now 12 TB partition, and another Core Storage partition taking up the remaining space. -
At this point, both the 12 TB and 2 TB partition is showing up in Disk Utility. Upon reformatting the 2 TB partition as a JHFS+ encrypted volume (so that it too has a booter), both partitions can be resized freely using Disk Utility with no errors.
-
The only caveat was that using First Aid (
diskutil verifyVolume
) on the 12 TB volume now persistently reported the error "Incorrect size for volume" (right after it says the volume "appears to be OK", too). This could not be fixed using anydiskutil
command. But despite the error, the two volumes can still be resized and normally mounted. -
At this point, it would seem that my original question has been solved, to a large extent.
This is what the partition scheme now looked like:
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *14.0 TB disk2
1: EFI EFI 209.7 MB disk2s1
2: Apple_CoreStorage - Main Partition - 12.0 TB disk2s2
3: Apple_Boot Boot OS X 134.2 MB disk2s3
4: Apple_CoreStorage Time Machine 2.0 TB disk2s4
5: Apple_Boot Boot OS X 134.2 MB disk2s5
(The rest of this is me not knowing what I am doing.)
-
After this, I came up with the hypothesis that the "Incorrect volume size" error that Disk Utility kept complaining about was about the Logical Volume, and not the Physical Volume.
- Core Storage has many abstraction layers, the order of which is Physical Volume → Logical Volume Group → Logical Volume Family → Logical Volume.
- If I am understanding it correctly, PV is the closest to the GPT partition, LVF is where FileVault encryption occurs, and LV is the volume that users can see.
-
To test this out, I deleted the 2 TB partition via Disk Utility. There was no issue. The hard drive now contained EFI, the 14 TB partition, and the 134.2 MB booter.
-
I attempted to use
diskutil cs resizeVolume
to resize only the Logical Volume back to 12 TB, and not the Physical Volume. The resizing could not proceed because of the "Incorrect volume size" error. (The same went fordiskutil cs resizeStack
). -
Then I had the horrible idea to revert not only the GPT table, but also the Core Storage volume header, to the state when this question was posted.
- This is in the hope that
diskutil verifyVolume
will then auto-adjust the volume size, and then I can start over the process, but resize the Logical Volume first. - I made a backup of the current volume header, then
dd
rewrote the first 512 bytes of the volume using the backup.img
file I made of the header at the start of this answer. -
After I did that, the 14 TB volume once again was unmountable. All
diskutil
operations would fail at this point.
- This is in the hope that
-
I attempted to revert this change by writing the 512-byte volume header back. Then, another kernel panic occurred.
-
This time, things got much more serious. For the next few hours, every time I connected the hard drive to my laptop, macOS immediately kernel panicked — whether it was in normal mode, Recovery, Safe Mode, or Single User Mode.
-
I tried booting into Ubuntu from USB, and examine and manipulate the Volume Header from there, to no avail.
-
I almost lost hope, when I remembered that, back when the partition first failed, I ran
dd
through the first few GBs of the partition and saved it as an image, which was since deleted. I recovered it using Disk Drill, and after painfully comparing hex dumps, I decided to copy the first62500
blocks (32 MB) of the image file back to the partition.- My idea was that this would cause macOS to recognize the volume in the same way that it recognized the original 14 TB volume when it first failed, and then if I redo the steps I described in the question (recreating the GPT table), I could get the volume back a second time, and I thought that the worst that could happen is I would lose all file changes since the partition first failed.
-
Shockingly, not only did this recover the partition, but all the files that were on the hard drive before I broke the partition the second time are still there.
-
Now I am in the process of backing up everything so that I can start the hard drive fresh.