bcache fails to attach local ssd as caching device in google cloud
I'm trying to setup bcache in an instance in google cloud using a raid of local SSDs as caching device but everything fails in the attach phase of the process. For testing purposes I created a new instance with blank disks (two SSDs and four Local SSDs). The objective is to have one regular SSD as backing device and form a raid with the 4 Local SSDs and use that device as caching.
In the output below you can see the steps taken (the outcome is the same when using a raid, I just used one Local SSD for simplicity here and because I believe the issue is not related to raid but to the disks themselves). When using the Local SSD as caching device, I fail to attach the caching device to the backing device. When instead I use a regular SSD for caching device, you can see that everything works as expected.
The question for you experts out there is: Are there any known limitations for the Local SSDs or am I doing anything wrong (or maybe there are some extra steps needed)?
For reference, these are devices being used:
/dev/sdb => Backing Device
/dev/sdc => SSD Caching Device
/dev/nvme0n1 => Local SSD Single Caching Device
# apt update && apt install mdadm bcache-tools -y
# make-bcache -B /dev/sdb
UUID: cb10650f-cf60-4a96-81eb-7149ae650f94
Set UUID: dc0a7f3a-de46-4b00-84f4-4aa40c203745
version: 1
block_size: 1
data_offset: 16
# mkfs.ext4 -L cached /dev/bcache0
# make-bcache -C /dev/nvme0n1
UUID: c5a33c1e-e1ef-4d3d-a5ac-5d0adc340f43
Set UUID: 228dcba5-6085-47a1-b2e9-eff68dd6ac14
version: 0
nbuckets: 768000
block_size: 8
bucket_size: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1
# bcache-super-show /dev/nvme0n1
sb.magic ok
sb.first_sector 8 [match]
sb.csum 674DD52F06C4562B [match]
sb.version 3 [cache device]
dev.label (empty)
dev.uuid c5a33c1e-e1ef-4d3d-a5ac-5d0adc340f43
dev.sectors_per_block 8
dev.sectors_per_bucket 1024
dev.cache.first_sector 1024
dev.cache.cache_sectors 786430976
dev.cache.total_sectors 786432000
dev.cache.ordered yes
dev.cache.discard no
dev.cache.pos 0
dev.cache.replacement 0 [lru]
cset.uuid 228dcba5-6085-47a1-b2e9-eff68dd6ac14
# echo 228dcba5-6085-47a1-b2e9-eff68dd6ac14 > /sys/block/bcache0/bcache/attach
-bash: echo: write error: Invalid argument
# make-bcache -C /dev/sdc
UUID: 55c95063-9aa7-4d2c-8c8c-d4d34d35a7ad
Set UUID: 2de3ccef-a7eb-4620-8b6d-265d0a06da17
version: 0
nbuckets: 204800
block_size: 1
bucket_size: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1
# bcache-super-show /dev/sdc
sb.magic ok
sb.first_sector 8 [match]
sb.csum 11E99ECE7A83EABE [match]
sb.version 3 [cache device]
dev.label (empty)
dev.uuid 55c95063-9aa7-4d2c-8c8c-d4d34d35a7ad
dev.sectors_per_block 1
dev.sectors_per_bucket 1024
dev.cache.first_sector 1024
dev.cache.cache_sectors 209714176
dev.cache.total_sectors 209715200
dev.cache.ordered yes
dev.cache.discard no
dev.cache.pos 0
dev.cache.replacement 0 [lru]
cset.uuid 2de3ccef-a7eb-4620-8b6d-265d0a06da17
# echo 2de3ccef-a7eb-4620-8b6d-265d0a06da17 > /sys/block/bcache0/bcache/attach
# bcache-super-show /dev/sdc
sb.magic ok
sb.first_sector 8 [match]
sb.csum 11E99ECE7A83EABE [match]
sb.version 3 [cache device]
dev.label (empty)
dev.uuid 55c95063-9aa7-4d2c-8c8c-d4d34d35a7ad
dev.sectors_per_block 1
dev.sectors_per_bucket 1024
dev.cache.first_sector 1024
dev.cache.cache_sectors 209714176
dev.cache.total_sectors 209715200
dev.cache.ordered yes
dev.cache.discard no
dev.cache.pos 0
dev.cache.replacement 0 [lru]
cset.uuid 2de3ccef-a7eb-4620-8b6d-265d0a06da17
# bcache-super-show /dev/sdb
sb.magic ok
sb.first_sector 8 [match]
sb.csum 2E55F82F4131C19B [match]
sb.version 1 [backing device]
dev.label (empty)
dev.uuid cb10650f-cf60-4a96-81eb-7149ae650f94
dev.sectors_per_block 1
dev.sectors_per_bucket 1024
dev.data.first_sector 16
dev.data.cache_mode 0 [writethrough]
dev.data.cache_state 1 [clean]
cset.uuid 2de3ccef-a7eb-4620-8b6d-265d0a06da17
Solution 1:
After some detailed reviewing I realized the problem is with the block size from the different devices. When adjusted the make-bcache commands, everything worked as expected:
make-bcache --block 4k -B /dev/sdb
make-bcache --block 4k -C /dev/nvme0n1