file system is going into read only mode frequently

My server file system is going into read only mode frequently. If I reboot my system, It is performing a fsck operation on disks. But, It is exited with exit status 4 ( File system errors left uncorrected) and it is asking to perform fsck manually. When I performing a manual fsck, It is clearing orphaned inodes and doing file systems modification. Then, It is working fine for 1 or 2 days only. Again the same problem is coming.

How to give permanent solution to fix this issue?

Or should I change my hard disk?

In the dmesg output, I could see some of the repeated errors. Is this the reason of this read only filesystem error.

For your reference here is dmesg output.

[ 1755.676318] sd 1:0:1:0: [sda] 1953525168 512-byte hardware sectors: (1.00 TB/931 GiB)
[ 1755.677167] sd 1:0:1:0: [sda] Write Protect is off
[ 1755.677171] sd 1:0:1:0: [sda] Mode Sense: 00 3a 00 00
[ 1755.677492] sd 1:0:1:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 1761.064636] ata2.01: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
[ 1761.064671] ata2.01: BMDMA stat 0x46
[ 1761.064699] ata2.01: cmd 35/00:00:88:c3:c3/00:04:25:00:00/f0 tag 0 dma 524288 out
[ 1761.064700]          res 51/84:80:88:c3:c3/84:02:25:00:00/f0 Emask 0x30 (host bus error)
[ 1761.064794] ata2.01: status: { DRDY ERR }
[ 1761.064820] ata2.01: error: { ICRC ABRT }
[ 1761.064852] ata2: soft resetting link
[ 1761.245018] ata2.01: configured for UDMA/33
[ 1761.245027] ata2: EH complete
[ 1761.249232] sd 1:0:1:0: [sda] 1953525168 512-byte hardware sectors: (1.00 TB/931 GiB)
[ 1761.249709] sd 1:0:1:0: [sda] Write Protect is off
[ 1761.249712] sd 1:0:1:0: [sda] Mode Sense: 00 3a 00 00
[ 1761.253105] sd 1:0:1:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 1794.770383] ata2.01: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
[ 1794.770419] ata2.01: BMDMA stat 0x46
[ 1794.770447] ata2.01: cmd ca/00:b8:30:de:b5/00:00:00:00:00/f8 tag 0 dma 94208 out
[ 1794.770448]          res 51/84:98:30:de:b5/84:02:25:00:00/f8 Emask 0x30 (host bus error)
[ 1794.770542] ata2.01: status: { DRDY ERR }
[ 1794.770568] ata2.01: error: { ICRC ABRT }
[ 1794.770600] ata2: soft resetting link
[ 1794.949090] ata2.01: configured for UDMA/33
[ 1794.949099] ata2: EH complete
[ 1794.950207] sd 1:0:1:0: [sda] 1953525168 512-byte hardware sectors: (1.00 TB/931 GiB)
[ 1794.950390] sd 1:0:1:0: [sda] Write Protect is off
[ 1794.950394] sd 1:0:1:0: [sda] Mode Sense: 00 3a 00 00
[ 1794.950777] sd 1:0:1:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 1799.948026] ata2.01: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
[ 1799.948062] ata2.01: BMDMA stat 0x46
[ 1799.948090] ata2.01: cmd ca/00:c0:f0:1e:b6/00:00:00:00:00/f8 tag 0 dma 98304 out
[ 1799.948092]          res 51/84:30:f0:1e:b6/84:02:25:00:00/f8 Emask 0x30 (host bus error)
[ 1799.948185] ata2.01: status: { DRDY ERR }
[ 1799.948211] ata2.01: error: { ICRC ABRT }
[ 1799.948244] ata2: soft resetting link
[ 1800.129181] ata2.01: configured for UDMA/33
[ 1800.129191] ata2: EH complete
[ 1800.129981] sd 1:0:1:0: [sda] 1953525168 512-byte hardware sectors: (1.00 TB/931 GiB)
[ 1800.145952] sd 1:0:1:0: [sda] Write Protect is off
[ 1800.145955] sd 1:0:1:0: [sda] Mode Sense: 00 3a 00 00
[ 1800.146327] sd 1:0:1:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 1874.851961] ata2.01: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
[ 1874.851996] ata2.01: BMDMA stat 0x46
[ 1874.852032] ata2.01: cmd ca/00:d0:50:e7:b5/00:00:00:00:00/f8 tag 0 dma 106496 out
[ 1874.852033]          res 51/84:70:50:e7:b5/84:02:25:00:00/f8 Emask 0x30 (host bus error)
[ 1874.852128] ata2.01: status: { DRDY ERR }
[ 1874.852153] ata2.01: error: { ICRC ABRT }
[ 1874.852186] ata2: soft resetting link
[ 1875.032851] ata2.01: configured for UDMA/33
[ 1875.032860] ata2: EH complete
[ 1875.033772] sd 1:0:1:0: [sda] 1953525168 512-byte hardware sectors: (1.00 TB/931 GiB)
[ 1875.034200] sd 1:0:1:0: [sda] Write Protect is off
[ 1875.034207] sd 1:0:1:0: [sda] Mode Sense: 00 3a 00 00
[ 1875.034630] sd 1:0:1:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 1887.046322] ata2.01: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
[ 1887.046357] ata2.01: BMDMA stat 0x46
[ 1887.046385] ata2.01: cmd 35/00:00:e8:91:53/00:04:15:00:00/f0 tag 0 dma 524288 out
[ 1887.046386]          res 51/84:60:e8:91:53/84:00:15:00:00/f0 Emask 0x30 (host bus error)
[ 1887.046481] ata2.01: status: { DRDY ERR }
[ 1887.046506] ata2.01: error: { ICRC ABRT }
[ 1887.046538] ata2: soft resetting link
[ 1887.224789] ata2.01: configured for UDMA/33
[ 1887.224798] ata2: EH complete

Solution 1:

Yes, your disk is dying. Replace it.