Disk usage in LVM: Which directories contains most data?
You can do a sneaky:
mount /dev/mapper/rhel-root /mnt
find /mnt -xdev -type f -size +20M -exec ls -lh {} \;
umount /mnt
Then go through the results, of course you can change the size you are looking for.
There is also the possibility that you have something still clinging to a fd, i.e deleted a log instead of rotating it.
For this you can check:
lsof | grep deleted
Example:
[root@xx-xxxxx51:/var/log/wildfly]$ lsof | grep -i delete
S80wildfl 2224 root 1w REG 253,3 64821642 2432 /var/log/wildfly/console.log-20170331 (deleted)
S80wildfl 2224 root 2w REG 253,3 64822168 2432 /var/log/wildfly/console.log-20170331 (deleted)
runuser 2228 root 1w REG 253,3 64825188 2432 /var/log/wildfly/console.log-20170331 (deleted)
runuser 2228 root 2w REG 253,3 64825188 2432 /var/log/wildfly/console.log-20170331 (deleted)
bash 2235 wildfly 1w REG 253,3 64825188 2432 /var/log/wildfly/console.log-20170331 (deleted)
bash 2235 wildfly 2w REG 253,3 64825188 2432 /var/log/wildfly/console.log-20170331 (deleted)
standalon 2237 wildfly 1w REG 253,3 64825188 2432 /var/log/wildfly/console.log-20170331 (deleted)
standalon 2237 wildfly 10w REG 253,3 64825188 2432 /var/log/wildfly/console.log-20170331 (deleted)
java 2307 wildfly 1w REG 253,3 64909558 2432 /var/log/wildfly/console.log-20170331 (deleted)
java 2307 wildfly 2w REG 253,3 64909558 2432 /var/log/wildfly/console.log-20170331 (deleted)
su 19197 root 2u CHR 136,1 0t0 4 /dev/pts/1 (deleted)
standalon 19200 terinopadm 10u CHR 136,1 0t0 4 /dev/pts/1 (deleted)
java 19269 terinopadm 2u CHR 136,1 0t0 4 /dev/pts/1 (deleted)
Temporary solution to recover the used space:
[root@xx-xxxxx51:/var/log/wildfly]$ lsof -p 2235
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 2235 wildfly cwd DIR 253,0 4096 2 /
bash 2235 wildfly rtd DIR 253,0 4096 2 /
bash 2235 wildfly txt REG 253,0 941880 138736 /bin/bash
bash 2235 wildfly mem REG 253,0 161704 131467 /lib64/ld-2.12.so
bash 2235 wildfly mem REG 253,0 1930416 131482 /lib64/libc-2.12.so
bash 2235 wildfly mem REG 253,0 23088 131492 /lib64/libdl-2.12.so
bash 2235 wildfly mem REG 253,0 134792 132059 /lib64/libtinfo.so.5.7
bash 2235 wildfly mem REG 253,0 99164480 9874 /usr/lib/locale/locale-archive
bash 2235 wildfly mem REG 253,0 26060 2707 /usr/lib64/gconv/gconv-modules.cache
bash 2235 wildfly 0r CHR 1,3 0t0 3845 /dev/null
bash 2235 wildfly 1w REG 253,3 4159791104 2432 /var/log/wildfly/console.log-20170331 (deleted)
bash 2235 wildfly 2w REG 253,3 4159791104 2432 /var/log/wildfly/console.log-20170331 (deleted)
[root@xx-xxxxx51:/var/log/wildfly]$ ll /proc/2235/fd/2
l-wx------ 1 wildfly wildfly 64 Mar 31 08:05 /proc/2235/fd/2 -> /var/log/wildfly/console.log-20170331 (deleted)
[root@xx-xxxxx51:/var/log/wildfly]$ : > "/proc/2235/fd/2"
Note in this case, this does not solve the issue, you need to either restart your application or reboot the server.
Hope it helps.