linux – rhel + dmesg + Buffer I/O error on dev dm-0, logical block


we installed some new VM machines with rhel 7.6

but from dmesg we found the following errors

(14706.075255) sd 1:0:0:0: timing out command, waited 1080s
(14706.075370) sd 1:0:0:0: timing out command, waited 1080s
(14706.075881) sd 1:0:0:0: timing out command, waited 1080s


(14706.076007) Buffer I/O error on dev dm-2, logical block 19736964, lost async page write
(14706.076081) Buffer I/O error on dev dm-2, logical block 1268620, lost async page write
(14706.076128) Buffer I/O error on dev dm-2, logical block 19736968, lost async page write
(14706.076172) Buffer I/O error on dev dm-2, logical block 19736966, lost async page write
(14706.076909) Buffer I/O error on dev dm-2, logical block 19736967, lost async page write
(14706.077721) Buffer I/O error on dev dm-2, logical block 19736969, lost async page write
(14706.077760) Buffer I/O error on dev dm-2, logical block 1268621, lost async page write
(14706.077775) Buffer I/O error on dev dm-2, logical block 19736972, lost async page write
(14706.077936) Buffer I/O error on dev dm-2, logical block 1268624, lost async page write
(14706.079369) XFS (dm-2): metadata I/O error: block 0x9694c78 ("xfs_buf_iodone_callback_error") error 5 numblks 8
(14858.217013) buffer_io_error: 3198 callbacks suppressed
(14858.217051) Buffer I/O error on dev dm-0, logical block 978837, lost async page write
(14858.217058) Buffer I/O error on dev dm-0, logical block 978838, lost async page write
(14858.217061) Buffer I/O error on dev dm-0, logical block 978839, lost async page write
(14858.217064) Buffer I/O error on dev dm-0, logical block 978840, lost async page write
(14858.217066) Buffer I/O error on dev dm-0, logical block 978841, lost async page write
(14858.217069) Buffer I/O error on dev dm-0, logical block 978842, lost async page write
(14858.217072) Buffer I/O error on dev dm-0, logical block 978843, lost async page write
(14858.217075) Buffer I/O error on dev dm-0, logical block 978844, lost async page write
(14858.217080) Buffer I/O error on dev dm-0, logical block 978845, lost async page write
(14858.217083) Buffer I/O error on dev dm-0, logical block 978846, lost async page write

is that errors are related to disks ? or file-system or memory cards ?

any advice how to continue from this situation to solution ?