Linux-x86_64 Error: 5: Input/output error的问题
今天朋友的测试库报错， ORA-27072: File I/O error， Linux-x86_64 Error: 5: Input/output error，数据块已经不能正常启动，看到 Linux-x86_64 Error: 5: Input/output error多数是磁盘空间满导致的
KCF: read, write or open error, block=0x283980 >
file=5 '/app/oradata1/general_1.dbf'
error=27072 txt: 'Linux-x86_64 Error: 5: Input/output error
Additional information: 4
Additional information: 2636160
Additional information: -1’
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_dbw1_30299.trc:
ORA-01242: data file suffered media failure: database in NOARCHIVELOG mode
ORA-01114: IO error writing block to file 5 (block # 2636160)
ORA-01110: data file 5: ‘/app/oradata1/general_1.dbf’
ORA-27072: File I/O error
Linux-x86_64 Error: 5: Input/output error
Additional information: 4
Additional information: 2636160
Additional information: -1
DBW1 (ospid: 30299): terminating the instance due to error 1242
Termination issued to instance processes. Waiting for the processes to exit
Instance terminated by DBW1, pid = 30299
查看磁盘使用情况，看空间还都比较富于，貌似不是空间引起的
[root@template app]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda3 16G 12G 3.5G 77% /
tmpfs 32G 17G 15G 53% /dev/shm
/dev/vda1 194M 32M 153M 17% /boot
/dev/vdb1 2.0T 907G 963G 49% /app
查看主机是虚拟机还是物理机，发现是个虚拟机，如果是虚拟机的话可能出现存储超分的情况，（存储超分就是看着使用率可能不高但是实际上物理磁盘的空间可能已经满了）
[root@template app]# dmidecode |grep -i prod
Product Name: OpenStack Nova
删除了/app下的部分文件，如果删除部分文件能启动的话就是存储超分了
SQL> startup
ORACLE instance started.
Total System Global Area 2.6991E+10 bytes
Fixed Size 2213976 bytes
Variable Size 1.9596E+10 bytes
Database Buffers 7247757312 bytes
Redo Buffers 145174528 bytes
Database mounted.
Database opened.
数据库能正常启动了说明了存储还是存在超分的情况，删除部分文件释放了空间部分空间数据库就能正常启动了。
有点奇怪的是dbv检测数据文件居然还有坏块
[oracle@template ~]$ dbv file=/app/oradata1/general_1.dbf blocksize=8192
DBVERIFY: Release 11.2.0.1.0 - Production on ÐÇÆÚ¶þ 10ÔÂ 29 11:19:44 2019
Copyright © 1982, 2009, Oracle and/or its affiliates. All rights reserved.
DBVERIFY - Verification starting : FILE = /app/oradata1/general_1.dbf
Page 2760430 is influx - most likely media corrupt
Corrupt block relative dba: 0x016a1eee (file 5, block 2760430)
Fractured block found during dbv:
Data in bad block:
type: 0 format: 2 rdba: 0x016a1eee
last change scn: 0x0000.00000000 seq: 0x1 flg: 0x05
spare1: 0x0 spare2: 0x0 spare3: 0x0
consistency value in tail: 0x3454286e
check value in block header: 0xb884
computed block checksum: 0x5843
Page 2760431 is marked corrupt
Corrupt block relative dba: 0x016a1eef (file 5, block 2760431)
Bad header found during dbv:
Data in bad block:
type: 67 format: 4 rdba: 0x45726576
last change scn: 0x6973.6e657478 seq: 0x6f flg: 0x6e
spare1: 0x72 spare2: 0x69 spare3: 0x7661
consistency value in tail: 0x43746567
check value in block header: 0x6a2e
computed block checksum: 0x21
Page 2760432 is marked corrupt
Corrupt block relative dba: 0x016a1ef0 (file 5, block 2760432)
Bad header found during dbv:
Data in bad block:
type: 111 format: 6 rdba: 0x6f697463
last change scn: 0x786f.7250286e seq: 0x6f flg: 0x6f
spare1: 0x6e spare2: 0x65 spare3: 0x7461
consistency value in tail: 0x664f2e65
check value in block header: 0x446c
computed block checksum: 0x3f34
Page 2760433 is marked corrupt
Corrupt block relative dba: 0x016a1ef1 (file 5, block 2760433)
Bad header found during dbv:
Data in bad block:
type: 115 format: 3 rdba: 0x6f666e49
last change scn: 0x6369.76726553 seq: 0x65 flg: 0x2e
spare1: 0x79 spare2: 0x73 spare3: 0x4874
consistency value in tail: 0x6f687465
check value in block header: 0x6567
computed block checksum: 0x435d
Page 2760434 is marked corrupt
Corrupt block relative dba: 0x016a1ef2 (file 5, block 2760434)
Bad header found during dbv:
Data in bad block:
type: 100 format: 6 rdba: 0x353a6176
last change scn: 0x325b.0a293739 seq: 0x30 flg: 0x31
spare1: 0x6a spare2: 0x61 spare3: 0x3930
consistency value in tail: 0x486e6f69
check value in block header: 0x2e38
block checksum disabled
Page 2760435 is marked corrupt
Corrupt block relative dba: 0x016a1ef3 (file 5, block 2760435)
Bad header found during dbv:
Data in bad block:
type: 97 format: 6 rdba: 0x6a2e7265
last change scn: 0x3031.3a617661 seq: 0x39 flg: 0x29
spare1: 0x64 spare2: 0x6c spare3: 0x3032
consistency value in tail: 0x53707474
check value in block header: 0x5b0a
block checksum disabled
Page 2760436 is marked corrupt
Corrupt block relative dba: 0x016a1ef4 (file 5, block 2760436)
Bad header found during dbv:
Data in bad block:
type: 101 format: 2 rdba: 0x732e7465
last change scn: 0x6563.69767265 seq: 0x28 flg: 0x48
spare1: 0x76 spare2: 0x6c spare3: 0x5370
consistency value in tail: 0x74696e6f
check value in block header: 0x7474
block checksum disabled
Page 2760437 is marked corrupt
Corrupt block relative dba: 0x016a1ef5 (file 5, block 2760437)
Bad header found during dbv:
Data in bad block:
type: 111 format: 2 rdba: 0x69546c72
last change scn: 0x746c.6946656d seq: 0x65 flg: 0x72
spare1: 0x58 spare2: 0x55 spare3: 0x7661
consistency value in tail: 0x7270732e
check value in block header: 0x6a2e
block checksum disabled
Page 2760438 is marked corrupt
Corrupt block relative dba: 0x016a1ef6 (file 5, block 2760438)
Bad header found during dbv:
Data in bad block:
type: 105 format: 6 rdba: 0x656d6172
last change scn: 0x772e.6b726f77 seq: 0x65 flg: 0x62
spare1: 0x67 spare2: 0x66 spare3: 0x6c69
consistency value in tail: 0x542e6c69
check value in block header: 0x662e
block checksum disabled
Page 2760439 is marked corrupt
Corrupt block relative dba: 0x016a1ef7 (file 5, block 2760439)
Bad header found during dbv:
Data in bad block:
type: 104 format: 2 rdba: 0x6f6f5064
last change scn: 0x6d65.7449246c seq: 0x2e flg: 0x72
spare1: 0x65 spare2: 0x61 spare3: 0x5428
consistency value in tail: 0x72456573
check value in block header: 0x6e75
block checksum disabled
Page 2760440 is marked corrupt
Corrupt block relative dba: 0x016a1ef8 (file 5, block 2760440)
Bad header found during dbv:
Data in bad block:
type: 114 format: 7 rdba: 0x6f726874
last change scn: 0x7845.6c715377 seq: 0x63 flg: 0x65
spare1: 0x72 spare2: 0x2e spare3: 0x6f69
consistency value in tail: 0x33302e33
check value in block header: 0x7470
computed block checksum: 0xe79
Page 2760441 is marked corrupt
Corrupt block relative dba: 0x016a1ef9 (file 5, block 2760441)
Bad header found during dbv:
Data in bad block:
type: 50 format: 5 rdba: 0x616a2074
last change scn: 0x6c71.732e6176 seq: 0x2e flg: 0x44
spare1: 0x9 spare2: 0x61 spare3: 0x6576
consistency value in tail: 0x0a293830
check value in block header: 0x6972
computed block checksum: 0xb3c
Page 2760442 is marked corrupt
Corrupt block relative dba: 0x016a1efa (file 5, block 2760442)
Bad header found during dbv:
Data in bad block:
type: 91 format: 2 rdba: 0x39302e38
last change scn: 0x3231.2039322e seq: 0x3a flg: 0x33
spare1: 0x30 spare2: 0x31 spare3: 0x3334
consistency value in tail: 0x00000000
check value in block header: 0x3a35
block checksum disabled
Page 2760443 is marked corrupt
Corrupt block relative dba: 0x016a1efb (file 5, block 2760443)
Completely zero block found during dbv:
Page 2760444 is marked corrupt
Corrupt block relative dba: 0x016a1efc (file 5, block 2760444)
Completely zero block found during dbv:
Page 2760445 is marked corrupt
Corrupt block relative dba: 0x016a1efd (file 5, block 2760445)
Completely zero block found during dbv:
Page 2760446 is marked corrupt
Corrupt block relative dba: 0x016a1efe (file 5, block 2760446)
Completely zero block found during dbv:
Page 2760447 is marked corrupt
Corrupt block relative dba: 0x016a1eff (file 5, block 2760447)
Completely zero block found during dbv:
DBVERIFY - Verification complete
Total Pages Examined : 4194302
Total Pages Processed (Data) : 2010582
Total Pages Failing (Data) : 0
Total Pages Processed (Index): 509952
Total Pages Failing (Index): 0
Total Pages Processed (Other): 392725
Total Pages Processed (Seg) : 0
Total Pages Failing (Seg) : 0
Total Pages Empty : 1281025
Total Pages Marked Corrupt : 18
Total Pages Influx : 1
Total Pages Encrypted : 0
Highest block SCN : 1998427953 (3594.1998427953)
[oracle@template ~]$