The problem occurs when replaying log records from a REORG
INDEX(ES) operation when the RECLAIM EXTENTS option is used.
The replay must be triggered by either crash recovery or by
executing ROLLFORWARD DATABASE after restoring a backup imagine
that was taken using the ONLINE option of the BACKUP DATABASE
command while REORG INDEX(ES) was running. The problem does not
occur when executing ROLLFORWARD DATABASE after restoring an
offline backup. This is a rare scenario because it requires an
unlikely timing scenario occurring before a recovery is needed.
The corruption can be detected by the db2dart tool. An example
of the errors that the db2dart tool might raise are:
Index inspection phase start. Index obj: 4 In pool: 3
Page 8: Error: Incorrect Total Free Space
Error: in page 8, pool page 132, of obj 4, in tablespace 3.
Error: Page data will be dumped to report.
...
Scanning pages for unique index itoken(0) root page:1p.
Scanning pages for index itoken(1) root page:2p.
Error: previous-node-pointer for page(8) is (498) but in
reality the previous node at this level is (3)
Error: last key on page 132p,entries=18 prefixNumber=-1 is
higher than the highkey on
page=147p,slot=1,rid=xFF00FFFFFF000000,entries=135.
Error: Index keys misordered, compare rc=2. Lvl(1)
Page,slot,rid current:135p,0,
3 keyErrs found in index itoken(1) root page:2p. Dumping
Page 2p.
Error: Page data will be dumped to report.
There are many other symptoms that might result from this page
corruption. Possible symptoms include but are not limited to:
- Crash recovery might fail if DB2 tries to use the corrupted
page. An example of an error you might see in the db2diag.log
file if that occurs is: sqliRedoAddKey, probe:1101.
- Incorrect results might be returned queries using this index.
- An infinite loop might occur while scanning or updating this
index.
- The SQL0901N Reason "Key data mismatch error might be
encountered during key delete". Note that this is a common
symptom to other problems so seeing this is not conclusive that
you hit this problem.
- Further corruption might occur as a result of this corruption.
- The db2dart tool might report other index problems such as
'Offset Location = 0 (x0) Error: invalid offset!'. |
If you have this problem you will need to recreate the corrupted
indexes. If this problem causes crash recovery to fail, you
will need to mark the problem index invalid in order to get
crash recovery to complete. Please contact db2support for help
with this task.
To avoid this problem in the future avoid running REORG
INDEX(ES) with the RECLAIM EXTENTS option concurrently with a
BACKUP DATABASE command that is using the ONLINE option. If a
crash occurs while running (or shortly after running) REORG
INDEX(ES) with the RECLAIM EXTENTS option, then after crash
recovery is complete run the db2dart tool with the /T option
against the table that was being reorganized to check for index
errors. |
