DB2 - Problem description
Problem IC70482 | Status: Closed |
Instance abend and possible crash recovery failure on V9.7 Fix Pack 2 using circular logging. | |
product: | |
DB2 FOR LUW / DB2FORLUW / 970 - DB2 | |
Problem description: | |
This problem may manifest itself in numerous ways. As a first symptom , the instance will abend in a variety of possible locations. Some known function in the stack traces are : 1. sqlp_get_log_filepath_for_lsn db2diag.log entry : 2010-07-01-01.13.49.945856+540 I14975442A499 LEVEL: Warning PID : 1781890 TID : 13624 PROC : db2sysc 4 INSTANCE: db2inst1 NODE : 004 DB : DBNAME APPHDL : 0-157 APPID: *N0.db2inst1.100729011146 AUTHID : DB2INST1 EDUID : 13624 EDUNAME: db2lload 4 FUNCTION: DB2 UDB, RAS/PD component, pdEDUIsInDB2KernelOperation, probe:600 DATA #1 : String, 33 bytes sqlp_get_log_filepath_for_lsn__F DATA #2 : String, 4 bytes sqlp 2. sqlpgwlp sqlpgasn2 2010-07-01-09.56.58.161004-240 I59812464E384 LEVEL: Warning PID : 31052 TID : 479683 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 EDUID : 75 EDUNAME: db2loggw (SAMPLE) 0 FUNCTION: DB2 UDB, RAS/PD component, pdEDUIsInDB2KernelOperation, probe:600 DATA #1 : String, 12 bytes _Z8sqlpgwlp DATA #2 : String, 4 bytes sqlp 3. sqlpshrBuildRecord sqlpshrEdu 2010-07-01-08.20.01.832248+540 I15631649A503 LEVEL: Warning PID : 835836 TID : 13880 PROC : db2sysc INSTANCE: db2inst1 NODE : 000 DB : SAMPLE APPHDL : 2-51 APPID: *N0.db2inst1.100802232000 AUTHID : DB2INST1 EDUID : 13880 EDUNAME: db2shred (SAMPLE) FUNCTION: DB2 UDB, RAS/PD component, pdEDUIsInDB2KernelOperation, probe:600 DATA #1 : String, 27 bytes @109@sqlpshrBuildRecord__F DATA #2 : String, 4 bytes sqlp - sqlpshrBuildRecord sqlpshrEdu other traps are possible as well. A db2diag.log message which also occurs when the problem happens is the following : 2010-07-01-09.46.35.712009+540 I13910007A467 LEVEL: Error PID : 1417464 TID : 3600 PROC : db2sysc 4 INSTANCE: db2inst1 NODE : 004 DB : SAMPLE EDUID : DB2INST1 EDUNAME: db2loggr (SAMPLE) 4 FUNCTION: DB2 UDB, data protection services, sqlpgspr, probe:550 DATA #1 : String, 117 bytes Reclaimed too much log space. Recalculating LogBytesInUse. Old LogBytesInUse: 74753276 New LogBytesInUse: 8124749448 and following this : 2010-07-01-07.06.17.705235+000 E34053A598 LEVEL: Severe PID : 417804 TID : 5142 PROC : db2sysc 0 INSTANCE: db2cbp NODE : 000 EDUID : 5142 EDUNAME: db2loggw (SAMPLE) 0 FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk, probe:1010 MESSAGE : ZRC=0x85100009=-2062548983=SQLP_NOSPACE "Log File has reached its saturation point" DIA8309C Log file was full. DATA #1 : <preformatted> Error getting next log file to write to. Filecount 20, active 20, inactive 40, tailindex 18446744073709551596 currentRecord 16 Note that this exhaust the available transaction log space ( SQLP_NOSPACE error ) and will bring the database down. An eyecatcher is the very high tailindex value which , when converted to hex, is close to 0xFFFF FFFF FFFF FFFF. When subsequently crash recovery is attempted, this may fail with the following message during the undo phase : 2010-12-20-09.34.11.751803+000 I5359A415 LEVEL: Info PID : 12093 TID : 13 PROC : db2sysc 0 INSTANCE: db2inst3 NODE : 000 DB : SAMPLE EDUID : 13 EDUNAME: db2loggr (SAMPLE) 0 FUNCTION: DB2 UDB, recovery manager, sqlpgSwitchFromRedoToUndo, probe:806 DATA #1 : <preformatted> TailIndex 1 firstLso 39333401 nextLso 39349705 logGapSize 0 2010-07-01-09.34.11.962973+000 I7301A532 LEVEL: Severe PID : 12093 TID : 14 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 EDUID : 14 EDUNAME: db2loggw (SAMPLE) 0 FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk, probe:909 MESSAGE : ZRC=0x8610000D=-2045771763=SQLP_BADLOG "Log File cannot be used" DIA8414C Logging can not continue due to an error. DATA #1 : <preformatted> TailPage 0 does not match pagePso 39349778 and firstLso 39007321 | |
Problem Summary: | |
**************************************************************** * USERS AFFECTED: * * All * **************************************************************** * PROBLEM DESCRIPTION: * * This APAR only applies to V9.7 FP2 database that use * * circular logging. * * * * When the database is circular logging(sqlpgicl), * * thefirstLso in FCB * * array are all initialized to * * baselso.2010-07-29-10.13.49.945856+540 I14975442A499 * * LEVEL:WarningPID : 1781890 TID : 13624 * * PROC :db2sysc 4INSTANCE: db2inst1 NODE : 004 * * DB : XXXXXXAPPHDL : 0-157 * * APPID:*N0.db2inst1.100729011146AUTHID : XXXXXEDUID : 13624 * * EDUNAME: db2lload 4FUNCTION: DB2 UDB, RAS/PD * * component,pdEDUIsInDB2KernelOperation, probe:600DATA #1 : * * String, 33 bytessqlp_get_log_filepath_for_lsn__FDATA #2 : * * String, 4 bytessqlpStacks<StackTrace>------Function + * * Offset------sqlp_get_log_filepath_for_lsnsqluRegisterLoadEndca * + 0x260</StackTrace> * **************************************************************** * RECOMMENDATION: * * (1) Upgrade to V9.7 Fix Pack 3, or, * * (2) Use archiving log * **************************************************************** | |
Local Fix: | |
One of the following methods may be used to attempt to work around the issue: 1) Restore database from a backup image 2) Reset log control file(SQLOGCTL.LFH) and bypass Crash recovery. However, db2 should be rebuild after resetting log control file. 2) Contact DB2 Support for assistance. Transaction log may be patched with internal tool. | |
available fix packs: | |
DB2 Version 9.7 Fix Pack 3 for Linux, UNIX, and Windows | |
Solution | |
Problem was first fixed in Version 9.7 Fix Pack 3. | |
Workaround | |
not known / see Local fix | |
BUG-Tracking | |
forerunner : APAR is sysrouted TO one or more of the following: IC76438 IC77551 follow-up : | |
Timestamps | |
Date - problem reported : Date - problem closed : Date - last modified : | 10.08.2010 23.09.2010 05.10.2011 |
Problem solved at the following versions (IBM BugInfos) | |
9.7.FP3, 9.7.FP3, 9.7.FP3.2 | |
Problem solved according to the fixlist(s) of the following version(s) | |
9.7.0.3 | |
9.7.0.3 |