DB2 - Problem description
Problem IC80745 | Status: Closed |
DB2LOGTS DAEMON ON HADR PRIMARY COULD BE BROUGHT DOWN DUE TO LOG INFO ARRAY BEING OUT OF SYNC | |
product: | |
DB2 FOR LUW / DB2FORLUW / 970 - DB2 | |
Problem description: | |
On HADR primary, db2logts daemon on HADR primary could be brought down due to log info array being out of sync. The issue can be hit when following events happened: 1. HADR standby restarts from last byte of a log file. When this event happens, we can see below message in db2diag.log: 2011-12-13-18.09.58.511431+540 E920335A494 LEVEL: Warning PID : 9699938 TID : 71547 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB APPHDL : 0-2523 APPID: *LOCAL.DB2.111213092048 EDUID : 71547 EDUNAME: db2shred (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpshrEdu, probe:45320 DATA #1 : <preformatted> Scan starting from the last byte of a truncated log file. Moving to next file.2130982527201 2. Takeover happens on the standby, and it becomes the new primary. 3. Transactions continue and new logs are generated on new primary. Finally, when log info array is full and DB2 trys to reuse one entry, the error will be hit, and following messages will appear in db2diag.log: 2011-12-15-00.13.22.849529+540 I2986417A454 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:64 MESSAGE : Extent number: DATA #1 : Hexdump, 4 bytes 0x070000001EFFD460 : 0000 6E1E ..n. 2011-12-15-00.13.22.849903+540 I2986872A469 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:65 MESSAGE : Entry number (readEntry): DATA #1 : Hexdump, 8 bytes 0x07000008E0027228 : 0000 0000 0000 0034 .......4 2011-12-15-00.13.22.850071+540 I2987342A483 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:66 MESSAGE : read entry extent number (readEntryExtNum): DATA #1 : Hexdump, 4 bytes 0x07000008E0027230 : 0000 6D14 ..m. 2011-12-15-00.13.22.850240+540 I2987826A1084 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:67 DATA #1 : String, 128 bytes The db2logts log info array is out of sync. Bringing down the db2logts daemon. From this point on, LOG SKIPPING is DISABLED. DATA #2 : Hexdump, 112 bytes 0x07000008E0029BA0 : 0000 6D12 0000 0000 0000 0997 EAB2 8001 ..m............. 0x07000008E0029BB0 : 0000 0000 0000 0000 0000 0016 0000 0000 ................ 0x07000008E0029BC0 : 0000 0000 0000 0000 0000 0000 0165 0000 .............e.. 0x07000008E0029BD0 : 0700 000D 0650 AF80 0700 000D 0650 B1A0 .....P.......P.. 0x07000008E0029BE0 : 0700 000D 0650 B3C0 0700 000D 0650 B5E0 .....P.......P.. 0x07000008E0029BF0 : 0700 000D 0650 B800 0700 000D 0650 5460 .....P.......PT` 0x07000008E0029C00 : 0700 000D 0650 5680 0700 000D 0650 58A0 .....PV......PX. 2011-12-15-00.13.22.850416+540 I2988911A165 LEVEL: Severe PID:9699938 TID:5798 NODE:000 Title: SQLP_DBCB Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin 2011-12-15-00.13.22.907328+540 I2989077A166 LEVEL: Severe PID:9699938 TID:5798 NODE:000 Title: SQLPG_XHDR Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin 2011-12-15-00.13.22.907523+540 I2989244A375 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:0 RETCODE : ZRC=0x00000000=0=PSM_OK "Unknown" | |
Problem Summary: | |
On HADR primary, db2logts daemon on HADR primary could be brought down due to log info array being out of sync. The issue can be hit when following events happened: 1. HADR standby restarts from last byte of a log file. When this event happens, we can see below message in db2diag.log: 2011-12-13-18.09.58.511431+540 E920335A494 LEVEL: Warning PID : 9699938 TID : 71547 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB APPHDL : 0-2523 APPID: *LOCAL.DB2.111213092048 EDUID : 71547 EDUNAME: db2shred (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpshrEdu, probe:45320 DATA #1 : <preformatted> Scan starting from the last byte of a truncated log file. Moving to next file.2130982527201 2. Takeover happens on the standby, and it becomes the new primary. 3. Transactions continue and new logs are generated on new primary. Finally, when log info array is full and DB2 trys to reuse one entry, the error will be hit, and following messages will appear in db2diag.log: 2011-12-15-00.13.22.849529+540 I2986417A454 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:64 MESSAGE : Extent number: DATA #1 : Hexdump, 4 bytes 0x070000001EFFD460 : 0000 6E1E ..n. 2011-12-15-00.13.22.849903+540 I2986872A469 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:65 MESSAGE : Entry number (readEntry): DATA #1 : Hexdump, 8 bytes 0x07000008E0027228 : 0000 0000 0000 0034 .......4 2011-12-15-00.13.22.850071+540 I2987342A483 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:66 MESSAGE : read entry extent number (readEntryExtNum): DATA #1 : Hexdump, 4 bytes 0x07000008E0027230 : 0000 6D14 ..m. 2011-12-15-00.13.22.850240+540 I2987826A1084 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:67 DATA #1 : String, 128 bytes The db2logts log info array is out of sync. Bringing down the db2logts daemon. From this point on, LOG SKIPPING is DISABLED. DATA #2 : Hexdump, 112 bytes 0x07000008E0029BA0 : 0000 6D12 0000 0000 0000 0997 EAB2 8001 ..m............. 0x07000008E0029BB0 : 0000 0000 0000 0000 0000 0016 0000 0000 ................ 0x07000008E0029BC0 : 0000 0000 0000 0000 0000 0000 0165 0000 .............e.. 0x07000008E0029BD0 : 0700 000D 0650 AF80 0700 000D 0650 B1A0 .....P.......P.. 0x07000008E0029BE0 : 0700 000D 0650 B3C0 0700 000D 0650 B5E0 .....P.......P.. 0x07000008E0029BF0 : 0700 000D 0650 B800 0700 000D 0650 5460 .....P.......PT` 0x07000008E0029C00 : 0700 000D 0650 5680 0700 000D 0650 58A0 .....PV......PX. 2011-12-15-00.13.22.850416+540 I2988911A165 LEVEL: Severe PID:9699938 TID:5798 NODE:000 Title: SQLP_DBCB Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin 2011-12-15-00.13.22.907328+540 I2989077A166 LEVEL: Severe PID:9699938 TID:5798 NODE:000 Title: SQLPG_XHDR Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin 2011-12-15-00.13.22.907523+540 I2989244A375 LEVEL: Error PID : 9699938 TID : 5798 PROC : db2sysc 0 INSTANCE: db2inst1 NODE : 000 DB : HADRDB EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0 FUNCTION: DB2 UDB, recovery manager, sqlpInformLogTsAboutNewLogHeader, probe:0 RETCODE : ZRC=0x00000000=0=PSM_OK "Unknown" | |
Local Fix: | |
NA | |
available fix packs: | |
DB2 Version 9.7 Fix Pack 6 for Linux, UNIX, and Windows | |
Solution | |
Workaround | |
not known / see Local fix | |
BUG-Tracking | |
forerunner : APAR is sysrouted TO one or more of the following: IC85207 follow-up : | |
Timestamps | |
Date - problem reported : Date - problem closed : Date - last modified : | 10.01.2012 19.04.2013 19.04.2013 |
Problem solved at the following versions (IBM BugInfos) | |
Problem solved according to the fixlist(s) of the following version(s) | |
9.7.0.6 |