On HADR primary, db2logts daemon on HADR primary could be
brought down due to log info array being out of sync.
The issue can be hit when following events happened:
1. HADR standby restarts from last byte of a log file. When this
event happens, we can see below message in db2diag.log:
2011-12-13-18.09.58.511431+540 E920335A494 LEVEL: Warning
PID : 9699938 TID : 71547 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
APPHDL : 0-2523 APPID: *LOCAL.DB2.111213092048
EDUID : 71547 EDUNAME: db2shred (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager, sqlpshrEdu, probe:45320
DATA #1 : <preformatted>
Scan starting from the last byte of a truncated log file. Moving
to next file.2130982527201
2. Takeover happens on the standby, and it becomes the new
primary.
3. Transactions continue and new logs are generated on new
primary. Finally, when log info array is full and DB2 trys to
reuse one entry, the error will be hit, and following messages
will appear in db2diag.log:
2011-12-15-00.13.22.849529+540 I2986417A454 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:64
MESSAGE : Extent number:
DATA #1 : Hexdump, 4 bytes
0x070000001EFFD460 : 0000 6E1E
..n.
2011-12-15-00.13.22.849903+540 I2986872A469 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:65
MESSAGE : Entry number (readEntry):
DATA #1 : Hexdump, 8 bytes
0x07000008E0027228 : 0000 0000 0000 0034
.......4
2011-12-15-00.13.22.850071+540 I2987342A483 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:66
MESSAGE : read entry extent number (readEntryExtNum):
DATA #1 : Hexdump, 4 bytes
0x07000008E0027230 : 0000 6D14
..m.
2011-12-15-00.13.22.850240+540 I2987826A1084 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:67
DATA #1 : String, 128 bytes
The db2logts log info array is out of sync.
Bringing down the db2logts daemon.
From this point on, LOG SKIPPING is DISABLED.
DATA #2 : Hexdump, 112 bytes
0x07000008E0029BA0 : 0000 6D12 0000 0000 0000 0997 EAB2 8001
..m.............
0x07000008E0029BB0 : 0000 0000 0000 0000 0000 0016 0000 0000
................
0x07000008E0029BC0 : 0000 0000 0000 0000 0000 0000 0165 0000
.............e..
0x07000008E0029BD0 : 0700 000D 0650 AF80 0700 000D 0650 B1A0
.....P.......P..
0x07000008E0029BE0 : 0700 000D 0650 B3C0 0700 000D 0650 B5E0
.....P.......P..
0x07000008E0029BF0 : 0700 000D 0650 B800 0700 000D 0650 5460
.....P.......PT`
0x07000008E0029C00 : 0700 000D 0650 5680 0700 000D 0650 58A0
.....PV......PX.
2011-12-15-00.13.22.850416+540 I2988911A165 LEVEL: Severe
PID:9699938 TID:5798 NODE:000 Title: SQLP_DBCB
Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin
2011-12-15-00.13.22.907328+540 I2989077A166 LEVEL: Severe
PID:9699938 TID:5798 NODE:000 Title: SQLPG_XHDR
Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin
2011-12-15-00.13.22.907523+540 I2989244A375 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:0
RETCODE : ZRC=0x00000000=0=PSM_OK "Unknown" |
On HADR primary, db2logts daemon on HADR primary could be
brought down due to log info array being out of sync.
The issue can be hit when following events happened:
1. HADR standby restarts from last byte of a log file. When this
event happens, we can see below message in db2diag.log:
2011-12-13-18.09.58.511431+540 E920335A494 LEVEL: Warning
PID : 9699938 TID : 71547 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
APPHDL : 0-2523 APPID: *LOCAL.DB2.111213092048
EDUID : 71547 EDUNAME: db2shred (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager, sqlpshrEdu, probe:45320
DATA #1 : <preformatted>
Scan starting from the last byte of a truncated log file. Moving
to next file.2130982527201
2. Takeover happens on the standby, and it becomes the new
primary.
3. Transactions continue and new logs are generated on new
primary. Finally, when log info array is full and DB2 trys to
reuse one entry, the error will be hit, and following messages
will appear in db2diag.log:
2011-12-15-00.13.22.849529+540 I2986417A454 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:64
MESSAGE : Extent number:
DATA #1 : Hexdump, 4 bytes
0x070000001EFFD460 : 0000 6E1E
..n.
2011-12-15-00.13.22.849903+540 I2986872A469 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:65
MESSAGE : Entry number (readEntry):
DATA #1 : Hexdump, 8 bytes
0x07000008E0027228 : 0000 0000 0000 0034
.......4
2011-12-15-00.13.22.850071+540 I2987342A483 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:66
MESSAGE : read entry extent number (readEntryExtNum):
DATA #1 : Hexdump, 4 bytes
0x07000008E0027230 : 0000 6D14
..m.
2011-12-15-00.13.22.850240+540 I2987826A1084 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:67
DATA #1 : String, 128 bytes
The db2logts log info array is out of sync.
Bringing down the db2logts daemon.
From this point on, LOG SKIPPING is DISABLED.
DATA #2 : Hexdump, 112 bytes
0x07000008E0029BA0 : 0000 6D12 0000 0000 0000 0997 EAB2 8001
..m.............
0x07000008E0029BB0 : 0000 0000 0000 0000 0000 0016 0000 0000
................
0x07000008E0029BC0 : 0000 0000 0000 0000 0000 0000 0165 0000
.............e..
0x07000008E0029BD0 : 0700 000D 0650 AF80 0700 000D 0650 B1A0
.....P.......P..
0x07000008E0029BE0 : 0700 000D 0650 B3C0 0700 000D 0650 B5E0
.....P.......P..
0x07000008E0029BF0 : 0700 000D 0650 B800 0700 000D 0650 5460
.....P.......PT`
0x07000008E0029C00 : 0700 000D 0650 5680 0700 000D 0650 58A0
.....PV......PX.
2011-12-15-00.13.22.850416+540 I2988911A165 LEVEL: Severe
PID:9699938 TID:5798 NODE:000 Title: SQLP_DBCB
Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin
2011-12-15-00.13.22.907328+540 I2989077A166 LEVEL: Severe
PID:9699938 TID:5798 NODE:000 Title: SQLPG_XHDR
Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin
2011-12-15-00.13.22.907523+540 I2989244A375 LEVEL: Error
PID : 9699938 TID : 5798 PROC : db2sysc
0
INSTANCE: db2inst1 NODE : 000 DB : HADRDB
EDUID : 5798 EDUNAME: db2loggr (HADRDB) 0
FUNCTION: DB2 UDB, recovery manager,
sqlpInformLogTsAboutNewLogHeader, probe:0
RETCODE : ZRC=0x00000000=0=PSM_OK "Unknown" |
