Home

Latest versions	fixlist
11.1.0.7
10.5.0.9
10.1.0.6
9.8.0.5
9.7.0.11
9.5.0.10
9.1.0.12

Have problems? - contact us.
Register for free
Contact form

DB2 - Problem description

Problem IC74633	Status: Closed
A PANIC MAY OCCUR AFTER ISSUING DB2STOP FORCE ON HADR STANDBY WHILE TAKEOVER HADR DB OPERATION IS IN PROGRESS
product:
DB2 FOR LUW / DB2FORLUW / 950 - DB2
Problem description:
Running db2 takeover hadr on db on a standby machine followed by db2 stop force before the takeover command completes successfully can cause memory to be freed twice. This would lead to a panic in the system. This APAR can only occur on DB2 9.5 FP7. 2010-11-25-17.42.00.931548-300 I68908668E377 LEVEL: Warning PID : 21039 TID : 47382024677696PROC : db2sysc INSTANCE: svtdbm NODE : 000 EDUID : 310 EDUNAME: db2hadrp (SAKE) FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrSDoTakeover, probe:47003 MESSAGE : Info: Standby has completed takeover (now primary). 2010-11-25-17.42.00.944158-300 I68909046E342 LEVEL: Warning PID : 21039 TID : 47382024677696PROC : db2sysc INSTANCE: svtdbm NODE : 000 EDUID : 310 EDUNAME: db2hadrp (SAKE) FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrEduS, probe:20342 MESSAGE : Info: Standby Finished. 2010-11-25-17.42.00.945189-300 I68909389E553 LEVEL: Warning PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, base sys utilities, sqeAgentServices::RequestAgent, probe:12 MESSAGE : Coordinator request rejected. DATA #1 : Hexdump, 4 bytes 0x00002B17E2FE9400 : 0700 0000 .... 2010-11-25-17.42.00.948897-300 I68909943E461 LEVEL: Error PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, base sys utilities, sqeAgentServices::RequestAgent, probe:70 MESSAGE : Agent not allocated, sqlcode = -6036 2010-11-25-17.42.00.950313-300 E68910405E495 LEVEL: Warning PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, base sys utilities, sqeAgentServices::RequestAgent, probe:100 MESSAGE : ADM7519W DB2 could not allocate an agent. The SQLCODE is "-6036". 2010-11-25-17.42.00.954150-300 I68910901E456 LEVEL: Severe PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, base sys utilities, sqeIcoordCB::DispatchIndependentDBAgent, probe:110 RETCODE : ZRC=0xFFFFE86C=-6036 2010-11-25-17.42.00.955750-300 I68911358E544 LEVEL: Error PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrSpawnICoord, probe:30100 MESSAGE : HADR ICoord Error: DATA #1 : Hexdump, 4 bytes 0x00002B17E2FEA9B0 : 6D00 0580 m... 2010-11-25-17.42.00.957140-300 I68911903E533 LEVEL: Error PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrSpawnICoord, probe:30101 MESSAGE : sqlcode: DATA #1 : Hexdump, 4 bytes 0x00002B17E2FEA9F0 : 6CE8 FFFF l... 2010-11-25-17.42.00.958549-300 I68912437E558 LEVEL: Warning PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrTakeoverHdrRouteIn, probe:55603 RETCODE : ZRC=0x8005006D=-2147155859=SQLE_CA_BUILT "SQLCA has been built and saved in component specific control block." 2010-11-25-17.42.00.954201-300 I68912996E341 LEVEL: Warning PID : 21039 TID : 47382024677696PROC : db2sysc INSTANCE: svtdbm NODE : 000 EDUID : 310 EDUNAME: db2hadrp (SAKE) FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrEduP, probe:20301 MESSAGE : Info: Primary Started. 2010-11-25-17.42.00.965391-300 I68913338E389 LEVEL: Warning PID : 21039 TID : 47382024677696PROC : db2sysc INSTANCE: svtdbm NODE : 000 EDUID : 310 EDUNAME: db2hadrp (SAKE) FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrSetTcpWindowSize, probe:32201 MESSAGE : Info: HADR Socket send buffer size, SO_SNDBUF: 16384 bytes 2010-11-25-17.42.00.966825-300 I68913728E392 LEVEL: Warning PID : 21039 TID : 47382024677696PROC : db2sysc INSTANCE: svtdbm NODE : 000 EDUID : 310 EDUNAME: db2hadrp (SAKE) FUNCTION: DB2 UDB, High Availability Disaster Recovery, hdrSetTcpWindowSize, probe:32251 MESSAGE : Info: HADR Socket receive buffer size, SO_RCVBUF: 87380 bytes 2010-11-25-17.42.00.965075-300 I68914121E1738 LEVEL: Severe PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, SQO Memory Management, sqloDiagnoseFreeBlockFailure, probe:10 MESSAGE : Possible memory corruption detected. DATA #1 : ZRC, PD_TYPE_ZRC, 4 bytes 0x820F0002 DATA #2 : Corrupt block address, PD_TYPE_CORRUPT_BLK_PTR, 8 bytes 0x00002b1789615100 DATA #3 : Block header, PD_TYPE_BLK_HEADER, 24 bytes 0x00002B17896150E8 : 1596 78B1 0200 B0FA 0000 0000 0000 0000 ..x............. 0x00002B17896150F8 : 0000 0000 0000 0000 ........ DATA #4 : Data header, PD_TYPE_BLK_DATA_HEAD, 48 bytes 0x00002B1789615100 : 8000 0000 0000 0000 DDDD DDDD DDDD DDDD ................ 0x00002B1789615110 : DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD ................ CALLSTCK: [0] 0x00002B1761A3CEEC pdLog + 0x18C [1] 0x00002B1763F7042A /home/svtdbm/sqllib/lib64/libdb2e.so.1 + 0x382E42A [2] 0x00002B1763F70F52 sqlofmblkEx + 0x130 [3] 0x00000000004147B6 _Z9sqlofmblkPv + 0x18 [4] 0x00002B1763EE040A sqlo_acs_fblk + 0x1C [5] 0x00002B176259A889 _Z26sqlexAgentCleanUserSecInfoP16SQLEX_AUTHINFO_TP5sqlca + 0xA3 [6] 0x00002B1762385917 _ZN14sqeApplication12AppStopUsingEP8sqeAgenthP5sqlca + 0xF67 [7] 0x00002B1762532B3C _Z11sqlesrspWrpP14db2UCinterface + 0x2D8 [8] 0x00002B176253306D sqleUCagentConnectReset + 0x17D [9] 0x00002B17626B852C /home/svtdbm/sqllib/lib64/libdb2e.so.1 + 0x1F7652C 2010-11-25-17.42.00.976550-300 I68915860E456 LEVEL: Severe PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, SQO Memory Management, sqloDiagnoseFreeBlockFailure, probe:30 DATA #1 : String, 12 bytes Double free. 2010-11-25-17.42.00.978148-300 E68916317E928 LEVEL: Critical PID : 21039 TID : 47381592664384PROC : db2sysc INSTANCE: svtdbm NODE : 000 DB : SAKE APPHDL : 0-28727 APPID: LOCAL.svtdbm.101126110514 AUTHID : SVTDBM EDUID : 630 EDUNAME: db2agent (SAKE) FUNCTION: DB2 UDB, base sys utilities, sqle_panic, probe:10 MESSAGE : ADM14001C An unexpected and critical error has occurred: "Panic". The instance may have been shutdown as a result. "Automatic" FODC (First Occurrence Data Capture) has been invoked and diagnostic information has been recorded in directory "/home/svtdbm/sqllib/db2dump/FODC_Panic_2010-11-25-17.42.00.9780 28/". Please look in this directory for detailed evidence about what happened and contact IBM support if necessary to diagnose the problem.
Problem Summary:
**************************************************************** * USERS AFFECTED: * * HADR environments * **************************************************************** * PROBLEM DESCRIPTION: * * Running db2 takeover hadr on db on a standby machine * * followed by * * db2 stop force before the takeover command completes * * * * successfully can cause memory to be freed twice. This would * * lead * * to a panic in the system. This APAR can only occur on DB2 * * 9.5 * * FP7. * **************************************************************** * RECOMMENDATION: * * Upgrade to DB2 9.5 FP8 or higher * ****************************************************************
Local Fix:
Don't issue db2stop force on standby instance while HADR takeover database operation is being performed.
available fix packs:
DB2 Version 9.5 Fix Pack 8 for Linux, UNIX, and Windows DB2 Version 9.5 Fix Pack 9 for Linux, UNIX, and Windows DB2 Version 9.5 Fix Pack 10 for Linux, UNIX, and Windows
Solution
The fix to avoid the double free of memory in this situation is included in DB2 9.5 FP8.
Workaround
not known / see Local fix
Timestamps
Date - problem reported : Date - problem closed : Date - last modified :	22.02.2011 27.06.2011 27.06.2011
Problem solved at the following versions (IBM BugInfos)
9.5.FP8
Problem solved according to the fixlist(s) of the following version(s)
9.5.0.8