As the instance has run out of memory, Db2 attempts to dump
diagnostics to FODC_AppErr. In this error code path, -901 was
hit, further prolonging an outage. FODC_AppErr during stress run
on the BLUMPP configuration, SQL901: Error unholding variation
lock.
db2set -all
[e] DB2COMM=TCPIP
[i] DB2_COL_CONFIG_LIMIT=SELLEN 64K, SORTKEY 32K, GRPKEY 32K
[i] DB2_DEFER_MEMORY_COMMIT=ON
[i] DB2_USE_BUFFERED_READ_FOR_ACTIVE_LOG=YES
[i]
DB2_CDE_REDUCED_LOGGING=SYNC_FIRST_PAGE_EXTENT_CRASH_REC:FALSE
[i] DB2_WORKLOAD=ANALYTICS
[i] DB2RSHCMD=/bin/ssh
[i] DB2_USE_ALTERNATE_PAGE_CLEANING=ON [DB2_WORKLOAD]
[i] DB2_REDUCED_OPTIMIZATION=COL_RTS ON
[i] DB2_EXTENDED_OPTIMIZATION=CDETQ NOENCODE
[i] DB2_ANTIJOIN=EXTEND [DB2_WORKLOAD]
[i] DB2SLEEP=ON
[i] DB2COMM=TCPIP
[g] DB2SYSTEM=MYHOST1
Stack Trace
=============
-------Frame------ ------Function + Offset------
0x09000000005C6884 pthread_kill + 0xC4
0x090000000B5CC51C sqloDumpEDU + 0x9C
0x090000000BB96B74 sqldDumpContext__FP9sqeBsuEduiN42PCcPvT2 +
0x13F4
0x090000000B9DD440 sqlrr_dump_ffdc__FP8sqlrr_cbiT2 + 0xC50
0x090000000B9DC798 sqlzeDumpFFDC__FP8sqeAgentUiP5sqlcai + 0x188
0x090000000B9DB8B8 sqlzeSqlCode__FP8sqeAgentUiUlT2P5sqlcaiUsPc +
0x288
0x090000000B9F8FEC sqlrrSqlCode + 0x18C
0x090000000F81B058
sqlra_get_section_subagent__FP8sqlrr_cbPUcsT2T3T2UsUliUi + 0x618
0x09000000114CB354
sqlrr_subagent_router__FP8sqeAgentP12SQLE_DB2RA_T + 0xF84
0x09000000114C859C IPRA.$sqleSubRequestRouter__FP8sqeAgentPUiPUl
+ 0x88C
0x0900000011474ACC sqleProcessSubRequest__FP8sqeAgent + 0x25FC
0x09000000114658C4 RunEDU__8sqeAgentFv + 0xAF4
0x090000000BC706DC EDUDriver__9sqzEDUObjFv + 0x2FC
0x090000000BC703A0 sqlzRunEDU__FPcUi + 0x10
0x090000000BC725A4 sqloEDUEntry + 0x374
db2daig.log
===========
2021-03-31-16.19.37.577430-240 I7917094A566 LEVEL:
Warning
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, SQO Memory Management,
SqloMemController::getPartitionStats, probe:16
DATA #1 :
DB-BLUDB - Current size : 74447616 KB, HWM : 79305728 KB,
Cached : 0 KB
2021-03-31-16.19.37.580511-240 I7917661A1038 LEVEL: Error
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, access plan manager,
sqlra_sqlW_mem_get_block, probe:9999
MESSAGE : ZRC=0x8B0F0014=-1961951212=SQLO_NOMEM_APPSHRH
"No memory available in 'Applications Shared Heap'"
DIA8300C A memory heap error has occurred.
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: 0 sqlerrml: 0
sqlerrmc:
sqlerrp : SQL11056
sqlerrd : (1) 0x00000000 (2) 0x00000000 (3)
0x00000000
(4) 0x00000000 (5) 0x00000000 (6)
0x00000000
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate:
2021-03-31-16.19.37.585871-240 I7918700A621 LEVEL: Error
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, access plan manager, sqlra_sqlW_copy_section,
probe:200
RETCODE : ZRC=0x8B0F0014=-1961951212=SQLO_NOMEM_APPSHRH
"No memory available in 'Applications Shared Heap'"
DIA8300C A memory heap error has occurred.
2021-03-31-16.19.37.591570-240 I7919914A628 LEVEL: Error
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, access plan manager,
sqlra_sqlW_get_dynamic_section, probe:900
RETCODE : ZRC=0x8B0F0014=-1961951212=SQLO_NOMEM_APPSHRH
"No memory available in 'Applications Shared Heap'"
DIA8300C A memory heap error has occurred.
2021-03-31-16.19.37.595075-240 I7920543A1049 LEVEL: Error
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, access plan manager, sqlra_load_var,
probe:200
MESSAGE : ZRC=0x8B0F0014=-1961951212=SQLO_NOMEM_APPSHRH
"No memory available in 'Applications Shared Heap'"
DIA8300C A memory heap error has occurred.
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: -973 sqlerrml: 18
sqlerrmc: application shared
sqlerrp : SQLRA116
sqlerrd : (1) 0x8B0F0014 (2) 0x00000014 (3)
0x00000000
(4) 0x00000000 (5) 0x00000000 (6)
0x00000000
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate:
2021-03-31-16.19.37.597982-240 I7921593A1568 LEVEL:
Severe
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, lock manager, sqlpluhd, probe:30
MESSAGE : ZRC=0x82100001=-2112880639=SQLP_NONSEVERE_PRGERR
"DPS detects non-severe programming error"
DIA8532C An internal processing error has occurred.
DATA #1 : String, 15 bytes
Cannot find LRB
DATA #2 : String, 8 bytes
SQLPLUHD
CALLSTCK: (Static functions may not be resolved correctly, as
they are resolved to the nearest symbol)
[0] 0x090000000B5D2D20 pdLog + 0x6A0
[1] 0x090000000B9DB28C sqlzSetAndLog901 + 0x1DC
[2] 0x090000000BABC22C sqlpluhd__FP9sqeBsuEduP14SQLP_LOCK_INFO
+ 0xAAC
[3] 0x090000000F81AFD4
sqlra_get_section_subagent__FP8sqlrr_cbPUcsT2T3T2UsUliUi + 0x594
[4] 0x09000000114CB354
sqlrr_subagent_router__FP8sqeAgentP12SQLE_DB2RA_T + 0xF84
[5] 0x09000000114C859C
IPRA.$sqleSubRequestRouter__FP8sqeAgentPUiPUl + 0x88C
[6] 0x0900000011474ACC sqleProcessSubRequest__FP8sqeAgent +
0x25FC
2021-03-31-16.19.37.615841-240 I7925084A493 LEVEL: Info
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, lock manager, sqlpluhd, probe:30
DATA #1 :
clientInfo.appl_name GreenTea temporal.se
2021-03-31-16.19.37.622151-240 I7926200A875 LEVEL:
Severe
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, relation data serv, sqlrrSqlCode, probe:15
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: -973 sqlerrml: 18
sqlerrmc: application shared
sqlerrp : SQLRA116
sqlerrd : (1) 0x8B0F0014 (2) 0x00000014 (3)
0x00000000
(4) 0x00000000 (5) 0x00000000 (6)
0x00000000
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate:
2021-03-31-16.19.37.630601-240 E7929347A803 LEVEL: Error
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, relation data serv, sqlrr_dump_ffdc, probe:30
MESSAGE : ADM14005E The following error occurred: "AppErr".
First Occurrence
Data Capture (FODC) has been invoked in the following
mode:
"Automatic". Diagnostic information has been recorded
in the
directory named
"/home/db2inst1/sqllib/db2dump/FODC_AppErr_2021-03-31-16.19.37.6
28221_
6750968_640459_000/".
2021-03-31-16.19.37.657913-240 I7931541A891 LEVEL:
Severe
PID : 6750968 TID : 640459 PROC :
db2sysc 0
INSTANCE: db2inst1 NODE : 000 DB :
BLUDB
APPHDL : 1-193 APPID: X.X.X.X.0000.210331202803
AUTHID : DB2INST1 HOSTNAME: MYHOST1
EDUID : 640459 EDUNAME: db2agntp (BLUDB) 0
FUNCTION: DB2 UDB, relation data serv, sqlrr_dump_ffdc,
probe:250
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: -901 sqlerrml: 30
sqlerrmc: Error unholding variation lock
sqlerrp : SQLRA146
sqlerrd : (1) 0x00000000 (2) 0x00000000 (3)
0x00000000
(4) 0x00000000 (5) 0xFFFFF40D (6)
0x00000000
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate: |
