The handling of escape sequences within the BSON functions has
had issues (see APAR IT28448 and APAR IT33799).
There still seems to be a problem with the GenBSON function.
If we try to do:
Short example to reproduce the problem:
echo "UNLOAD TO 'test.unl' SELECT data, data::JSON as json,
bson_value_lvarchar(data, 'test') as bvstring, teststring as
output_wished FROM (SELECT test as teststring,
genbson(ROW(test),1,1) as data FROM (SELECT 'abc/def\'ghi\'"jkl'
as test FROM systables where tabid = 1))" | dbaccess -e
sysmaster
This shows that the GenBSON function returns the following byte
representation:
22000000027465737400130000006162635c2f6465665c5c6768695c226a6b6c
0000
This sequence is wrong. The correct sequence is:
1f000000027465737400100000006162632f6465665c676869226a6b6c0000
Note that there are 3 additional backslash bytes (0x5C) inside
the BSON byte representation (despite the length changes from
0x00000022 to 0x0000001f and 0x00000013 to 0x00000010). Those
additional backslash bytes seem to have been added as encoding
the JSON string in a 1:1 manner - but these are really only
needed to create a valid JSON representation. So these bytes
should under no circumstances be stored inside the BSON field
BUT need to be added if handling the field with the ::JSON cast
(as well as that they should be discarded if a JSON string is
presented to the database and written into the BSON object).
If we "omit" the GenBSON function for this test and present the
JSON (obviously with the correct needed escaping) and use this
command:
echo "UNLOAD TO 'test.unl' SELECT data, data::JSON as json,
bson_value_lvarchar(data, 'test') as bvstring FROM (SELECT
'{\'"test\'":\'"abc\'\'/def\'\'\'\'ghi\'\'\'"jkl\'"}'::JSON::BSON as data
FROM systables where tabid = 1)" | dbaccess -e sysmaster
We still see the same problem:
Database selected.
UNLOAD TO 'test.unl' SELECT data, data::JSON as json,
bson_value_lvarchar(data, 'test') as bvstring FROM (SELECT
'{"test":"abc\'/def\'\'ghi\'"jkl"}'::JSON::BSON as data FROM
systables where tabid = 1)
1 row(s) unloaded.
Database closed.
Result in test.unl:
22000000027465737400130000006162635c2f6465665c5c6768695c226a6b6c
0000|{"test":"abc\'\'/def\'\'\'\'ghi\'\'"jkl"}|abc\'\'/def\'\'\'\'ghi\'\'"jkl|
So the problem seems not to be related to the GenBSON function
but to the internal JSONBSON cast. |
