Skip to content

Jackson: Deserialize map with non String keys #244

Closed

Description

@fdinoff

I'm trying to deserialize a map that contains keys that are not Strings. They do not seem to be deserialized properly.

For instance the library I am interfacing with, treats all strings as the binary type. (The encoding of the bytes not guaranteed to be utf-8 which is why they are not using the string type.)

So assuming I have a map of {a:1, b:2, c:3} where a, b and c are binary payloads. The following code tries to deserialize that into a Map<byte[], Integer>

ByteArrayOutputStream out = new ByteArrayOutputStream();
MessagePacker messagePacker = MessagePack.newDefaultPacker(out).packMapHeader(3);
for (int i = 0; i < 3; i++) {
    messagePacker
            .packBinaryHeader(1).writePayload(new byte[]{(byte) (i + 'a')})
            .packInt(i);
}
messagePacker.close();

ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
Map<byte[], Integer> map = objectMapper.readValue(
        out.toByteArray(), new TypeReference<Map<byte[], Integer>>() {});

This results in an exception

com.fasterxml.jackson.databind.JsonMappingException: Can not find a (Map) Key deserializer for type [array type, component type: [simple type, class byte]]
    at com.fasterxml.jackson.databind.deser.DeserializerCache._handleUnknownKeyDeserializer(DeserializerCache.java:608)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findKeyDeserializer(DeserializerCache.java:169)
    at com.fasterxml.jackson.databind.DeserializationContext.findKeyDeserializer(DeserializationContext.java:462)
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer.createContextual(MapDeserializer.java:231)
    at com.fasterxml.jackson.databind.DeserializationContext.handleSecondaryContextualization(DeserializationContext.java:653)
    at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:444)
    at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3666)
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3558)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2688)

A quick search of this on google looks like I would need to add a KeyDeserializer. http://stackoverflow.com/questions/11246748/deserializing-non-string-map-keys-with-jackson. So I created a SimpleModule and registered it with the ObjectMapper.

ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
SimpleModule module = new SimpleModule();
module.addKeyDeserializer(byte[].class, new KeyDeserializer() {
            @Override
            public Object deserializeKey(String key, DeserializationContext ctxt) throws
                    IOException,
                    JsonProcessingException {
                throw new IOException("KeyDeserializer Called");
            }
        });
objectMapper.registerModule(module);

Map<byte[], Integer> map = objectMapper.readValue(
        out.toByteArray(), new TypeReference<Map<byte[], Integer>>() {});

However it doesn't appear the KeyDeserializer ever called called and the returned map is empty.


This is also a problem with Integer keys (and I assume all other non string keys)

ByteArrayOutputStream out = new ByteArrayOutputStream();
MessagePacker messagePacker = MessagePack.newDefaultPacker(out).packMapHeader(3);
for (int i = 0; i < 3; i++) {
    messagePacker
            .packInt(i)
            .packInt(i);
}
messagePacker.close();

ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
Map<Integer, Integer> map = objectMapper.readValue(
        out.toByteArray(), new TypeReference<Map<Integer, Integer>>() {});

Exception is thrown without the KeyDeserializer. Adding the KeyDeserializer the map is empty.

(Side comment (and possibly another issue): Serializing a Map<Integer, Integer> with jackson actually outputs a map of Map<String, Integer> on the msgpack side which is undesirable.)

Activity

  1. added this to the v07 milestone on Jun 23, 2015
  2. komamitsu commented on Aug 9, 2015

    @komamitsu
    Member

    @fdinoff In #271, jackson-dataformat-msgpack supports Boolean, Integer and Float type key while byte array key seemed to work before the change.

    BTW, you tried to use Map<byte[], Integer> map but I don't think it's easy to use byte[] as a key of Map since byte[]#equals doesn't match other byte arrays which have the same values.

  3. simpleseeker commented on Aug 10, 2015

    @simpleseeker

    Iy
    Bl MDM a
    N,b. Et

    On Sunday, August 9, 2015, Mitsunori Komatsu notifications@github.com
    wrote:

    @fdinoff https://github.com/fdinoff In #271
    #271,
    jackson-dataformat-msgpack supports Boolean, Integer and Float type key
    while byte array key seemed to work before the change.

    BTW, you tried to use Map<byte[], Integer> map but I don't think it's
    easy to use byte[] as a key of Map since byte[]#equals doesn't match
    other byte arrays which have the same values.

    —
    Reply to this email directly or view it on GitHub
    #244 (comment)
    .

  4. fdinoff commented on Aug 20, 2015

    @fdinoff
    ContributorAuthor

    @komamitsu Thanks for adding this.

    Would it be possible to get a Map<String, Integer> map back if byte[] was the key of the map? The user would have to provide the enocding (By passing a Charset) so that you could decode it properly.

  5. komamitsu commented on Aug 21, 2015

    @komamitsu
    Member

    Map<String, Integer> can be obtained from Map<byte[], Integer> easily like this and I don't think this library doesn't need to provide it.

            Map<byte[], Integer> orig = new HashMap<>();
            orig.put("aa".getBytes(), 1111);
            orig.put("bb".getBytes(), 2222);
    
            Map<String, Integer> converted = orig.entrySet().stream().collect(
                    Collectors.toMap(e -> new String(e.getKey()), Map.Entry::getValue));
    
  6. fdinoff commented on Aug 23, 2015

    @fdinoff
    ContributorAuthor

    @komamitsu Sorry, didn't realize that byte[] was a valid key now.

    The problem now is what happens if the bytes weren't valid utf8? is the conversion to String and back to a byte array lossless? The doc for String#getBytes() says its unspecified, so the conversion isn't lossless. Losing valid data would be undesirable.

    Also I don't think you can deserialize into an actual object. (Is there?)

    Example: You have the following class

    class SomeObject {
        public Integer hello;
    }

    I would expect something like the following to work (the byte array is utf16 encoded.

    ByteArrayOutputStream out = new ByteArrayOutputStream();
    MessagePacker messagePacker = MessagePack.newDefaultPacker(out).packMapHeader(1);
    byte[] bytes = "Hello".getBytes(StandardCharsets.UTF_16); 
    messagePacker.packBinaryHeader(bytes.length).writePayload(bytes);
    messagePacker.packInt(42);
    messagePacker.close();
    
    SomeObject o = objectMapper.readValue(out.toByteArray(), SomeObject.class);
    assertEquals(42, o.hello);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions