{"action":"created","issue":{"url":"https://api.github.com/repos/nanopb/nanopb/issues/174","labels_url":"https://api.github.com/repos/nanopb/nanopb/issues/174/labels{/name}","comments_url":"https://api.github.com/repos/nanopb/nanopb/issues/174/comments","events_url":"https://api.github.com/repos/nanopb/nanopb/issues/174/events","html_url":"https://github.com/nanopb/nanopb/issues/174","id":110323438,"number":174,"title":"Encode callback is called twice","user":{"login":"oije2333","id":14018415,"avatar_url":"https://avatars.githubusercontent.com/u/14018415?v=3","gravatar_id":"","url":"https://api.github.com/users/oije2333","html_url":"https://github.com/oije2333","followers_url":"https://api.github.com/users/oije2333/followers","following_url":"https://api.github.com/users/oije2333/following{/other_user}","gists_url":"https://api.github.com/users/oije2333/gists{/gist_id}","starred_url":"https://api.github.com/users/oije2333/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oije2333/subscriptions","organizations_url":"https://api.github.com/users/oije2333/orgs","repos_url":"https://api.github.com/users/oije2333/repos","events_url":"https://api.github.com/users/oije2333/events{/privacy}","received_events_url":"https://api.github.com/users/oije2333/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-10-07T21:37:44Z","updated_at":"2015-10-08T04:20:38Z","closed_at":null,"body":"When encoding a message containing a sub-message that has a string field, the encode callback is called twice.\r\n\r\nModified the \"simple\" example to demonstrate:\r\n\r\nsimple.proto:\r\n\r\n    // A very simple protocol definition, consisting of only\r\n    // one message.\r\n\r\n    syntax = \"proto2\";\r\n\r\n    message SimpleMessage {\r\n        optional string lucky_number = 1;\r\n    }\r\n\r\n    message Response {\r\n      optional SimpleMessage simpleMessage = 1;\r\n    }\r\n\r\nsimple.c:\r\n\r\n    #include <stdio.h>\r\n    #include <pb_encode.h>\r\n    #include <pb_decode.h>\r\n    #include \"simple.pb.h\"\r\n\r\n    bool encode_string(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)\r\n    {\r\n        printf(\"encode_string...\\n\");\r\n\r\n        char *str = \"Hello world!\";\r\n    \r\n        if (!pb_encode_tag_for_field(stream, field))\r\n            return false;\r\n    \r\n        return pb_encode_string(stream, (uint8_t*)str, strlen(str));\r\n    }\r\n\r\n    bool decode_string(pb_istream_t *stream, const pb_field_t *field, void **arg)\r\n    {\r\n        printf(\"decode_string...\\n\");\r\n\r\n        uint8_t buffer[1024] = {0};\r\n    \r\n        /* We could read block-by-block to avoid the large buffer... */\r\n        if (stream->bytes_left > sizeof(buffer) - 1)\r\n            return false;\r\n    \r\n        if (!pb_read(stream, buffer, stream->bytes_left))\r\n            return false;\r\n    \r\n        /* Print the string, in format comparable with protoc --decode.\r\n         * Format comes from the arg defined in main().\r\n         */\r\n        printf((char*)*arg, buffer);\r\n        return true;\r\n    }\r\n\r\n    int main()\r\n    {\r\n        /* This is the buffer where we will store our message. */\r\n        uint8_t buffer[128];\r\n        size_t message_length;\r\n        bool status;\r\n    \r\n        /* Encode our message */\r\n        {\r\n            /* Allocate space on the stack to store the message data.\r\n             *\r\n             * Nanopb generates simple struct definitions for all the messages.\r\n             * - check out the contents of simple.pb.h!\r\n             * It is a good idea to always initialize your structures\r\n             * so that you do not have garbage data from RAM in there.\r\n             */\r\n            Response message = Response_init_zero;\r\n            message.has_simpleMessage = true;\r\n        \r\n            /* Create a stream that will write to our buffer. */\r\n            pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));\r\n        \r\n            /* Fill in the callback */\r\n            message.simpleMessage.lucky_number.funcs.encode = &encode_string;\r\n        \r\n            /* Now we are ready to encode the message! */\r\n            status = pb_encode(&stream, Response_fields, &message);\r\n            message_length = stream.bytes_written;\r\n        \r\n            /* Then just check for any errors.. */\r\n            if (!status)\r\n            {\r\n                printf(\"Encoding failed: %s\\n\", PB_GET_ERROR(&stream));\r\n                return 1;\r\n            }\r\n        }\r\n    \r\n        /* Now we could transmit the message over network, store it in a file or\r\n         * wrap it to a pigeon's leg.\r\n         */\r\n\r\n        /* But because we are lazy, we will just decode it immediately. */\r\n    \r\n        {\r\n            /* Allocate space for the decoded message. */\r\n            Response message = Response_init_zero;\r\n        \r\n            /* Create a stream that reads from the buffer. */\r\n            pb_istream_t stream = pb_istream_from_buffer(buffer, message_length);\r\n        \r\n            /* Fill in the callback and arg */\r\n            message.simpleMessage.lucky_number.funcs.decode = &decode_string;\r\n            message.simpleMessage.lucky_number.arg = \"stringvalue: \\\"%s\\\"\\n\";\r\n\r\n            /* Now we are ready to decode the message. */\r\n            status = pb_decode(&stream, Response_fields, &message);\r\n        \r\n            /* Check for errors... */\r\n            if (!status)\r\n            {\r\n                printf(\"Decoding failed: %s\\n\", PB_GET_ERROR(&stream));\r\n                return 1;\r\n            }\r\n        }\r\n    \r\n        return 0;\r\n    }\r\n"},"comment":{"url":"https://api.github.com/repos/nanopb/nanopb/issues/comments/146415454","html_url":"https://github.com/nanopb/nanopb/issues/174#issuecomment-146415454","issue_url":"https://api.github.com/repos/nanopb/nanopb/issues/174","id":146415454,"user":{"login":"PetteriAimonen","id":922265,"avatar_url":"https://avatars.githubusercontent.com/u/922265?v=3","gravatar_id":"","url":"https://api.github.com/users/PetteriAimonen","html_url":"https://github.com/PetteriAimonen","followers_url":"https://api.github.com/users/PetteriAimonen/followers","following_url":"https://api.github.com/users/PetteriAimonen/following{/other_user}","gists_url":"https://api.github.com/users/PetteriAimonen/gists{/gist_id}","starred_url":"https://api.github.com/users/PetteriAimonen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PetteriAimonen/subscriptions","organizations_url":"https://api.github.com/users/PetteriAimonen/orgs","repos_url":"https://api.github.com/users/PetteriAimonen/repos","events_url":"https://api.github.com/users/PetteriAimonen/events{/privacy}","received_events_url":"https://api.github.com/users/PetteriAimonen/received_events","type":"User","site_admin":false},"created_at":"2015-10-08T04:20:38Z","updated_at":"2015-10-08T04:20:38Z","body":"It is normal that encoding callbacks are called multiple times. This is necessary to obtain the submessage size, which in protobuf format must be encoded before the submessage itself. This is documented in e.g. https://koti.kapsi.fi/jpa/nanopb/docs/concepts.html#encoding-callbacks\r\n\r\nI'm not aware of any alternate approaches which would support encoding into streams. When encoding into memory buffers, the re-encoding would be possible to avoid. However such optimization has not been implemented so far. (Listed as a limitation in https://koti.kapsi.fi/jpa/nanopb/docs/index.html )"}}