X-Git-Url: https://gerrit.automotivelinux.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2Freference.rst;h=6c38f6b40558be53b964fd6d202af1edfe0a60c5;hb=b663909fb6e86f0ae0f450523e72fb7fbfb719ab;hp=3331c6d3b735da2e32bf77c44f31bdd0ef8f2532;hpb=9b1e1b440ab6a21bacab939b9c7bef0fa4ca5c90;p=apps%2Fagl-service-can-low-level.git diff --git a/docs/reference.rst b/docs/reference.rst index 3331c6d3..6c38f6b4 100644 --- a/docs/reference.rst +++ b/docs/reference.rst @@ -6,25 +6,187 @@ Nanopb: API reference .. contents :: + + + Compilation options =================== -The following options can be specified using -D switch given to the C compiler: - -============================ ================================================================================================ -__BIG_ENDIAN__ Set this if your platform stores integers and floats in big-endian format. - Mixed-endian systems (different layout for ints and floats) are currently not supported. -NANOPB_INTERNALS Set this to expose the field encoder functions that are hidden since nanopb-0.1.3. -PB_MAX_REQUIRED_FIELDS Maximum number of required fields to check for presence. Default value is 64. Increases stack - usage 1 byte per every 8 fields. Compiler warning will tell if you need this. -PB_FIELD_16BIT Add support for tag numbers > 255 and fields larger than 255 bytes or 255 array entries. - Increases code size 3 bytes per each field. Compiler error will tell if you need this. -PB_FIELD_32BIT Add support for tag numbers > 65535 and fields larger than 65535 bytes or 65535 array entries. - Increases code size 9 bytes per each field. Compiler error will tell if you need this. -============================ ================================================================================================ - -The PB_MAX_REQUIRED_FIELDS, PB_FIELD_16BIT and PB_FIELD_32BIT settings allow raising some datatype limits to suit larger messages. -Their need is recognized automatically by C-preprocessor #if-directives in the generated .pb.h files. The default setting is to use -the smallest datatypes (least resources used). +The following options can be specified in one of two ways: + +1. Using the -D switch on the C compiler command line. +2. By #defining them at the top of pb.h. + +You must have the same settings for the nanopb library and all code that +includes pb.h. + +============================ ================================================ +__BIG_ENDIAN__ Set this if your platform stores integers and + floats in big-endian format. Mixed-endian + systems (different layout for ints and floats) + are currently not supported. +NANOPB_INTERNALS Set this to expose the field encoder functions + that are hidden since nanopb-0.1.3. +PB_MAX_REQUIRED_FIELDS Maximum number of required fields to check for + presence. Default value is 64. Increases stack + usage 1 byte per every 8 fields. Compiler + warning will tell if you need this. +PB_FIELD_16BIT Add support for tag numbers > 255 and fields + larger than 255 bytes or 255 array entries. + Increases code size 3 bytes per each field. + Compiler error will tell if you need this. +PB_FIELD_32BIT Add support for tag numbers > 65535 and fields + larger than 65535 bytes or 65535 array entries. + Increases code size 9 bytes per each field. + Compiler error will tell if you need this. +PB_NO_ERRMSG Disables the support for error messages; only + error information is the true/false return + value. Decreases the code size by a few hundred + bytes. +PB_BUFFER_ONLY Disables the support for custom streams. Only + supports encoding and decoding with memory + buffers. Speeds up execution and decreases code + size slightly. +PB_OLD_CALLBACK_STYLE Use the old function signature (void\* instead + of void\*\*) for callback fields. This was the + default until nanopb-0.2.1. +PB_SYSTEM_HEADER Replace the standard header files with a single + header file. It should define all the required + functions and typedefs listed on the + `overview page`_. Value must include quotes, + for example *#define PB_SYSTEM_HEADER "foo.h"*. +============================ ================================================ + +The PB_MAX_REQUIRED_FIELDS, PB_FIELD_16BIT and PB_FIELD_32BIT settings allow +raising some datatype limits to suit larger messages. Their need is recognized +automatically by C-preprocessor #if-directives in the generated .pb.h files. +The default setting is to use the smallest datatypes (least resources used). + +.. _`overview page`: index.html#compiler-requirements + + +Proto file options +================== +The generator behaviour can be adjusted using these options, defined in the +'nanopb.proto' file in the generator folder: + +============================ ================================================ +max_size Allocated size for *bytes* and *string* fields. +max_count Allocated number of entries in arrays + (*repeated* fields). +type Type of the generated field. Default value + is *FT_DEFAULT*, which selects automatically. + You can use *FT_CALLBACK*, *FT_STATIC* or + *FT_IGNORE* to force a callback field, a static + field or to completely ignore the field. +long_names Prefix the enum name to the enum value in + definitions, i.e. *EnumName_EnumValue*. Enabled + by default. +packed_struct Make the generated structures packed. + NOTE: This cannot be used on CPUs that break + on unaligned accesses to variables. +============================ ================================================ + +These options can be defined for the .proto files before they are converted +using the nanopb-generatory.py. There are three ways to define the options: + +1. Using a separate .options file. + This is the preferred way as of nanopb-0.2.1, because it has the best + compatibility with other protobuf libraries. +2. Defining the options on the command line of nanopb_generator.py. + This only makes sense for settings that apply to a whole file. +3. Defining the options in the .proto file using the nanopb extensions. + This is the way used in nanopb-0.1, and will remain supported in the + future. It however sometimes causes trouble when using the .proto file + with other protobuf libraries. + +The effect of the options is the same no matter how they are given. The most +common purpose is to define maximum size for string fields in order to +statically allocate them. + +Defining the options in a .options file +--------------------------------------- +The preferred way to define options is to have a separate file +'myproto.options' in the same directory as the 'myproto.proto'. :: + + # myproto.proto + message MyMessage { + required string name = 1; + repeated int32 ids = 4; + } + +:: + + # myproto.options + MyMessage.name max_size:40 + MyMessage.ids max_count:5 + +The generator will automatically search for this file and read the +options from it. The file format is as follows: + +* Lines starting with '#' or '//' are regarded as comments. +* Blank lines are ignored. +* All other lines should start with a field name pattern, followed by one or + more options. For example: *"MyMessage.myfield max_size:5 max_count:10"*. +* The field name pattern is matched against a string of form *'Message.field'*. + For nested messages, the string is *'Message.SubMessage.field'*. +* The field name pattern may use the notation recognized by Python fnmatch(): + + - *\** matches any part of string, like 'Message.\*' for all fields + - *\?* matches any single character + - *[seq]* matches any of characters 's', 'e' and 'q' + - *[!seq]* matches any other character + +* The options are written as *'option_name:option_value'* and several options + can be defined on same line, separated by whitespace. +* Options defined later in the file override the ones specified earlier, so + it makes sense to define wildcard options first in the file and more specific + ones later. + +If preferred, the name of the options file can be set using the command line +switch *-f* to nanopb_generator.py. + +Defining the options on command line +------------------------------------ +The nanopb_generator.py has a simple command line option *-s OPTION:VALUE*. +The setting applies to the whole file that is being processed. + +Defining the options in the .proto file +--------------------------------------- +The .proto file format allows defining custom options for the fields. +The nanopb library comes with *nanopb.proto* which does exactly that, allowing +you do define the options directly in the .proto file:: + + import "nanopb.proto"; + + message MyMessage { + required string name = 1 [(nanopb).max_size = 40]; + repeated int32 ids = 4 [(nanopb).max_count = 5]; + } + +A small complication is that you have to set the include path of protoc so that +nanopb.proto can be found. This file, in turn, requires the file +*google/protobuf/descriptor.proto*. This is usually installed under +*/usr/include*. Therefore, to compile a .proto file which uses options, use a +protoc command similar to:: + + protoc -I/usr/include -Inanopb/generator -I. -omessage.pb message.proto + +The options can be defined in file, message and field scopes:: + + option (nanopb_fileopt).max_size = 20; // File scope + message Message + { + option (nanopb_msgopt).max_size = 30; // Message scope + required string fieldsize = 1 [(nanopb).max_size = 40]; // Field scope + } + + + + + + + + pb.h ==== @@ -33,22 +195,23 @@ pb_type_t --------- Defines the encoder/decoder behaviour that should be used for a field. :: - typedef enum { ... } pb_type_t; + typedef uint8_t pb_type_t; -The low-order byte of the enumeration values defines the function that can be used for encoding and decoding the field data: +The low-order nibble of the enumeration values defines the function that can be used for encoding and decoding the field data: ==================== ===== ================================================ LTYPE identifier Value Storage format ==================== ===== ================================================ PB_LTYPE_VARINT 0x00 Integer. PB_LTYPE_SVARINT 0x01 Integer, zigzag encoded. -PB_LTYPE_FIXED 0x02 Integer or floating point. -PB_LTYPE_BYTES 0x03 Structure with *size_t* field and byte array. -PB_LTYPE_STRING 0x04 Null-terminated string. -PB_LTYPE_SUBMESSAGE 0x05 Submessage structure. +PB_LTYPE_FIXED32 0x02 32-bit integer or floating point. +PB_LTYPE_FIXED64 0x03 64-bit integer or floating point. +PB_LTYPE_BYTES 0x04 Structure with *size_t* field and byte array. +PB_LTYPE_STRING 0x05 Null-terminated string. +PB_LTYPE_SUBMESSAGE 0x06 Submessage structure. ==================== ===== ================================================ -The high-order byte defines whether the field is required, optional, repeated or callback: +The bits 4-5 define whether the field is required, optional or repeated: ==================== ===== ================================================ HTYPE identifier Value Field handling @@ -56,13 +219,24 @@ HTYPE identifier Value Field handling PB_HTYPE_REQUIRED 0x00 Verify that field exists in decoded message. PB_HTYPE_OPTIONAL 0x10 Use separate *has_* boolean to specify whether the field is present. -PB_HTYPE_ARRAY 0x20 A repeated field with preallocated array. + (Unless it is a callback) +PB_HTYPE_REPEATED 0x20 A repeated field with preallocated array. Separate *_count* for number of items. -PB_HTYPE_CALLBACK 0x30 A field with dynamic storage size, data is - actually a pointer to a structure containing a - callback function. + (Unless it is a callback) +==================== ===== ================================================ + +The bits 6-7 define the how the storage for the field is allocated: + +==================== ===== ================================================ +ATYPE identifier Value Allocation method +==================== ===== ================================================ +PB_ATYPE_STATIC 0x00 Statically allocated storage in the structure. +PB_ATYPE_CALLBACK 0x40 A field with dynamic storage size. Struct field + actually contains a pointer to a callback + function. ==================== ===== ================================================ + pb_field_t ---------- Describes a single structure field with memory position in relation to others. The descriptions are usually autogenerated. :: @@ -79,7 +253,7 @@ Describes a single structure field with memory position in relation to others. T } pb_packed; :tag: Tag number of the field or 0 to terminate a list of fields. -:type: LTYPE and HTYPE of the field. +:type: LTYPE, HTYPE and ATYPE of the field. :data_offset: Offset of field data, relative to the end of the previous field. :size_offset: Offset of *bool* flag for optional fields or *size_t* count for arrays, relative to field data. :data_size: Size of a single data entry, in bytes. For PB_LTYPE_BYTES, the size of the byte array inside the containing structure. For PB_HTYPE_CALLBACK, size of the C data type if known. @@ -106,14 +280,16 @@ Part of a message structure, for fields with type PB_HTYPE_CALLBACK:: typedef struct _pb_callback_t pb_callback_t; struct _pb_callback_t { union { - bool (*decode)(pb_istream_t *stream, const pb_field_t *field, void *arg); - bool (*encode)(pb_ostream_t *stream, const pb_field_t *field, const void *arg); + bool (*decode)(pb_istream_t *stream, const pb_field_t *field, void **arg); + bool (*encode)(pb_ostream_t *stream, const pb_field_t *field, void * const *arg); } funcs; void *arg; }; -The *arg* is passed to the callback when calling. It can be used to store any information that the callback might need. +A pointer to the *arg* is passed to the callback when calling. It can be used to store any information that the callback might need. + +Previously the function received just the value of *arg* instead of a pointer to it. This old behaviour can be enabled by defining *PB_OLD_CALLBACK_STYLE*. When calling `pb_encode`_, *funcs.encode* is used, and similarly when calling `pb_decode`_, *funcs.decode* is used. The function pointers are stored in the same memory location but are of incompatible types. You can set the function pointer to NULL to skip the field. @@ -128,6 +304,41 @@ Protocol Buffers wire types. These are used with `pb_encode_tag`_. :: PB_WT_32BIT = 5 } pb_wire_type_t; +PB_GET_ERROR +------------ +Get the current error message from a stream, or a placeholder string if +there is no error message:: + + #define PB_GET_ERROR(stream) (string expression) + +This should be used for printing errors, for example:: + + if (!pb_decode(...)) + { + printf("Decode failed: %s\n", PB_GET_ERROR(stream)); + } + +The macro only returns pointers to constant strings (in code memory), +so that there is no need to release the returned pointer. + +PB_RETURN_ERROR +--------------- +Set the error message and return false:: + + #define PB_RETURN_ERROR(stream,msg) (sets error and returns false) + +This should be used to handle error conditions inside nanopb functions +and user callback functions:: + + if (error_condition) + { + PB_RETURN_ERROR(stream, "something went wrong"); + } + +The *msg* parameter must be a constant string. + + + pb_encode.h =========== @@ -277,6 +488,17 @@ In Protocol Buffers format, the submessage size must be written before the subme If the submessage contains callback fields, the callback function might misbehave and write out a different amount of data on the second call. This situation is recognized and *false* is returned, but garbage will be written to the output before the problem is detected. + + + + + + + + + + + pb_decode.h =========== @@ -320,15 +542,15 @@ In addition to EOF, the pb_decode implementation supports terminating a message For optional fields, this function applies the default value and sets *has_* to false if the field is not present. -pb_decode_varint +pb_decode_noinit ---------------- -Read and decode a varint_ encoded integer. :: +Same as `pb_decode`_, except does not apply the default values to fields. :: - bool pb_decode_varint(pb_istream_t *stream, uint64_t *dest); + bool pb_decode_noinit(pb_istream_t *stream, const pb_field_t fields[], void *dest_struct); -:stream: Input stream to read from. 1-10 bytes will be read. -:dest: Storage for the decoded integer. Value is undefined on error. -:returns: True on success, false if value exceeds uint64_t range or an IO error happens. +(parameters are the same as for `pb_decode`_.) + +The destination structure should be filled with zeros before calling this function. Doing a *memset* manually can be slightly faster than using `pb_decode`_ if you don't need any default values. pb_skip_varint -------------- @@ -373,48 +595,41 @@ Remove the data for a field from the stream, without actually decoding it:: :wire_type: Type of field to skip. :returns: True on success, false on IO error. -.. sidebar:: Field decoders +.. sidebar:: Decoding fields manually - The functions with names beginning with *pb_dec_* are called field decoders. Each PB_LTYPE has an own field decoder, which handles translating from Protocol Buffers data to C data. + The functions with names beginning with *pb_decode_* are used when dealing with callback fields. The typical reason for using callbacks is to have an array of unlimited size. In that case, `pb_decode`_ will call your callback function repeatedly, which can then store the values into e.g. filesystem in the order received in. - Each field decoder reads and decodes a single value. For arrays, the decoder is called repeatedly. + For decoding numeric (including enumerated and boolean) values, use `pb_decode_varint`_, `pb_decode_svarint`_, `pb_decode_fixed32`_ and `pb_decode_fixed64`_. They take a pointer to a 32- or 64-bit C variable, which you may then cast to smaller datatype for storage. - You can use the decoders from your callbacks. Just be aware that the pb_field_t passed to the callback is not directly compatible - with the *varint* field decoders. Instead, you must create a new pb_field_t structure and set the data_size according to the data type - you pass to *dest*, e.g. *field.data_size = sizeof(int);*. Other fields in the *pb_field_t* don't matter. + For decoding strings and bytes fields, the length has already been decoded. You can therefore check the total length in *stream->bytes_left* and read the data using `pb_read`_. - The field decoder interface is a bit messy as a result of the interface required inside the nanopb library. - Eventually they may be replaced by separate wrapper functions with a more friendly interface. + Finally, for decoding submessages in a callback, simply use `pb_decode`_ and pass it the *SubMessage_fields* descriptor array. -pb_dec_varint -------------- -Field decoder for PB_LTYPE_VARINT. :: +pb_decode_varint +---------------- +Read and decode a varint_ encoded integer. :: - bool pb_dec_varint(pb_istream_t *stream, const pb_field_t *field, void *dest) + bool pb_decode_varint(pb_istream_t *stream, uint64_t *dest); :stream: Input stream to read from. 1-10 bytes will be read. -:field: Field description structure. Only *field->data_size* matters. -:dest: Pointer to destination integer. Must have size of *field->data_size* bytes. -:returns: True on success, false on IO errors or if `pb_decode_varint`_ fails. - -This function first calls `pb_decode_varint`_. It then copies the first bytes of the 64-bit result value to *dest*, or on big endian architectures, the last bytes. +:dest: Storage for the decoded integer. Value is undefined on error. +:returns: True on success, false if value exceeds uint64_t range or an IO error happens. -pb_dec_svarint --------------- -Field decoder for PB_LTYPE_SVARINT. Similar to `pb_dec_varint`_, except that it performs zigzag-decoding on the value. :: +pb_decode_svarint +----------------- +Similar to `pb_decode_varint`_, except that it performs zigzag-decoding on the value. This corresponds to the Protocol Buffers *sint32* and *sint64* datatypes. :: - bool pb_dec_svarint(pb_istream_t *stream, const pb_field_t *field, void *dest); + bool pb_decode_svarint(pb_istream_t *stream, int64_t *dest); -(parameters are the same as `pb_dec_varint`_) +(parameters are the same as `pb_decode_varint`_) -pb_dec_fixed32 --------------- -Field decoder for PB_LTYPE_FIXED32. :: +pb_decode_fixed32 +----------------- +Decode a *fixed32*, *sfixed32* or *float* value. :: - bool pb_dec_fixed32(pb_istream_t *stream, const pb_field_t *field, void *dest); + bool pb_decode_fixed32(pb_istream_t *stream, void *dest); :stream: Input stream to read from. 4 bytes will be read. -:field: Not used. :dest: Pointer to destination *int32_t*, *uint32_t* or *float*. :returns: True on success, false on IO errors. @@ -422,9 +637,9 @@ This function reads 4 bytes from the input stream. On big endian architectures, it then reverses the order of the bytes. Finally, it writes the bytes to *dest*. -pb_dec_fixed64 --------------- -Field decoder for PB_LTYPE_FIXED64. :: +pb_decode_fixed64 +----------------- +Decode a *fixed64*, *sfixed64* or *double* value. :: bool pb_dec_fixed(pb_istream_t *stream, const pb_field_t *field, void *dest); @@ -433,53 +648,28 @@ Field decoder for PB_LTYPE_FIXED64. :: :dest: Pointer to destination *int64_t*, *uint64_t* or *double*. :returns: True on success, false on IO errors. -Same as `pb_dec_fixed32`_, except this reads 8 bytes. - -pb_dec_bytes ------------- -Field decoder for PB_LTYPE_BYTES. Reads a length-prefixed block of bytes. :: - - bool pb_dec_bytes(pb_istream_t *stream, const pb_field_t *field, void *dest); - -**Note:** This is an internal function that is not useful in decoder callbacks. To read bytes fields in callbacks, use -*stream->bytes_left* and `pb_read`_. +Same as `pb_decode_fixed32`_, except this reads 8 bytes. -:stream: Input stream to read from. -:field: Field description structure. Only *field->data_size* matters. -:dest: Pointer to a structure similar to pb_bytes_array_t. -:returns: True on success, false on IO error or if length exceeds the array size. - -This function expects a pointer to a structure with a *size_t* field at start, and a variable sized byte array after it. It will deduce the maximum size of the array from *field->data_size*. - -pb_dec_string -------------- -Field decoder for PB_LTYPE_STRING. Reads a length-prefixed string. :: - - bool pb_dec_string(pb_istream_t *stream, const pb_field_t *field, void *dest); +pb_make_string_substream +------------------------ +Decode the length for a field with wire type *PB_WT_STRING* and create a substream for reading the data. :: -**Note:** This is an internal function that is not useful in decoder callbacks. To read string fields in callbacks, use -*stream->bytes_left* and `pb_read`_. + bool pb_make_string_substream(pb_istream_t *stream, pb_istream_t *substream); -:stream: Input stream to read from. -:field: Field description structure. Only *field->data_size* matters. -:dest: Pointer to a character array of size *field->data_size*. -:returns: True on success, false on IO error or if length exceeds the array size. - -This function null-terminates the string when successful. On error, the contents of the destination array is undefined. +:stream: Original input stream to read the length and data from. +:substream: New substream that has limited length. Filled in by the function. +:returns: True on success, false if reading the length fails. -pb_dec_submessage ------------------ -Field decoder for PB_LTYPE_SUBMESSAGE. Calls `pb_decode`_ to perform the actual decoding. :: +This function uses `pb_decode_varint`_ to read an integer from the stream. This is interpreted as a number of bytes, and the substream is set up so that its `bytes_left` is initially the same as the length, and its callback function and state the same as the parent stream. - bool pb_dec_submessage(pb_istream_t *stream, const pb_field_t *field, void *dest) +pb_close_string_substream +------------------------- +Close the substream created with `pb_make_string_substream`_. :: -**Note:** This is an internal function that is not useful in decoder callbacks. To read submessage fields in callbacks, use -`pb_decode`_ directly. - -:stream: Input stream to read from. -:field: Field description structure. Only *field->ptr* matters. -:dest: Pointer to the destination structure. -:returns: True on success, false on IO error or if `pb_decode`_ fails. + void pb_close_string_substream(pb_istream_t *stream, pb_istream_t *substream); -The *field->ptr* should be a pointer to *pb_field_t* array describing the submessage. +:stream: Original input stream to read the length and data from. +:substream: Substream to close +This function copies back the state from the substream to the parent stream. +It must be called after done with the substream.