3 High Performance, Feature-Rich Idiomatic Go codec/encoding library for
4 binc, msgpack, cbor, json.
6 Supported Serialization formats are:
8 - msgpack: https://github.com/msgpack/msgpack
9 - binc: http://github.com/ugorji/binc
10 - cbor: http://cbor.io http://tools.ietf.org/html/rfc7049
11 - json: http://json.org http://tools.ietf.org/html/rfc7159
16 go get github.com/ugorji/go/codec
18 This package will carefully use 'unsafe' for performance reasons in specific places.
19 You can build without unsafe use by passing the safe or appengine tag
20 i.e. 'go install -tags=safe ...'. Note that unsafe is only supported for the last 3
21 go sdk versions e.g. current go release is go 1.9, so we support unsafe use only from
22 go 1.7+ . This is because supporting unsafe requires knowledge of implementation details.
24 Online documentation: http://godoc.org/github.com/ugorji/go/codec
25 Detailed Usage/How-to Primer: http://ugorji.net/blog/go-codec-primer
27 The idiomatic Go support is as seen in other encoding packages in
28 the standard library (ie json, xml, gob, etc).
30 Rich Feature Set includes:
32 - Simple but extremely powerful and feature-rich API
33 - Support for go1.4 and above, while selectively using newer APIs for later releases
34 - Excellent code coverage ( > 90% )
35 - Very High Performance.
36 Our extensive benchmarks show us outperforming Gob, Json, Bson, etc by 2-4X.
37 - Careful selected use of 'unsafe' for targeted performance gains.
38 100% mode exists where 'unsafe' is not used at all.
39 - Lock-free (sans mutex) concurrency for scaling to 100's of cores
40 - In-place updates during decode, with option to zero the value in maps and slices prior to decode
41 - Coerce types where appropriate
42 e.g. decode an int in the stream into a float, decode numbers from formatted strings, etc
44 Overflows, nil maps/slices, nil values in streams are handled correctly
45 - Standard field renaming via tags
46 - Support for omitting empty fields during an encoding
47 - Encoding from any value and decoding into pointer to any value
48 (struct, slice, map, primitives, pointers, interface{}, etc)
49 - Extensions to support efficient encoding/decoding of any named types
50 - Support encoding.(Binary|Text)(M|Unm)arshaler interfaces
51 - Support IsZero() bool to determine if a value is a zero value.
52 Analogous to time.Time.IsZero() bool.
53 - Decoding without a schema (into a interface{}).
54 Includes Options to configure what specific map or slice type to use
55 when decoding an encoded list or map into a nil interface{}
56 - Mapping a non-interface type to an interface, so we can decode appropriately
57 into any interface type with a correctly configured non-interface value.
58 - Encode a struct as an array, and decode struct from an array in the data stream
59 - Option to encode struct keys as numbers (instead of strings)
60 (to support structured streams with fields encoded as numeric codes)
61 - Comprehensive support for anonymous fields
62 - Fast (no-reflection) encoding/decoding of common maps and slices
63 - Code-generation for faster performance.
64 - Support binary (e.g. messagepack, cbor) and text (e.g. json) formats
65 - Support indefinite-length formats to enable true streaming
66 (for formats which support it e.g. json, cbor)
67 - Support canonical encoding, where a value is ALWAYS encoded as same sequence of bytes.
68 This mostly applies to maps, where iteration order is non-deterministic.
69 - NIL in data stream decoded as zero value
70 - Never silently skip data when decoding.
71 User decides whether to return an error or silently skip data when keys or indexes
72 in the data stream do not map to fields in the struct.
73 - Encode/Decode from/to chan types (for iterative streaming support)
74 - Drop-in replacement for encoding/json. `json:` key in struct tag supported.
75 - Provides a RPC Server and Client Codec for net/rpc communication protocol.
76 - Handle unique idiosyncrasies of codecs e.g.
77 - For messagepack, configure how ambiguities in handling raw bytes are resolved
78 - For messagepack, provide rpc server/client codec to support
79 msgpack-rpc protocol defined at:
80 https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md
84 Users can register a function to handle the encoding or decoding of
87 There are no restrictions on what the custom type can be. Some examples:
92 type MyStructWithUnexportedFields struct { a int; b bool; c []int; }
93 type GifImage struct { ... }
95 As an illustration, MyStructWithUnexportedFields would normally be
96 encoded as an empty map because it has no exported fields, while UUID
97 would be encoded as a string. However, with extension support, you can
98 encode any of these however you like.
100 ## Custom Encoding and Decoding
102 This package maintains symmetry in the encoding and decoding halfs.
103 We determine how to encode or decode by walking this decision tree
105 - is type a codec.Selfer?
106 - is there an extension registered for the type?
107 - is format binary, and is type a encoding.BinaryMarshaler and BinaryUnmarshaler?
108 - is format specifically json, and is type a encoding/json.Marshaler and Unmarshaler?
109 - is format text-based, and type an encoding.TextMarshaler?
110 - else we use a pair of functions based on the "kind" of the type e.g. map, slice, int64, etc
112 This symmetry is important to reduce chances of issues happening because the
113 encoding and decoding sides are out of sync e.g. decoded via very specific
114 encoding.TextUnmarshaler but encoded via kind-specific generalized mode.
116 Consequently, if a type only defines one-half of the symmetry
117 (e.g. it implements UnmarshalJSON() but not MarshalJSON() ),
118 then that type doesn't satisfy the check and we will continue walking down the
123 RPC Client and Server Codecs are implemented, so the codecs can be used
124 with the standard net/rpc package.
130 // create and configure Handle
133 mh codec.MsgpackHandle
137 mh.MapType = reflect.TypeOf(map[string]interface{}(nil))
139 // configure extensions
140 // e.g. for msgpack, define functions and enable Time support for tag 1
141 // mh.SetExt(reflect.TypeOf(time.Time{}), 1, myExt)
143 // create and use decoder/encoder
148 h = &bh // or mh to use msgpack
151 dec = codec.NewDecoder(r, h)
152 dec = codec.NewDecoderBytes(b, h)
155 enc = codec.NewEncoder(w, h)
156 enc = codec.NewEncoderBytes(&b, h)
162 conn, err := listener.Accept()
163 rpcCodec := codec.GoRpc.ServerCodec(conn, h)
164 //OR rpcCodec := codec.MsgpackSpecRpc.ServerCodec(conn, h)
165 rpc.ServeCodec(rpcCodec)
169 //RPC Communication (client side)
170 conn, err = net.Dial("tcp", "localhost:5555")
171 rpcCodec := codec.GoRpc.ClientCodec(conn, h)
172 //OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h)
173 client := rpc.NewClientWithCodec(rpcCodec)
177 To run tests, use the following:
181 To run the full suite of tests, use the following:
183 go test -tags alltests -run Suite
185 You can run the tag 'safe' to run tests or build in safe mode. e.g.
187 go test -tags safe -run Json
188 go test -tags "alltests safe" -run Suite
190 ## Running Benchmarks
192 Please see http://github.com/ugorji/go-codec-bench .
196 Struct fields matching the following are ignored during encoding and decoding
198 - struct tag value set to -
199 - func, complex numbers, unsafe pointers
200 - unexported and not embedded
201 - unexported and embedded and not struct kind
202 - unexported and embedded pointers (from go1.10)
204 Every other field in a struct will be encoded/decoded.
206 Embedded fields are encoded as if they exist in the top-level struct,
207 with some caveats. See Encode documentation.