Bitmessage Wiki - User contributions [en]

Bitmessage Protocol Version 3

2016-11-16T01:57:58Z

Bmng-dev: Distinguish identity from pubkey structure, minor corrections

==Common standards==

=== Hashes ===

Most of the time [http://en.wikipedia.org/wiki/SHA-2 SHA-512] hashes are used, however [http://en.wikipedia.org/wiki/RIPEMD RIPEMD-160] is also used when creating an address.

A double-round of SHA-512 is used for the Proof Of Work. Example of double-SHA-512 encoding of string "hello":

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round of sha-512)
0592a10584ffabf96539f3d780d776828c67da1ab5b169e9e8aed838aaecc9ed36d49ff1423c55f019e050c66c6324f53588be88894fef4dcffdb74b98e2b200(second round of sha-512)

For Bitmessage addresses (RIPEMD-160) this would give:

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round is sha-512)
79a324faeebcbf9849f310545ed531556882487e (with ripemd-160)

== Common structures ==

All integers are encoded in big endian

=== Message structure ===

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 || magic || uint32_t || Magic value indicating message origin network, and used to seek to next message when stream state is unknown
|-
| 12 || command || char[12] || ASCII string identifying the packet content, NULL padded (non-NULL padding results in packet rejected)
|-
| 4 || length || uint32_t || Length of payload in number of bytes. The maximum allowed value is 1,600,003 bytes
|-
| 4 || checksum || uint32_t || First 4 bytes of sha512(payload)
|-
| ? || message_payload || uchar[] || The actual data, a [[#Message_types|message]]. Not to be confused with objectPayload.
|}

Known magic values:

{|class="wikitable"
! Magic value !! Sent over wire as
|-
| 0xE9BEB4D9 || E9 BE B4 D9
|}

=== Variable length integer ===

An integer can be encoded depending on the represented value to save space. Variable length integers always precede an array/vector of a type of data that may vary in length. Varints MUST use the minimum possible number of bytes to encode a value. For example; the value 6 can be encoded with one byte therefore a varint that uses three bytes to encode the value 6 is malformed and the decoding task must be aborted.

{|class="wikitable"
! Value !! Storage length !! Format
|-
| < 0xfd || 1 || uint8_t
|-
| <= 0xffff || 3 || 0xfd followed by the integer as uint16_t
|-
| <= 0xffffffff || 5 || 0xfe followed by the integer as uint32_t
|-
| - || 9 || 0xff followed by the integer as uint64_t
|}

=== Variable length string ===

A variable length string can be stored using a variable length integer to encode the length followed by the string itself.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1 - 9 || length || [[#Variable_length_integer|var_int]] || Length of the string
|-
| length || string || char[] || The string itself (can be empty)
|}

=== Variable length list of integers===

n integers can be stored using n+1 [[#Variable_length_integer|variable length integers]] where the first var_int equals n.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1 - 9 || count|| [[#Variable_length_integer|var_int]] || Number of var_ints below
|-
| 1 - 9 || || [[#Variable_length_integer|var_int]] || The first value stored
|-
| 1 - 9 || || [[#Variable_length_integer|var_int]] || The second value stored...
|-
| 1 - 9 || || [[#Variable_length_integer|var_int]] || etc...
|}

=== Network address ===

When a network address is needed somewhere, this structure is used. Network addresses are not prefixed with a timestamp or stream in the version message.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8 || time || uint64 || the Time.
|-
| 4 || stream || uint32 || Stream number for this node
|-
| 8 || services || uint64_t || same service(s) listed in [[#version|version]]
|-
| 16 || IPv6/4 || char[16] || IPv6 address. IPv4 addresses are written into the message as a 16 byte [http://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses IPv4-mapped IPv6 address]
(12 bytes ''00 00 00 00 00 00 00 00 00 00 FF FF'', followed by the 4 bytes of the IPv4 address). Hidden Service addresses can be represented as an IPv6 address with a 48-bit routing prefix of fd87:d87e:eb43::48 under the Unique Local Address block (fc00::/7) with the remaining 10 bytes being the Base256 encoding of the Hidden Service address
|-
| 2 || port || uint16_t || port number
|}

=== Inventory Vectors ===

Inventory vectors are used for notifying other nodes about objects they have or data which is being requested. Two rounds of SHA-512 are used, resulting in a 64 byte hash. Only the first 32 bytes are used; the remaining 32 bytes are ignored.

Inventory vectors consist of the following data format:

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || hash || char[32] || Hash of the object
|}

=== Identity ===

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 || [[#Pubkey_bitfield_features|behavior bitfield]] || uint32_t || A bitfield of optional behaviors and features of the identity.
|-
| 64 || public signing key || uchar[64] || The ECC public key used for signing in uncompressed format without the point compression prefix
|-
| 64 || public encryption key || uchar[64] || The ECC public key used for encryption in uncompressed format without the point compression prefix
|-
| 3 - 9 || nonce_trials_per_byte || [[#Variable_length_integer|var_int]] || Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000.
|-
| 3 - 9 || extra_bytes || [[#Variable_length_integer|var_int]] || Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000.
|}

=== Envelope ===

Bitmessage uses [https://en.wikipedia.org/wiki/Integrated_Encryption_Scheme ECIES] to encrypt its messages. For more information see [[Encryption]]

==== Plain Envelope ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 16 || IV || uchar[16] || Initialization Vector used for AES-256-CBC
|-
| 2 || elliptic curve || uint16_t || Elliptic Curve secp256k1. This is the NID (numerical identifier) 714 (0x02CA) assigned by OpenSSL to represent secp256k1
|-
| 2 || X length || uint16_t || Length of X component of public key R
|-
| X length || X || uchar[X length] || X component of public key R
|-
| 2 || Y length || uint16_t || Length of Y component of public key R
|-
| Y length || Y || uchar[Y length] || Y component of public key R
|-
| ? || encrypted || uchar[] || Cipher text
|-
| 32 || mac || uchar[32] || HMACSHA256 Message Authentication Code
|}

==== Tagged Envelope ====

A tagged envelope is identical to an [[#Plain_Envelope|plain envelope]] but prepended with a tag. Tagged envelopes are only used by [[#v4_pubkey|v4 pubkeys]] and [[#v5 broadcast|v5 broadcasts]].

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || tag || uchar[32] || The recipients tag
|-
| 1+ || envelope || [[#Plain_Envelope|plain_envelope]] ||
|}

== Enumerateds and Flags ==

=== Message Encodings ===

{|class="wikitable"
! Value !! Name !! Description
|-
| 0 || IGNORE || Any data with this number may be ignored. The sending node might simply be sharing its public key with you.
|-
| 1 || TRIVIAL|| UTF-8. No 'Subject' or 'Body' sections. Useful for simple strings of data, like URIs or magnet links.
|-
| 2 || SIMPLE || UTF-8. Uses 'Subject' and 'Body' sections. No MIME is used.
<code>messageToTransmit = 'Subject:' + subject + '\n' + 'Body:' + message</code>
|}

Further values for the message encodings can be decided upon by the community. Any MIME or MIME-like encoding format, should they be used, should make use of Bitmessage's 8-bit bytes.

=== Identity bitfield features ===

{|class="wikitable"
! Bit!! Name !! Description
|-
| 0 || undefined || The most significant bit at the beginning of the structure. Undefined
|-
| 1 || undefined || The next most significant bit. Undefined
|-
| ... || ... || ...
|-
| 30 || include_destination || Receiving node expects that the RIPE hash encoded in their address preceedes the encrypted message data of msg messages bound for them.
|-
| 31 || does_ack|| If true, the receiving node does send acknowledgements (rather than dropping them).
|}

=== Node services ===

The following services are currently assigned:

{|class="wikitable"
! Value !! Name !! Description
|-
| 1 || NODE_NETWORK || This is a normal network node.
|-
| 2 || NODE_SSL || This node supports SSL/TLS in the current connect
|}

=== Object Types ===

{|class="wikitable"
! Value !! Name !! Description
|-
| 0 || [[#getpubkey|getpubkey]] ||
|-
| 1 || [[#pubkey|pubkey]] ||
|-
| 2 || [[#msg|msg]] || A msg or msg ack
|-
| 3 || [[#broadcast|broadcast]] ||
|}

=== Error Levels ===

{|class="wikitable"
! Value !! Name !! Description
|-
| 0 || WARNING ||
|-
| 1 || ERROR ||
|-
| 2 || FATAL || A fatal or fatal-like error has occured. The connection usually terminated following this error.
|}

== Message types ==

Undefined messages received on the wire must be ignored.

=== error ===

error is used to convey the reason for a following disconnection.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || error level || [[#Variable_length_integer|var_int]] || The [[#Error_levels|error level]] of this error
|-
| 1+ || ban time || [[#Variable_length_integer|var_int]] || The length of time the emitting node will refuse connections from the receiving node
|-
| 1+ || inv_vector_length || [[#Variable_length_integer|var_int]] || The length of the inventory vector (max: 100) Inventory vectors are fixed length at 32 bytes. Why does the size need to specified? Perhaps this was intended to be a count of inventory vectors?
|-
| inv_vector_length || inv_vector || [[#Inventory Vectors|inv_vect]] || The inventory vector of the offending object this error relates to
|-
| 1+ || errorText || [[#Variable_length_string|var_str]] || The error text (max length: 1000)
|}
The only error PyBitmessage sends is a FATAL error when it receives a version message where the timestamp is out by more than hour from its own

=== version ===

When a node creates an outgoing connection, it will immediately advertise its version. The remote node will respond with its version. No further communication is possible until both peers have exchanged their version. A PyBitmessage server responds with verack then version. A bmd server responds with version then verack

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 || version || int32_t || Identifies protocol version being used by the node. The current protocol version is 3. Old nodes will always assume that future protocol version nodes are compatible; it is up to the new client to judge whether the previous version is incompatible and disconnect if it thinks that it is a good idea.
|-
| 8 || services || uint64_t || bitfield of features to be enabled for this connection
|-
| 8 || timestamp || int64_t || standard UNIX timestamp in seconds
|-
| 26 || addr_recv || [[#Network_address|net_addr]] || The network address of the node receiving this message (not including the time or stream number)
|-
| 26 || addr_from || [[#Network_address|net_addr]] || The network address of the node emitting this message (not including the time or stream number and the ip itself is ignored by the receiver)
|-
| 8 || nonce || uint64_t || Random nonce used to detect connections to self.
|-
| 1+ || user_agent || [[#Variable_length_string|var_str]] || User Agent generally in the form of /Application:Version/ (max length: 5000)
|-
| 1+ || stream_numbers || [[#Variable_length_list_of_integers|var_int_list]] || The stream numbers that the emitting node is interested in. Sending nodes must not include more than 160,000 stream numbers.
|}

A "verack" packet shall be sent if the version packet was accepted. Once you have sent and received a verack messages with the remote node, send an addr message advertising up to 1,000 peers of which you are aware, and one or more inv messages advertising all of the valid objects of which you are aware.

=== verack ===

This message is sent in reply to [[#version|version]] and has no payload. The TCP timeout starts out at 20 seconds; after verack messages are exchanged, the timeout is raised to 10 minutes.

If both sides announce that they support SSL, they MUST perform a SSL handshake immediately after they both send and receive verack. During this SSL handshake, the TCP client acts as a SSL client, and the TCP server acts as a SSL server. PyBitmessage v0.5.4 or later requires the AECDH-AES256-SHA cipher using the secp256k1 curve over TLSv1.

=== addr ===

Provide information on known nodes of the network. Only nodes that have been known to be on the network in the last 3 hours should be advertised. This command is regularly abused and any entries should not be relied upon as being nodes

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1 - 3 || count || [[#Variable_length_integer|var_int]] || Number of address entries (max: 1,000)
|-
| 38 x count || list of net_addr || [[#Network_address|net_addr]][] || Address of other nodes on the network.
|}

=== inv ===

Allows a node to advertise its knowledge of one or more objects.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1 - 3 || count || [[#Variable_length_integer|var_int]] || Number of inventory entries (max: 50,000)
|-
| 32 x count || list of inv_vect || [[#Inventory Vectors|inv_vect]][] || Inventory vectors
|}

=== getdata ===

getdata is used in response to an [[#inv|inv]] message to retrieve the content of a specific object after filtering known elements.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1 - 3 || count || [[#Variable_length_integer|var_int]] || Number of inventory entries (max: 50,000)
|-
| 32 x count || list of inv_vect || [[#Inventory_Vectors|inv_vect]][] || Inventory vectors
|}
Current usage reveals getdata to only ever contain 1 entry

=== ping ===

ping is used to check whether a peer is still responsive and has no payload. A ping should be sent if a peer has not sent anything for more than 5 minutes. A peer receiving a ping may either close the connection or keep it open. To keep a connection open a peer should respond with either a pong (if it has nothing to send) or any contextually valid message. If a peer is silent for a full 10 minutes the connection should be closed.

=== pong ===

pong may be sent in response to a ping. pong may also be sent pre-emptively when a node recognises it hasn't sent anything to a peer for up to 5 minutes and wishes to keep the connection open. pong has no payload.

=== object ===

An object is a message which is shared throughout a stream. It is the only message which propagates; all others are only between two nodes. Objects have a type, like 'msg', or 'broadcast'. To be a valid object, the [[Proof of work|Proof Of Work]] must be done. The maximum allowable length of an object (not to be confused with the objectPayload) is 218 bytes (256 KiB).

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8 || nonce || uint64_t || A nonce that satisfies the [[Proof of work|Proof Of Work]]
|-
| 8 || expiresTime || uint64_t || The "end of life" time of this object. Objects shall be shared with peers until its end-of-life time has been reached. The node should store the inventory vector of that object for some extra period of time to avoid reloading it from another node with a small time delay. The time may be no further than 28 days + 3 hours in the future.
|-
| 4 || objectType || uint32_t || The [[#Object_type_enum|object type]]. Nodes should relay objects even if they use an undefined object type.
|-
| 1 - 9 || version || [[#Variable_length_integer|var_int]] || The object's version.
|-
| 1 - 9 || stream number || [[#Variable_length_integer|var_int]] || The stream number in which this object may propagate
|-
| ? || objectPayload || uchar[] || This field varies depending on the object type; see below.
|-
|}

== Object types ==

Here are the payloads for various object types.

=== getpubkey ===

When a node has the hash of a public key (from an address) but not the public key itself, it must send out a request for the public key.

==== v2 and v3 getpubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 20 || ripe || uchar[20] || The ripemd hash of the public key
|}

==== v4 getpubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || tag || uchar[32] || The tag derived from the address version, stream number, and ripe
|}

=== pubkey ===

==== v2 pubkey ====
This is the first 3 fields of an [[#Identity|identity]] (behavior bitfield, public signing key, and public encryption key). The proof of work difficulty fields (nonce_trials_per_byte and extra_bytes) are omitted, inferring the network default values. For clarity
This is still in use and supported by PyBitmessage but ''new'' v2 addresses are not generated by PyBitmessage.

{|class="wikitable"
! Field Size !! Description !! Data type
|-
| 4 || [[#Pubkey_bitfield_features|behavior bitfield]] || uint32_t
|-
| 64 || public signing key || uchar[64]
|-
| 64 || public encryption key || uchar[64]
|}

==== v3 pubkey ====
This is the full identity structure protected by a signature

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 138 - 150 || identity || [[#Identity|Identity]] || This is the full identity structure
|-
| 7 - 73 || signature || [[#Variable_length_string|var_str]] || The ECDSA signature covering this structure prepended with the object header (excluding the nonce). The signature is actually two signed positive integers ''r'' and ''s'' encoded in ASN.1 according to DER
|}

==== v4 pubkey ====
This is basically an encrypted [[#v3_pubkey|v3 pubkey]] except the version is is 4.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Tagged_Envelope|tagged_envelope]] || Encrypted pubkey data
|-
|}

When version 4 pubkeys are created, most of the data in the pubkey is encrypted. This is done in such a way that only someone who has the Bitmessage address which corresponds to a pubkey can decrypt and use that pubkey. This prevents people from gathering pubkeys sent around the network and using the data from them to create messages to be used in spam or in flooding attacks.

In order to encrypt the pubkey data, a double SHA-512 hash is calculated from the address version number, stream number, and ripe hash of the Bitmessage address that the pubkey corresponds to. The first 32 bytes of this hash are used to create a public and private key pair with which to encrypt and decrypt the pubkey data, using the same algorithm as message encryption (see [[Encryption]]). The remaining 32 bytes of this hash are added to the unencrypted part of the pubkey and used as a tag, as above. This allows nodes to determine which pubkey to decrypt when they wish to send a message.

In PyBitmessage, the double hash of the address data is calculated using the python code below:

doubleHashOfAddressData = hashlib.sha512(hashlib.sha512(encodeVarint(addressVersionNumber) + encodeVarint(streamNumber) + hash).digest()).digest()

=== msg ===

Used for person-to-person messages.

==== v1 msg ====
{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Plain_Envelope|plain_envelope]] || Encrypted msg data
|}

==== Decrypted msg ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 139 - 151 || sender || [[#Identity|Identity]] || Sender's identity prepend with the version. The proof of work difficulty (nonce_trials and extra_padding fields) may be personalised for the recipient
|-
| 20 || destination ripe || uchar[20] || The ripe hash of the public key of the receiver of the message
|-
| 1+ || encoding || [[#Variable_length_integer|var_int]] || [[#Message_Encodings|Message encoding]]
|-
| 1+ || message || [[#Variable_length_string|var_str]] || The message encoded as per encoding
|-
| 1+ || ack_data || [[#Variable_length_string|var_str]] || The acknowledgement data to be transmitted. This is a fully qualified [[#object|object]] with [[Proof of work|Proof Of Work]] completed
|-
| 7 - 73 || signature || [[#Variable_length_string|var_str]] || The ECDSA signature covering this structure prepended with the object header (excluding the nonce). The signature is actually two signed positive integers ''r'' and ''s'' encoded in ASN.1 according to DER
|}

==== msg ack ====

A special form of msg used as an acknowledgement receipt. The objectType and version fields in the object header are set exactly the same as for a [[#v1 msg|v1 msg]]

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || ack data || uchar[32] || A random sequence of bytes that the sender waits for as an indication that the recipient has received their msg
|}

=== broadcast ===

Users who are subscribed to the sending address will see the message appear in their inbox.

Pubkey objects and v5 broadcast objects are encrypted the same way: The data encoded in the sender's Bitmessage address is hashed twice. The first 32 bytes of the resulting hash constitutes the "private" encryption key and the last 32 bytes constitute a '''tag''' so that anyone listening can easily decide if this particular message is interesting. The sender calculates the public key from the private key and then encrypts the object with this public key. Thus anyone who knows the Bitmessage address of the sender of a broadcast or pubkey object can decrypt it.

Having a broadcast version of 5 indicates that a tag is used which, in turn, is used when the sender's address version is >=4.

==== v4 broadcast ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Plain_Envelope|plain_envelope]] ||
|}

==== v5 broadcast ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Tagged_Envelope|tagged_envelope]] ||
|}

==== Decrypted broadcast ====

A decrypted broadcast is nearly identical to a decrypted msg. The decrypted broadcast does not have destination ripe field nor an acknowlegement field.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 139 - 151 || sender || [[#Identity|Identity]] || Sender's identity
|-
| 1+ || encoding || [[#Variable_length_integer|var_int]] || [[#Message_Encodings|Message encoding]]
|-
| 1+ || message || [[#Variable_length_string|var_str]] || The message encoded as per encoding
|-
| 7 - 73 || signature || [[#Variable_length_string|var_str]] || The ECDSA signature covering this structure prepended with the object header (excluding the nonce). The signature is actually two signed positive integers ''r'' and ''s'' encoded in ASN.1 according to DER
|}

Bitmessage Protocol Version 3

2016-06-27T04:41:47Z

Bmng-dev: Removed proposed message encoding. Added ping and pong messages. Other fixes and clarifications

==Common standards==

=== Hashes ===

Most of the time [http://en.wikipedia.org/wiki/SHA-2 SHA-512] hashes are used, however [http://en.wikipedia.org/wiki/RIPEMD RIPEMD-160] is also used when creating an address.

A double-round of SHA-512 is used for the Proof Of Work. Example of double-SHA-512 encoding of string "hello":

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round of sha-512)
0592a10584ffabf96539f3d780d776828c67da1ab5b169e9e8aed838aaecc9ed36d49ff1423c55f019e050c66c6324f53588be88894fef4dcffdb74b98e2b200(second round of sha-512)

For Bitmessage addresses (RIPEMD-160) this would give:

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round is sha-512)
79a324faeebcbf9849f310545ed531556882487e (with ripemd-160)

== Common structures ==

All integers are encoded in big endian

=== Message structure ===

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 || magic || uint32_t || Magic value indicating message origin network, and used to seek to next message when stream state is unknown
|-
| 12 || command || char[12] || ASCII string identifying the packet content, NULL padded (non-NULL padding results in packet rejected)
|-
| 4 || length || uint32_t || Length of payload in number of bytes. The maximum allowed value is 1,600,003 bytes
|-
| 4 || checksum || uint32_t || First 4 bytes of sha512(payload)
|-
| ? || message_payload || uchar[] || The actual data, a [[#Message_types|message]]. Not to be confused with objectPayload.
|}

Known magic values:

{|class="wikitable"
! Magic value !! Sent over wire as
|-
| 0xE9BEB4D9 || E9 BE B4 D9
|}

=== Variable length integer ===

An integer can be encoded depending on the represented value to save space. Variable length integers always precede an array/vector of a type of data that may vary in length. Varints MUST use the minimum possible number of bytes to encode a value. For example; the value 6 can be encoded with one byte therefore a varint that uses three bytes to encode the value 6 is malformed and the decoding task must be aborted.

{|class="wikitable"
! Value !! Storage length !! Format
|-
| < 0xfd || 1 || uint8_t
|-
| <= 0xffff || 3 || 0xfd followed by the integer as uint16_t
|-
| <= 0xffffffff || 5 || 0xfe followed by the integer as uint32_t
|-
| - || 9 || 0xff followed by the integer as uint64_t
|}

=== Variable length string ===

A variable length string can be stored using a variable length integer to encode the length followed by the string itself.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || length || [[#Variable_length_integer|var_int]] || Length of the string
|-
| length || string || char[] || The string itself (can be empty)
|}

=== Variable length list of integers===

n integers can be stored using n+1 [[#Variable_length_integer|variable length integers]] where the first var_int equals n.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || count|| [[#Variable_length_integer|var_int]] || Number of var_ints below
|-
| 1+ || || [[#Variable_length_integer|var_int]] || The first value stored
|-
| 1+ || || [[#Variable_length_integer|var_int]] || The second value stored...
|-
| 1+ || || [[#Variable_length_integer|var_int]] || etc...
|}

=== Network address ===

When a network address is needed somewhere, this structure is used. Network addresses are not prefixed with a timestamp or stream in the version message.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8 || time || uint64 || the Time.
|-
| 4 || stream || uint32 || Stream number for this node
|-
| 8 || services || uint64_t || same service(s) listed in [[#version|version]]
|-
| 16 || IPv6/4 || char[16] || IPv6 address. IPv4 addresses are written into the message as a 16 byte [http://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses IPv4-mapped IPv6 address]
(12 bytes ''00 00 00 00 00 00 00 00 00 00 FF FF'', followed by the 4 bytes of the IPv4 address). Hidden Service addresses can be represented as an IPv6 address with a 48-bit routing prefix of fd87:d87e:eb43::48 under the Unique Local Address block (fc00::/7) with the remaining 10 bytes being the Base256 encoding of the Hidden Service address
|-
| 2 || port || uint16_t || port number
|}

=== Inventory Vectors ===

Inventory vectors are used for notifying other nodes about objects they have or data which is being requested. Two rounds of SHA-512 are used, resulting in a 64 byte hash. Only the first 32 bytes are used; the remaining 32 bytes are ignored.

Inventory vectors consist of the following data format:

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || hash || char[32] || Hash of the object
|}

=== Envelope ===

Bitmessage uses [https://en.wikipedia.org/wiki/Integrated_Encryption_Scheme ECIES] to encrypt its messages. For more information see [[Encryption]]

==== Plain Envelope ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 16 || IV || uchar[16] || Initialization Vector used for AES-256-CBC
|-
| 2 || elliptic curve || uint16_t || Elliptic Curve secp256k1. This is the NID (numerical identifier) 714 (0x02CA) assigned by OpenSSL to represent secp256k1
|-
| 2 || X length || uint16_t || Length of X component of public key R
|-
| X length || X || uchar[X length] || X component of public key R
|-
| 2 || Y length || uint16_t || Length of Y component of public key R
|-
| Y length || Y || uchar[Y length] || Y component of public key R
|-
| ? || encrypted || uchar[] || Cipher text
|-
| 32 || mac || uchar[32] || HMACSHA256 Message Authentication Code
|}

==== Tagged Envelope ====

A tagged envelope is identical to an [[#Plain_Envelope|plain envelope]] but prepended with a tag. Tagged envelopes are only used by [[#v4_pubkey|v4 pubkeys]] and [[#v5 broadcast|v5 broadcasts]].

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || tag || uchar[32] || The recipients tag
|-
| 1+ || envelope || [[#Plain_Envelope|plain_envelope]] ||
|}

== Enumerations and Flags ==

=== Message Encodings ===

{|class="wikitable"
! Value !! Name !! Description
|-
| 0 || IGNORE || Any data with this number may be ignored. The sending node might simply be sharing its public key with you.
|-
| 1 || TRIVIAL|| UTF-8. No 'Subject' or 'Body' sections. Useful for simple strings of data, like URIs or magnet links.
|-
| 2 || SIMPLE || UTF-8. Uses 'Subject' and 'Body' sections. No MIME is used.
<code>messageToTransmit = 'Subject:' + subject + '\n' + 'Body:' + message</code>
|}

Further values for the message encodings can be decided upon by the community. Any MIME or MIME-like encoding format, should they be used, should make use of Bitmessage's 8-bit bytes.

=== Identity bitfield features ===

{|class="wikitable"
! Bit!! Name !! Description
|-
| 0 || undefined || The most significant bit at the beginning of the structure. Undefined
|-
| 1 || undefined || The next most significant bit. Undefined
|-
| ... || ... || ...
|-
| 30 || include_destination || Receiving node expects that the RIPE hash encoded in their address preceedes the encrypted message data of msg messages bound for them.
|-
| 31 || does_ack|| If true, the receiving node does send acknowledgements (rather than dropping them).
|}

=== Node services ===

The following services are currently assigned:

{|class="wikitable"
! Value !! Name !! Description
|-
| 1 || NODE_NETWORK || This is a normal network node.
|-
| 2 || NODE_SSL || This node supports SSL/TLS in the current connect
|}

=== Object Types ===

{|class="wikitable"
! Value !! Name !! Description
|-
| 0 || [[#getpubkey|getpubkey]] ||
|-
| 1 || [[#pubkey|pubkey]] ||
|-
| 2 || [[#msg|msg]] || A msg or msg ack
|-
| 3 || [[#broadcast|broadcast]] ||
|}

=== Error Levels ===

{|class="wikitable"
! Value !! Name !! Description
|-
| 0 || WARNING ||
|-
| 1 || ERROR ||
|-
| 2 || FATAL || A fatal or fatal-like error has occured. The connection usually terminated following this error.
|}

== Message types ==

Undefined messages received on the wire must be ignored.

=== error ===

error is used to convey the reason for a following disconnection.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || error level || [[#Variable_length_integer|var_int]] || The [[#Error_levels|error level]] of this error
|-
| 1+ || ban time || [[#Variable_length_integer|var_int]] || The length of time the emitting node will refuse connections from the receiving node
|-
| 1+ || inv_vector_length || [[#Variable_length_integer|var_int]] || The length of the inventory vector (max: 100) Inventory vectors are fixed length at 32 bytes. Why does the size need to specified? Perhaps this was intended to be a count of inventory vectors?
|-
| inv_vector_length || inv_vecter || [[#Inventory Vectors|inv_vect]] || The inventory vector of the offending object this error relates to
|-
| 1+ || errorText || [[#Variable_length_string|var_str]] || The error text (max length: 1000)
|}
The only error PyBitmessage sends is a FATAL error when it receives a version message where the timestamp is out by more than hour from its own

=== version ===

When a node creates an outgoing connection, it will immediately advertise its version. The remote node will respond with its version. No further communication is possible until both peers have exchanged their version. A PyBitmessage server responds with verack then version. A bmd server responds with version then verack

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 || version || int32_t || Identifies protocol version being used by the node. The current protocol version is 3. Old nodes will always assume that future protocol version nodes are compatible; it is up to the new client to judge whether the previous version is incompatible and disconnect if it thinks that it is a good idea.
|-
| 8 || services || uint64_t || bitfield of features to be enabled for this connection
|-
| 8 || timestamp || int64_t || standard UNIX timestamp in seconds
|-
| 26 || addr_recv || [[#Network_address|net_addr]] || The network address of the node receiving this message (not including the time or stream number)
|-
| 26 || addr_from || [[#Network_address|net_addr]] || The network address of the node emitting this message (not including the time or stream number and the ip itself is ignored by the receiver)
|-
| 8 || nonce || uint64_t || Random nonce used to detect connections to self.
|-
| 1+ || user_agent || [[#Variable_length_string|var_str]] || User Agent generally in the form of /Application:Version/ (max length: 5000)
|-
| 1+ || stream_numbers || [[#Variable_length_list_of_integers|var_int_list]] || The stream numbers that the emitting node is interested in. Sending nodes must not include more than 160,000 stream numbers.
|}

A "verack" packet shall be sent if the version packet was accepted. Once you have sent and received a verack messages with the remote node, send an addr message advertising up to 1,000 peers of which you are aware, and one or more inv messages advertising all of the valid objects of which you are aware.

=== verack ===

This message is sent in reply to [[#version|version]] and has no payload. The TCP timeout starts out at 20 seconds; after verack messages are exchanged, the timeout is raised to 10 minutes.

If both sides announce that they support SSL, they MUST perform a SSL handshake immediately after they both send and receive verack. During this SSL handshake, the TCP client acts as a SSL client, and the TCP server acts as a SSL server. PyBitmessage v0.5.4 or later requires the AECDH-AES256-SHA cipher using the secp256k1 curve over TLSv1.

=== addr ===

Provide information on known nodes of the network. Only nodes that have been known to be on the network in the last 3 hours should be advertised. This command is regularly abused and any entries should not be relied upon as being nodes

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || count || [[#Variable_length_integer|var_int]] || Number of address entries (max: 1,000)
|-
| 38 x count || list of net_addr || [[#Network_address|net_addr]][] || Address of other nodes on the network.
|}

=== inv ===

Allows a node to advertise its knowledge of one or more objects.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || count || [[#Variable_length_integer|var_int]] || Number of inventory entries (max: 50,000)
|-
| 32 x count || list of inv_vect || [[#Inventory Vectors|inv_vect]][] || Inventory vectors
|}

=== getdata ===

getdata is used in response to an [[#inv|inv]] message to retrieve the content of a specific object after filtering known elements.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || count || [[#Variable_length_integer|var_int]] || Number of inventory entries (max: 50,000)
|-
| 32 x count || list of inv_vect || [[#Inventory_Vectors|inv_vect]][] || Inventory vectors
|}
Current usage reveals getdata to only ever contain 1 entry

=== ping ===

ping is used to check whether a peer is still responsive and has no payload. A ping should be sent if a peer has not sent anything for more than 5 minutes. A peer receiving a ping may either close the connection or keep it open. To keep a connection open a peer should respond with either a pong (if it has nothing to send) or any contextually valid message. If a peer is silent for a full 10 minutes the connection should be closed.

=== pong ===

pong may be sent in response to a ping. pong may also be sent pre-emptively when a node recognises it hasn't sent anything to a peer for up to 5 minutes and wishes to keep the connection open. pong has no payload.

=== object ===

An object is a message which is shared throughout a stream. It is the only message which propagates; all others are only between two nodes. Objects have a type, like 'msg', or 'broadcast'. To be a valid object, the [[Proof of work|Proof Of Work]] must be done. The maximum allowable length of an object (not to be confused with the objectPayload) is 218 bytes (256 KiB).

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8 || nonce || uint64_t || A nonce that satisfies the [[Proof of work|Proof Of Work]]
|-
| 8 || expiresTime || uint64_t || The "end of life" time of this object. Objects shall be shared with peers until its end-of-life time has been reached. The node should store the inventory vector of that object for some extra period of time to avoid reloading it from another node with a small time delay. The time may be no further than 28 days + 3 hours in the future.
|-
| 4 || objectType || uint32_t || The [[#Object_type_enum|object type]]. Nodes should relay objects even if they use an undefined object type.
|-
| 1+ || version || [[#Variable_length_integer|var_int]] || The object's version.
|-
| 1+ || stream number || [[#Variable_length_integer|var_int]] || The stream number in which this object may propagate
|-
| ? || objectPayload || uchar[] || This field varies depending on the object type; see below.
|-
|}

== Object types ==

Here are the payloads for various object types.

=== getpubkey ===

When a node has the hash of a public key (from an address) but not the public key itself, it must send out a request for the public key.

==== v2 and v3 getpubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 20 || ripe || uchar[20] || The ripemd hash of the public key
|}

==== v4 getpubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || tag || uchar[32] || The tag derived from the address version, stream number, and ripe
|}

=== pubkey ===

==== v2 pubkey ====
This is still in use and supported by current clients but ''new'' v2 addresses are not generated by clients.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 || [[#Pubkey_bitfield_features|behavior bitfield]] || uint32_t || A bitfield of optional behaviors and features that can be expected from the node receiving the message.
|-
| 64 || public signing key || uchar[64] || The ECC public key used for signing in uncompressed format without the point compression prefix
|-
| 64 || public encryption key || uchar[64] || The ECC public key used for encryption in uncompressed format without the point compression prefix
|}

==== v3 pubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 132 || public keys || [[#v2_pubkey|v2 pubkey]] || This is the same three fields as a [[#v2_pubkey|v2 pubkey]]
|-
| 1+ || nonce_trials_per_byte || [[#Variable_length_integer|var_int]] || Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000.
|-
| 1+ || extra_bytes || [[#Variable_length_integer|var_int]] || Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000.
|-
| 1+ || signature || [[#Variable_length_string|var_str]] || The ECDSA signature covering this structure prepended with the object header (excluding the nonce). The signature is actually two signed integers ''r'' and ''s'' encoded in ASN.1 according to DER
|}

==== v4 pubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Tagged_Envelope|tagged_envelope]] || Encrypted pubkey data
|-
|}

A decrypted v4 pubkey is identical to a [[#v3_pubkey|v3 pubkey]] (except the version is 4).

When version 4 pubkeys are created, most of the data in the pubkey is encrypted. This is done in such a way that only someone who has the Bitmessage address which corresponds to a pubkey can decrypt and use that pubkey. This prevents people from gathering pubkeys sent around the network and using the data from them to create messages to be used in spam or in flooding attacks.

In order to encrypt the pubkey data, a double SHA-512 hash is calculated from the address version number, stream number, and ripe hash of the Bitmessage address that the pubkey corresponds to. The first 32 bytes of this hash are used to create a public and private key pair with which to encrypt and decrypt the pubkey data, using the same algorithm as message encryption (see [[Encryption]]). The remaining 32 bytes of this hash are added to the unencrypted part of the pubkey and used as a tag, as above. This allows nodes to determine which pubkey to decrypt when they wish to send a message.

In PyBitmessage, the double hash of the address data is calculated using the python code below:

doubleHashOfAddressData = hashlib.sha512(hashlib.sha512(encodeVarint(addressVersionNumber) + encodeVarint(streamNumber) + hash).digest()).digest()

=== msg ===

Used for person-to-person messages.

==== v1 msg ====
{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Plain_Envelope|plain_envelope]] || Encrypted msg data
|}

==== Decrypted msg ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || sender || [[#pubkey|pubkey]] || Sender's pubkey. A v4 pubkey should be not be encrypted
|-
| 20 || destination ripe || uchar[20] || The ripe hash of the public key of the receiver of the message
|-
| 1+ || encoding || [[#Variable_length_integer|var_int]] || [[#Message_Encodings|Message encoding]]
|-
| 1+ || message || [[#Variable_length_string|var_str]] || The message encoded as per encoding
|-
| 1+ || ack_data || [[#Variable_length_string|var_str]] || The acknowledgement data to be transmitted. This is a fully qualified [[#object|object]] with [[Proof of Work]] completed
|-
| 1+ || signature || [[#Variable_length_string|var_str]] || The ECDSA signature covering this structure prepended with the object header (excluding the nonce). The signature is actually two signed integers ''r'' and ''s'' encoded in ASN.1 according to DER
|}

==== msg ack ====

A special form of msg used as an acknowledgement receipt. The objectType and version fields in the object header are set exactly the same as for a [[#v1 msg|v1 msg]]

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || ack data || uchar[32] || A random sequence of bytes that the sender waits for as an indication that the recipient has received their msg
|}

=== broadcast ===

Users who are subscribed to the sending address will see the message appear in their inbox.

Pubkey objects and v5 broadcast objects are encrypted the same way: The data encoded in the sender's Bitmessage address is hashed twice. The first 32 bytes of the resulting hash constitutes the "private" encryption key and the last 32 bytes constitute a '''tag''' so that anyone listening can easily decide if this particular message is interesting. The sender calculates the public key from the private key and then encrypts the object with this public key. Thus anyone who knows the Bitmessage address of the sender of a broadcast or pubkey object can decrypt it.

Having a broadcast version of 5 indicates that a tag is used which, in turn, is used when the sender's address version is >=4.

==== v4 broadcast ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Plain_Envelope|plain_envelope]] ||
|}

==== v5 broadcast ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Tagged_Envelope|tagged_envelope]] ||
|}

==== Decrypted broadcast ====

A decrypted broadcast is nearly identical to a decrypted msg. The decrypted broadcast does not have destination ripe field nor an acknowlegement field.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || sender || [[#pubkey|pubkey]] || Sender's pubkey. A v4 pubkey should be not be encrypted
|-
| 1+ || encoding || [[#Variable_length_integer|var_int]] || [[#Message_Encodings|Message encoding]]
|-
| 1+ || message || [[#Variable_length_string|var_str]] || The message encoded as per encoding
|-
| 1+ || signature || [[#Variable_length_string|var_str]] || The ECDSA signature covering this structure prepended with the object header (excluding the nonce). The signature is actually two signed integers ''r'' and ''s'' encoded in ASN.1 according to DER
|}

Bitmessage Protocol Version 3

2016-06-12T03:53:56Z

Bmng-dev: Cleaned and updated Protocol v3 for posterity

==Common standards==

=== Hashes ===

Most of the time [http://en.wikipedia.org/wiki/SHA-2 SHA-512] hashes are used, however [http://en.wikipedia.org/wiki/RIPEMD RIPEMD-160] is also used when creating an address.

A double-round of SHA-512 is used for the Proof Of Work. Example of double-SHA-512 encoding of string "hello":

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round of sha-512)
0592a10584ffabf96539f3d780d776828c67da1ab5b169e9e8aed838aaecc9ed36d49ff1423c55f019e050c66c6324f53588be88894fef4dcffdb74b98e2b200(second round of sha-512)

For Bitmessage addresses (RIPEMD-160) this would give:

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round is sha-512)
79a324faeebcbf9849f310545ed531556882487e (with ripemd-160)

== Common structures ==

All integers are encoded in big endian

=== Message structure ===

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 || magic || uint32_t || Magic value indicating message origin network, and used to seek to next message when stream state is unknown
|-
| 12 || command || char[12] || ASCII string identifying the packet content, NULL padded (non-NULL padding results in packet rejected)
|-
| 4 || length || uint32_t || Length of payload in number of bytes. The maximum allowed value is 1,600,003 bytes
|-
| 4 || checksum || uint32_t || First 4 bytes of sha512(payload)
|-
| ? || message_payload || uchar[] || The actual data, a [[#Message_types|message]]. Not to be confused with objectPayload.
|}

Known magic values:

{|class="wikitable"
! Magic value !! Sent over wire as
|-
| 0xE9BEB4D9 || E9 BE B4 D9
|}

=== Variable length integer ===

An integer can be encoded depending on the represented value to save space. Variable length integers always precede an array/vector of a type of data that may vary in length. Varints MUST use the minimum possible number of bytes to encode a value. For example; the value 6 can be encoded with one byte therefore a varint that uses three bytes to encode the value 6 is malformed and the decoding task must be aborted.

{|class="wikitable"
! Value !! Storage length !! Format
|-
| < 0xfd || 1 || uint8_t
|-
| <= 0xffff || 3 || 0xfd followed by the integer as uint16_t
|-
| <= 0xffffffff || 5 || 0xfe followed by the integer as uint32_t
|-
| - || 9 || 0xff followed by the integer as uint64_t
|}

=== Variable length string ===

A variable length string can be stored using a variable length integer to encode the length followed by the string itself.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || length || [[#Variable_length_integer|var_int]] || Length of the string
|-
| length || string || char[] || The string itself (can be empty)
|}

=== Variable length list of integers===

n integers can be stored using n+1 [[#Variable_length_integer|variable length integers]] where the first var_int equals n.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || count|| [[#Variable_length_integer|var_int]] || Number of var_ints below
|-
| 1+ || || [[#Variable_length_integer|var_int]] || The first value stored
|-
| 1+ || || [[#Variable_length_integer|var_int]] || The second value stored...
|-
| 1+ || || [[#Variable_length_integer|var_int]] || etc...
|}

=== Network address ===

When a network address is needed somewhere, this structure is used. Network addresses are not prefixed with a timestamp or stream in the version message.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8 || time || uint64 || the Time.
|-
| 4 || stream || uint32 || Stream number for this node
|-
| 8 || services || uint64_t || same service(s) listed in [[#version|version]]
|-
| 16 || IPv6/4 || char[16] || IPv6 address. IPv4 addresses are written into the message as a 16 byte [http://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses IPv4-mapped IPv6 address]
(12 bytes ''00 00 00 00 00 00 00 00 00 00 FF FF'', followed by the 4 bytes of the IPv4 address). Hidden Service addresses can be represented as an IPv6 address with a 48-bit routing prefix of fd87:d87e:eb43::48 under the Unique Local Address block (fc00::/7) with the remaining 10 bytes being the Base256 encoding of the Hidden Service address
|-
| 2 || port || uint16_t || port number
|}

=== Inventory Vectors ===

Inventory vectors are used for notifying other nodes about objects they have or data which is being requested. Two rounds of SHA-512 are used, resulting in a 64 byte hash. Only the first 32 bytes are used; the remaining 32 bytes are ignored.

Inventory vectors consist of the following data format:

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || hash || char[32] || Hash of the object
|}

=== Envelope ===

Bitmessage uses [https://en.wikipedia.org/wiki/Integrated_Encryption_Scheme ECIES] to encrypt its messages. For more information see [[Encryption]]

==== Plain Envelope ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 16 || IV || uchar[16] || Initialization Vector used for AES-256-CBC
|-
| 2 || elliptic curve || uint16_t || Elliptic Curve secp256k1. This is the NID (numerical identifier) 714 (0x02CA) assigned by OpenSSL to represent secp256k1
|-
| 2 || X length || uint16_t || Length of X component of public key R
|-
| X length || X || uchar[X length] || X component of public key R
|-
| 2 || Y length || uint16_t || Length of Y component of public key R
|-
| Y length || Y || uchar[Y length] || Y component of public key R
|-
| ? || encrypted || uchar[] || Cipher text
|-
| 32 || mac || uchar[32] || HMACSHA256 Message Authentication Code
|}

==== Tagged Envelope ====

A tagged envelope is identical to an [[#Plain_Envelope|plain envelope]] but prepended with a tag. Tagged envelopes are only used by [[#v4_pubkey|v4 pubkeys]] and [[#v5 broadcast|v5 broadcasts]].

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || tag || uchar[32] || The recipients tag
|-
| 1+ || envelope || [[#Plain_Envelope|plain_envelope]] ||
|}

== Enumerations and Flags ==

=== Message Encodings ===

{|class="wikitable"
! Value !! Name !! Description
|-
| 0 || IGNORE || Any data with this number may be ignored. The sending node might simply be sharing its public key with you.
|-
| 1 || TRIVIAL|| UTF-8. No 'Subject' or 'Body' sections. Useful for simple strings of data, like URIs or magnet links.
|-
| 2 || SIMPLE || UTF-8. Uses 'Subject' and 'Body' sections. No MIME is used.
<code>messageToTransmit = 'Subject:' + subject + '\n' + 'Body:' + message</code>
|-
| 3 || EXTENDED || A data structure in bencode, then compressed with zlib. Null data type is encoded as an empty string, and booleans as an integer 0 (false) or 1 (true). Text fields are encoded using UTF-8. v5 and newer address versions MUST support this. Proposal, exact structure pending standardisation.
|}

Further values for the message encodings can be decided upon by the community. Any MIME or MIME-like encoding format, should they be used, should make use of Bitmessage's 8-bit bytes.

=== Identity bitfield features ===

{|class="wikitable"
! Bit!! Name !! Description
|-
| 0 || undefined || The most significant bit at the beginning of the structure. Undefined
|-
| 1 || undefined || The next most significant bit. Undefined
|-
| ... || ... || ...
|-
| 30 || include_destination || Receiving node expects that the RIPE hash encoded in their address preceedes the encrypted message data of msg messages bound for them.
|-
| 31 || does_ack|| If true, the receiving node does send acknowledgements (rather than dropping them).
|}

=== Node services ===

The following services are currently assigned:

{|class="wikitable"
! Value !! Name !! Description
|-
| 1 || NODE_NETWORK || This is a normal network node.
|-
| 2 || NODE_SSL || This node supports SSL/TLS in the current connect
|}

=== Object Types ===

{|class="wikitable"
! Value !! Name !! Description
|-
| 0 || [[#getpubkey|getpubkey]] ||
|-
| 1 || [[#pubkey|pubkey]] ||
|-
| 2 || [[#msg|msg]] || A msg or msg ack
|-
| 3 || [[#broadcast|broadcast]] ||
|}

=== Error Levels ===

{|class="wikitable"
! Value !! Name !! Description
|-
| 0 || WARNING ||
|-
| 1 || ERROR ||
|-
| 2 || FATAL || A fatal or fatal-like error has occured. The connection usually terminated following this error.
|}

== Message types ==

Undefined messages received on the wire must be ignored.

=== error ===
The only error PyBitmessage sends is a FATAL error when it receives a version message where the timestamp is out by more than hour from its own
{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || error level || [[#Variable_length_integer|var_int]] || The [[#Error_levels|error level]] of this error
|-
| 1+ || ban time || [[#Variable_length_integer|var_int]] || The length of time the emitting node will refuse connections from the receiving node
|-
| 1+ || inv_vector_length || [[#Variable_length_integer|var_int]] || The length of the inventory vector (max: 100) Inventory vectors are fixed length at 32 bytes. Why does the size need to specified? Perhaps this should be a boolean value to indicate the presence of an inventory vector instead?
|-
| inv_vector_length || inv_vecter || [[#Inventory Vectors|inv_vect]] || The inventory vector of the offending object this error relates to
|-
| 1+ || errorText || [[#Variable_length_string|var_str]] || The error text (max length: 1000)
|}

=== version ===

When a node creates an outgoing connection, it will immediately advertise its version. The remote node will respond with its version. No further communication is possible until both peers have exchanged their version. A PyBitmessage server responds with verack then version. A bmd server responds with version then verack

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 || version || int32_t || Identifies protocol version being used by the node. The current protocol version is 3. Nodes should disconnect if the remote node's version is lower but continue with the connection if it is higher. What is the intent here?
|-
| 8 || services || uint64_t || bitfield of features to be enabled for this connection
|-
| 8 || timestamp || int64_t || standard UNIX timestamp in seconds
|-
| 26 || addr_recv || [[#Network_address|net_addr]] || The network address of the node receiving this message (not including the time or stream number)
|-
| 26 || addr_from || [[#Network_address|net_addr]] || The network address of the node emitting this message (not including the time or stream number and the ip itself is ignored by the receiver)
|-
| 8 || nonce || uint64_t || Random nonce used to detect connections to self.
|-
| 1+ || user_agent || [[#Variable_length_string|var_str]] || User Agent generally in the form of /Application:Version/ (max length: 5000)
|-
| 1+ || stream_numbers || [[#Variable_length_list_of_integers|var_int_list]] || The stream numbers that the emitting node is interested in. Sending nodes must not include more than 160,000 stream numbers.
|}

A "verack" packet shall be sent if the version packet was accepted. Once you have sent and received a verack messages with the remote node, send an addr message advertising up to 1,000 peers of which you are aware, and one or more inv messages advertising all of the valid objects of which you are aware.

=== verack ===

This message is sent in reply to [[#version|version]] and has no payload. The TCP timeout starts out at 20 seconds; after verack messages are exchanged, the timeout is raised to 10 minutes.

If both sides announce that they support SSL, they MUST perform a SSL handshake immediately after they both send and receive verack. During this SSL handshake, the TCP client acts as a SSL client, and the TCP server acts as a SSL server. PyBitmessage v0.5.4 or later requires the AECDH-AES256-SHA cipher over TLSv1, and prefers the secp256k1 curve (but other curves may be accepted, depending on the version of python and OpenSSL used).

=== addr ===

Provide information on known nodes of the network. Only nodes that have been known to be on the network in the last 3 hours should be advertised. This command is easily abused and any entries should be treated as unreliable

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || count || [[#Variable_length_integer|var_int]] || Number of address entries (max: 1,000)
|-
| 38 x count || list of net_addr || [[#Network_address|net_addr]][] || Address of other nodes on the network.
|}

=== inv ===

Allows a node to advertise its knowledge of one or more objects.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || count || [[#Variable_length_integer|var_int]] || Number of inventory entries (max: 50,000)
|-
| 32 x count || list of inv_vect || [[Protocol specification#Inventory Vectors|inv_vect]][] || Inventory vectors
|}

=== getdata ===

getdata is used in response to an [[#inv|inv]] message to retrieve the content of a specific object after filtering known elements.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || count || [[#Variable_length_integer|var_int]] || Number of inventory entries (max: 50,000)
|-
| 32 x count || list of inv_vect || [[#Inventory_Vectors|inv_vect]][] || Inventory vectors
|}
Current usage reveals getdata to only ever contain 1 entry

=== object ===

An object is a message which is shared throughout a stream. It is the only message which propagates; all others are only between two nodes. Objects have a type, like 'msg', or 'broadcast'. To be a valid object, the [[Proof of work|Proof Of Work]] must be done. The maximum allowable length of an object (not to be confused with the objectPayload) is 218 bytes (256 KiB).

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8 || nonce || uint64_t || A nonce that satisfies the [[Proof of work|Proof Of Work]]
|-
| 8 || expiresTime || uint64_t || The "end of life" time of this object. Objects shall be shared with peers until its end-of-life time has been reached. The node should store the inventory vector of that object for some extra period of time to avoid reloading it from another node with a small time delay. The time may be no further than 28 days + 3 hours in the future.
|-
| 4 || objectType || uint32_t || The [[#Object_type_enum|object type]]. Nodes should relay objects even if they use an undefined object type.
|-
| 1+ || version || [[#Variable_length_integer|var_int]] || The object's version.
|-
| 1+ || stream number || [[#Variable_length_integer|var_int]] || The stream number in which this object may propagate
|-
| ? || objectPayload || uchar[] || This field varies depending on the object type; see below.
|-
|}

== Object types ==

Here are the payloads for various object types.

=== getpubkey ===

When a node has the hash of a public key (from an address) but not the public key itself, it must send out a request for the public key.

==== v2 and v3 getpubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 20 || ripe || uchar[20] || The ripemd hash of the public key
|}

==== v4 getpubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || tag || uchar[32] || The tag derived from the address version, stream number, and ripe
|}

=== pubkey ===

==== v2 pubkey ====
This is still in use and supported by current clients but ''new'' v2 addresses are not generated by clients.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 || [[#Pubkey_bitfield_features|behavior bitfield]] || uint32_t || A bitfield of optional behaviors and features that can be expected from the node receiving the message.
|-
| 64 || public signing key || uchar[64] || The ECC public key used for signing in uncompressed format without the point compression prefix
|-
| 64 || public encryption key || uchar[64] || The ECC public key used for encryption in uncompressed format without the point compression prefix
|}

==== v3 pubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 132 || public keys || [[#v2_pubkey|v2 pubkey]] || This is the same three fields as a [[#v2_pubkey|v2 pubkey]]
|-
| 1+ || nonce_trials_per_byte || [[#Variable_length_integer|var_int]] || Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000.
|-
| 1+ || extra_bytes || [[#Variable_length_integer|var_int]] || Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000.
|-
| 1+ || signature || [[#Variable_length_string|var_str]] || The ECDSA signature covering this structure prepended with the object header (excluding the nonce). The signature is actually two signed integers ''r'' and ''s'' encoded in ASN.1 according to DER
|}

==== v4 pubkey ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Tagged_Envelope|tagged_envelope]] || Encrypted pubkey data
|-
|}

A decrypted v4 pubkey is identical to a [[#v3_pubkey|v3 pubkey]] (except the version is 4).

When version 4 pubkeys are created, most of the data in the pubkey is encrypted. This is done in such a way that only someone who has the Bitmessage address which corresponds to a pubkey can decrypt and use that pubkey. This prevents people from gathering pubkeys sent around the network and using the data from them to create messages to be used in spam or in flooding attacks.

In order to encrypt the pubkey data, a double SHA-512 hash is calculated from the address version number, stream number, and ripe hash of the Bitmessage address that the pubkey corresponds to. The first 32 bytes of this hash are used to create a public and private key pair with which to encrypt and decrypt the pubkey data, using the same algorithm as message encryption (see [[Encryption]]). The remaining 32 bytes of this hash are added to the unencrypted part of the pubkey and used as a tag, as above. This allows nodes to determine which pubkey to decrypt when they wish to send a message.

In PyBitmessage, the double hash of the address data is calculated using the python code below:

doubleHashOfAddressData = hashlib.sha512(hashlib.sha512(encodeVarint(addressVersionNumber) + encodeVarint(streamNumber) + hash).digest()).digest()

=== msg ===

Used for person-to-person messages.

==== v1 msg ====
{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Plain_Envelope|plain_envelope]] || Encrypted msg data
|}

==== Decrypted msg ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || sender || [[#pubkey|pubkey]] || Sender's pubkey. A v4 pubkey should be not be encrypted
|-
| 20 || destination ripe || uchar[20] || The ripe hash of the public key of the receiver of the message
|-
| 1+ || encoding || [[#Variable_length_integer|var_int]] || [[#Message_Encodings|Message encoding]]
|-
| 1+ || message || [[#Variable_length_string|var_str]] || The message encoded as per encoding
|-
| 1+ || ack_data || [[#Variable_length_string|var_str]] || The acknowledgement data to be transmitted. This is a fully qualified [[#object|object]] with [[Proof of Work]] completed
|-
| 1+ || signature || [[#Variable_length_string|var_str]] || The ECDSA signature covering this structure prepended with the object header (excluding the nonce). The signature is actually two signed integers ''r'' and ''s'' encoded in ASN.1 according to DER
|}

==== msg ack ====

A special form of msg used as an acknowledgement receipt. The objectType and version fields in the object header are set exactly the same as for a [[#v1 msg|v1 msg]]

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 32 || ack data || uchar[32] || A random sequence of bytes that the sender waits for as an indication that the recipient has received their msg
|}

=== broadcast ===

Users who are subscribed to the sending address will see the message appear in their inbox.

Pubkey objects and v5 broadcast objects are encrypted the same way: The data encoded in the sender's Bitmessage address is hashed twice. The first 32 bytes of the resulting hash constitutes the "private" encryption key and the last 32 bytes constitute a '''tag''' so that anyone listening can easily decide if this particular message is interesting. The sender calculates the public key from the private key and then encrypts the object with this public key. Thus anyone who knows the Bitmessage address of the sender of a broadcast or pubkey object can decrypt it.

Having a broadcast version of 5 indicates that a tag is used which, in turn, is used when the sender's address version is >=4.

==== v4 broadcast ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Plain_Envelope|plain_envelope]] ||
|}

==== v5 broadcast ====

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| ? || envelope || [[#Tagged_Envelope|tagged_envelope]] ||
|}

==== Decrypted broadcast ====

A decrypted broadcast is nearly identical to a decrypted msg. The decrypted broadcast does not have destination ripe field nor an acknowlegement field.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 1+ || sender || [[#pubkey|pubkey]] || Sender's pubkey. A v4 pubkey should be not be encrypted
|-
| 1+ || encoding || [[#Variable_length_integer|var_int]] || [[#Message_Encodings|Message encoding]]
|-
| 1+ || message || [[#Variable_length_string|var_str]] || The message encoded as per encoding
|-
| 1+ || signature || [[#Variable_length_string|var_str]] || The ECDSA signature covering this structure prepended with the object header (excluding the nonce). The signature is actually two signed integers ''r'' and ''s'' encoded in ASN.1 according to DER
|}

Category:Proposals

2014-08-05T07:18:50Z

Bmng-dev: Recreate Proposals category

Proposals are technical specifications for enhancing the protocol that are under consideration.

Protocol specification v3

2014-07-22T04:39:29Z

Bmng-dev: Fix links

{{Template:Stub}}

{{Template:Proposal}}

== Introduction ==
This is a DRAFT for the protocol version 3. It describes the changes in protocol version 3 versus version 2. Things which are unchanged from version 2 are not described in detail. So you should use the [[Protocol specification|version 2 protocol specification]] as a reference for all formats which are not mentioned in this description.

== New features in version 3 ==

Here are the new features of the version 3 of Bitmessage protocol in keywords:

* object type is now coded inside the message
* objects may have a variable time to live
* The POW is more difficult but can be easier if you lower the time to live
* The protocol is tolerant for further message extension
* The protocol is tolerant for further object extension
* A lower maximum object size

== Common structures ==

=== Variable length integer ===

The shortest possible encoding MUST be used. In version 2 the decimal value 10 could be encoded as 0x0A, 0xFD000A, 0xFE0000000A, or 0xFF000000000000000A. In version 3 only the shortest representation, 0x0A, is allowed. The 9-byte form is no longer useful and SHOULD NOT be used.

== Message types ==

Most message types are unchanged from version 2 to version 3. Only the four "objecttype" messages are not valid any more. They are summarized into one single message.

=== version ===

The version message is identical to [[Protocol specification#version|version 2 version message]].

=== verack ===

The verack message is identical to [[Protocol specification#verack|version 2 verack message]].

=== addr ===

The addr message is identical to [[Protocol specification#addr|version 2 addr message]].

=== inv ===

The inv message is identical to [[Protocol specification#inv|version 2 inv message]].

=== getdata ===

The getdata message is identical to [[Protocol specification#getdata|version 2 getdata message]].

=== Unsupported messages ===
If a node receives an unknown message it must silently ignore it. This is for further extensions of the protocol with other messages. Nodes that don't understand such a new message type shall be able to work correct with the message types they understand.

Maybe some version 2 nodes did already implement it that way, but in version 3 it is '''part of the protocol specification''', that a node must silently ignore unsupported messages.

== Object type ==

The four former object types "getpubkey", "pubkey", "msg" and "broadcast" are summarized into a single Message type "object". The four Messages as they did exist in version 2 protocol are not valid any more.

Objects are shared throughout a stream. A client should advertise objects until their end of life time is reached. To be a valid object, the [[Proof of work|Proof Of Work]] has to be done.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8||POW nonce||uint64_t||
Random nonce used for the [[Proof of work|Proof Of Work]]
|-
| 8||time||uint64_t||
The "end of life" time of this object (be aware, in version 2 of the protocol this was the generation time). Objects shall be broadcast until its end of life time has been reached. The node shall store the inventory vector of that object for at least another hour to avoid reloading it from another node with a small time delay.
The maximum value for the "time to life" of an object is 28 days. so the "end of life time" is 28 days in the future at maximum.
|-
| 4||object type||uint32_t||
This field specifies the content of the object.
Valid values are (at the moment) 0-"getpubkey", 1-"pubkey", 2-"msg", 3-"broadcast". all other values are reserved. Nodes shall transport objects with unknown types, too. This will make further protocol changes more easy.
|-
|-
| ?||payload||uchar[]||
This field varies depending on the object type. For a detailed description of their content refer to [[Protocol specification#Object_types|version 2 object types]]
|-
|}

== Proof of Work ==

Generally the POW is done exactly like in [[Proof of work|version 2]]

The "target" (the difficulty of the POW) is defined a little bit lower (more difficult) in version 3. This is, because practice did show, it is to easy to flood the network with data.
In addition to that it is possible in version 3 to lower the time to live of a message (for example when doing a live chat) and getting an easier POW for that.

Setting the live time e.g. to 360 seconds (6 Minutes), which is totally sufficient for a live chat, the POW will be 10 times easier than in version 2.

The "time to live" is the difference between the current time and the "end of life" time. For POW check and generation the "time to live" value is considered to be at least 300 seconds, even if the object has to live less than 300 seconds.

payload = EndOfLifeTime + ObjectType + payload

PayloadLenInBytes * TimeToLiveInSeconds
loadvalue = --------------------------------------- + payloadLengthExtraBytes + 8
3600

2^64
target = ------------------------------------------------
loadvalue * averageProofOfWorkNonceTrialsPerByte

== Maximum object size ==

In version 2 the maximum object size was defined to be 170 MBytes. This object size is totally unrealistic for a normal use (POW), but is perfectly for a network attack. It will be to big to handle for clients with low bandwidth.
Currently an average bitmessage "msg" object has the size of 2 kBytes.
So version 3 limits the objects to a maximum size of 64 kBytes.

Protocol specification v3

2014-07-18T10:56:19Z

Bmng-dev: Formatting, spelling corrections

{{Template:Stub}}

{{Template:Proposal}}

== Introduction ==
This is a DRAFT for the protocol version 3. It describes the changes in protocol version 3 versus version 2. Things which are unchanged from version 2 are not described in detail. So you should use [https://bitmessage.org/wiki/Protocol_specification this] as a reference for all formats which are not mentioned in this description.

== New features in version 3 ==

Here are the new features of the version 3 of Bitmessage protocol in keywords:

* object type is now coded inside the message
* objects may have a variable time to live
* The POW is more difficult but can be easier if you lower the time to live
* The protocol is tolerant for further message extension
* The protocol is tolerant for further object extension
* A lower maximum object size

== Common structures ==

=== Variable length integer ===

The shortest possible encoding MUST be used. In version 2 the decimal value 10 could be encoded as 0x0A, 0xFD000A, 0xFE0000000A, or 0xFF000000000000000A. In version 3 only the shortest representation, 0x0A, is allowed. The 9-byte form is no longer useful and SHOULD NOT be used.

== Message types ==

Most message types are unchanged from version 2 to version 3. Only the four "objecttype" messages are not valid any more. They are summarized into one single message.

=== version ===

The version message is identical to protocol version 2. you can look up details [https://bitmessage.org/wiki/Protocol_specification#version here]

=== verack ===

The verack message is identical to protocol version 2. you can look up details [https://bitmessage.org/wiki/Protocol_specification#verack here]

=== addr ===

The addr message is identical to protocol version 2. you can look up details [https://bitmessage.org/wiki/Protocol_specification#addr here]

=== inv ===

The inv message is identical to protocol version 2. you can look up details [https://bitmessage.org/wiki/Protocol_specification#inv here]

=== getdata ===

The getdata message is identical to protocol version 2. you can look up details [https://bitmessage.org/wiki/Protocol_specification#getdata here]

=== Unsupported messages ===
If a node receives an unknown message it must silently ignore it. This is for further extensions of the protocol with other messages. Nodes that don't understand such a new message type shall be able to work correct with the message types they understand.

Maybe some version 2 nodes did already implement it that way, but in version 3 it is '''part of the protocol specification''', that a node must silently ignore unsupported messages.

== Object type ==

The four former object types "getpubkey", "pubkey", "msg" and "broadcast" are summarized into a single Message type "object". The four Messages as they did exist in version 2 protocol are not valid any more.

Objects are shared throughout a stream. A client should advertise objects until their end of life time is reached. To be a valid object, the [[Proof of work|Proof Of Work]] has to be done.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8||POW nonce||uint64_t||
Random nonce used for the [[Proof of work|Proof Of Work]]
|-
| 8||time||uint64_t||
The "end of life" time of this object (be aware, in version 2 of the protocol this was the generation time). Objects shall be broadcast until its end of life time has been reached. The node shall store the inventory vector of that object for at least another hour to avoid reloading it from another node with a small time delay.
The maximum value for the "time to life" of an object is 28 days. so the "end of life time" is 28 days in the future at maximum.
|-
| 4||object type||uint32_t||
This field specifies the content of the object.
Valid values are (at the moment) 0-"getpubkey", 1-"pubkey", 2-"msg", 3-"broadcast". all other values are reserved. Nodes shall transport objects with unknown types, too. This will make further protocol changes more easy.
|-
|-
| ?||payload||uchar[]||
This field varies depending on the object type. For a detailed description of their content look [https://bitmessage.org/wiki/Protocol_specification#Object_types here]
|-
|}

== Proof of Work ==

Generally the POW is done exactly like in [[Proof of work|version 2]]

The "target" (the difficulty of the POW) is defined a little bit lower (more difficult) in version 3. This is, because practice did show, it is to easy to flood the network with data.
In addition to that it is possible in version 3 to lower the time to live of a message (for example when doing a live chat) and getting an easier POW for that.

Setting the live time e.g. to 360 seconds (6 Minutes), which is totally sufficient for a live chat, the POW will be 10 times easier than in version 2.

The "time to live" is the difference between the current time and the "end of life" time. For POW check and generation the "time to live" value is considered to be at least 300 seconds, even if the object has to live less than 300 seconds.

payload = EndOfLifeTime + ObjectType + payload

PayloadLenInBytes * TimeToLiveInSeconds
loadvalue = --------------------------------------- + payloadLengthExtraBytes + 8
3600

2^64
target = ------------------------------------------------
loadvalue * averageProofOfWorkNonceTrialsPerByte

== Maximum object size ==

In version 2 the maximum object size was defined to be 170 MBytes. This object size is totally unrealistic for a normal use (POW), but is perfectly for a network attack. It will be to big to handle for clients with low bandwidth.
Currently an average bitmessage "msg" object has the size of 2 kBytes.
So version 3 limits the objects to a maximum size of 64 kBytes.

Mobile Protocol specification

2014-07-18T10:49:42Z

Bmng-dev: Formatting, spelling corrections

{{Template:Stub}}

{{Template:Proposal}}

This is a DRAFT for protocol extensions to enable mobile devices to participate in bitmessage communication.

== Intention ==

If you want to let mobile devices participate in bitmessage communication, you need a gateway device. Theoretically the device could participate like a standard node, but this will cause too much traffic. Even if you have a flat rate, it'll cost to much battery.
This page describes a way, the standard protocol could be enhanced, to let mobile devices participate, at least through a gateway.

== The Gateway ==

The gateway first of all is a standard bitmessage node. As a bitmessage node it connects to other nodes and collects and exchanges objects. The mobile gateway functionality is an extension, an additional protocol variant.

When a mobile device connects to a gateway it will identify itself in the "services" field of the version message to be a mobile device. The gateway will answer in the same field, that it is a gateway.

Therefore the service table has to be enhanced:

{|class="wikitable"
! Value !! Name !! Description
|-
| 1 || NODE_NETWORK || This is a normal network node.
|-
| 2 || NODE_MOBILE || This is a mobile device with limited bandwidth.
|-
| 4 || NODE_GATEWAY || This node can send reduced traffic and work as a gateway.
|}

== Differences in protocol behavior ==

=== Network address ===

Because we have to share private keys between mobile node and gateway, we do ssh encryption on the connection. For that reason the "Network Address" gets an additional (optional) field for the public ssh key.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 4 (or 8)|| time || uint32 || the Time. Protocol version 1 clients use 4 byte time while protocol version 2 clients use 8 byte time.
|-
| 4 || stream|| uint32 || Stream number for this node
|-
| 8 || services || uint64_t || same service(s) listed in [[#version|version]]
|-
| 16 || IPv6/4 || char[16] || IPv6 address. The original client only supports IPv4 and only reads the last 4 bytes to get the IPv4 address. However, the IPv4 address is written into the message as a 16 byte [http://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses IPv4-mapped IPv6 address]
(12 bytes ''00 00 00 00 00 00 00 00 00 00 FF FF'', followed by the 4 bytes of the IPv4 address).
|-
| 2 || port || uint16_t || port number
|-
| 1+ || ssh fingerprint || [[#Variable length string|var_str]] || This field is optional. It contains the ssh fingerprint, for nodes that support ssh
|}

=== Sending messages ===

When accessed by a NODE_MOBILE the NODE_GATEWAY will not send any "addr" message, because the NODE_MOBILE will connect to the bitmessage network only through well known gateways. It will neither send a "inv" message, because it wants to reduce the traffic to the NODE_MOBILE.

When accessed to a NODE_GATEWAY the NODE_MOBILE will not send a "addr" message, too. But it might send an "inv" message, when it has objects to share. and it can share already all types of objects with the network.

With this very small protocol changes the NODE_MOBILE is already able to send messages using the bitmessage protocol, but with very reduced traffic load.

=== Receiving messages ===

For the receive direction we must bring the NODE_GATEWAY in the position to select the objects the NODE_MOBILE is interested in.

For that we introduce a new Message type:

=== setreceivekey ===

Receiving this message, the NODE_GATEWAY starts to send "inv" messages and offers the NODE_MOBILE objects. In opposite to a standard node connection, it offers only objects, which are already found to be for this node.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 64||private encryption key||uchar[]||The ECC private key used for encryption (uncompressed format; normally prepended with \x04 )
|}

there might be multiple of this messages.

NODE_GATEWAY stores this information only as long as the connection exists. It can now filter the objects that are intended for NODE_MOBILE.

Whenever the NODE_GATEWAY notifies, that NODE_MOBILE might be interested in objects, it sends a "inv" message to offer them (must be only the objects, NODE_MOBILE is interested in!).

How does NODE_GATEWAY know, which objects are of interest: It uses the private key received in "setreceivekey" to filter private messages. Every "getpubkey" object is translated into a request. When the object arrives at NODE_GATEWAY or if its already there, it is offered in an "inv" message.

Message Tagging Proposal

2014-07-18T10:01:44Z

Bmng-dev: Formatting, spelling corrections

{{Template:Proposal}}

== Introduction ==
This is a draft proposal to allow Bitmessage messages to be 'tagged' so they can be received by 'lite' clients, such as mobile phones. This page describes the changes proposed and the reasoning behind them.

=== Summary of the proposal ===

* Bitmessage messages can be 'tagged' with a hash derived from the message's destination address and a time value.
* This tag allows 'lite' clients, such as mobile phones, to receive messages sent to them without downloading and processing large amounts of data.
* The time component of the tag is 00:00 on the current day in Unix time.
* Message tagging is only necessary when sending messages to 'lite' clients. Otherwise the tag field is filled with random data.
* A lite client can signal that messages to its addresses should be tagged by setting a flag in the behavior bitfield of each of its address's pubkeys.

== Reasoning behind the proposal ==

Currently, Bitmessage clients such as PyBitmessage act as full nodes in the Bitmessage peer-to-peer network. This involves transmitting, receiving, and processing a large amount of data. This workload may be manageable for more powerful devices, such as laptops, desktops, and servers, but it is typically too great for less powerful devices such as mobile phones and tablets to reasonably handle. This is particularly true in cases where mobile internet connections mean that bandwidth is very expensive. Because of this, a high proportion of the computing devices in use today are not suitable to act as full nodes in the Bitmessage peer-to-peer network. Therefore, in order for the users of these devices to be able to use Bitmessage, there must be some way of doing so that does not involve their device acting as a full node in the network.

One approach to solving this problem is the creation of 'lite' clients. These are Bitmessage clients which do not do all the processing and transmission work that full nodes do. Instead, the burden of this work is shifted to servers, which supply lite clients with the data they need and relay data sent by lite clients to the rest of the Bitmessage network. Thankfully the design of Bitmessage allows this to be implemented in such a way that very little security is lost. All message encryption and decryption can be done locally by the lite client, and it can retain sole control over its addresses and cryptographic keys. Data provided by servers can be cryptographically authenticated as being correct, and data sent to servers can be disseminated to several at once to ensure that it reaches the rest of the network.

However, this arrangement does require one significant compromise. In order for 'lite' clients such as mobile phones to receive messages, there needs to be some way for them to identify messages that have been sent to their addresses. Full nodes in Bitmessage do not require this because they receive and automatically attempt to decrypt all messages in the streams (segments of the network) which they are a part of. As described above however, the work required for this approach is too great for many devices, and therefore there must be some way for lite clients to identify which messages are destined for them, so that they can request those messages from a server. This identification of messages sent to lite clients can be accomplished by introducing a 'tag' to Bitmessage messages which is derived from the destination address.

Tagging messages in this way is clearly a trade-off in terms of security, as it reveals some extra information, albeit obfuscated information, about the message's destination. However, this trade-off delivers a substantial benefit in allowing mobile phones and other lite clients a viable part of Bitmessage. Adding a tag to messages when the destination is a lite client can be a user-configurable option (there is already an option for this in the PyBitmessage settings page), so it will not be forced on anyone.

Under this proposal, message tags are calculated by hashing two sets of data. The first set is data taken from the message's destination address. The second is a time value based on the day the message is sent.

Adding a time component to the data used to calculate the tag reduces the information leakage created by tagging messages. It is proposed that the time value should be based on the day on which the message is sent. Thus, if a lite client is offline for 5 days, it can calculate 5 different tag values for each of its addresses and request any messages marked with those tags from servers. The balance here is between the level of obfuscation introduced into the tag and the burden created for a lite client requesting messages with those tags. Unless an adversary discovers the address that the messages are being sent to, there should be no way to link messages sent on different days to each other.

Where a message is not being sent to a lite client's address and therefore doesn't need to be tagged, the 'tag' field of the message can be filled with random data. If the hashing function used to calculate the tag is secure, then message tags and random data of the same length should be indistinguishable, except in cases where multiple messages are sent to a particular address on the same day. Even then, this tagging scheme reveals only a little information to attackers, while massively reducing the burden of work required for devices such as mobile phones to use Bitmessage.

== Proposed changes in detail ==

=== Recognizing lite client addresses ===
A Bitmessage client can determine whether an address is a 'lite client' address that requires messages sent to it to be tagged by examining the behavior bitfield in that address's pubkey. See https://bitmessage.org/wiki/Protocol_specification#Pubkey_bitfield_features. Bit 30, which is currently designated as 'include_destination' would be changed to 'include_tag'. The 'include_destination' flag relates to an earlier way of tagging messages (simply using the destination's address's ripe hash as the tag). This tagging scheme was never implemented, so it can be safely replaced.

=== Address data ===
The message tagging proposed is very similar to the tagging of version 4 pubkeys that is already done in Bitmessage (see https://bitmessage.org/wiki/Protocol_specification#pubkey). The 'address data' component of the data used to calculate the message tag is the same as that in an encrypted pubkey. The address version number, stream number, and ripe hash of the address are concatenated to give the 'address data'.

In order to create the input data for the hash, the address version number and stream number are encoded using variable length integer encoding (see https://bitmessage.org/wiki/Protocol_specification#Variable_length_integer) and the time value is converted to its byte value in big-endian order.

=== Time value ===
The 'time' component of the tag is proposed to be based on Unix time (see https://en.wikipedia.org/wiki/Unix_time), which is already used throughout Bitmessage. It is proposed that the time value should be the Unix timestamp for the beginning of the current day in Unix time. A pseudocode expression of the proposed formula to calculate the time value follows:
unixTime = getUnixTime()
remainderSeconds = unixTime mod 86,400
timeValue = unixTime - remainderSeconds

There are 86,400 seconds in a day, so calculating unixTime mod 86,400 gives the number of seconds since the beginning of the current day in Unix time. Subtracting that number of seconds from unixTime gives the Unix timestamp of the beginning of the current day.

For example, if the current Unix time is 1,404,786,854 then the time value would be calculated as follows:
1,404,786,854 mod 86,400 = 9,254
1,404,786,854 - 9,254 = 1,404,777,600

The time value used in the hash calculation to create the message tag would therefore be 1,404,777,600. The following day it would be 1,404,864,000,000.

==== Pseudocode ====
A pseudocode expression of the proposed procedure to calculate a message tag follows:
fullTag = sha512(sha512(addressVersionNumber + streamNumber + ripeHash + unixTimestamp))
messageTag = fullTag[0-31]

==== Example ====
For example, if the current Unix time were the same as the above (1,404,786,854), then the message tag for address BM-87ozvCK4Jkx9Pc4dP7cd6y3T33DcSdmWPaq would be as follows. Note that the values below are hexadecimal representations of the actual values, not the values themselves:
addressVersion = 04
streamNumber = 01
ripeHash = ec87a1475401c88030f0a1efd0cf85ecdfd7bbca
timeValue = 0000000053bb3480
dataToHash = 0401ec87a1475401c88030f0a1efd0cf85ecdfd7bbca0000000053bb3480
fullTag = 8d2a1e2284791ec0fbbec41b58b1ecba341a9ccde1306d861a1702253f2a21e1f1382c3b4f4377262dabb0768bab87d56e54bb26d5c84f44b36a92c695bbb4a7
messageTag = 8d2a1e2284791ec0fbbec41b58b1ecba341a9ccde1306d861a1702253f2a21e1

=== Msg object ===
Under this proposal, a msg object would be composed as follows. This can be compared to the current specification found at https://bitmessage.org/wiki/Protocol_specification#msg.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8||POW nonce||uint64_t||Random nonce used for the [[Proof of work|Proof Of Work]]
|-
| 4 (or 8)||time||uint32_t||The time that this message was generated and broadcast. We are transitioning to 8 byte time.
|-
| 1+ ||streamNumber||var_int||The stream number of the destination address.
|-
| 32||tag||uchar[]||The message tag, made up of bytes 0-31 of sha512(sha512(addressVersionNumber + streamNumber + ripeHash + unixTimestamp)). If the message has no tag, this field is filled with 32 bytes of random data.
|-
| ?||encrypted||uchar[]||Encrypted data. See [[Protocol_specification#Encrypted_payload|Encrypted payload]]. See also [[Protocol_specification#Unencrypted_Message_Data|Unencrypted Message Data Format]]
|}

== Conclusion ==

All feedback, ideas, and criticism about this proposal are welcome. A discussion thread can be found on the Bitmessage forum here: https://bitmessage.org/forum/index.php?topic=4075.0.

Argothas Architecture Proposal

2014-07-17T14:00:47Z

Bmng-dev: Insert Proposal template

{{Template:Proposal}}
<h1>Introduction</h1>
After a lot of reading and thinking, I have an idea that might help resolve some of the issues that we are encountering. I am calling the collection of these the new "system", however it is more a change of both architecture and protocol. The proposed system is still very much an idea forming so please bear with me.

This post has used ideas from:
<ul>
<li>BitBoards</li>
<li>Message tagging proposal</li>
<li>The non-stream proposal for streams</li> <li>Onion routing</li> </ul>

One of the main issues we are encountering is the ability to make the BM protocol smartphone friendly whilst keeping the recipient hidden. I beleive this is flawed as the it requires a user to trust the server they are using as their bitboard. The operator of a BitBoard will have vision over all messages sent to any of their addresses, and although these messages are encrypted, vision of this data will reveal some information about the user.

<h1>The Proposed System</h1>
<h2>Overview</h2>
The system offers the following advantages:
<ul>
<li>Scalability</li>
<li>Spam protection</li>
<li>Distributed trust of server (i.e. not trusting a single server)</li> <li>The ability for businesses (and individuals) to operate their own servers</li> </ul>

The system makes the following trade-offs:
<ul>
<li>Complete anonyminity of recpient, the ability to "check" for messages</li> </ul>

The system has the following diadvantages:
<ul>
<li>See trade-offs</li>
<li>???</li>
</ul>

<h2>Custom Notation</h2>
KEY1, KEY2 -> DATA1, DATA2 = Table of keys mapped to data
 [TYPE | FIELD1, FIELD2] = Message with type and fields
 E:OBJECT = Encrypted object
 S:OBJECT = Signed object (i.e. tamper-proof)
 IP(MACHINE) = IP address of a machine
 TO>> FROM MESSAGE = message transmission
 HASH(OBJECT) = Hash of an object
 BMADDR = BitMessage Address
 BMMESSAGE = BitMessage style message
 NMESSAGE = Notification message
 MTAG = Message tag
 C = Client
 S = Sender
 R = Reciever
 TTL = Time to live
 TTD = Time at which point to die
 NONCE = Random data
 PERIOD = Period of time in seconds
 POWVAL = Proof of work value (as described in the POW description)

<h2>System Description</h2>
<h3>The Building Blocks</h3>
The new system is designed in a much more client/server architecture (with similarities to hidden services in TOR), and is done so to enable light-weight clients (lite clients). Whilst I have seperated out clients and servers, they would likely be able to be merged, although I have not thought of the ramifactions of merging them. I have also seperated out server roles, this is to help define roles and processes, agian they may be able to be merged.

 Data Server (DS) - These hold the actual messages
 Introduction Server (IS) - These hold notifcation messages
 Directory Server (DIR) - These hold information about registered servers for the use of clients to connect to their desired service.

<h3>New Address Process</h3>
<ol>
<li>The client creates a new BMADDR as per usual</li>
<li>The client connects to a DIR to request a list of known ISs</li>
<li>The client selects a number of ISs that it wishes to use</li>
<li>The client connects to each IS and requests to use it</li>
<li>The IS accepts or rejects the request</li>
<ul>
<li>On denial the client should add another IS to its pool</li>
</ul>
<li>On acceptance the client prepares to store the IS, TTD, NONCE, PERIOD and POWVAL with its BMADDR</li>
<li>The client publishes its BMADDR and IS data (to a DIR)</li>
</ol>

Why do we use multiple ISs? There are two reasons, the first is redundancy. If a sender cannot connect to one IS is can choose another (or many others), also if an IS is down when a client attempts to recieve, it messages sent to multiple IS can be recieved. The second reason stop a single IS from seeing all messaes that are sent, by distrbuting the "load" among many. A business could operate its own IS, only accepting requests from employees, employees would only need to use the one IS for their BMADDR.

What is the data being stored with the BMADDR? For a sender to send messages to the reciever, the sender must know which ISs the reciever is using. So the IP of the IS is stored. The TTD indicated at what point in time this IS should stop being used. The NONCE is used in the calculation of the MTAG for this IS. This is so that if the same NMESSAGE is sent to multiple ISs it will have different MTAGs and as such reduce the amount of information available. PERIOD is used in the MTAG calculation. The current time should be modded with the PERIOD, with this result being subtracted from the current time. This has been described in the MTAG proposal. The reason for including this data is it allows a cleint to specify different PERIODs for different ISs.

Possible Attack: An attacker that wishes to bring attention to a particular MTAG (and hence potentially the BMADDR) could send messages to all of its listed ISs. The operators of each IS would then be able to see a large volume of messages destined for one MTAG.

How does the IS and the client negotiate use? The client connects to the IS and provides it with a passphrase (of preset number of bytes). Based on this passphrase a server can accept or deny the request.

Possible Attack: A malicious IS operator may only accept one request. As a result the number of NMESSAGEs sent to this IS will be aproximately equal to the number of NMESSAGEs sent to the single client (aproxiately, as junk messages can be sent and will be accepted). This is why it is important for a client to use multiple ISs

Possible Attack: A malicious IS opertor may only accept one request. the operator can then monitor the IP address of any connections. This is why onion routing is encourage (to be discussed later)

Possible Attacks: Other possible attacks using knowledge obtained from connections and MTAGs have been discussed in the MTAG proposal thread. Most of these can be mitigated through the use of onion routing and VPNs.

Possible Attack: A malicious IS operator may accept requests but then delete any NMESSAGEs sent to the IS. This is why it is important for a client to use multiple ISs

Why isnt the server notified of the TTD? In order to protect the clients, the server does hold any information it can use to identify the clients (except possibly client specific passwords). By ensuring this, the IS cannot make assocatations between MTAGs over a period of time (except in the case that a static IP is used). As a result, the server cannot determine what MTAGs are for a particular client and has no way to selectively remove them.

If the server can't stop messages for a particular client, how can it prevent itself from being DOSed (potentially through normal use)? As shown later, the server will indicate the minimum proof of work value that it will accept. Thus a server under highload may continue to increase its POW until it can sustain its use. Clients should be aware of this and add more IS servers to its pool if it wishes to keep a low POW for some of its senders.

<h3>New Message Process</h3>
<ol>
<li>The sender creates the BMMESSAGE</li>
<li>The sender connects to a DIR to get a list of DSs</li>
<li>The sender selects a DS (*) and requests to store the size of the BMMESSAGE on it</li>
<li>The DS accepts or denies the request</li>
<ul>
<li>On denial, the sender must select a new DS</li>
<li>On acceptance the DS replues with its minimum POWVAL</li>
</ul>
<li>The sender calculates POW and uploads the encrypted BMESSAGE to the DS</li>
<li>The sender prepares the NMESSAGE</li>
<li>The sender connects to a DIR to collect the Rs ISs</li>
<li>The sender selects one or more (but prefably not all) of the ISs</li>
<li>For each IS, the sender connects and requests the ISs POWVAL</li>
<li>The sender calculates the POW and uploads the NMESSAGE</li>
</ol>

Why seperate DS and IS? This has been done as an IS has an obligation to store every NMESSAGE it recieves. If it were to also store the BMESSAGEs it could rapidly run out of room for NMESSAGEs. It also prevent the operator of a node to guess teh size of a BMESSAGE being sent to a particular address.

* Variation: Instead of using one DS a sender could use multiple, this creates redundancy. This would require the content of the NMESSAGE to map multiple DSs to portnetially different hashes of the same message.

What is a NMESSAGE? This iss similar to a BMMESSAGE in terms of security, however its content is in a strict format. The content of the message will tell the reciever which HASH to request from each DS. It will also reveal the MTAG

Variation: The format of the NMESSAGE could allow the sender to bundle multiple BMESSAGESs into one NMESSAGE. This would disguise any multipart messages. It could also be used, by a client/server that only sends messages periodically and bundles where it can. This would allow a sender to obscure that they are sending multipart messages (see below).

Possible attack: By examining the size of the NMESSAGE, an attacker could information about the contents. For example, a larger NMESSAGE would indicate the possibility of a multipart message. This could be partially mitigated by including padding up to the average size. (Of course by increasing the the size of messages to the average size, will slowly increase the message size to the max size).

Possible attack:If the size of NMESSAGES is not fixed (in order to allow multipart messages, and redancy of parts), then an attacker could take advantage of this to use up an ISs resources. As such NMESSAGES must be a fixed size or must have an upperlimit on their size.

NMESSAGES vs BMMESSAGE: Based on creating a low maximum size for NMESSAGE to allow ISs to store a high volume of them, DSs could allow a much larger maximum size of BMMESSAGEs. Unlike the current network a 10mb message would not severly damage the overal network health, indeed it may not even effect the node that it is uploading to).

<h3>Recieving Messages</h3>
<ol>
<li>Periodically the reciever connects to its ISs (*) and requests any NMESSAGEs for its MTAGS</li>
<li>The reciever decrypts the NMESSAGE</li>
<li>The reciever connects to the listed DSs to get the lsited BMMESSAGEs</li>
</ol>

* Variation: A IS could require that a valid server pasphrase is supplied in order to retrieve NMESSAGES. This comes at a cost to a users anonyminity as if different passphrases are supplied to different clients, a operator could the determine what MTAGs are destined for the same client. It would then be preferable for anyone wishing to operate a private server (e.g. businesses) to use BitMessage software that supports ?SOCKS?.

Possible attack: If a client does not using a changing IP address or reuses the same connection in its request for MTAGs, the IS (or DS in the case of hashes), would be able to determine that the objects are linked to the one address (and potentially IP).

<h2>Where does onion routing come into all of this?</h2>
With the above architecture, an attack can oobserve connections to the various servers to determine who is using it (and by examning the size of data exchanges, what they are using it for). (Note: it would be highly suggested that servers self publish their public key to a DIR so that clients can establish secure communications). In BitMessage, this was mitigated by all nodes transferring all data, irrespective of whether it was destined to them or not. This however does not scale well, and makes smartphone cleints prohibitive. This new system has a heavy need for onion routing, so that the IP address of a conenction cannot be used to identify a user, or group different MTAGs or BMMESSAGE hashes together.

This then asks an important question: Should clients know to use TOR (and hence it would be prefferable for the maintainers of client software to build in TOR), or should the BM network build its own onion routing network?.

<h1>Conclusion</h1>
Whilst I have done my best to nut out most of the questions, possibilities and issues I doubt I have covered them all. If I have missed anything feel free to point it out. Also, although this may seem like a proposal, it is a rather drastic change to the concept of BitMessage and should be up for lots of discussion.

Template:Proposal

2014-07-17T08:38:29Z

Bmng-dev: Added Proposals category

<div class="plainlinks" style="border: 1px solid #ee0; width: 50%; background-color: #ffc; padding: 10px; margin-bottom: 25px">This page is a proposal or recommendation. Its content is most likely not representing an actual situation.</div>
<includeonly>[[Category:Proposals]]</includeonly>

Protocol specification v3

2014-07-05T11:07:06Z

Bmng-dev: Variable length integer encoding constraint

{{Template:Stub}}

{{Template:Proposal}}

== Introduction ==
This is a DRAFT for the protocol version 3. It describes the changes in protocol version 3 versus version 2. Things which are unchanged from version 2 are not described in detail. So you should use [https://bitmessage.org/wiki/Protocol_specification this] as a reference for all formates which are not mentioned in this description.

== new features in version 3 ==

Here are the new features of the version 3 of Bitmessage protocol in keywords:

* object type is now coded inside the message
* objects may have a variable time to live
* The POW is more difficult but can be easier if you lower the time to live
* The protocol is tolerant for further message extension
* The protocol is tolerant for further object extension
* A lower maximum object size

== Common structures ==

=== Variable length integer ===

The shortest possible encoding MUST be used. In version 2 the decimal value 10 could be encoded as 0x0A, 0xFD000A, 0xFE0000000A, or 0xFF000000000000000A. In version 3 only the shortest representation, 0x0A, is allowed. The 9-byte form is no longer useful and SHOULD NOT be used.

== Message types ==

Most message types are unchanged from version 2 to version 3. Only the four "objecttype" messages are not valid any more. They are summerized into one single message.

=== version ===

The version message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#version here]

=== verack ===

The verack message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#verack here]

=== addr ===

The addr message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#addr here]

=== inv ===

The inv message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#inv here]

=== getdata ===

The getdata message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#getdata here]

=== unsupported messages ===
If a node receives an unknown message it must silently ignore it. This is for further extensions of the protocol with other messages. Nodes that don't understand such a new message type shall be able to work correct with the message types they understand.

Maybe some version 2 nodes did already implement it that way, but in version 3 it is '''part of the protocol specification''', that a node must silently ignore unsupported messages.

== Object type ==

The four former object types "getpubkey", "pubkey", "msg" and "broadcast" are summarizes into a single Message type "object". The four Messages as they did exist in version 2 protocol are not valid any more.

Objects are shared throughout a stream. A client should advertise objects until their end of life time is reached. To be a valid object, the [[Proof of work|Proof Of Work]] has to be done.

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 8||POW nonce||uint64_t||
Random nonce used for the [[Proof of work|Proof Of Work]]
|-
| 8||time||uint64_t||
The "end of life" time of this object (be aware, in version 2 of the protocol this was the generation time). Objects shall be broadcast until its end of life time has been reached. The node shall store the inventory vector of that object for at least another hour to avoid reloading it from another node with a small time delay.
The maximum value for the "time to life" of an object is 28 days. so the "end of life time" is 28 days in the future at maximum.
|-
| 4||object type||uint32_t||
This field specifies the content of the object.
Valid values are (at the moment) 0-"getpubkey", 1-"pubkey", 2-"msg", 3-"broadcast". all other values are reserved. Nodes shall transport objects with unknown types, too. This will make further protocol changes more easy.
|-
|-
| ?||payload||uchar[]||
This field varies depending on the object type. For a detailed description of their content look [https://bitmessage.org/wiki/Protocol_specification#Object_types here]
|-
|}

=== Do a POW ===

Generally the POW is done exactly like in [[Proof of work|version 2]]

The "target" (the difficulty of the POW) is defined a little bit lower (more difficult) in version 3. This is, because practice did show, it is to easy to float the network with data.
In addition to that it is possible in version 3 to lower the time to life of a message (for example when doing a life chat) and getting an easier POW for that.

Setting the live time e.g. to 360 seconds (6 Minutes), which is totally suffient for a life chat, the POW will be 10 times easier than in version 2.

The "time to live" is the difference between the current time and the "end of life" time. For POW check and generation the "time to live" value is considered to be at least 300 seconds, even if the object has to life less than 300 seconds.

payload = EndOfLifeTime + ObjectType + payload

PayloadLenInBytes * TimeToLiveInSeconds
loadvalue = --------------------------------------- + payloadLengthExtraBytes + 8
3600

2^64
target = ------------------------------------------------
loadvalue * averageProofOfWorkNonceTrialsPerByte

=== Maximum object size ===

In version 2 the maximum object size was defined to be 170 MBytes. This object size is totally unrealistic for a normal use (POW), but is perfectly for a network attack. It will be to big to handle for clients with low bandwidth.
Currently an average bitmessage "msg" object has the size of 2 kBytes.
So version 3 limits the objects to a maximum size of 64 kBytes.