Protocol specification

From Bitmessage Wiki
Revision as of 18:39, 20 August 2013 by Apatomoose (talk | contribs) (PyBitmessage uses 0xf* followed by the integer for values >= 0xfd, not 0xf* followed by length. See https://github.com/Bitmessage/PyBitmessage/blob/master/src/addresses.py lines 58-88 encodeVarint and decodeVarint)
Jump to navigation Jump to search

Common standards

Hashes

Most of the time SHA-512 hashes are used, however RIPEMD-160 is also used when creating an address.

A double-round of SHA-512 is used for the Proof Of Work. Example of double-SHA-512 encoding of string "hello":

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round of sha-512)
0592a10584ffabf96539f3d780d776828c67da1ab5b169e9e8aed838aaecc9ed36d49ff1423c55f019e050c66c6324f53588be88894fef4dcffdb74b98e2b200(second round of sha-512)

For Bitmessage addresses (RIPEMD-160) this would give:

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round is sha-512)
79a324faeebcbf9849f310545ed531556882487e (with ripemd-160)

Common structures

All integers are encoded in big endian. (This is different from Bitcoin).

Message structure

Field Size Description Data type Comments
4 magic uint32_t Magic value indicating message origin network, and used to seek to next message when stream state is unknown
12 command char[12] ASCII string identifying the packet content, NULL padded (non-NULL padding results in packet rejected)
4 length uint32_t Length of payload in number of bytes
4 checksum uint32_t First 4 bytes of sha512(payload)
? payload uchar[] The actual data, a message or an object

Known magic values:

Magic value Sent over wire as
0xE9BEB4D9 E9 BE B4 D9

Variable length integer

Integer can be encoded depending on the represented value to save space. Variable length integers always precede an array/vector of a type of data that may vary in length.

Value Storage length Format
< 0xfd 1 uint8_t
<= 0xffff 3 0xfd followed by the integer as uint16_t
<= 0xffffffff 5 0xfe followed by the integer as uint32_t
- 9 0xff followed by the integer as uint64_t

Variable length string

Variable length string can be stored using a variable length integer followed by the string itself.

Field Size Description Data type Comments
1+ length var_int Length of the string
? string char[] The string itself (can be empty)

Variable length list of integers

n integers can be stored using n+1 variable length integers where the first var_int equals n.

Field Size Description Data type Comments
1+ count var_int Number of var_ints below
1+ var_int The first value stored
1+ var_int The second value stored...
1+ var_int etc...

Network address

When a network address is needed somewhere, this structure is used. This protocol and structure supports IPv6, but note that the original client currently only supports IPv4 networking. Network addresses are not prefixed with a timestamp or stream in the version message.

Field Size Description Data type Comments
4 (or 8) time uint32 the Time. Protocol version 1 clients use 4 byte time while protocol version 2 clients use 8 byte time.
4 stream uint32 Stream number for this node
8 services uint64_t same service(s) listed in version
16 IPv6/4 char[16] IPv6 address. The original client only supports IPv4 and only reads the last 4 bytes to get the IPv4 address. However, the IPv4 address is written into the message as a 16 byte IPv4-mapped IPv6 address

(12 bytes 00 00 00 00 00 00 00 00 00 00 FF FF, followed by the 4 bytes of the IPv4 address).

2 port uint16_t port number

Inventory Vectors

Inventory vectors are used for notifying other nodes about objects they have or data which is being requested. Two rounds of SHA-512 are used, resulting in a 64 byte hash. Only the first 32 bytes are used; the later 32 bytes are ignored.

Inventory vectors consist of the following data format:

Field Size Description Data type Comments
32 hash char[32] Hash of the object

Encrypted payload

Bitmessage uses ECIES to encrypt its messages. For more information see Encryption

Field Size Description Data type Comments
16 IV uchar[] Initialization Vector used for AES-256-CBC
2 uint16_t Curve type Elliptic Curve type 0x02CA (714)
2 uint16_t X length Length of X component of public key R
X length uchar[] X X component of public key R
2 uint16_t Y length Length of Y component of public key R
Y length uchar[] Y Y component of public key R
? encrypted uchar[] Cipher text
32 MAC uchar[] HMACSHA256 Message Authentication Code

Unencrypted Message Data

Field Size Description Data type Comments
1+ msg_version var_int Message format version
1+ address_version var_int Sender's address version number. This is needed in order to calculate the sender's address to show in the UI, and also to allow for forwards compatible changes to the public-key data included below.
1+ stream var_int Sender's stream number
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the node with this pubkey included in this msg message (the sender's pubkey).
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
1+ nonce_trials_per_byte var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 320 is the network minimum so any lower values will be automatically raised to 320. This field is new and is only included when the address_version >= 3.
1+ extra_bytes var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 14000 is the network minimum so any lower values will be automatically raised to 14000. This field is new and is only included when the address_version >= 3.
20 destination ripe uchar[] The ripe hash of the public key of the receiver of the message
1+ encoding var_int Message Encoding type
1+ message_length var_int Message Length
message_length message uchar[] The message.
1+ ack_length var_int Length of the acknowledgement data
ack_length ack_data uchar[] The acknowledgement data to be transmitted. This takes the form of a Bitmessage protocol message, like another msg message. The POW therein must already be completed.
1+ sig_length var_int Length of the signature
sig_length signature uchar[] The ECDSA signature which covers everything from the msg_version to the ack_data.

Message Encodings

Value Name Description
0 IGNORE Any data with this number may be ignored. The sending node might simply be sharing its public key with you.
1 TRIVIAL UTF-8. No 'Subject' or 'Body' sections. Useful for simple strings of data, like URIs or magnet links.
2 SIMPLE UTF-8. Uses 'Subject' and 'Body' sections. No MIME is used.

messageToTransmit = 'Subject:' + subject + '\n' + 'Body:' + message

Further values for the message encodings can be decided upon by the community. Any MIME or MIME-like encoding format, should they be used, should make use of Bitmessage's 8-bit bytes.

Pubkey bitfield features

Bit Name Description
0 undefined The most significant bit at the beginning of the structure. Undefined
1 undefined The next most significant bit. Undefined
... ... ...
30 include_destination Receiving node expects that the RIPE hash encoded in their address preceedes the encrypted message data of msg messages bound for them.
31 does_ack If true, the receiving node does send acknowledgements (rather than dropping them).

Message types

version

When a node creates an outgoing connection, it will immediately advertise its version. The remote node will respond with its version. No futher communication is possible until both peers have exchanged their version.

Payload:

Field Size Description Data type Comments
4 version int32_t Identifies protocol version being used by the node
8 services uint64_t bitfield of features to be enabled for this connection
8 timestamp int64_t standard UNIX timestamp in seconds
26 addr_recv net_addr The network address of the node receiving this message (not including the time or stream number)
26 addr_from net_addr The network address of the node emitting this message (not including the time or stream number and the ip itself is ignored by the receiver)
8 nonce uint64_t Random nonce used to detect connections to self.
1+ user_agent var_str User Agent (0x00 if string is 0 bytes long)
1+ stream numbers var_int_list The stream numbers that the emitting node is interested in.

A "verack" packet shall be sent if the version packet was accepted. Once you have sent and received a verack messages with the remote node, send an addr message advertising up to 1000 peers of which you are aware, and one or more inv messages advertising all of the valid objects of which you are aware.

The following services are currently assigned:

Value Name Description
1 NODE_NETWORK This is a normal network node.

verack

The verack message is sent in reply to version. This message consists of only a message header with the command string "verack".

addr

Provide information on known nodes of the network. Non-advertised nodes should be forgotten after typically 3 hours

Payload:

Field Size Description Data type Comments
1+ count var_int Number of address entries (max: 1000)
34x? addr_list net_addr Address of other nodes on the network.

inv

Allows a node to advertise its knowledge of one or more objects. Payload (maximum payload length: 50000 items):

Field Size Description Data type Comments
? count var_int Number of inventory entries
32x? inventory inv_vect[] Inventory vectors

getdata

getdata is used in response to an inv message to retrieve the content of a specific object after filtering known elements.

Payload (maximum payload length: 50000 entries):

Field Size Description Data type Comments
? count var_int Number of inventory entries
32x? inventory inv_vect[] Inventory vectors

Object types

Objects are a subset of network messages. They are shared throughout a stream. A client should advertise objects that are not older than 2.5 days. To be a valid object, the Proof Of Work has to be done.

getpubkey

When a node has the hash of a public key (from an address) but not the public key itself, it must send out a request for the public key.

Field Size Description Data type Comments
8 POW nonce uint64_t Random nonce used for the Proof Of Work
4 (or 8) time uint32_t The time that this message was generated and broadcast. We are transitioning to 8 byte time.
1+ address version var_int The address' version
1+ stream number var_int The address' stream number
20 pub key hash uchar[] The ripemd hash of the public key

pubkey

A version 2 public key. This is still in use and supported by current clients but new v2 addresses are not generated by clients.

Field Size Description Data type Comments
8 POW nonce uint64_t Random nonce used for the Proof Of Work
4 (or 8) time uint32_t The time that this message was generated and broadcast. We are transitioning to 8 byte time.
1+ address version var_int The address' version which is set to 2.
1+ stream number var_int The address' stream number
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the node receiving the message.
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )

A version 3 pubkey

Field Size Description Data type Comments
8 POW nonce uint64_t Random nonce used for the Proof Of Work
4 (or 8) time uint32_t The time that this message was generated and broadcast. We are transitioning to 8 byte time.
1+ address version var_int The address' version which is set to 3.
1+ stream number var_int The address' stream number
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the node receiving the message.
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
1+ nonce_trials_per_byte var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 320 is the network minimum so any lower values will be automatically raised to 320.
1+ extra_bytes var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 14000 is the network minimum so any lower values will be automatically raised to 14000.
1+ sig_length var_int Length of the signature
sig_length signature uchar[] The ECDSA signature which covers everything from the time to the extra_bytes.

msg

Used for person-to-person messages.

Field Size Description Data type Comments
8 POW nonce uint64_t Random nonce used for the Proof Of Work
4 (or 8) time uint32_t The time that this message was generated and broadcast. We are transitioning to 8 byte time.
1+ streamNumber var_int The stream number of the destination address.
? encrypted uchar[] Encrypted data. See Encrypted payload. See also Unencrypted Message Data Format

broadcast

Version 1 broadcast messages are sent in-the-clear. Version 2 are encrypted. Users who are subscribed to the sending address will see the message appear in their inbox.

Version 1 broadcast format:

Field Size Description Data type Comments
8 POW nonce uint64_t The Proof Of Work nonce
4 (or 8) time uint32_t The time that the message was broadcast. We are transitioning to 8 byte time.
1+ broadcast version var_int The version number of this broadcast protocol message which is equal to 1 in this case.
1+ address version var_int The sender's address version
1+ stream number var_int The sender's stream number
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the owner of this pubkey.
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
20 address hash uchar[] The sender's address hash. This is included so that nodes can more cheaply detect whether this is a broadcast message for which they are listening, although it must be verified with the public key above.
1+ encoding var_int The encoding type of the message
1+ messageLength var_int The message length in bytes
messageLength message uchar[] The message
1+ sig_length var_int Length of the signature
sig_length signature uchar[] The signature which covers everything from the broadcast version down through the message.

Version 2 broadcasts:

Field Size Description Data type Comments
8 POW nonce uint64_t The Proof Of Work nonce
4 (or 8) time uint32_t The time that the message was broadcast. We are transitioning to 8 byte time.
1+ broadcast version var_int The version number of this broadcast protocol message which is equal to 2 in this case.
1+ stream number var_int The sender's stream number
? encrypted uchar[] Encrypted broadcast data. See Encrypted payload. See also Unencrypted Broadcast Data Format

Unencrypted data format:

Field Size Description Data type Comments
1+ broadcast version var_int The version number of this broadcast protocol message which is equal to 2 in this case. This is included here so that it can be signed.
1+ address version var_int The sender's address version
1+ stream number var_int The sender's stream number
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the owner of this pubkey.
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
1+ nonce_trials_per_byte var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 320 is the network minimum so any lower values will be automatically raised to 320. This field is new and is only included when the address_version >= 3.
1+ extra_bytes var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 14000 is the network minimum so any lower values will be automatically raised to 14000. This field is new and is only included when the address_version >= 3.
1+ encoding var_int The encoding type of the message
1+ messageLength var_int The message length in bytes
messageLength message uchar[] The message
1+ sig_length var_int Length of the signature
sig_length signature uchar[] The signature which covers everything from the broadcast version down through the message.