Difference between revisions of "Protocol specification v3"

From Bitmessage Wiki
Jump to navigation Jump to search
 
(43 intermediate revisions by 3 users not shown)
Line 4: Line 4:
  
 
== Introduction ==
 
== Introduction ==
This is a DRAFT for the protocol version 3. It describes the changes in protocol version 3 versus version 2. Things which are unchanged from version 2 are not described in detail. So you should use [https://bitmessage.org/wiki/Protocol_specification this] as a reference for all formates which are not mentioned in this description.
+
This is a DRAFT for the protocol version 3. It describes the changes in protocol version 3 versus version 2. Things which are unchanged from version 2 are not described in detail. So you should use the [[Protocol specification|current protocol specification]], which includes the version 3 changes, as a reference for all formats which are not mentioned in this description.
 +
 
 +
== New features in version 3 ==
 +
 
 +
Here are the new features of the version 3 of Bitmessage protocol in keywords:
 +
 
 +
* object type is now coded inside the message
 +
* objects may have a variable time to live
 +
* The POW is more difficult but can be easier if you lower the time to live
 +
* The protocol is tolerant for further message extension
 +
* The protocol is tolerant for further object extension
 +
* A lower maximum object size
 +
 
 +
 
 +
== Common structures ==
 +
 
 +
=== Variable length integer ===
 +
 
 +
The shortest possible encoding MUST be used. In version 2 the decimal value 10 could be encoded as 0x0A, 0xFD000A, 0xFE0000000A, or 0xFF000000000000000A. In version 3 only the shortest representation, 0x0A, is allowed. The 9-byte form is no longer useful and SHOULD NOT be used.
 +
 
 +
Here is an example address which uses malformed varints: BM-CVA3RC7Mvy7JDNSpQChktwrSe4KNMaEdDdcymfUo. This address is invalid.
 +
 
  
 
== Message types ==
 
== Message types ==
  
Most message types are unchanged from version 2 to version 3. Only the four "objecttype" messages are not valid any more. They are summerized into one single message.
+
Most message types are unchanged from version 2 to version 3. Only the four "objecttype" messages are not valid any more. They are summarized into one single message.
  
 
=== version ===
 
=== version ===
  
The version message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#version here]
+
The version message is identical to [[Protocol specification#version|version 2 version message]].
  
 
=== verack ===
 
=== verack ===
  
The verack message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#verack here]
+
The verack message is identical to [[Protocol specification#verack|version 2 verack message]].
  
 
=== addr ===
 
=== addr ===
  
The addr message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#addr here]
+
The addr message is identical to [[Protocol specification#addr|version 2 addr message]].
  
 
=== inv ===
 
=== inv ===
  
The inv message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#inv here]
+
The inv message is identical to [[Protocol specification#inv|version 2 inv message]].
  
 
=== getdata ===
 
=== getdata ===
  
The getdata message is identically to protocol version 2. you can lookup details [https://bitmessage.org/wiki/Protocol_specification#getdata here]
+
The getdata message is identical to [[Protocol specification#getdata|version 2 getdata message]].
 +
 
 +
=== error ===
 +
 
 +
Version 3 of the protocol defined a special error (or debug) message. This message may be silently ignored (and therefor handled like any other "unknown" message).
 +
The message is intended to inform the other node about protocol errors and can be used for debugging and improving code.
 +
 
 +
{|class="wikitable"
 +
! Field Size !! Description !! Data type !! Comments
 +
|-
 +
| 1+||fatal||var_int||
 +
This qualifies the error. If set to 0, than its just a "warning". You can expect, everything still worked fine. If set to 1, than it's an error, so you may expect, something was going wrong (e.g. an object got lost). If set to 2, it's a fatal error. The node will drop the line for that error and maybe ban you for some time.
 +
|-
 +
 
 +
|-
 +
| 1+||ban time||var_int||
 +
If the error is fatal, you can specify the ban time in seconds, here. You inform the other node, that you will not accept further connections for this number of seconds. For non fatal errors this field has no meaning and should be zero.
 +
|-
 +
 
 +
|-
 +
| 1+||inventory vector||var_str||
 +
If the error is related to an object, this Variable length string contains the inventory vector of that object. If the error is not related to an object, this string is empty.
 +
|-
 +
 
 +
|-
 +
| 1+||error text||var_str||
 +
A human readable string in English, which describes the error.
 +
|-
 +
 
 +
|}
 +
 
 +
=== Unsupported messages ===
 +
If a node receives an unknown message it <u>must</u> silently ignore it. This is for further extensions of the protocol with other messages. Nodes that don't understand such a new message type shall be able to work correct with the message types they understand.
 +
 
 +
Maybe some version 2 nodes did already implement it that way, but in version 3 it is '''part of the protocol specification''', that a node <u>must</u> silently ignore unsupported messages.
  
 
== Object type ==
 
== Object type ==
  
The four former object types "getpubkey", "pubkey", "msg" and "broadcast" are summarizes into a single Message type "object". The four Messages as they did exist in version 2 protocol are not valid any more.
+
The four former object types "getpubkey", "pubkey", "msg" and "broadcast" are summarized into a single Message type "object". The four Messages as they did exist in version 2 protocol are not valid any more.
  
Objects are shared throughout a stream.  A client should advertise objects until thair end of life time is reached. To be a valid object, the [[Proof of work|Proof Of Work]] has to be done.
+
Objects are shared throughout a stream.  A client should advertise objects until their end of life time is reached. To be a valid object, the [[Proof of work|Proof Of Work]] has to be done.
  
 
{|class="wikitable"
 
{|class="wikitable"
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
| 8||POW nonce||uint64_t||Random nonce used for the [[Proof of work|Proof Of Work]]
+
| 8||POW nonce||uint64_t||
 +
Random nonce used for the [[Proof of work|Proof Of Work]]
 
|-
 
|-
| 8||time||uint32_t||The "end of life" time of this object. Objects shall be broadcast until its end of life time has been reached. The node shall store the inventory vector of that object for at least anouther hour to avoid reloading it from another node with a small time delay.
+
| 8||time||uint64_t||
 +
The "end of life" time of this object (be aware, in version 2 of the protocol this was the generation time). Objects shall be broadcast until its end of life time has been reached. The node shall store the inventory vector of that object for at least another hour to avoid reloading it from another node with a small time delay.
 +
The maximum value for the "time to life" of an object is 28 days. so the "end of life time" is 28 days in the future at maximum.
 
|-
 
|-
| 4||object type||uint32_t||this field specifies the content of the object.
+
| 4||object type||uint32_t||
Valis valies are (at the moment) 0-"getpubkey", 1-"pubkey", 2-"msg", 3-"broadcast". all other values are reserved. Nodes shall transport objects with unknown types, too, This will make furter protocol changes more easy.
+
This field specifies the content of the object.
 +
Valid values are (at the moment) 0-"getpubkey", 1-"pubkey", 2-"msg", 3-"broadcast". all other values are reserved. Nodes shall transport objects with unknown types, too. This will make further protocol changes more easy.
 
|-
 
|-
 
|-
 
|-
| ?||payload||uchar[]||this field varies depending on the object type.  
+
| ?||payload||uchar[]||
 +
This field varies depending on the object type. For a detailed description of their content refer to [[Protocol specification#Object_types|version 2 object types]]
 
|-
 
|-
 
|}
 
|}
 +
 +
 +
== Proof of Work ==
 +
 +
Generally the POW is done exactly like in [[Proof of work|version 2]]
 +
 +
The "target" (the difficulty of the POW) is defined a little bit lower (more difficult) in version 3. This is, because practice did show, it is to easy to flood the network with data.
 +
In addition to that it is possible in version 3 to lower the time to live of a message (for example when doing a live chat) and getting an easier POW for that.
 +
 +
 +
[[File:POW forumla 2.png|none|caption]]
 +
 +
 +
payloadLengthExtraBytes = 1000
 +
 +
nonceTrialsPerByte = 1000
 +
 +
== Maximum object size ==
 +
 +
In version 2 the maximum object size was defined to be 170 MBytes. This object size is totally unrealistic for a normal use (POW), but is perfectly for a network attack. It will be to big to handle for clients with low bandwidth.
 +
Currently an average bitmessage "msg" object has the size of 2 kBytes.
 +
So version 3 limits the objects to a maximum size of 256 KiB(the payload of the object, starting from the POW nonce can have 262144 bytes at maximum).

Latest revision as of 20:14, 17 October 2014

Introduction

This is a DRAFT for the protocol version 3. It describes the changes in protocol version 3 versus version 2. Things which are unchanged from version 2 are not described in detail. So you should use the current protocol specification, which includes the version 3 changes, as a reference for all formats which are not mentioned in this description.

New features in version 3

Here are the new features of the version 3 of Bitmessage protocol in keywords:

  • object type is now coded inside the message
  • objects may have a variable time to live
  • The POW is more difficult but can be easier if you lower the time to live
  • The protocol is tolerant for further message extension
  • The protocol is tolerant for further object extension
  • A lower maximum object size


Common structures

Variable length integer

The shortest possible encoding MUST be used. In version 2 the decimal value 10 could be encoded as 0x0A, 0xFD000A, 0xFE0000000A, or 0xFF000000000000000A. In version 3 only the shortest representation, 0x0A, is allowed. The 9-byte form is no longer useful and SHOULD NOT be used.

Here is an example address which uses malformed varints: BM-CVA3RC7Mvy7JDNSpQChktwrSe4KNMaEdDdcymfUo. This address is invalid.


Message types

Most message types are unchanged from version 2 to version 3. Only the four "objecttype" messages are not valid any more. They are summarized into one single message.

version

The version message is identical to version 2 version message.

verack

The verack message is identical to version 2 verack message.

addr

The addr message is identical to version 2 addr message.

inv

The inv message is identical to version 2 inv message.

getdata

The getdata message is identical to version 2 getdata message.

error

Version 3 of the protocol defined a special error (or debug) message. This message may be silently ignored (and therefor handled like any other "unknown" message). The message is intended to inform the other node about protocol errors and can be used for debugging and improving code.

Field Size Description Data type Comments
1+ fatal var_int

This qualifies the error. If set to 0, than its just a "warning". You can expect, everything still worked fine. If set to 1, than it's an error, so you may expect, something was going wrong (e.g. an object got lost). If set to 2, it's a fatal error. The node will drop the line for that error and maybe ban you for some time.

1+ ban time var_int

If the error is fatal, you can specify the ban time in seconds, here. You inform the other node, that you will not accept further connections for this number of seconds. For non fatal errors this field has no meaning and should be zero.

1+ inventory vector var_str

If the error is related to an object, this Variable length string contains the inventory vector of that object. If the error is not related to an object, this string is empty.

1+ error text var_str

A human readable string in English, which describes the error.

Unsupported messages

If a node receives an unknown message it must silently ignore it. This is for further extensions of the protocol with other messages. Nodes that don't understand such a new message type shall be able to work correct with the message types they understand.

Maybe some version 2 nodes did already implement it that way, but in version 3 it is part of the protocol specification, that a node must silently ignore unsupported messages.

Object type

The four former object types "getpubkey", "pubkey", "msg" and "broadcast" are summarized into a single Message type "object". The four Messages as they did exist in version 2 protocol are not valid any more.

Objects are shared throughout a stream. A client should advertise objects until their end of life time is reached. To be a valid object, the Proof Of Work has to be done.

Field Size Description Data type Comments
8 POW nonce uint64_t

Random nonce used for the Proof Of Work

8 time uint64_t

The "end of life" time of this object (be aware, in version 2 of the protocol this was the generation time). Objects shall be broadcast until its end of life time has been reached. The node shall store the inventory vector of that object for at least another hour to avoid reloading it from another node with a small time delay. The maximum value for the "time to life" of an object is 28 days. so the "end of life time" is 28 days in the future at maximum.

4 object type uint32_t

This field specifies the content of the object. Valid values are (at the moment) 0-"getpubkey", 1-"pubkey", 2-"msg", 3-"broadcast". all other values are reserved. Nodes shall transport objects with unknown types, too. This will make further protocol changes more easy.

? payload uchar[]

This field varies depending on the object type. For a detailed description of their content refer to version 2 object types


Proof of Work

Generally the POW is done exactly like in version 2

The "target" (the difficulty of the POW) is defined a little bit lower (more difficult) in version 3. This is, because practice did show, it is to easy to flood the network with data. In addition to that it is possible in version 3 to lower the time to live of a message (for example when doing a live chat) and getting an easier POW for that.


caption


payloadLengthExtraBytes = 1000

nonceTrialsPerByte = 1000

Maximum object size

In version 2 the maximum object size was defined to be 170 MBytes. This object size is totally unrealistic for a normal use (POW), but is perfectly for a network attack. It will be to big to handle for clients with low bandwidth. Currently an average bitmessage "msg" object has the size of 2 kBytes. So version 3 limits the objects to a maximum size of 256 KiB(the payload of the object, starting from the POW nonce can have 262144 bytes at maximum).