getblocktemplate

The original getwork mining protocol simply issues block headers for a miner to solve. The miner is kept in the dark as to what is actually in this block, and has no influence over it. In effect, the authority of the miner to decide which transactions are accepted, etc, is all transferred blindly to the pool operator. A corrupt (or compromised) pool operator could use the combined hash power of all the miners to execute double spend attacks or other similar attacks.

getblocktemplate moves block creation to the miner, while giving pools a way to set down the rules for participation. While pools can do just as much as they could before by expressing it in these rules, miners can not be kept in the dark and are enabled to freely choose what they participate in mining. This improves the security of the Bitcoin network by making blocks decentralized again.

The original getwork protocol only provides a single block header, which is sufficient for a total of about 4 GH of mining. With the "rollntime" extension, this can be extended to 4 GH *per second*, but even that is far from sufficient for the next generation of mining equipment (ASICs) which are capable of 1000 GH/s on the high end.

By moving block creation to the miners, they are enabled to create as much work as they need locally, thus overcoming this limitation.

Due to scalability problems, bitcoind's JSON-RPC stack has not been able to keep up with the hashrates needed for solo mining today. Since getblocktemplate drastically reduces the load required to a single request per new block on the network, direct solo mining on bitcoind is again possible. Poolservers likewise benefit from having to meet much lower demands of miners who can make their own blocks.

The original getwork protocol was designed in a way that was very incompatible with extensions. As a result, as new functionality was needed, extensions were "hacked in" out-of-band using HTTP headers. getblocktemplate is designed from the start to be flexible for future extensions, and the BIP 23 specification already covers how the established getwork extensions can be implemented cleanly, regardless of transport protocol.

Mining software with GBT support:

• BFGMiner 2.8+
• cgminer 2.9+
• Ufasoft Bitcoin Miner 0.39+
• Butterfly Labs's EasyMiner

To take advantage of getblocktemplate, you also need a compatible pool:

• BitMinter
• EclipseMC
• Eligius
• itzod.ru
• PolMine
• Encourage your favourite pool(s) to support decentralized mining!

See also: Poolservers

If you are implementing your own pool server, see the section for poolserver development.

If your miner can include C libraries, you can harness libblkmaker to do all the GBT interpretation for you: all your miner needs to do then is handle the networking (libblkmaker can prepare the JSON for you) and ask libblkmaker for data (block headers to search). Note that libblkmaker does not provide a SHA256 implementation, and your miner needs to provide one for it to work. libblkmaker currently only supports the Jansson JSON library, but was designed such that it can easily be ported to others;

If your miner is written in Python, you can harness python-blkmaker, a native Python port of libblkmaker, to do all the GBT interpretation for you: all your miner needs to do then is handle the networking (python-blkmaker can prepare the JSON for you) and ask the blkmaker module for data (block headers to search). A short example Python script is included that finds a prepared share using CPU mining.

To start participating, the miner contacts the pool and requests an initial template:

{"id": 0, "method": "getblocktemplate", "params": [{"capabilities": ["coinbasetxn", "workid", "coinbase/append"]}]}

The server will respond with the full details needed to immediately begin mining blocks:

{
 "error": null,
 "result": {
   "coinbasetxn": {
     "data": "0100000001000000000000000000000000000000000000000000000000000000
0000000000ffffffff1302955d0f00456c6967697573005047dc66085fffffffff02fff1052a01
0000001976a9144ebeb1cd26d6227635828d60d3e0ed7d0da248fb88ac01000000000000001976
a9147c866aee1fa2f3b3d5effad576df3dbf1f07475588ac00000000"
   },
   "previousblockhash": "000000004d424dec1c660a68456b8271d09628a80cc62583e5904f5894a2483c",
   "transactions": [],
   "expires": 120,
   "target": "00000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
   "longpollid": "some gibberish",
   "height": 23957,
   "version": 2,
   "curtime": 1346886758,
   "mutable": ["coinbase/append"],
   "bits": "ffff001d"
 },
 "id": 0
}

If the pool allows the "coinbase/append" mutation by including it in the "mutable" key, you can rebuild the coinbase transaction to append any data your miner wants, such as an extranonce - you can use as much space as you need so long as the coinbase data does not overflow the 100 byte hard limit. If the server does *not* allow "coinbase/append", or you don't care to change it, you can skip this step entirely :)

The coinbase data always begins after exactly 42 bytes of the coinbase transaction. The 42nd byte (that is, the byte immediately before the data) is the length of the data. So when/if you want to add on to the coinbase data, simply insert it 42+DataLen bytes into the transaction, and increment the 42nd byte by the length of your inserted data.

Python example:

import binascii
coinbase = binascii.a2b_hex(template['coinbasetxn']['data'])
extradata = b'my block'
# The following can be done better in both Python 2 and Python 3, but this way works with both
origLen = ord(coinbase[41:42])
newLen = origLen + len(extradata)
coinbase = coinbase[0:41] + chr(newLen).encode('ascii') + coinbase[42:42 + origLen] + extradata + coinbase[42 + origLen:]

Collect your coinbase transaction (modified or not) at the front of the "transactions" list provided by the server. Apply a double-SHA256 hash to each transaction in the list.

Now, as long as the list has more than 1 hash remaining, go through each pair and hash them together. That is, concatenate the first two, double-SHA256 that, repeat for the next two, and so on. If you encounter an odd pair (that is, the hash list ends with a single item and no pairing), concatenate it with itself and hash that. Continue to do that until there is only one hash left: that is your merkle root.

Python example:

import hashlib
def dblsha(data):
 	return hashlib.sha256(hashlib.sha256(data).digest()).digest()

txnlist = [coinbase] + [binascii.a2b_hex(a['data']) for a in template['transactions']]
merklehashes = [dblsha(t) for t in txnlist]
while len(merklehashes) > 1:
	 if len(merklehashes) % 2:
	  	merklehashes.append(merklehashes[-1])
 	merklehashes = [dblsha(merklehashes[i] + merklehashes[i + 1]) for i in range(0, len(merklehashes), 2)]
merkleroot = merklehashes[0]

Assemble the block header as laid out in the Bitcoin block hashing algorithm, using the data provided in the block template along with your very own merkle root. Note that miners are expected to check the "version" number, and should not create blocks with versions they do not understand unless the server instructs them to do so with the "version/force" or "version/reduce" mutations - you don't need to support those, but if you don't support the version the server has provided, understand that the server may reject submissions if they don't meet some unknown future rules.

Python example:

import struct
blkheader = struct.pack('<I', template['version']) + \
            binascii.a2b_hex(template['previousblockhash']) + \
            merkleroot + \
            struct.pack('<I', template['curtime']) + \
            binascii.a2b_hex(template['bits']) + \
            b'NONC'

Since you're making all the blocks yourself, you really don't ever need to get another template... until it's invalid. Generally, it's a good idea to refresh more often anyway to get more transactions, but it's better to let the pool decide when to do that, since it knows what's changed in the meantime already.

If your template included a "longpollid" key, you can queue a request for a new template to be executed as soon as the pool decides you should change. This is the same as any other template request, except that you include the "longpollid" parameter that the pool provided in your request. If there is already a new template, the pool might respond immediately, so be sure not to depend on a delay!

So to amend the template request we used initially, now we use:

{"id": 0, "method": "getblocktemplate", "params": [{
    "capabilities": ["coinbasetxn", "workid", "coinbase/append"],
    "longpollid": "some gibberish",
}]}

When miner find the job which meets requested difficulty, it can submit the block to the server as a share:

{"id": 0, "method": "submitblock", "params": [
     "020000003c48a294584f90e58325c60ca82896d071826b45680a661cec4d424d00000000
de6433d46c0c7f50d84a05aec77be0199176cdd47f77e344b6f50c84380fddba66dc47501d00ff
ff0000010001010000000100000000000000000000000000000000000000000000000000000000
00000000ffffffff1302955d0f00456c6967697573005047dc66085fffffffff02fff1052a0100
00001976a9144ebeb1cd26d6227635828d60d3e0ed7d0da248fb88ac01000000000000001976a9
147c866aee1fa2f3b3d5effad576df3dbf1f07475588ac00000000"
]}

To assemble the block data, simply concatenate your block header, number of transactions encoded in Bitcoin varint format, followed by each of the transactions in your block (beginning with the coinbase). If the server has listed "submit/coinbase" in its "mutable" key, you may opt to omit the transactions after the coinbase.

Python example:

def varintEncode(n):
  if n < 0xfd:
    return struct.pack('<B', n)
  # NOTE: Technically, there are more encodings for numbers bigger than
  # 16-bit, but transaction counts can't be that high with version 2 Bitcoin
  # blocks
  return b'\xfd' + struct.pack('<H', n)
blkdata = blkheader + varintEncode(len(txnlist)) + coinbase
if 'submit/coinbase' not in template.get('mutable', ()):
  for txn in txnlist[1:]:
    blkdata += txn

Recommended standards to start with (in order of importance):

• BIP 22 (non-optional sections)
• BIP 22 Long Polling
• BIP 23 Basic Pool Extensions
• BIP 23 Mutation "coinbase/append" (required for miner extranonce rolling)
• BIP 23 Submission Abbreviation "submit/coinbase"
• BIP 23 Mutation "time/increment" (be sure to specify "maxtime" or "maxtimeoff"!)