Transactions

Transactions are at the core of the Bitcoin ecosystem. Transactions can be as simple as just sending some bitcoins to a Bitcoin address, or can be quite complex, depending on the requirements. Each transaction is composed of at least one input and output. Inputs can be thought of as coins being spent that have been created in a previous transaction, and outputs as coins being created. If a transaction is minting new coins, then there is no input, and therefore no signature is needed. If a transaction should send coins to some other user (a Bitcoin address), then it needs to be signed by the sender with their private key. In this case, a reference is also required to the previous transaction to show the origin of the coins. Coins are unspent transaction outputs represented in Satoshis.

Transactions are not encrypted and are publicly visible on the blockchain. Blocks are made up of transactions, and these can be viewed using any online blockchain explorer.

The transaction lifecycle

Now, let's look at the lifecycle of a Bitcoin transaction. The steps of the process are as follows:

1. A user/sender sends a transaction using wallet software or some other interface.
2. The wallet software signs the transaction using the sender's private key.
3. The transaction is broadcasted to the Bitcoin network using a flooding algorithm.
4. Mining nodes (miners) who are listening for the transactions verify and include this transaction in the next block to be mined. Just before the transactions are placed in the block, they are placed in a special memory buffer called the transaction pool. The purpose of the transaction pool is explained in the next section.
5. Next, the mining starts, which is the process through which the blockchain is secured and new coins are generated as a reward for the miners who spend appropriate computational resources. Once a miner solves the PoW problem, it broadcasts the newly mined block to the network. PoW is explained in detail in the Mining section. The nodes verify the block and propagate the block further, and confirmations start to generate.
6. Finally, the confirmations start to appear in the receiver's wallet and after approximately three confirmations, the transaction is considered finalized and confirmed. However, three to six is just a recommended number; the transaction can be considered final even after the first confirmation. The key idea behind waiting for six confirmations is that the probability of double spending is virtually eliminated after six confirmations.

When a transaction is created by a user and sent to the network, it ends up in a special area on each Bitcoin software client. This special area is called the transaction pool or memory pool.

Transaction pool

Also known as memory pools, these pools are basically created in local memory (computer RAM) by nodes (Bitcoin clients) in order to maintain a temporary list of transactions that have not yet been added to a block. Miners pick up transactions from these memory pools to create candidate blocks. Miners select transactions from the pool after they pass the verification and validity checks. The selection of which transactions to choose is based on the fee and their place in the order of transactions in the pool. Miners prefer to pick up transactions with higher fees.

To send transactions on the Bitcoin network, the sender needs to pay a fee to the miners. This fee is an incentive mechanism for the miners.

Transaction fees

Transaction fees are charged by the miners. The fee charged is dependent upon the size and weight of the transaction. Transaction fees are calculated by subtracting the sum of the inputs from the sum of the outputs.

A simple formula can be used:

fee = sum(inputs) – sum(outputs)

The fees are used as an incentive for miners to encourage them to include users' transactions in the block the miners are creating. All transactions end up in the memory pool, from where miners pick up transactions based on their priority to include them in the proposed block. The calculation of priority is introduced later in this chapter; however, from a transaction fee point of view, a transaction with a higher fee will be picked up sooner by the miners. There are different rules based on which fee is calculated for various types of actions, such as sending transactions, inclusion in blocks, and relaying by nodes.

Fees are not fixed by the Bitcoin protocol and are not mandatory; even a transaction with no fee will be processed in due course, but may take a very long time. This is, however, no longer practical due to the high volume of transactions and competing investors on the Bitcoin network, therefore it is advisable to always provide a fee. The time for transaction confirmation usually ranges from 10 minutes to over 12 hours in some cases. Transaction time is dependent on transaction fees and network activity. If the network is very busy, then naturally, transactions will take longer to process, and if you pay a higher fee then your transaction is more likely to be picked by miners first due to the additional incentive of the higher fee.

A transaction on the Bitcoin network is represented by a data structure that consists of several fields. We will now introduce the transaction data structure.

The transaction data structure

A transaction at a high level contains metadata, inputs, and outputs. Transactions are combined to create a block's body.

The transaction data structure is shown in the following table:

The following sample transaction is the decoded transaction from the first example of a payment provided at the start of this chapter:

{
   "lock_time":0,
   "size":226,
   "inputs":[
      {
         "prev_out":{
            "index":139,
            "hash":"40120e43f00ff96e098a9173f14f1371655b3478bc0a558d6dc17a4ab176387d"
         },
  "script":"483045022100de6fd8120d9f142a82d5da9389e271caa3a757b01757c8e4fa7afbf92e74257c02202a78d4fbd52ae9f3a0083760d76f84643cf8ab80f5ef971e3f98ccba2c71758d012102c16942555f5e633645895c9affcb994ea7910097b7734a6c2d25468622f25e12"
      }
   ],
   "version":1,
   "vin_sz":1,
   "hash":"d28ca5a59b2239864eac1c96d3fd1c23b747f0ded8f5af0161bae8a616b56a1d",
   "vout_sz":2,
   "out":[
      {
         "script_string":"OP_DUP OP_HASH160 c568ffeb46c6a9362e44a5a49deaa6eab05a619a OP_EQUALVERIFY OP_CHECKSIG",
         "address":"1JzouJCVmMQBmTcd8K4Y5BP36gEFNn1ZJ3",
         "value":33324,
         "script":"76a914c568ffeb46c6a9362e44a5a49deaa6eab05a619a88ac"
      },
      {
         "script_string":"OP_DUP OP_HASH160 9386c8c880488e80a6ce8f186f788f3585f74aee OP_EQUALVERIFY OP_CHECKSIG",
         "address":"1ET3oBGf8JpunjytE7owyVtmBjmvcDycQe",
         "value":93376,
         "script":"76a9149386c8c880488e80a6ce8f186f788f3585f74aee88ac"
      }
   ]
}

As shown in the preceding decoded transaction, there are a number of structures that make up the transaction. All these elements will be described now.

Metadata

This part of the transaction contains values such as the size of the transaction, the number of inputs and outputs, the hash of the transaction, and a lock_time field. Every transaction has a prefix specifying the version number. These fields are shown in the preceding example as lock_time, size, and version.

Inputs

Generally, each input spends a previous output. Each output is considered an Unspent Transaction Output (UTXO) until an input consumes it. A UTXO is an unspent transaction output that can be spent as an input to a new transaction.

The transaction input data structure is explained in the following table:

In the previous sample decoded transaction, the inputs are defined under the "inputs" : [ section.

Outputs

Outputs have three fields, and they contain instructions for sending bitcoins. The first field contains the amount of Satoshis, whereas the second field contains the size of the locking script. Finally, the third field contains a locking script that holds the conditions that need to be met in order for the output to be spent. More information on transaction spending using locking and unlocking scripts and producing outputs is discussed later in this section.

The transaction output data structure is explained in the following table:

In the previous sample decoded transaction, two outputs are shown under the "out":[ section.

Verification

Verification is performed using Bitcoin's scripting language, which we will now describe in the next section in detail.

The Script language

Bitcoin uses a simple stack-based language called Script to describe how bitcoins can be spent and transferred. It is not Turing complete and has no loops to avoid any undesirable effects of long-running/hung scripts on the Bitcoin network. This scripting language is based on a Forth programming language-like syntax and uses a reverse polish notation in which every operand is followed by its operators. It is evaluated from left to right using a Last in, First Out (LIFO) stack.

Scripts are composed of two components, namely elements and operations. Scripts use various operations (opcodes) or instructions to define their operations. Elements simply represent data such as digital signatures. Opcodes are also known as words, commands, or functions. Earlier versions of the Bitcoin node software had a few opcodes that are no longer used due to bugs discovered in their design.

The various categories of the scripting opcodes are constants, flow control, stack, bitwise logic, splice, arithmetic, cryptography, and lock time.

A transaction script is evaluated by combining ScriptSig and ScriptPubKey. ScriptSig is the unlocking script, whereas ScriptPubKey is the locking script. We will now describe how a transaction is unlocked and spent:

• ScriptSig is provided by the user who wishes to unlock the transaction
• ScriptPubKey is part of the transaction output and specifies the conditions that need to be fulfilled in order to spend the output

In other words, outputs are locked by the ScriptPubKey (the locking script), which contains the conditions that when met, will unlock the output, and the coins can then be redeemed. We will see the script execution in detail shortly.

Commonly used opcodes

In a computer, an opcode is an instruction to perform some operation. For example, ADD is an opcode, which is used for integer addition in Intel CPUs and various other architectures. Similarly, in Bitcoin design, opcodes are introduced that perform several operations related to Bitcoin transaction verification.

A description of some of the most commonly used opcodes is listed in the following table, extracted from the Bitcoin Developer's Guide:

There are many opcodes in the Bitcoin scripting language, and covering all of them is out of scope of this book. Interested readers can refer to the complete list and relevant details at https://en.bitcoin.it/wiki/Script#Opcodes.

Now that we've covered the transaction life cycle and data structure, let's move on to talk about the types of scripts used to undertake these transactions.

Types of scripts

There are various standard scripts available in Bitcoin to handle the verification and value transfer from the source to the destination. These scripts range from very simple to quite complex depending upon the requirements of the transaction. Standard transaction types are discussed here. Standard transactions are evaluated using the IsStandard() and IsStandardTx() tests and only those transactions that pass the test are allowed to be broadcasted or mined on the Bitcoin network. However, nonstandard transactions are also allowed on the network, as long as they pass the validity checks:

• Pay-to-Public-Key Hash (P2PKH): P2PKH is the most commonly used transaction type and is used to send transactions to Bitcoin addresses. The format of this type of transaction is shown as follows:

  ScriptPubKey: OP_DUP OP_HASH160 <pubKeyHash> OP_EQUALVERIFY OP_CHECKSIG
  ScriptSig: <sig> <pubKey>
  

  The ScriptPubKey and ScriptSig parameters are concatenated together and executed. An example will follow shortly in this section, where this is explained in more detail.

• Pay-to-Script Hash (P2SH): P2SH is used in order to send transactions to a script hash (that is, addresses starting with 3) and was standardized in BIP16. In addition to passing the script, the redeem script is also evaluated and must be valid. The template is shown as follows:

  ScriptPubKey: OP_HASH160 <redeemScriptHash> OP_EQUAL
  ScriptSig: [<sig>…<sign>] <redeemScript>
  

• MultiSig (Pay to MultiSig): The M of N multisignature transaction script is a complex type of script where it is possible to construct a script that requires multiple signatures to be valid in order to redeem a transaction. Various complex transactions such as escrow and deposits can be built using this script. The template is shown here:

  ScriptPubKey: <m> <pubKey> [<pubKey> . . . ] <n> OP_CHECKMULTISIG
  ScriptSig: 0 [<sig > . . . <sign>]
  

  Raw multisig is obsolete, and multisig is now usually part of the P2SH redeem script, mentioned in the previous bullet point.

• Pay to Pubkey: This script is a very simple script that is commonly used in coinbase transactions. It is now obsolete and was used in an old version of Bitcoin. The public key is stored within the script in this case, and the unlocking script is required to sign the transaction with the private key. The template is shown as follows:

  <PubKey> OP_CHECKSIG
  

• Null data/OP_RETURN: This script is used to store arbitrary data on the blockchain for a fee. The limit of the message is 40 bytes. The output of this script is unredeemable because OP_RETURN will fail the validation in any case. ScriptSig is not required in this case. The template is very simple and is shown as follows:

  OP_RETURN <data>
  

The P2PKH script execution is shown in the following diagram:

Figure 6.15: P2PKH script execution

In the preceding diagram, we have a standard P2PKH script presented on top, which shows both the unlocking (ScriptSig) and locking (ScriptPubKey) parts of the script.

Recall that in the introduction of the Script language; we mentioned that a Bitcoin script is composed of two parts, an unlocking script and a locking script. The unlocking script is composed of <sig> and <pubkey> elements, which are part of all transaction inputs. The unlocking script satisfies the conditions that are required to consume the output. The locking script defines the conditions that are required to be met to spend the bitcoins. Transactions are authorized by executing both of these parts collectively.

Now focusing again on the previous diagram, we see that in the middle we have a visualization of the stack where the data elements are pushed and popped from. In the bottom section, we show the execution of the script. This diagram shows the step-by-step execution of the script with its outcome on the stack.

Now let's examine how this script is executed:

1. In the first step of the data elements, <sig> and <pubkey> are placed on the stack.
2. The stack item at the top <pubkey> is duplicated due to the OP_DUP instruction, which duplicates the top stack item.
3. After this, the instruction OP_HASH160 executes, which produces the hash of <pubkey>, which is the top element in the stack.
4. <pubkeyhash> is then pushed on to the stack. At this stage, we have two hashes on the stack: the one that is produced as a result of executing OP_HASH160 on <pubkey> from the unlocking script, and the other one provided by the locking script.
5. Now the OP_EQUALVERIFY opcode instruction executes and checks that whether the top two elements (that is, the hashes) are equal or not. If they are equal, the script continues, otherwise it fails.
6. Finally, OP_CHECKSIG executes to check the validity of the signatures of the top two elements of the stack. If the signature is valid then the stack will contain the value True that is, 1, otherwise False , that is, 0.

All transactions are encoded into hex format before being transmitted over the Bitcoin network. The following sample transaction is shown in hex format, and is retrieved using bitcoin-cli on the Bitcoin node running on mainnet by using the following command:

$ bitcoin-cli getrawtransaction "d28ca5a59b2239864eac1c96d3fd1c23b747f0ded8f5af0161bae8a616b56a1d"
{
  "result":"01000000017d3876b14a7ac16d8d550abc78345b6571134ff173918a096ef90ff043 0e12408b0000006b483045022100de6fd8120d9f142a82d5da9389e271caa3a757b01 757c8e4fa7afbf92e74257c02202a78d4fbd52ae9f3a0083760d76f84643cf8ab80f5 ef971e3f98ccba2c71758d012102c16942555f5e633645895c9affcb994ea7910097b 7734a6c2d25468622f25e12ffffffff022c820000000000001976a914c568ffeb46c6 a9362e44a5a49deaa6eab05a619a88acc06c0100000000001976a9149386c8c880488 e80a6ce8f186f788f3585f74aee88ac00000000", "error": null, "id": null }

Scripting is quite limited and can only be used to program one thing—the transfer of bitcoins from one address to other addresses. However, there is some flexibility possible when creating these scripts, which allows for certain conditions to be put on the spending of the bitcoins. This set of conditions can be considered a basic form of a financial contract. In other words, we can say that there is a basic support for building contracts in Bitcoin. However, it is not same as smart contracts, which allow the writing of arbitrary programs on the blockchain.

We will discuss smart contracts in detail later in Chapter 10, Smart Contracts, but for now, we will introduce the concept of contracts in Bitcoin.

Contracts

Contracts are Bitcoin scripts that use the Bitcoin blockchain to enforce a financial agreement. This is a simple definition but has far-reaching consequences as it allows users to programmatically create complex contracts that can be used in many real-world scenarios. Contracts allow the development of completely decentralized, independent, and reduced-risk platforms by programmatically enforcing different conditions for unlocking the money (bitcoins). With the security guarantees provided by the Bitcoin blockchain, it is almost impossible to circumvent these conditions.

Various contracts, such as escrow, arbitration, and micropayment channels, can be built using the Bitcoin scripting language. The current implementation of the script language is minimal, but it is still possible to develop various types of complex contracts. For example, we could set the release of funds to be allowed only when multiple parties sign the transaction, or release the funds only after a specific amount of time has elapsed. Both of these scenarios can be realized using the multisig and transaction lock time options.

Even though the scripting language in Bitcoin can be used to create complex contracts, it is quite limited and is not at all on par with the smart contracts, which are Turing complete constructs and allow arbitrary program development.

Recently, however, there has some more advancement in this area. A new smart contract language for Bitcoin has been announced, called Miniscript, which will be discussed next.

Miniscript

This language allows for a more structured approach to writing Bitcoin scripts. Even though Bitcoin Script supports combinations of different spending conditions, such as time and hash locks, it is not easy to analyze existing scripts or build new complex scripts. Miniscript makes it easier to write complex spending rules. It also makes it easier to ascertain the correctness of the script.

Currently, Miniscript supports P2WSH and P2SH-P2WSH, and offers limited support for P2SH.

Coinbase transactions

A coinbase transaction or generation transaction is always created by a miner and is the first transaction in a block. It is used to create new coins. It includes a special field, also called the coinbase, which acts as an input to the coinbase transaction. This transaction also allows up to 100 bytes of arbitrary data storage.

A coinbase transaction input has the same number of fields as a usual transaction input, but the structure contains the coinbase data size and fields instead of the unlocking script size and fields. Also, it does not have a reference pointer to the previous transaction. This structure is shown in the following table:

All transactions in the Bitcoin system go under a validation mechanism. We introduce how Bitcoin transactions are validated in the next section.

Transaction validation

This verification process is performed by Bitcoin nodes. There are three main things that nodes check when verifying a transaction:

1. That transaction inputs are previously unspent. This validation step prevents double spending by verifying that the transaction inputs have not already been spent by someone else.
2. That the sum of the transaction outputs is not more than the total sum of the transaction inputs. However, both input and output sums can be the same, or the sum of the input (total value) could be more than the total value of the outputs. This check ensures that no new bitcoins are created out of thin air.
3. That the digital signatures are valid, which ensures that the script is valid.

Even though transaction construction and validation are generally a secure and sound process, some vulnerabilities exist in Bitcoin. We will now introduce some Bitcoin's infamous shortcomings.

Transaction bugs

The following are two major Bitcoin vulnerabilities that have been infamously exploited.

Transaction malleability

Transaction malleability is a Bitcoin attack that was introduced due to a bug in the Bitcoin implementation. Due to this bug, it became possible for an adversary to change the transaction ID of a transaction, thus resulting in a scenario where it would appear that a certain transaction has not been executed.

This can allow scenarios where double deposits or withdrawals can occur. In other words, this bug allows the changing of the unique ID of a Bitcoin transaction before it is confirmed. If the ID is changed before confirmation without making the transaction invalid, it would seem that the transaction did not occur at all, which can then give the false impression that the transaction has not been executed, thus allowing double-deposit or withdrawal attacks.

Value overflow

This incident is one of the most well-known events in Bitcoin history. On 15 August 2010, a transaction was discovered that created roughly 184 billion bitcoins. This problem occurred due to the integer overflow bug where the amount field in the Bitcoin code was defined as a signed integer instead of an unsigned integer. This bug means that the amount can also be negative, and resulted in a situation where the outputs were so large that the total value resulted in an overflow. To the validation logic in Bitcoin code, all appeared to be correct, and it looked like the fee was also positive (after the overflow). This bug was fixed via a soft fork (more on this in the Blockchain section) quickly after its discovery.

Now that we understand the fundamentals of how Bitcoin transactions work, and we appreciate some of the security-related aspects of Bitcoin, let's now look at the Bitcoin blockchain, which is the foundation of the Bitcoin cryptocurrency.