Transaction Confirmation & Expiration | Solana (2024)

Problems relating totransaction confirmation arecommon with many newer developers while building applications. This article aimsto boost the overall understanding of the confirmation mechanism used on theSolana blockchain, including some recommended best practices.

Brief background on transactions #

Before diving into how Solana transaction confirmation and expiration works,let's briefly set the base understanding of a few things:

• what a transaction is
• the lifecycle of a transaction
• what a blockhash is
• and a brief understanding of Proof of History (PoH) and how it relates toblockhashes

What is a transaction? #

Transactions consist of two components: amessage and alist of signatures. The transaction message iswhere the magic happens and at a high level it consists of four components:

• a header with metadata about the transaction,
• a list of instructions to invoke,
• a list of accounts to load, and
• a “recent blockhash.”

In this article, we’re going to be focusing a lot on a transaction’srecent blockhash because it plays a big rolein transaction confirmation.

Transaction lifecycle refresher #

Below is a high level view of the lifecycle of a transaction. This article willtouch on everything except steps 1 and 4.

1. Create a header and a list of instructions along with the list of accountsthat instructions need to read and write
2. Fetch a recent blockhash and use it to prepare a transaction message
3. Simulate the transaction to ensure it behaves as expected
4. Prompt user to sign the prepared transaction message with their private key
5. Send the transaction to an RPC node which attempts to forward it to thecurrent block producer
6. Hope that a block producer validates and commits the transaction into theirproduced block
7. Confirm the transaction has either been included in a block or detect when ithas expired

What is a Blockhash? #

A “blockhash” refers to the last Proof ofHistory (PoH) hash for a “slot” (descriptionbelow). Since Solana uses PoH as a trusted clock, a transaction’s recentblockhash can be thought of as a timestamp.

Proof of History refresher #

Solana’s Proof of History mechanism uses a very long chain of recursive SHA-256hashes to build a trusted clock. The “history” part of the name comes from thefact that block producers hash transaction id’s into the stream to record whichtransactions were processed in their block.

PoH hash calculation:next_hash = hash(prev_hash, hash(transaction_ids))

PoH can be used as a trusted clock because each hash must be producedsequentially. Each produced block contains a blockhash and a list of hashcheckpoints called “ticks” so that validators can verify the full chain ofhashes in parallel and prove that some amount of time has actually passed.

Transaction Expiration #

By default, all Solana transactions will expire if not committed to a block in acertain amount of time. The vast majority of transaction confirmation issuesare related to how RPC nodes and validators detect and handle expiredtransactions. A solid understanding of how transaction expiration works shouldhelp you diagnose the bulk of your transaction confirmation issues.

How does transaction expiration work? #

Each transaction includes a “recent blockhash” which is used as a PoH clocktimestamp and expires when that blockhash is no longer “recent enough”.

As each block is finalized (i.e. the maximum tick heightis reached,reaching the "block boundary"), the final hash of the block is added to theBlockhashQueue which stores a maximum of the300 most recent blockhashes.During transaction processing, Solana Validators will check if eachtransaction's recent blockhash is recorded within the most recent 151 storedhashes (aka "max processing age"). If the transaction's recent blockhash isolder than thismax processing age, the transaction is not processed.

Since slots (aka the time period a validator canproduce a block) are configured to last about400ms,but may fluctuate between 400ms and 600ms, a given blockhash can only be used bytransactions for about 60 to 90 seconds before it will be considered expired bythe runtime.

Example of transaction expiration #

Let’s walk through a quick example:

1. A validator is actively producing a new block for the current slot
2. The validator receives a transaction from a user with the recent blockhashabcd...
3. The validator checks this blockhash abcd... against the list of recentblockhashes in the BlockhashQueue and discovers that it was created 151blocks ago
4. Since it is exactly 151 blockhashes old, the transaction has not expired yetand can still be processed!
5. But wait: before actually processing the transaction, the validator finishedcreating the next block and added it to the BlockhashQueue. The validatorthen starts producing the block for the next slot (validators get to produceblocks for 4 consecutive slots)
6. The validator checks that same transaction again and finds it is now 152blockhashes old and rejects it because it’s too old :(

Why do transactions expire? #

There’s a very good reason for this actually, it’s to help validators avoidprocessing the same transaction twice.

A naive brute force approach to prevent double processing could be to checkevery new transaction against the blockchain’s entire transaction history. Butby having transactions expire after a short amount of time, validators only needto check if a new transaction is in a relatively small set of recentlyprocessed transactions.

Other blockchains #

Solana’s approach of prevent double processing is quite different from otherblockchains. For example, Ethereum tracks a counter (nonce) for each transactionsender and will only process transactions that use the next valid nonce.

Ethereum’s approach is simple for validators to implement, but it can beproblematic for users. Many people have encountered situations when theirEthereum transactions got stuck in a pending state for a long time and all thelater transactions, which used higher nonce values, were blocked fromprocessing.

Advantages on Solana #

There are a few advantages to Solana’s approach:

1. A single fee payer can submit multiple transactions at the same time that areallowed to be processed in any order. This might happen if you’re usingmultiple applications at the same time.
2. If a transaction doesn’t get committed to a block and expires, users can tryagain knowing that their previous transaction will NOT ever be processed.

By not using counters, the Solana wallet experience may be easier for users tounderstand because they can get to success, failure, or expiration statesquickly and avoid annoying pending states.

Disadvantages on Solana #

Of course there are some disadvantages too:

1. Validators have to actively track a set of all processed transaction id’s toprevent double processing.
2. If the expiration time period is too short, users might not be able to submittheir transaction before it expires.

These disadvantages highlight a tradeoff in how transaction expiration isconfigured. If the expiration time of a transaction is increased, validatorsneed to use more memory to track more transactions. If expiration time isdecreased, users don’t have enough time to submit their transaction.

Currently, Solana clusters require that transactions use blockhashes that are nomore than 151 blocks old.

Transaction confirmation tips #

As mentioned before, blockhashes expire after a time period of only 151 blockswhich can pass as quickly as one minute when slots are processed within thetarget time of 400ms.

One minute is not a lot of time considering that a client needs to fetch arecent blockhash, wait for the user to sign, and finally hope that thebroadcasted transaction reaches a leader that is willing to accept it. Let’s gothrough some tips to help avoid confirmation failures due to transactionexpiration!

Fetch blockhashes with the appropriate commitment level #

Given the short expiration time frame, it’s imperative that clients andapplications help users create transactions with a blockhash that is as recentas possible.

When fetching blockhashes, the current recommended RPC API is calledgetLatestBlockhash. By default, thisAPI uses the finalized commitment level to return the most recently finalizedblock’s blockhash. However, you can override this behavior bysetting the commitment parameterto a different commitment level.

Recommendation

The confirmed commitment level should almost always be used for RPC requestsbecause it’s usually only a few slots behind the processed commitment and hasa very low chance of belonging to a droppedfork.

But feel free to consider the other options:

• Choosing processed will let you fetch the most recent blockhash compared toother commitment levels and therefore gives you the most time to prepare andprocess a transaction. But due to the prevalence of forking in the Solanablockchain, roughly 5% of blocks don’t end up being finalized by the clusterso there’s a real chance that your transaction uses a blockhash that belongsto a dropped fork. Transactions that use blockhashes for abandoned blockswon’t ever be considered recent by any blocks that are in the finalizedblockchain.
• Using the default commitment level finalizedwill eliminate any risk that the blockhash you choose will belong to a droppedfork. The tradeoff is that there is typically at least a 32 slot differencebetween the most recent confirmed block and the most recent finalized block.This tradeoff is pretty severe and effectively reduces the expiration of yourtransactions by about 13 seconds but this could be even more during unstablecluster conditions.

Use an appropriate preflight commitment level #

If your transaction uses a blockhash that was fetched from one RPC node then yousend, or simulate, that transaction with a different RPC node, you could runinto issues due to one node lagging behind the other.

When RPC nodes receive a sendTransaction request, they will attempt todetermine the expiration block of your transaction using the most recentfinalized block or with the block selected by the preflightCommitmentparameter. A VERY common issue is that a received transaction’s blockhashwas produced after the block used to calculate the expiration for thattransaction. If an RPC node can’t determine when your transaction expires, itwill only forward your transaction one time and afterwards will thendrop the transaction.

Similarly, when RPC nodes receive a simulateTransaction request, they willsimulate your transaction using the most recent finalized block or with theblock selected by the preflightCommitment parameter. If the block chosen forsimulation is older than the block used for your transaction’s blockhash, thesimulation will fail with the dreaded “blockhash not found” error.

Recommendation

Even if you use skipPreflight, ALWAYS set the preflightCommitmentparameter to the same commitment level used to fetch your transaction’sblockhash for both sendTransaction and simulateTransaction requests.

Be wary of lagging RPC nodes when sending transactions #

When your application uses an RPC pool service or when the RPC endpoint differsbetween creating a transaction and sending a transaction, you need to be wary ofsituations where one RPC node is lagging behind the other. For example, if youfetch a transaction blockhash from one RPC node then you send that transactionto a second RPC node for forwarding or simulation, the second RPC node might belagging behind the first.

Recommendation

For sendTransaction requests, clients should keep resending a transaction to aRPC node on a frequent interval so that if an RPC node is slightly laggingbehind the cluster, it will eventually catch up and detect your transaction’sexpiration properly.

For simulateTransaction requests, clients should use thereplaceRecentBlockhash parameter totell the RPC node to replace the simulated transaction’s blockhash with ablockhash that will always be valid for simulation.

Avoid reusing stale blockhashes #

Even if your application has fetched a very recent blockhash, be sure thatyou’re not reusing that blockhash in transactions for too long. The idealscenario is that a recent blockhash is fetched right before a user signs theirtransaction.

Recommendation for applications

Poll for new recent blockhashes on a frequent basis to ensure that whenever auser triggers an action that creates a transaction, your application already hasa fresh blockhash that’s ready to go.

Recommendation for wallets

Poll for new recent blockhashes on a frequent basis and replace a transaction’srecent blockhash right before they sign the transaction to ensure the blockhashis as fresh as possible.

Use healthy RPC nodes when fetching blockhashes #

By fetching the latest blockhash with the confirmed commitment level from anRPC node, it’s going to respond with the blockhash for the latest confirmedblock that it’s aware of. Solana’s block propagation protocol prioritizessending blocks to staked nodes so RPC nodes naturally lag about a block behindthe rest of the cluster. They also have to do more work to handle applicationrequests and can lag a lot more under heavy user traffic.

Lagging RPC nodes can therefore respond togetLatestBlockhash requests withblockhashes that were confirmed by the cluster quite awhile ago. By default, alagging RPC node detects that it is more than 150 slots behind the cluster willstop responding to requests, but just before hitting that threshold they canstill return a blockhash that is just about to expire.

Recommendation

Monitor the health of your RPC nodes to ensure that they have an up-to-date viewof the cluster state with one of the following methods:

1. Fetch your RPC node’s highest processed slot by using thegetSlot RPC API with the processedcommitment level and then call the`getMaxShredInsertSlot RPC API toget the highest slot that your RPC node has received a “shred” of a blockfor. If the difference between these responses is very large, the cluster isproducing blocks far ahead of what the RPC node has processed.
2. Call the getLatestBlockhash RPC API with the confirmed commitment levelon a few different RPC API nodes and use the blockhash from the node thatreturns the highest slot for itscontext slot.

Wait long enough for expiration #

Recommendation

When calling the getLatestBlockhashRPC API to get a recent blockhash for your transaction, take note of thelastValidBlockHeight in the response.

Then, poll the getBlockHeight RPC APIwith the confirmed commitment level until it returns a block height greaterthan the previously returned last valid block height.

Consider using “durable” transactions #

Sometimes transaction expiration issues are really hard to avoid (e.g. offlinesigning, cluster instability). If the previous tips are still not sufficient foryour use-case, you can switch to using durable transactions (they just require abit of setup).

To start using durable transactions, a user first needs to submit a transactionthatinvokes instructions that create a special on-chain “nonce” accountand stores a “durable blockhash” inside of it. At any point in the future (aslong as the nonce account hasn’t been used yet), the user can create a durabletransaction by following these 2 rules:

1. The instruction list must start with an“advance nonce” system instructionwhich loads their on-chain nonce account
2. The transaction’s blockhash must be equal to the durable blockhash stored bythe on-chain nonce account

Here’s how these durable transactions are processed by the Solana runtime:

1. If the transaction’s blockhash is no longer “recent”, the runtime checks ifthe transaction’s instruction list begins with an “advance nonce” systeminstruction
2. If so, it then loads the nonce account specified by the “advance nonce”instruction
3. Then it checks that the stored durable blockhash matches the transaction’sblockhash
4. Lastly it makes sure to advance the nonce account’s stored blockhash to thelatest recent blockhash to ensure that the same transaction can never beprocessed again

For more details about how these durable transactions work, you can read theoriginal proposalandcheck out an examplein the Solana docs.