Ethash | ethereum.org (2024)

Ethash(opens in a new tab) is a modified version of the Dagger-Hashimoto algorithm. Ethash proof-of-work is memory hard(opens in a new tab), which was thought to make the algorithm ASIC resistant. Ethash ASICs were eventually developed but GPU mining was still a viable option until proof-of-work was switched off. Ethash is still used to mine other coins on other non-Ethereum proof-of-work networks.

Memory hardness is achieved with a proof of work algorithm that requires choosing subsets of a fixed resource dependent on the nonce and block header. This resource (a few gigabytes in size) is called a DAG. The DAG is changed every 30000 blocks, a ~125-hour window called an epoch (roughly 5.2 days) and takes a while to generate. Since the DAG only depends on block height, it can be pre-generated, but if it's not the client needs to wait until the end of this process to produce a block. If clients do not pre-generate and cache DAGs ahead of time the network may experience massive block delay on each epoch transition. Note that the DAG does not need to be generated for verifying the proof-of-work essentially allowing for verification with both low CPU and small memory.

The large dataset is updated once every 30000 blocks, so the vast majority of a miner's effort will be reading the dataset, not making changes to it.

WORD_BYTES = 4 # bytes in word

2DATASET_BYTES_INIT = 2**30 # bytes in dataset at genesis

3DATASET_BYTES_GROWTH = 2**23 # dataset growth per epoch

4CACHE_BYTES_INIT = 2**24 # bytes in cache at genesis

5CACHE_BYTES_GROWTH = 2**17 # cache growth per epoch

6CACHE_MULTIPLIER=1024 # Size of the DAG relative to the cache

7EPOCH_LENGTH = 30000 # blocks per epoch

8MIX_BYTES = 128 # width of mix

9HASH_BYTES = 64 # hash length in bytes

10DATASET_PARENTS = 256 # number of parents of each dataset element

11CACHE_ROUNDS = 3 # number of rounds in cache production

12ACCESSES = 64 # number of accesses in hashimoto loop

Show all

Ethereum's development coincided with the development of the SHA3 standard, and thestandards process made a late change in the padding of the finalized hash algorithm, so that Ethereum's"sha3_256" and "sha3_512" hashes are not standard sha3 hashes, but a variant often referredto as "Keccak-256" and "Keccak-512" in other contexts. See discussion, e.g. here(opens in a new tab), here(opens in a new tab), or here(opens in a new tab).

Please keep that in mind as "sha3" hashes are referred to in the description of the algorithm below.

The parameters for Ethash's cache and dataset depend on the block number. The cache size and dataset size both grow linearly; however, we always take the highest prime below the linearly growing threshold in order to reduce the risk of accidental regularities leading to cyclic behavior.

1def get_cache_size(block_number):
2 sz = CACHE_BYTES_INIT + CACHE_BYTES_GROWTH * (block_number // EPOCH_LENGTH)
3 sz -= HASH_BYTES
4 while not isprime(sz / HASH_BYTES):
5 sz -= 2 * HASH_BYTES
6 return sz
7
8def get_full_size(block_number):
9 sz = DATASET_BYTES_INIT + DATASET_BYTES_GROWTH * (block_number // EPOCH_LENGTH)
10 sz -= MIX_BYTES
11 while not isprime(sz / MIX_BYTES):
12 sz -= 2 * MIX_BYTES
13 return sz
Show all
 Copy

Tables of dataset and cache size values are provided in the appendix.

1def mkcache(cache_size, seed):
2 n = cache_size // HASH_BYTES
3
4 # Sequentially produce the initial dataset
5 o = [sha3_512(seed)]
6 for i in range(1, n):
7 o.append(sha3_512(o[-1]))
8
9 # Use a low-round version of randmemohash
10 for _ in range(CACHE_ROUNDS):
11 for i in range(n):
12 v = o[i][0] % n
13 o[i] = sha3_512(map(xor, o[(i-1+n) % n], o[v]))
14
15 return o
Show all
 Copy

The cache production process involves first sequentially filling up 32 MB of memory, then performing two passes of Sergio Demian Lerner's RandMemoHash algorithm from Strict Memory Hard Hashing Functions (2014)(opens in a new tab). The output is a set of 524288 64-byte values.

We use an algorithm inspired by the FNV hash(opens in a new tab) in some cases as a non-associative substitute for XOR. Note that we multiply the prime with the full 32-bit input, in contrast with the FNV-1 spec which multiplies the prime with one byte (octet) in turn.

1FNV_PRIME = 0x01000193
2
3def fnv(v1, v2):
4 return ((v1 * FNV_PRIME) ^ v2) % 2**32
 Copy

Please note, even the yellow paper specifies fnv as v1*(FNV_PRIME ^ v2), all current implementations consistently use the above definition.

1def calc_dataset_item(cache, i):
2 n = len(cache)
3 r = HASH_BYTES // WORD_BYTES
4 # initialize the mix
5 mix = copy.copy(cache[i % n])
6 mix[0] ^= i
7 mix = sha3_512(mix)
8 # fnv it with a lot of random cache nodes based on i
9 for j in range(DATASET_PARENTS):
10 cache_index = fnv(i ^ j, mix[j % r])
11 mix = map(fnv, mix, cache[cache_index % n])
12 return sha3_512(mix)
Show all
 Copy

Essentially, we combine data from 256 pseudorandomly selected cache nodes, and hash that to compute the dataset node. The entire dataset is then generated by:

1def calc_dataset(full_size, cache):
2 return [calc_dataset_item(cache, i) for i in range(full_size // HASH_BYTES)]
 Copy

Now, we specify the main "hashimoto"-like loop, where we aggregate data from the full dataset in order to produce our final value for a particular header and nonce. In the code below, header represents the SHA3-256 hash of the RLP representation of a truncated block header, that is, of a header excluding the fields mixHash and nonce. nonce is the eight bytes of a 64 bit unsigned integer in big-endian order. So nonce[::-1] is the eight-byte little-endian representation of that value:

1def hashimoto(header, nonce, full_size, dataset_lookup):
2 n = full_size / HASH_BYTES
3 w = MIX_BYTES // WORD_BYTES
4 mixhashes = MIX_BYTES / HASH_BYTES
5 # combine header+nonce into a 64 byte seed
6 s = sha3_512(header + nonce[::-1])
7 # start the mix with replicated s
8 mix = []
9 for _ in range(MIX_BYTES / HASH_BYTES):
10 mix.extend(s)
11 # mix in random dataset nodes
12 for i in range(ACCESSES):
13 p = fnv(i ^ s[0], mix[i % w]) % (n // mixhashes) * mixhashes
14 newdata = []
15 for j in range(MIX_BYTES / HASH_BYTES):
16 newdata.extend(dataset_lookup(p + j))
17 mix = map(fnv, mix, newdata)
18 # compress mix
19 cmix = []
20 for i in range(0, len(mix), 4):
21 cmix.append(fnv(fnv(fnv(mix[i], mix[i+1]), mix[i+2]), mix[i+3]))
22 return {
23 "mix digest": serialize_hash(cmix),
24 "result": serialize_hash(sha3_256(s+cmix))
25 }
26
27def hashimoto_light(full_size, cache, header, nonce):
28 return hashimoto(header, nonce, full_size, lambda x: calc_dataset_item(cache, x))
29
30def hashimoto_full(full_size, dataset, header, nonce):
31 return hashimoto(header, nonce, full_size, lambda x: dataset[x])
Show all
 Copy

Essentially, we maintain a "mix" 128 bytes wide, and repeatedly sequentially fetch 128 bytes from the full dataset and use the fnv function to combine it with the mix. 128 bytes of sequential access are used so that each round of the algorithm always fetches a full page from RAM, minimizing translation lookaside buffer misses which ASICs would theoretically be able to avoid.

If the output of this algorithm is below the desired target, then the nonce is valid. Note that the extra application of sha3_256 at the end ensures that there exists an intermediate nonce which can be provided to prove that at least a small amount of work was done; this quick outer PoW verification can be used for anti-DDoS purposes. It also serves to provide statistical assurance that the result is an unbiased, 256-bit number.

1def mine(full_size, dataset, header, difficulty):
2 # zero-pad target to compare with hash on the same digit
3 target = zpad(encode_int(2**256 // difficulty), 64)[::-1]
4 from random import randint
5 nonce = randint(0, 2**64)
6 while hashimoto_full(full_size, dataset, header, nonce) > target:
7 nonce = (nonce + 1) % 2**64
8 return nonce
 Copy

In order to compute the seed hash that would be used to mine on top of a given block, we use the following algorithm:

1 def get_seedhash(block):
2 s = '\x00' * 32
3 for i in range(block.number // EPOCH_LENGTH):
4 s = serialize_hash(sha3_256(s))
5 return s
 Copy

Note that for smooth mining and verifying, we recommend pre-computing future seedhashes and datasets in a separate thread.

The following code should be prepended if you are interested in running the above python spec as code.

1import sha3, copy
2
3# Assumes little endian bit ordering (same as Intel architectures)
4def decode_int(s):
5 return int(s[::-1].encode('hex'), 16) if s else 0
6
7def encode_int(s):
8 a = "%x" % s
9 return '' if s == 0 else ('0' * (len(a) % 2) + a).decode('hex')[::-1]
10
11def zpad(s, length):
12 return s + '\x00' * max(0, length - len(s))
13
14def serialize_hash(h):
15 return ''.join([zpad(encode_int(x), 4) for x in h])
16
17def deserialize_hash(h):
18 return [decode_int(h[i:i+WORD_BYTES]) for i in range(0, len(h), WORD_BYTES)]
19
20def hash_words(h, sz, x):
21 if isinstance(x, list):
22 x = serialize_hash(x)
23 y = h(x)
24 return deserialize_hash(y)
25
26def serialize_cache(ds):
27 return ''.join([serialize_hash(h) for h in ds])
28
29serialize_dataset = serialize_cache
30
31# sha3 hash function, outputs 64 bytes
32def sha3_512(x):
33 return hash_words(lambda v: sha3.sha3_512(v).digest(), 64, x)
34
35def sha3_256(x):
36 return hash_words(lambda v: sha3.sha3_256(v).digest(), 32, x)
37
38def xor(a, b):
39 return a ^ b
40
41def isprime(x):
42 for i in range(2, int(x**0.5)):
43 if x % i == 0:
44 return False
45 return True
Show all
 Copy

The following lookup tables provide approximately 2048 tabulated epochs of data sizes and cache sizes.

1def get_datasize(block_number):

2 return data_sizes[block_number // EPOCH_LENGTH]

4def get_cachesize(block_number):

5 return cache_sizes[block_number // EPOCH_LENGTH]

7data_sizes = [

81073739904, 1082130304, 1090514816, 1098906752, 1107293056,

91115684224, 1124070016, 1132461952, 1140849536, 1149232768,

101157627776, 1166013824, 1174404736, 1182786944, 1191180416,

111199568512, 1207958912, 1216345216, 1224732032, 1233124736,

121241513344, 1249902464, 1258290304, 1266673792, 1275067264,

131283453312, 1291844992, 1300234112, 1308619904, 1317010048,

141325397376, 1333787776, 1342176128, 1350561664, 1358954368,