From: thepipeline_xyz
The development of high-performance blockchain technology, particularly in the EVM (Ethereum Virtual Machine) ecosystem, has been significantly influenced by concepts and engineering principles derived from high-frequency trading (HFT) systems [00:05:55]. This article explores the challenges inherent in scaling blockchain performance and the advancements proposed by projects like Monad Labs, drawing parallels with the demanding world of HFT.
High-Frequency Trading (HFT) and Performance Optimization
Keone and James, founders of Monad Labs, gained extensive experience building full trading systems for high-frequency trading at Jump Trading, later transitioning to Jump Crypto [00:00:09]. This field is characterized by extreme competitiveness and a constant drive for minimal latency [00:06:00].
In HFT, the goal is to process market data, make rapid trading decisions, and send orders back to exchanges faster than competitors [00:05:23]. Even nanoseconds of delay can determine success or failure in securing a trade [00:05:47]. This environment necessitates building highly performant systems from scratch, constantly optimizing and shaving off latency [00:06:03].
Key aspects of HFT and their relevance to blockchain include:
- Extreme Latency Competition: Small differences in processing speed dictate trade outcomes [00:05:40].
- Optimization at the Core: A constant focus on improving system performance, often involving the creation of custom data structures and avoiding general-purpose libraries [00:13:51].
- Massive Volume with Thin Margins: Daily notional volumes can reach tens of billions of dollars, with profit margins often less than a basis point per trade [00:25:43]. This requires systems capable of handling tens of millions of orders daily across hundreds of instruments [00:27:56].
- Deep Hardware Understanding: Optimizing performance involves a quantitative approach, running experiments, and understanding how code translates to assembly instructions at a low level to maximize hardware capabilities [00:30:42].
This background in Highfrequency trading systems and performance optimization provided a unique foundation for identifying and tackling performance bottlenecks in blockchain technology [00:06:55].
Blockchain Performance Challenges
Upon entering the crypto space, particularly working with Solana DeFi, it became evident that there was a significant need for more performant blockchains, especially within the EVM ecosystem, which was found to be very inefficient and unoptimized compared to HFT systems [00:06:40].
The Parallel EVM and Its Limitations
While parallel EVM has become a major narrative in crypto, Monad Labs’ early implementation of an optimistic parallel execution algorithm over one and a half years ago revealed that it alone offered limited improvement over serial execution [01:13:00]. This is because the core bottleneck lies elsewhere [03:24:00].
Optimistic parallel execution works by running transactions in parallel, generating pending results with input/output records [02:02:00]. These results are committed in the original transaction order, and if an input is invalidated due to a conflict, the work is re-executed [02:20:00]. Re-execution is often cheap as inputs are usually cached [02:51:00].
State Access: The Primary Bottleneck
The actual bottleneck for performance in blockchains, including Ethereum, is state access [03:30:00]. Transactions depend on accounts and slots stored on SSDs, and the cost of reading from an SSD is significant (e.g., 80-100 microseconds per read) [03:41:00].
Standard databases (like Pebble DB, RocksDB, LMDB, and mdbx derivatives) used by Ethereum and other EVM-compatible blockchains to store state do not support efficient parallel access [03:54:00]. When multiple virtual machines attempt parallel reads, they still bottleneck, effectively leading to single-file execution [04:05:00]. These general-purpose databases are not optimized for the specific access patterns of a blockchain, resulting in poor performance despite capable hardware [04:47:00]. This represents a significant challenge in challenges of standard databases in blockchain performance.
Other key challenges and propositions in blockchain technology identified include:
- Cryptographic Functions: Elliptic curve cryptography and hashing are expensive [09:08:00].
- Computation vs. State Access: The computational logic in most smart contracts is relatively cheap compared to state access [09:26:00].
- VM Choice Impact: The choice of VM (EVM, SVM, WASM) makes minor differences in performance [03:37:12].
- Access Lists: Counter-intuitively, access lists, intended to improve state prediction, may actually worsen performance due to overhead [03:22:01].
Monad’s Advancements and Solutions
Monad Labs addresses these challenges through several key technology advancements and infrastructure in blockchain and optimizations:
1. Custom State Database
Monad built a custom database from scratch, specifically designed for blockchain state storage [08:45:00]. This approach allows them to maximize the performance of modern SSDs (e.g., 500,000 I/O operations per second for a $200 SSD) [12:09:00]. By customizing the data structure to known usage patterns, they can extract significantly better performance than general-purpose databases, making far fewer requests to the hardware per lookup [16:27:00]. This is critical for realizing true performance gains from parallel execution [04:47:00].
2. Optimistic Parallel Execution (with State Access Optimization)
While Monad implemented optimistic parallel execution early on [01:34:00], they recognized that its impact on performance is limited without addressing the underlying state access bottleneck [03:18:00]. Their innovation lies in combining this algorithm with a highly optimized custom database, which makes re-execution cheap and efficient [02:51:00].
3. Separation of Execution and Consensus
Monad separates execution and consensus, allowing them to operate in parallel [01:17:00]. In current blockchains like Ethereum, the execution budget within a block is very small (e.g., ~100 milliseconds within a 12-second block, or 1% of the time) [01:19:00]. By not requiring execution to complete before consensus, Monad provides a much larger time budget for execution, significantly unlocking performance [01:19:00]. This approach is not a restriction but a relaxation of synchronization, handled deterministically [01:18:00].
4. No Shortcuts Approach
Monad’s development philosophy, “no shortcuts,” emphasizes fundamental engineering and scientific rigor [02:00:00]. This means:
- Commodity Hardware Focus: The goal is to run effectively on commodity hardware (e.g., $200 SSDs), avoiding the shortcut of simply requiring expensive, large-RAM machines, which hinders decentralization and scalability [02:11:00].
- Deep Understanding and Experimentation: Measuring everything, making no assumptions, and constantly experimenting are key [03:33:00]. This includes micro-optimizations like translation look-aside buffer optimization, even if they only yield small percentage gains, as they accumulate [03:10:00].
- Building from Scratch: Developing core components like the database from the ground up to achieve maximum performance [02:32:00].
- Iterative Development: Being willing to write and discard code to find the most optimal solutions [03:42:00].
Impact and Future Outlook
Monad’s approach aims to significantly expand the EVM ecosystem by providing a highly performant and scalable blockchain [05:35:00]. By meticulously rebuilding the execution stack, researching custom databases, implementing parallel and asynchronous execution, Monad explores a new, orthogonal direction for blockchain development [05:38:00]. This work could ultimately lead to changes being incorporated into Ethereum itself, pushing the entire space forward [05:47:00]. This aligns with evolution and impact of highperformance blockchain technology and future developments in crypto and blockchain technology.
A critical step for ensuring transparency and progress in the industry is the establishment of standardized benchmarks [04:26:00]. Currently, claims of high TPS (transactions per second) lack context, as the complexity of transactions (e.g., simple token transfers vs. complex Uniswap or borrowing protocol interactions) can drastically alter performance figures [04:42:00]. Monad plans to release a publicly available GitHub repository with benchmarks, likely using recent Ethereum history as a standard, to allow for verifiable and replicable performance comparisons [04:54:00]. This initiative aims to bring more rigor and scientific practice to the blockchain industry, moving beyond intuition and marketing claims [04:58:00]. This will also facilitate more meaningful Discussion on high throughput blockchains.
The goal is to enable a blockchain capable of handling the scale of traditional finance, supporting interactions like fully on-chain limit order books with sub-cent fees and thousands of orders per second [02:35:00]. Ultimately, users value responsiveness and low latency, and while throughput depends on the application, a high throughput is essential for broad user adoption [04:00:00].