From: thepipeline_xyz
Developing high-performance systems, especially in the context of blockchain technology, presents unique and significant challenges. Monad’s approach highlights several key areas where traditional methods fall short and innovation is required [00:01:41].
The “Build from Scratch” Approach
Monad’s decision to build its protocol from the ground up, rather than forking an existing open-source project, was driven by the need for greater technical control [00:01:46]. Forking would necessitate modifying so many aspects of existing projects that leveraging ongoing maintenance or new features would become impractical [00:02:17]. Building from scratch allows for:
- Precise Implementation Control [00:02:41]: This includes fine-tuning memory allocation, disk interaction, and other low-level details crucial for optimal performance [00:02:44].
- Avoiding Friction [00:03:22]: Existing EVM implementations, such as Ethereum clients, are often “arbitrarily rate-limited” due to design choices focused on low node requirements, rather than raw execution throughput [00:04:04]. To achieve high performance, a different design space is required [00:04:21].
Scarcity of Specialized Talent
One of the significant challenges in building high-performance blockchain systems is the difficulty in finding qualified low-level system engineers [00:05:21]. These specialists typically work in fields like:
- Big tech, focusing on operating systems (e.g., Linux) [00:05:26]
- Silicon manufacturing (e.g., Nvidia) [00:05:33]
- High-frequency trading firms [00:05:37]
Their expertise in creating high-performance systems is critical but not commonly found among general developers [00:05:55].
Technical Hurdles in Concurrency and Data Management
The core of building high-performance blockchain systems involves overcoming specific technical obstacles:
Concurrent Execution
Modern machines feature many cores, yet most existing blockchain clients only use a single core for transaction execution, with auxiliary threads for other tasks [00:06:22]. The challenge is to leverage these multiple cores by enabling concurrent execution of transactions [00:06:34]. This is complicated by:
- Transaction Dependencies [00:06:36]: Transactions within a block are not always independent tasks; they often have interdependencies that must be managed [00:06:44].
- Pipelining [00:06:47]: Implementing pipelining, where multiple steps of different transactions are processed simultaneously, is crucial for efficiency [00:08:40]. This requires understanding transaction relationships and potentially removing dependencies [00:06:57]. Monad’s architecture is “highly pipeline” to keep the CPU constantly active [00:09:00].
Database Performance
The database is a critical component, and waiting for disk access is a major performance bottleneck [00:07:07]. Even 30 microseconds of disk access time, when compounded across many transactions and accesses per transaction, quickly adds up, leading to an underutilized CPU [00:07:19]. Monad addressed this by building its own database, ensuring the CPU is always busy and not waiting for disk I/O [00:08:02]. This is a key innovation in their drive for high-performance systems [00:08:10].
Benchmarking Real-World Performance
Accurate performance benchmarking is another challenge, as many projects use misleading metrics [00:22:41]. Simple token transfers are easily achievable, even by unoptimized, single-core clients [00:20:01]. To truly assess performance, Monad benchmarks using replayed Ethereum history because it includes:
- Complex Contract Interactions [00:21:26]: Real Ethereum history involves diverse protocols, including AMMs, lending protocols, and computationally expensive ZK proofs, which are far more taxing than simple transfers [00:21:28].
- Realistic Usage Patterns [00:22:23]: This approach simulates what the system looks like with actual users, exposing potential bottlenecks and use patterns not anticipated during internal testing [00:17:17].
Overcoming Decentralization-Performance Trade-offs
A significant challenge in building crypto infrastructure is pushing the decentralization-performance trade-off curve forward [00:24:40]. Many projects employ “tricks” to inflate performance metrics without true decentralization:
- Geographical Collocation [00:23:04]: Having all nodes in one location, or concentrating stake weight in a few collocated nodes, can make consensus appear faster by reducing latency, but compromises decentralization [00:23:13].
- High Hardware Requirements [00:24:17]: Relying on a small number of nodes with very high hardware or bandwidth requirements limits participation [00:24:17].
Monad actively tests worst-case scenarios, like having highly staked nodes in geographically distant locations (e.g., Singapore and New York), to ensure the algorithm performs well under true decentralized conditions [00:25:40]. The aim is to make decentralization more performant through technological innovation, rather than by manipulating parameters or centralizing the network [00:24:56].
Anticipating Unforeseen Usage Patterns
Even with robust internal testing and Ethereum replay, real-world usage on a public testnet can expose unexpected behaviors [00:28:03]. Scenarios like:
- Thousands of concurrent users performing complex actions (e.g., NFT mints at the same time) [00:27:45].
- The emergence of new applications or actors, such as AI agents, that might “crash the testnet” through intense, automated activity [00:30:40], [00:30:50].
These events test the system’s ability to “gracefully degrade” rather than fall over completely, providing a poor user experience with errors or reverted transactions [00:29:07]. A robust system should simply “just work” without needing stories or excuses for failures across different layers of the stack (RPC, validator, distribution network) [00:32:22].
Ultimately, building high-performance blockchain systems is not a finish line but a starting line, with continuous work needed for optimizations, new features, and research to support increasing decentralization and usability [00:48:43].