From: thepipeline_xyz
Measuring the performance of blockchain platforms can be complex, with various metrics often leading to confusion and differing interpretations. One of the most frequently discussed, yet often misunderstood, metrics is Transactions Per Second (TPS) [00:46:56].
Understanding and Measuring TPS
The concept of TPS aims to quantify the throughput of a blockchain, indicating how many transactions it can process in a given second [00:46:56]. However, the definition of what constitutes a “transaction” can vary significantly across different platforms, leading to inflated or incomparable numbers [00:49:24].
The Solana Case: Voting Transactions
A prominent example of differing TPS reporting comes from Solana. Its main block explorers advertise a high TPS number, which includes not only user-initiated transactions (like swapping tokens or minting NFTs) but also votes from validators [00:47:16].
While these votes are technically formulated as Solana transactions, they are distinct from direct user interactions [00:47:42]. The “true TPS” for user-driven activities on Solana is approximately 500 transactions per second, with an additional 2500 or more votes occurring every second [00:47:59]. This means a reported TPS of 3000 would include both [00:48:15].
For Monad, the plan is to only count “real transactions”—smart contract interactions and transfers—and not include votes in any reported TPS figures [00:48:21].
Varied Transaction Definitions
Other platforms also adopt different definitions:
- Some, like Aptos or Sui, might count an “instruction” as a transaction [00:48:57]. This means a single smart contract invocation that executes several sub-instructions could be counted as ten transactions, distorting the actual throughput [00:49:09]. This non-uniformity makes direct comparison between different blockchain platforms difficult [00:49:24].
Capacity vs. Demand
Another layer of confusion in measuring blockchain performance is the difference between a system’s maximum possible throughput and the current demand it experiences [00:49:39]. If a system’s capacity far exceeds its current usage, the observed TPS will be much lower than its theoretical maximum [00:49:52].
Furthermore, when teams try to demonstrate maximum capacity by loading their testnets with transactions, questions arise about whether the testnet accurately reflects a real-world production environment [00:50:04].
Towards Standardization and Transparency
The current landscape is characterized by “information wars” where different teams advertise performance metrics in ways that favor their platforms [00:50:46].
To address this, a better approach would be to:
- Reproducible Benchmarks: Implement reproducible benchmarks with open-source GitHub repositories [00:50:49]. This involves publishing full scripts that define how servers are deployed globally and how transactions are sent to nodes, allowing others to verify and reproduce the results [00:51:01]. Monad plans to introduce such benchmarks for its own platform and potentially for competitive benchmarks [00:51:17].
- Standardized Metrics: Recognize that transaction sizes and complexities vary [00:51:47]. While simple transfers might allow for accurate comparison between different blockchain platforms, real-world activity involving complex smart contract interactions makes TPS a less reliable metric [00:52:10].
- “Bytes Per Second”: For a more consistent measure of raw throughput in a production system, “bytes per second” could be a better benchmark. This metric abstracts away the conditional definitions of a “transaction” across different blockchains [00:52:32].
- Historical Benchmarking: Monad uses the historical Ethereum transaction history as a benchmark, providing a proxy for real-life activity given the varying composition of transactions [00:52:56].
In conclusion, while TPS is a significant metric for blockchain performance, its effectiveness is often diluted by conflated definitions and measurement methodologies [00:53:25]. The need for performant blockchain requires a commitment to transparent and reproducible benchmarking practices for meaningful comparison of different database structures for blockchain efficiency and blockchain system design [00:51:40], ultimately driving the industry forward through better understanding of technical challenges and solutions in blockchain performance and current blockchain infrastructure.