Mysten Labs Technology Prototype on Sui Provides First Proof of Elastic Blockchain Scaling

Palo Alto, California, March 20th, 2024, Chainwire

Pilotfish, a prototype Sui extension, was able to grow smart contract execution throughput in direct proportion to the number of machines used.

Mysten Labs, a team of leading distributed systems, programming languages, and cryptography experts and originators of the Sui Network, today announced a landmark achievement in scaling blockchain capacity. During testing and development in a Sui blockchain environment, Pilotfish, a prototype Sui extension, increased throughput by 8x when backed by 8 machines, successfully illustrating the possibility of linear scaling. Notably, the latency per transaction decreased as more machines were added during the test, proving the viability of linear horizontal scaling for low latency blockchain transactions for the first time ever on any blockchain.

This proof-of-concept, was formally presented as an academic preprint titled, "Pilotfish: Distributed Transaction Execution for Lazy Blockchains," and initially shared on January 29, 2024 as an article on the open access archive arXiv.

Sui leads the industry in throughput by leveraging multiple cores for parallel transaction execution within individual validators. Using this approach, within months following mainnet, Sui reached 65.8 million transaction executions in a single day—the most of any blockchain ever. However, a single machine can only incorporate a finite number of cores, so throughput gains via this strategy are limited. Moreover, the machines capable of accommodating large numbers of cores are expensive to buy and use. The breakthrough solution that Pilotfish offers enables a single validator to employ multiple servers simultaneously. This new technology not only makes elastic capacity possible, but its auto-scaling architecture, akin to centralized solutions like AWS or Heroku, allows the use of smaller, cheaper servers available on demand.

Nearly all other blockchains aiming to scale use some form of batching—grouping transactions together to be added all at once. The downside of this method is significant additional latency. Instead of transactions moving directly to finality, they are delayed waiting for others to be batched for execution with them. Unlike these other scaling solutions, Pilotfish achieves potentially linear throughput scaling without increasing latency. In the simplest terms, with the horizontal scaling enabled by Pilotfish, simply by engaging the requisite number of machines, a validator can adapt its throughput and resource use to the use case at hand.

Pilotfish achieves its remarkable scalability by enabling a single validator to execute transactions across multiple machines. Specifically, Pilotfish partitions the validation function into three key roles: (1) the Primary, which handles sequencing transactions using a high-throughput consensus, (2) SequencingWorkers (SWs), which store transactions and dispatch them for execution; and (3) ExecutionWorkers (EWs), which store the blockchain state and execute transactions received from the SWs. Different components may reside on multiple different machines to increase the capacity of the system with more precision.

In experimental results, Pilotfish has already achieved remarkable results in terms of linear scaling, low latency, and support for various workloads. In testing, Pilotfish reduced network response times and maintained a latency envelope below 20ms. Future plans include enhancements like multiple SequencingWorkers, shard replication, and ultra-fast networking.

Contact

Mysten Labs
lexi.wangler@mystenlabs.com

Disclaimer: Press release sponsored by our commercial partners.