All You Need to Know About Solana's v1.16 Update

What’s this Article About?

Solana’s network of validators has successfully reached a super-majority in the adoption of version 1.16, the latest upgrade to the Solana Labs validator client. After an exhaustive period of auditing, bolstered by the dedicated efforts of volunteers and canary nodes, this milestone is the culmination of nearly ten months of rigorous development.

In the following sections, we’ll delve into how v1.16 was tested as well as Solana’s feature gate system; a framework that governs how new features are phased into the network. Then we’ll turn to the new features that were implemented in v1.16.

How v1.16 was Tested

v1.16 has been subject to rigorous testing over the past few months. The v1.16 release has been running on testnet since June 7, 2023, and has underwent numerous stress tests. In addition, a small subset of volunteer nodes updated to v1.16 starting on August 23, 2023. These volunteers were able to identify and resolve various issues such as slow startups for RPC nodes as well as general protection faults. Solana Labs also deployed several canary nodes to mainnet-beta. This was done to monitor the stability of the v1.16 nodes under real-world conditions. To see the past activity and progress these canary nodes made, visit the #canaries-monitoring channel in the Solana Tech Discord.

To catch edge cases or infrequent race conditions, multiple runtime fuzzers have been used to execute partially randomized transactions. These transactions were executed across different runtime versions to ensure consistent performance. v1.16 has also been audited extensively by Halborn. Audit reports are published to this repository as they become available over time.

Feature Gates

It is important to note that some of the features we will discuss in the following sections are not currently live. Instead, features are slowly rolled out using a feature gate system. Features are activated at certain epochs based upon relative priority, and the order in which they were activated on other networks. The feature gate activation scheduling has been ad-hoc so far based on a list of criteria, which you can find here. Essentially, these gates should be activated first on testnet, followed by devnet, and finally on mainnet-beta. To activate a feature, an engineer with the requisite activation keypair sends a transaction that gets processed and makes the feature go live on the next epoch. Only one feature gate is to be activated at a time per network to ensure proper performance. It is important to note that some features may require a “soak” period, which could delay the activation of lower-priority gates.

This feature gate system is in place to ensure that consensus-breaking changes do not cause a validator running a newer version to fork off from the canonical chain and continues producing blocks. For example, a v1.14 validator doesn’t know about new v1.16 features and could crash the network when disputes arise. A commit merged this week to Solana’s codebase encourages all consensus-breaking changes to have a Solana Improvement Document (SIMD). The feature gate issue template will now include a request to the issue’s SIMD. This helps in standardizing the development process and provides greater transparency via documentation to new changes.

Confidential Transfers

Confidential Transfers, a feature introduced by Token2022, utilizes zero-knowledge proofs to encrypt the balances and transaction amounts of SPL tokens. The primary focus of this feature is to improve user privacy through a renowned emphasis on confidentiality rather than anonymity.

Confidential Transfers leverage Twisted ElGamal Encryption for mathematical operations on the encrypted amounts. These transfers are validated using Sigma Protocols, a specialized category of zero-knowledge proofs where one party (the prover) can demonstrate to another party (the verifier) that they know of a certain secret without actually disclosing the secret itself. Be sure to read our article What is Token2022? for a deeper dive into the intricacies of Confidential Transfers.

A nice feature to accompany the rollout of Confidential Transfers is the addition of Command Line Interface (CLI) support. The create-token command has been extended to include a --enable-confidential-transfers flag, which allows users to mint tokens with confidential transfers enabled. Also, the update-confidential-transfer-settings command has been added to enable dynamic changes to confidential transfer configurations for a given mint. This allows for the auditor key and approval settings to be updated.

Better Runtime Support for Zero-Knowledge Proofs

The v1.16 release boosts Solana’s zero-knowledge capabilities with enhanced runtime support for zero-knowledge computations, specifically 128-bit elliptic curve operations. v1.16 introduces alt_bn128 syscalls, which are crucial for generating proofs efficiently.

alt_bn128 refers to a specific implementation of an elliptic curve used for cryptographic operations known as the Barreto-Naehrig curve (BN-128). BN-128 is a specific type of pairing-friendly elliptic curve that allows for the efficient implementation of zk-SNARKs (Zero-Knowledge Succinct Non-Interactive Argument of Knowledge). As an aside, an elliptic curve is considered “pairing-friendly” if it allows for certain calculations to be done more efficiently. So, in our case, using the BN-128 curve makes working with zero-knowledge math and proofs a lot faster.

Syscall, or a system call, is used to request services from the operating system kernel. Within the context of Solana, a syscall allows programs running in the Solana Virtual Machine (SVM) to interact with outside resources.

An alt_bn128 syscall, then, is a call that Solana programs can use to interact with the highly efficient BN-128 curve. This makes the verification of zero-knowledge proofs streamlined, offering better security and privacy features on Solana. alt_bn128 g1 and g2 syscalls were also added recently, which allows for the compression of Groth16 proofs. This is important as these types of proofs take up 256 bytes of instruction data per proof and private Solana programs (PSPs) currently need to verify two Groth16 proofs. With g1 and g2 compression, the bytes required per proof can be halved to 128 bytes, which is crucial for space efficiency.

Moreover, Solidity-based contracts face compatibility issues with Solana if they contain calls to the following precompiled contracts for elliptic curve operations:

bn256Add - Performs addition on the elliptic curve operations
bn256ScalarMult - Performs scalar multiplication on elliptic curve operations
bn256Pairing - Elliptic curve pairing operations to perform zkSNARKs verification within the block gas limit

These operations are standardized on Ethereum through EIP-196, EIP-197, and EIP-198. The introduction of alt_bn128 syscalls is a significant leap towards bridging the compatibility gap. Solidity contracts that rely on these elliptic curve operations may now find it easier to transition to or even interoperate with Solana.

The inclusion of the alt_bn128 syscall in Solana’s v1.16 update marks a major advancement in Solana’s ability to efficiently and securely handle zero-knowledge proofs. Check out the following pull requests if you want to learn more about alt_bn128 syscalls:

Validators

The v1.16 update dramatically reduces RAM usage for validators. Solana previously relied on RAM for indexing accounts. The system has now been reconfigured to index accounts on disk by default, which greatly reduces RAM usage. Yanshu from Luganodes notes that since the release of v1.16 their validator is running smoothly on just ~39 GB of RAM in comparison to ~120 GB on previous versions:

Source: https://x.com/0xyanshu/status/1707387005016121754?s=20

The v1.16 release also introduces a reworked gossip pull-requests peer sampling system. This new system is effective in reducing startup bandwith for validators. In previous versions validators could encounter bandwith constraints due to a high volume of gossip pull-requests. This could slow down or even overwhelm the validator. v1.16 addresses this issue by introducing a time since last request variable. This variable is used to gauge incoming traffic levels and apply rate-limiting to prevent the validator from becoming overwhelmed when it’s starting up.

Staked validators that fall behind the network can now catch up to the current state even faster with the new repair request feature that is proportional to one’s stake. Whenever a large stake validator is forked off of the network, they make a repair request. This validator will now get shreds faster because they hold a large stake. This makes sure that the validator is no longer forked off from the network and the can continue to contribute. Staked validators have a higher priority over RPC nodes for repair requests since they aren’t the ones producing blocks.

The delay threshold for issuing shred repairs has also been increased from 100ms to 200ms. This change was made to reduce the number of repair requests for shreds that would ultimately be served through Turbine. As an aside, Turbine refers to the multi-layer block propagation mechanism that Solana uses to broadcast ledger entries to all nodes. Here, the Solana cluster divides itself into layers of nodes, and each node in a given layer is responsible to propagate data to the next downstream layer. This is a crucial adjustment as it minimizes unnecessary repair requests, which enhances the efficiency of Turbine’s data propagation.

It has never been easier to run your own validator. For those keen on operating their own validator, Solana offers Solana Validator Education, a series of validator workshops that can be found on their YouTube channel.

Support For Resizable Accounts

When deploying a program on Solana, the amount allocated for the program is always two times the size of the program. v1.16 allows you to deploy programs with resizable data accounts. This means that you could deploy your program with a smaller sized account, and then expand its size later while paying for the memory difference. Support for resizable accounts provides more flexibility and better resource allocation for developers deploying applications on Solana.

Epoch Accounts Hash

In previous versions, there were issues with respect to blocks and verifying all accounts in state. If a validator went a prolonged period of time without interacting with a certain account, it’s possible for that validator to have a corrupted version of that account and not even realize it. This is due to the fact that account state would not be checked against the state held by other validator nodes since no transactions were made to modify its state.

v1.16 addresses this issue by introducing the Epoch Accounts Hash. This is hash of all accounts that is generated at the end of each epoch, even if those accounts have not been interacted with. The Epoch Accounts Hash allows the network to identify and fork off nodes with corrupted data, which increases Solana’s integrity and security.

System Tuning

System tuning is a process of optimizing a validator’s operating system and hardware configurations to achieve optimal performance. With the v1.16 release, solana-sys-tuner has been removed and now manual testing is recommended. This was removed because in previous versions the TransactionStatus and AddressSignature columns of the RocksDB were not cleaned properly. In addition, periodic compaction, a process that reclaims storage space, was disabled by default. This resulted in unbounded growth in these columns for nodes running the --enable-rpc-transaction-history flag. Now validators using this flag will have their storage space managed more efficiently due to the following commit. This is a significant improvement in streamlining validator storage requirements by removing the need to store unnecessary transaction statuses and address signatures.

Conclusion

Solana’s v1.16 release marks a significant milestone, encapsulating ten months of development. The reason for its slow release is due to the prioritization of QUIC, so these advancements in confidential transfers, zero-knowledge support, and validator optimizations are long overdue. Nevertheless, these improvements are nothing short of groundbreaking. This release helps to bring new levels of efficiency and privacy to Solana.

Looking ahead, Solana Labs is shifting to a more agile release cycle, targeting a new release roughly once every three months. These future releases are to be a lot smaller than v1.16. This allows for faster iteration and reduced risk when deploying. The release schedule for v1.17 can be found here, and proves to be another highly anticipated release with even better zero-knowledge support and the potential introduction of Posidon syscalls.

If you’ve read this far anon, thank you! With a more agile release cycle and a host of new features and improvements, the future of Solana looks better than ever. Whether you’re a developer, investor, validator, or a Solana enthusiast, be sure to keep an eye on the upcoming releases! Your Solana journey is just getting started.

Additional Resources / Further Reading

‍