Summarize
Solana is a high-performance blockchain platform that uses a unique technical architecture to achieve high throughput and low latency. Its core technologies include the Proof of History (POH) algorithm to ensure transaction order and global clock, Leader Rotation Schedule and Tower BFT consensus mechanism to increase the block rate. The Turbine mechanism optimizes large block propagation through Reed-solomon encoding. Solana Virtual Machine (SVM) and Sealevel parallel execution engine speed up transaction execution. These are all architectural designs for Solana to achieve high performance, but they also bring some problems, such as network downtime, transaction failures, MEV problems, rapid state growth and centralization problems. We also focus on the problems caused by this mechanism in this article.
Solanas ecosystem is developing rapidly, and various data indicators have grown rapidly in the first half of the year, especially in DeFi, infrastructure, GameFi/NFT, DePin/AI, and consumer applications. Solanas high TPS and consumer application-oriented strategy and the ecological environment with weak brand effect provide entrepreneurs and developers with abundant entrepreneurial opportunities. In terms of consumer applications, Solana has demonstrated its vision for promoting the application of blockchain technology in a wider range of fields. By supporting projects such as Solana Mobile and building SDKs specifically for consumer applications, Solana is committed to integrating blockchain technology into everyday applications to increase user acceptance and convenience. For example, applications such as Stepn provide users with novel fitness and social experiences by combining blockchain and mobile technologies. Although many consumer applications are still exploring the best business model and market positioning, the technical platform and ecosystem support provided by Solana undoubtedly provide strong support for these innovative attempts. As the technology further develops and the market matures, Solana is expected to achieve more breakthroughs and success cases in the field of consumer applications.
Although Solana has gained significant market share in the blockchain industry with its high throughput and low transaction costs, it is also facing fierce competition from other emerging public chains. As a potential rival in the EVM ecosystem, Base is seeing a rapid increase in the number of active addresses on the chain. At the same time, although Solanas total locked volume (TVL) in the DeFi field has hit a record high, competitors such as Base are also rapidly gaining market share, and the financing amount of the Base ecosystem also surpassed Solana for the first time in Q2.
Although Solana has made certain achievements in technology and market acceptance, it needs to continue to innovate and improve to meet the challenges from competitors such as Base. In particular, in terms of improving network stability, reducing transaction failure rate, solving MEV problems, and slowing down the growth rate of state, Solana needs to continue to optimize its technical architecture and network protocols to maintain its leading position in the blockchain industry.
Technology Architecture
Solana is known for its high TPS and fast finality brought by its POH algorithm, Tower BFT consensus mechanism, Trubine data transmission network, and SVM virtual machine. We will briefly introduce how each of its components works, how to achieve its high performance goals for architectural design, and the drawbacks and derived problems brought about by this architectural design.
POH Algorithm
POH (Proof of History) is a technology for determining global time. It is not a consensus mechanism, but an algorithm for determining the order of transactions. POH technology is derived from the most basic cryptographic SHA 256 technology. SHA 256 is usually used to calculate the integrity of data. Given an input X, there is only one output Y, so any change to X will result in a completely different Y.
POH sequence, source : Solana white paper
POH sequence diagram, source: Solana white paper
In Solanas POH sequence, the integrity of the entire sequence is ensured by applying the sha 256 algorithm, which determines the integrity of the transactions. For example, if we package the transactions into a block and generate the corresponding sha 256 hash value, the transactions in this block are determined, and any changes will cause the hash value to change. After that, this block hash is used as part of the X that will be used as the next sha 256 function, and then the hash of the next block is added, then the previous block and the next block are both determined, and any changes will result in a different new Y.
This is the core meaning of its Proof of History technology. The hash of the previous block will be used as part of the next sha 256 function, similar to a chain. The latest Y always contains proof of history.
Transaction Flow architecture diagram, source : Solana White Paper
In Solanas transaction flow architecture diagram, the transaction process under the POH mechanism is described. Under a rotation mechanism called Leader Rotation Schedule, a Leader node will be generated among all on-chain validators. The Leader node collects transactions and sorts them for execution, generates a POH sequence, and then generates a block to propagate to other nodes.
Leader election mechanism, source: Helius
In order to avoid single point failure at the Leader node, a time limit is introduced. In Solana, the time unit is divided by epoch. Each epoch contains 432,000 slots (time slots), and each slot lasts 400 ms. In each slot, the rotation system will assign a Leader node in each slot. The Leader node must publish a block within the given slot time (400 ms), otherwise, the slot will be skipped and the Leader node of the next slot will be re-elected.
In general, the Leader node uses the POH mechanism to finalize all historical transactions. Solanas basic time unit is Slot, and the Leader node needs to broadcast blocks within a slot. Users send transactions to the Leader through the RPC node, and the Leader node packages and sorts the transactions and then executes to generate blocks. The blocks are propagated to other validators. Validators need to reach a consensus through a mechanism to reach a consensus on the transactions and order within the block. The consensus uses the Tower BFT consensus mechanism.
Top BFT consensus mechanism
Tower BFT protocol, source: Helius
The Tower BFT consensus protocol comes from the BFT consensus algorithm and is a specific engineering implementation of it. The algorithm is still related to the POH algorithm. When voting on a block, if the validators vote itself is a transaction, then the block hash formed by the users transaction and the validators exchange can also be used as a historical proof, and the transaction details of the user and the voting details of the validator can be uniquely confirmed.
Voting Icon
The Tower BFT algorithm stipulates that if all validators vote on the block and more than 2/3 of the validators vote approve, then the block can be confirmed. The advantage of this mechanism is that it saves a lot of memory, because only the hash sequence needs to be voted on to confirm the block. However, in the traditional consensus mechanism, block flooding is generally used, that is, a validator receives a block and then sends it to the surrounding validators, which will cause a lot of redundancy in the network because a validator receives the same block more than once.
In Solana, due to the large number of validator voting transactions, the efficiency brought by the centralization of the Leader node and the 400 ms Slot time, the overall block size and block frequency are particularly high. When large blocks are propagated, they will also cause great pressure on the network. Solana uses the Turbine mechanism to solve the problem of large block propagation.
Turbine
Turbine block propagation mechanism , source: Helius
The Leader node splits the block into shredded sub-blocks through a process called Sharding, with the size of the block measured in MTU (maximum transmission unit, the maximum amount of data that can be sent from one node to the next without splitting it into smaller units). The integrity and availability of the data are then guaranteed by using the Reed-solomon erasure code scheme.
Reed-solomon coding scheme, source: Helius
By dividing the block into four Data Shreds, and then using Reed-solomon encoding to encode four packets into eight packets to prevent packet loss and damage during data transmission, this solution can tolerate a packet loss rate of up to 50%. In actual tests, Solanas packet loss rate is about 15%, so this solution is well compatible with the current Solana architecture.
In the underlying data transmission, the UDP/TCP protocol is generally considered. Since Solana has a high tolerance for packet loss, the UDP protocol is used for transmission. The disadvantage is that it will not retransmit when packets are lost, but the advantage is the faster transmission rate. On the contrary, the TCP protocol will retransmit multiple times when packets are lost, which will greatly reduce the transmission rate and throughput. With Reed-solomon, this solution can significantly increase Solanas throughput. In a real environment, the throughput can be increased by 9 times.
Schematic diagram of layered propagation, source : Helius
After Turbine shards the data, it uses a multi-layer propagation mechanism to propagate it. The Leader node will hand over the block to any block validator before the end of each slot. The validator will then shard the block into Shreds and generate erasure codes. The validator will then start Turbine propagation. First, it must be propagated to the root node, and then the root node will determine which validators are located at which layer. The process is as follows:
1. Create a node list: The root node aggregates all active validators into a list, and then sorts them according to each validator’s stake in the network (that is, the amount of SOL staked). Those with higher weights are placed in the first layer, and so on.
2. Node Grouping: Each validator in the first layer will then create its own node list to build its own first layer.
3. Layer formation: Divide the nodes into layers from the top of the list. By determining the depth and breadth values, the general shape of the entire tree can be determined. This parameter affects the propagation rate of shreds.
Nodes with a higher stake in the hierarchy can obtain complete shreds in advance and restore the complete block. However, due to transmission loss, the probability of obtaining complete shreds for nodes in the later layers will be reduced. If these shreds are not enough to build a complete fragment, the Leader will be required to retransmit directly. At this time, data transmission will be carried out inside the tree, and the nodes in the first layer have already built a complete block confirmation. The longer it takes for the validators in the later layers to vote after completing the block construction.
The idea of this mechanism is similar to the single-node mechanism of the Leader node. There are also some priority nodes in the block propagation process. These nodes first obtain shreds fragments to form a complete block to reach a voting consensus. Pushing redundancy to a deeper level can significantly speed up the finality process and maximize throughput and efficiency. Because in fact the first few layers may represent 2/3 of the nodes, then the votes of subsequent nodes are irrelevant.
SVM
Solana can process thousands of transactions per second, mainly due to its POH mechanism, Tower BFT consensus and Turbine data propagation mechanism. However, as a virtual machine for state transition, if the Leader node is executing transactions and the SVM is slow, the throughput of the entire system will be reduced. Therefore, Solana proposed the Sealevel parallel execution engine to speed up the execution of transactions for SVM.
Sealevel parallel execution diagram, source: Xangle
In SVM, instructions consist of 4 parts, including program ID, program instructions, and a list of accounts that read/write data. By determining whether the current account is in a read or write state and whether the operation to be changed for the state is conflicting, the accounts transaction instructions can be parallelized without conflicting states. Each instruction is represented by a Program ID. This is one of the reasons why Solanas validators have high requirements, because the validators GPU/CPU is required to support SIMD (single instruction multiple data) and AVX advanced vector expansion capabilities.
Ecological Development
Solana Ecosystem Lands cape, source: Gate Ventures
In the current development of the Solana ecosystem, it is increasingly leaning towards practical utility, such as Blinks, Actions, and even Solana Mobile, and the development direction of officially supported applications is also more inclined towards consumer applications rather than unlimited involution of infrastructure. With Solanas current sufficient performance, the types of applications are more abundant. As for Ethereum, due to its low TPS, the Ethereum ecosystem is still based on infrastructure and expansion technology. When the infrastructure cannot support applications, it is impossible to build consumer applications. This has caused an unbalanced state in which funds are invested too much in infrastructure but too little in applications.
DeFi
DeFi Landscape, source: Gate Ventures
Among the DeFi protocols on Solana, there are a large number of projects that have not issued tokens, including Kamino (First Lending), Marginfi (Lending + Restaking), SoLayer (Restaking), Meteora, etc. Due to Solana’s united ecological atmosphere, usually when a project is issuing tokens, other projects will try to avoid it as much as possible to attract enough market attention.
DEX market share, source: Dune
Currently, competition in the entire DEX sector is fierce, and its leaders have also undergone multiple migrations, from Raydium, Orca to the current dominant Jupiter.
The initiator of DEX transaction, source : Dune
It is worth noting that about 50% of DEX transactions are initiated by MEV bots, mainly because of their low fees and active Meme transactions, which have fostered the profitability of MEV. This is also one of the main reasons for the frequent failures and downtime of user peak transactions.
Solana TVL, Image source: Defillama
The DeFi protocol on Solana has experienced an explosive increase in USD nominal TVL as the price of SOL has risen. The upward trend of its TVL has not stopped, and a new wave of upward trend has formed.
In short, although the competition in the Solana track is fierce, there are still changes. Unlike Uniswap on Ethereum, which occupies the brand mind of users, DEX, which should be highly sticky and have network effects, will also face the risk of being replaced. The transactions of the Solana main chain are filled with MEV bots, which has caused great user experience problems for users and still needs to be solved. In the overall direction, Solanas TVL is still growing very rapidly, and its subsequent DeFi ecological development is still very worth looking forward to. In addition, the brand mind of these applications is not strong in occupying users, which is a potential driving force for entrepreneurs to choose chains.
Infrastructure
Infrastructure Landscape, Image source: Gate Ventures
In terms of infrastructure construction, the main leaders are the oracle Pyth and the cross-chain bridge Wormhole, as well as some targeted solutions that the public may not know much about, such as:
1. Jito Labs: Focusing on building MEV solutions on Solana, its Jito Labs client built Bundle and pseudo emempool to allow researchers to conduct MEV. Its current market share exceeds 50%. In addition, the staked SOL of its LSD protocol Jito is close to 12 million, and is still growing rapidly.
2. Helius: As an active contribution community on Solana, Helius has the most comprehensive research on Solana and contributes code through research.
3. GenesysGo: Its product ShdwDrive is a data storage project in Solana. It is committed to supporting projects that can be commercialized, including social data, website hosting and other businesses. It is still in the testnet stage. At the same time, its parent company GenesysGo is also building various public goods and research for the Solana community.
In addition, Solana still has a lot of projects worth exploring waiting for the Chinese community to explore. We do find that these infrastructures have a huge impact on Solanas protocol-level construction, ecological development, and community, and there may be opportunities to further explore their potential, whether through investment or cooperation.
Gaming / NFT
Gaming / NFT Landscape, Image source: Gate Ventures
Solana also has a rich GameFi and NFT ecosystem, among which Mad Labs occupies a relatively important position in the entire Solana ecosystem. Many project airdrops will give priority to Mad Labs holders. In the past, DeGods was the leader. The NFT market has also undergone changes. In the past, Magic Eden was the most used, but now it has changed to Tensor.
DePin / AI
DePin Landscape, Image source: Gate Ventures
DePin data overview, source: DePin Scan
Currently in Solanas DePin market, Render is the well-deserved leader with actual business. With Solanas development strategy centered on practical applications, it has also caught the narrative of DePin in this round of recovery. In the first half of the year, a large number of new DePin projects were built on Solana, including io.net, Nosana, Shadow, etc.
Consumer
Consumer Landscape, Image source: Gate Ventures
Whether it is Solana Mobile or the column built specifically for consumer applications on the Solana Ecosystem official website, the invention of Actions and Blinks, all illustrate Solanas vision for the commercialization and practicality of blockchain. The release of its Mobile collection is also putting web-side dapps on mobile terminals, which is very in line with human nature and the development trend of the Internet. Therefore, applications are prone to explode on this soil, the most typical example being Stepn.
Looking at the consumer applications currently in operation, most of them have not yet found a good breakthrough, so they are still unable to realize a real application in the business world. This includes many factors, including single product innovation, single business model, weak Web2 marketing, demand for Gas Fees, and entry barriers for tokens.
However, consumer applications are the ultimate landing scenario for blockchain technology, and also determine the ceiling of the public chain. Therefore, Solana’s exploration of mobile consumer applications is very necessary, and our long-term exploration in this direction is also necessary. Especially in the current Ethereum ecosystem, infrastructure is far greater than applications. Ultimately, infrastructure serves applications.
Payments
Payment Landscap e, Source: Gate Ventures
Wallets on Solana include Phantom, Backpack, TipLink, etc. Like DEX, the brand effect here is not strong, so entrepreneurs have more opportunities. In the past, the leading wallet was Phantom, but now it has changed to Backpack, which was built by Mad Labs. It is worth mentioning that Mad Labs is now also the NFT leader on Solana.
Solana stablecoin issuance status, source: Defillama
It currently cooperates with Paypal, Visa and others to conduct stablecoin payment transfers on the chain. This business scenario itself is very conducive to the Solana chain with fast finality and low gas fee. Currently, its on-chain stablecoin is in a state of slow growth.
Stablecoin transfer stack chart Y TD, source: Artemis
Solana had a remarkable stablecoin transfer market share in the first half of the year. However, its market share has declined significantly since June. Solana performed absolutely well in the first half of the year, but its initial transfer data in the second half of the year showed a clear downward trend.
Competitor data
Number of active addresses on the chain, source: Artemis
Among all the public chains, Base is seen as a potential competitor of Solana in the EVM ecosystem. The number of active addresses on Base is increasing rapidly, and Solana is still in a stage of rapid growth despite its first-mover advantage. NEAR remains at a high level, but Aptos and Sui lag behind in the competition among public chains.
TVL comparison, source: Artemis
Solana has also made significant progress in the Defi field. Its TVL has hit a record high and is still a long way from other public chains. However, it is worth noting that Base is also in a stage of rapid growth.
Public chain stablecoin reserves, source: Allium
Currently, Solanas market share of stablecoin supply has been sluggish. Ethereums market share has naturally shrunk due to the emergence of multiple chains, while Bases market share has been quietly increasing.
Financing data, source: Mess ari
In terms of capital market financing, the Base ecosystem’s financing frequency has increased significantly in the last quarter, and has surpassed the Solana ecosystem. Therefore, the market share and capital financing of various on-chain data also show that the competition between Base and Solana in the market is valid, and this competitive pressure will be greater as Base matures. In addition, Base and Solana have similar visions, both of which hope to achieve the Cryptio Native Consumer App vision of Mass Adoption with high TPS.
Technical challenges
Downtime
Solana has experienced multiple outages in history. We have sorted out the specific events and reasons for the outages:
May 4, 2021
The network performance has declined, resulting in a large number of transactions being unable to be completed.
September 3, 2021
The network is unstable and performance is degraded. It lasts about 1 hour.
September 14, 2021
Grape Protocols IDO activity on the Raydium platform was very hot. Many users sent a large number of transactions through written machine scripts. These transactions caused memory overflow, causing the verification node to crash. Ultimately, the entire network was unable to produce blocks, and the interruption time was as long as 17 hours.
January 21, 2022
Due to high market volatility, the network was flooded with transactions submitted by a large number of arbitrage robots, causing severe network load and an outage lasting up to 30 hours.
May 1, 2022
Due to the casting of a new NFT project, a large number of robot transactions emerged, causing the main network nodes to lose consensus and then suspend block production for up to 7 hours.
June 1, 2022
Due to a vulnerability in the durable nonce function in the transaction, the network was restarted and the interruption time was about 4.5 hours.
October 1, 2022
Network downtime due to node configuration errors
February 25, 2023
Solana mainnet performance issues eventually forced validator nodes to automatically enter a “voting only” safety mode, making it unable to process user transactions.
February 6, 2024
The BPF (Berkley Packet Filter) loader failed, with a downtime of 4 hours and 46 minutes
Solanas network architecture, such as the Gulfstream Leader election mechanism and the single-node risk of the Leader node, makes subsequent predictions of the Leader node feasible. As network transactions increase, the single-node Leader will experience great memory pressure, and the Leader node needs to be ready to retransmit blocks to the nodes in the Turbine tree at any time, otherwise the consensus vote cannot be completed. When a large number of DDOS attacks occur, system downtime caused by single-node failures becomes extremely frequent.
In short, the downtime is mainly due to the inability to generate blocks. It may be due to a single node failure caused by the Leader mechanism, which causes problems in the block formation, or it may be due to the inability of the consensus layer to reach a consensus on the block, resulting in the inability to generate blocks. Overall, this is closely related to Solanas own architecture and the software testing process.
transaction failed
The proportion of failed transactions by users, source: Dune
Users who have used Solana should know that our transactions often fail to be submitted normally. After a while, the transaction is found to have failed, which creates a very poor user experience. As shown in the figure above, according to statistics, about 35% of the transactions submitted by users fail and require users to submit multiple times. When there are large fluctuations on the chain, this proportion will be even greater.
The main reason for this is the network layer technology QUIC, which is a relatively new technology.
Network protocol hierarchy - 5-layer structure, source: Research Gate
QUIC (Quickl UDP Internet Connections) is a transport layer improvement proposed by Google for the HTTP 2.0 protocol. This experimental protocol is developed based on the UDP transport layer protocol and is also known as HTTP 3.0.
HTTP/2 and QUIC diagram , source: EMQX
TCP is more reliable than UDP, but UDP has a higher rate than TCP because TCP has a congestion control mechanism when packets are lost and retransmits lost packets. UDP has a high rate but low reliability. Google hopes to build a high-reliability and high-rate transport layer protocol, QUIC. The core feature of QUIC is independent logical streams. It allows multiple data streams to be transmitted in parallel on a single connection, and each stream can be processed independently. In contrast, TCP only supports a single data stream and needs to receive and confirm each message in the order it is sent.
Failed transaction diagram, source: bread
The main reason for Solanas downtime is the use of the QUIC experimental application layer protocol. Due to the speed of UDP and multi-channel transmission, and the desire to maintain the transmission of complete data, it will also design a mechanism to retransmit multiple times in the event of packet loss. When the Leader node receives multiple transactions, it opens multiple channels through the QUIC protocol, but the Leader node is a computer after all, especially the transaction capacity that can be processed online. Therefore, when a large number of transactions flow in, the Leader node will cut off certain channel connections, which will cause the transaction to be dropped. There is no set standard for how to choose the connection to be cut off (such as cutting off all connections with a fee lower than xxx), and whether all connections will be cut off is random. Therefore, this leads to a certain space for black box operations. The Leader node may prefer profitable MEV transactions and abandon users low-value transactions.
MEV
In Solanas block generation mechanism, since RPC interacts directly with the Leader and adopts the FCFS principle, it does not have a Mempool like Ethereum. Due to the existence of Mempool and Ethereums permissionless principle, Ethereum faces a more severe MEV problem in comparison.
MEV architecture, source: Helius
Jito Labs client currently occupies 50% of the client market share, so Jito Labs built a pseudo mempool by itself. Users enter a pseudo mempool through RPC and stay for about 200 ms. Jito Labs provides an off-chain inclusion guarantee to ensure that all transactions in the bundle are included in the block. Searchers can bid for the opportunity to attack pending transactions. Searchers bid for the Bundle that maximizes profits, and then the Block Engine is responsible for finding the Bundle with the highest bid and submitting it to the Leader running the Jito Labs client.
This is the root cause of MEV, but MEV has its positive externalities and demand. If Jito Labs does not do pseudo mempool, other projects will do it, so Jito Labs chooses to eat this market to improve the mechanism of MEV and reduce negative externalities. Of course, this demand for MEV bots puts users in the most vulnerable position, because validators will charge fees, mev bots will get arbitrage profits, but users will suffer higher slippage and possible failed transactions.
State Growth
Solanas POH mechanism and Turbine consensus cause its blocks to be too large, which will cause the problem of state growth. Currently, there is no definite answer to the size of the ledger, and the ledger is still growing at a rate of one block every 450 ms in a practical environment, about 4 PB per year (running at maximum performance of 1 GBPS). Currently, Solanas historical construction occurs after 2 epochs, which is about 4 days (a total of 100-200 GB). The past data is stored in the Google Bigtable database.
Solana’s ledger data is not transparent, and the official has not disclosed much about the extremely high block size and potential impact caused by the pursuit of large blocks and high TPS throughput. The storage of the ledger is also completely dependent on third parties, because the official has also found that centralized databases such as Google have higher performance than Genesys Go, Arweave, etc. At present, these decentralized databases still have problems with commercialization. This rapid growth and centralized hosting are one of the reasons why Solana has been criticized.
Outlook
Solana also released its future roadmap, including:
1. Improve the protocol for issuing tokens, including transfer encryption, Hooks, and metadata pointers.
2. Client improvements, including the lightweight client Tinydancer, the transitional client Frankendancer, and the final client Firedancer.
3. Supporting development components of the ecosystem: Gmaeshift focuses on game SDKs, armada markets focuses on token life cycle improvements, SPE focuses on enterprise-level SVM blockchains, virtual machine improvements, etc.
We can see that Solanas POH algorithm and Turbine consensus mechanism both prioritize performance in the blockchain trilemma. The benefit is that they have the best performance in the current environment, which brings a wider range of applications that can be explored. And with Solanas strategic goal of consumer applications, there is a great possibility that some mass adoption applications will emerge. At the same time, the brand effect of projects on Solana is weak, so there are more opportunities for entrepreneurs.
Solanas main advantages in ecological development are DePin/AI and Meme, but we can also see that its ecological development has not yet reached the expected development, and the Consumer App has not yet been commercialized. In terms of competitors, there are also rising stars such as Base, and Bases financing amount and market share are rapidly increasing.
Solana has also faced some technical problems, including downtime, transaction failures, MEV, rapid state growth, and centralization, but the positive side of Solana is that it does not focus on redundant infrastructure construction, but relies more on the current TPS capacity to build consumer-oriented applications, and its roadmap is also centered around this. With the construction of more and more Layer 2 and the launch of clients, the TPS of the SVM ecosystem will be taken to the next level. Solana is still an oasis, with many ecological projects that capital has not fully reached, and there are many opportunities for entrepreneurs to explore.
About Gate Ventures
Gate Ventures is the venture capital arm of Gate.io, focusing on investments in decentralized infrastructure, ecosystems, and applications that will reshape the world in the Web 3.0 era. Gate Ventures works with global industry leaders to empower teams and startups with innovative thinking and capabilities to redefine social and financial interaction models.
Official website: https://ventures.gate.io/
Twitter: https://x.com/gate_ventures
Medium: https://medium.com/gate_ventures
