Metaroids

    Subscribe to the Metaverse Newsletter

    Join me as we build a network of metaverse and web3 pioneers.

    What's Hot

    OpenAI Hired Developers to Train AI to Replace Developers

    January 28, 2023

    Ex-Google SVP Launched AI-Powered Search Engine

    January 28, 2023

    BuzzFeed Plans to ‘Hire’ ChatGPT as Newest Content Creator

    January 28, 2023
    Twitter
    Metaroids
    • Home
    • News

      OpenAI Hired Developers to Train AI to Replace Developers

      January 28, 2023

      Ex-Google SVP Launched AI-Powered Search Engine

      January 28, 2023

      BuzzFeed Plans to ‘Hire’ ChatGPT as Newest Content Creator

      January 28, 2023

      Lawyer Could Face Jail Time for Bringing AI in Court

      January 26, 2023

      Anand Mahindra Sparks Alarm Over Deepfake Videos

      January 25, 2023
    • Learn

      Midjourney Mastery: A Guide to Using Image Prompts

      January 22, 2023

      Design UI Like a Pro: 5 Advanced Midjourney Tricks

      January 13, 2023

      Reddit Moons: An Ultimate Guide

      January 12, 2023

      The ChatGPT Era: How Creatives Can Ensure Longevity & Thrive

      January 7, 2023

      Google Translate vs ChatGPT vs DeepL: Translator Ultimate Showdown

      January 6, 2023
    • Others
      1. Feature
      2. Press Release
      3. Opinion
      4. Lists
      5. View All

      ChatGPT: How AI Can Potentially Impact the World’s Religions

      January 27, 2023

      Humans’ Quest to Decode Animal Languages Through AI

      January 25, 2023

      ChatGPT + Wolfram Alpha: A Super Powerful Assistant

      January 22, 2023

      AI-Powered Sexbots and the Risks They Could Trigger in the Future

      January 21, 2023

      Hyped NFT Trippin’ Ape Tribe to Launch on Solana in Mid-Late May

      May 6, 2022

      Microsoft’s VALL-E May Trigger an Avalanche of Cyber Crimes

      January 18, 2023

      Why Web3 Gaming is Failing & How to Turn the Tide

      December 23, 2022

      Can the Metaverse Work Without Blockchain?

      August 17, 2022

      NFTs are 100% Sustainable: Here’s Why Critics are Wrong

      August 8, 2022

      Best Solana Wallets for NFTs, DeFi, & Security

      October 27, 2022

      Top Move-to-Earn Crypto Apps That Pay You to Exercise

      August 31, 2022

      Top 13 Blockchains for NFTs (Choose Wisely)

      August 16, 2022

      Top Selling NFT Collections of August 2022

      August 9, 2022

      OpenAI Hired Developers to Train AI to Replace Developers

      January 28, 2023

      Ex-Google SVP Launched AI-Powered Search Engine

      January 28, 2023

      BuzzFeed Plans to ‘Hire’ ChatGPT as Newest Content Creator

      January 28, 2023

      ChatGPT: How AI Can Potentially Impact the World’s Religions

      January 27, 2023
    • Contact
    Twitter Discord LinkedIn
    Metaroids
    Home » Why is Solana Having Outages? The Truth You Should Know
    Feature

    Why is Solana Having Outages? The Truth You Should Know

    By DamoclesOctober 12, 2022Updated:October 12, 2022No Comments9 Mins Read
    Share
    Facebook Twitter LinkedIn Email Reddit Telegram

    The Solana network has gone down eight times throughout its two years of existence. In 2022 alone, the network experienced five outages that repeatedly sent chills down the spines of investors, developers, and the community.

    Solana’s degraded performance isn’t something new. In fact, the first Solana outage occurred back in December 2020, when there were barely any users on the chain. You can only imagine how ‘heavier’ its network load is in 2022, as it peaked at 1.1M daily active users in the second quarter.

    The fear of another potential network freeze remains high for builders developing platforms on the chain, as well as end users who utilize Solana-based decentralized apps (dapps) on a daily basis. The next mandatory day off, so to speak, could happen at any moment, it seems, as the network becomes more unpredictable.

    So how bad is the outage situation? When will this be fixed? Is it something that users need to accept as the norm? We will address all of that in this Solana outage full report.

    This guide aims to get to the bottom of Solana’s downtime issues, learn what caused these failures, and uncover the ultimate solutions that are being proposed.

    Table of Contents

    • Why is Solana Having Outages?
    • List of Solana’s Outage Incidents
      • Solana Outage 1st Occurrence 
      • Solana Outage 2nd Occurrence 
      • Solana Outage 3rd Occurrence 
      • Solana Outage 4th Occurrence 
      • Solana Outage 5th Occurrence  
      • Solana Outage 6th Occurrence 
      • Solana Outage 7th Occurrence 
      • Solana Outage 8th Occurrence 
    • Will Solana Be Able to Fix Its Outage Problem? 

    Why is Solana Having Outages?

    Solana is experiencing multiple outages because of problems that emerge from high network demand coupled with relatively few validators to handle the load. That, and the bugs in its blockchain system.

    The network suffers every time transaction inflows become abnormally high, which normally happens whenever someone deliberately attacks Solana (DDoS attack) or utilizes bots to gain an advantage in non-fungible token (NFT) minting events.

    Unlike Ethereum, Solana has ultra-low fees (roughly $0.00025) that make it really affordable to launch an attack on the network or flood it with bots.

    But this issue can be potentially mitigated if the network has more validators, as each node would not be overwhelmed by the overflow. The common reason why Solana has few node operators is because of its expensive validator hardware requirements, which have pros and cons to it. 

    To run a Solana validator node, you need a device with at least 12 cores/24 threads CPU, 128GB RAM, and 500 GB disk space. RPC node recommendations are even more demanding. Check out the full hardware requirements here if you wish.

    The pricey requirement repels bad actors, but, as a consequence, also severely limits the number of participants that can help the network handle its massive transaction demand. This is a huge predicament since more validators translate to not only better performance and less probability of downtime, but also better decentralization.

    Solana’s longest outage incident lasted for 17-long hours, which occurred last September 2021. The shortest Solana downtime happened on January 2022, which interrupted the network for four hours. 

    List of Solana’s Outage Incidents

    We tracked the full history of Solana’s downtime events from the first occurrence down to the latest.

    IncidentDateReason
    1st OutageDecember 4, 2020Mainnet cluster failure to produce blocks
    2nd OutageSeptember 14, 2021DDoS attack or bots
    3rd OutageDecember 2021 Alleged DDoS attack
    4th OutageJanuary 2022DDoS attack or bots
    5th OutageMarch 2022Bug causing RPC nodes to fork
    6th OutageApril 30, 2022Bots
    7th OutageJune 2022Bug
    8th OutageSeptember 30, 2022Consensus Bug
    Solana’s Outage Incidents

    Let’s look at each downtime occurrence in greater detail.

    Solana Outage 1st Occurrence 

    Date of Occurrence: December 4, 2020

    Barely nine months after its launch, the so-called ‘Ethereum killer’ suffered a six-hour outage incident due to the failure of its Mainnet Beta cluster to produce new blocks. With its operations halted, the blockchain’s ability to confirm new transactions was paralyzed. 

    The incident, which happened on December 4, 2020, was immediately resolved the next day through Solana’s 200+ validators, who restarted the network. 

    Solana Outage 2nd Occurrence 

    Date of Occurrence: September 14, 2021

    On September 14, 2021, Solana experienced a 17-hour outage on its network due to distributed denial-of-service (DDoS) attack. This type of cyber attack involves flooding a network with overwhelming traffic to paralyze its operations. 

    1/ Solana Mainnet Beta encountered a large increase in transaction load which peaked at 400,000 TPS. These transactions flooded the transaction processing queue, and lack of prioritization of network-critical messaging caused the network to start forking.

    — Solana Status (@SolanaStatus) September 14, 2021

    In Solana’s case, Grape Protocol conducted an initial DEX offering (IDO) on Raydium, in which 400,000 transactions per second (tps) flooded that operation. The incident was so severe that it became the network’s longest outage so far. 

    The protocol’s validators opted to restart the network (again) to mitigate the problem in the most convenient way possible. 

    Note: Grape is a web3 social networking protocol, while Raydium is an automated market maker (AMM) and liquidity provider on the Solana blockchain. 

    Solana Outage 3rd Occurrence 

    Date of Occurrence: December 2021 

    Solana hasn’t logged this event on its Incident History, but on December 2021, there were reports that Solana had suffered again from a DDoS attack, temporarily clogging the platform. 

    But unlike the previous incident, the network appeared to have remained online and hasn’t shown any severe downtime. 

    Blockasset, a Solana-based NFT project, detected the incident and alarmed the community about the supposed DDoS attack. 

    There were conflicting claims about whether or not the network has suffered from a global outage again. It’s almost a year since this alleged downtime occurred, but it seems that the network doesn’t recognize this as an ‘official’ outage. 

    Solana Outage 4th Occurrence 

    Date of Occurrence: January 2022

    Barely a month after the last reported outage, Solana hogged the headlines again when it suffered from protocol problems causing network instability and degraded performance. 

    There were conflicting updates during this time, with Wu Blockchain, a web3 news outlet reporting a 4-hour network outage, while Solana Status maintained its stance of a 100% operational platform. 

    Anatoly Yakovenko
    Anatoly Yakovenko

    Anatoly Yakovenko, one of Solana’s co-founders, denied that another DDoS attack caused the incident. Solana Status has logged this issue two days later on its Incident History as a mere ‘performance degradation’.

    Solana Outage 5th Occurrence  

    Date of Occurrence: March 2022

    There weren’t many reports about this particular network problem, but the protocol logged this issue on its incident records. According to the data, the platform suffered from a degraded performance last March 10 and 21. It also registered that on March 28, some of its remote procedure call (RPC) nodes were down. 

    While not as serious as the other occurrences, this was caused (partly) by a faulty update that caused some RPC nodes to fork off.

    Solana Outage 6th Occurrence 

    Date of Occurrence: April 2022 

    On April 30, 2022, Solana reported that its Mainnet Beta cluster had frozen, preventing the network from creating new blocks and reaching consensus. 

    Validators later discovered that it was caused by the flooding of transactions in the network, clocking at 6 million tps, equivalent to over 100 Gbps of traffic. As a result, the Solana community suffered a seven-hour outage. 

    But Solana quickly clarified that there was ‘no evidence’ of a DDoS attack. Instead, it blamed the bots for the incident, which were programmed to win an NFT mint at Metaplex’s Candy Machine (an NFT launch program). The network was later restarted to resume the platform’s operation. 

    The downtime occurred a couple of days after the Okay Bears launch, which broke records as the most hyped (and botted) Solana mint at the time. Coincidence? Probably not.

    Solana Outage 7th Occurrence 

    Date of Occurrence: June 2022 

    This time, the outage lasted for four and a half hours due to a bug that generated different outputs that caused a major consensus failure. 

    Yakovenko explained that the incident stemmed from the ‘durable nonce intersection’, a feature that prompted a part of the network to consider a specific block invalid. To prevent this incident from happening again, this feature was disabled as the network restarted. 

    Solana Outage 8th Occurrence 

    Date of Occurrence: September 30, 2022

    Last September 30, 2022, Solana continued its ‘Outage Saga’ with its latest downtime episode, which lasted for six hours. This time, the network halt was caused by a misconfigured node, freezing the protocol’s entire operations. 

    The Solana network is experiencing an outage and not processing transactions. Developers across the ecosystem are working on diagnosing the issue and to restart the network. More information will be provided as it becomes available.

    — Solana Status (@SolanaStatus) October 1, 2022

    Solana Status initially reported that the platform was experiencing degraded performance and later updated it as a full-blown outage due to a consensus algorithm bug.

    Will Solana Be Able to Fix Its Outage Problem? 

    The Solana founder himself (Yakovenko ) assured the community that fixing the outage issues is the “number one priority” of Solana Labs. It has introduced three major upgrades to help contain DDoS attacks and bots and proposed easier ways for users to run validator nodes.

    As of press time, there are two solutions (stake-weighted QoS and Fee markets) that are “in progress” and one (QUIC) launched on testnet.

    If there is one thing that can significantly reduce SOL’s outage problems noticeably in the short term, it would be the speedy increase of its validators. No one can deny the power of numbers, as Ethereum, which currently has 8,373 nodes in its network, remains to be one of the most robust blockchain networks in the field. In contrast, Solana merely has 2,065 validator nodes despite handling 30-40x the transaction volume of ETH!

    As a response, the Solana Foundation opened its server rental program to bring down the cost for would-be validators. While this isn’t the best compromise since the servers will still be highly centralized (controlled by fewer entities), it will help the performance massively if more people can become independent validators.

    In the long run, this issue will still sort itself out as the cost of running a server is trending down thanks to Moore’s Law. But patience isn’t the primary virtue of most web3 users.

    As for the systemic bugs, the current solution is to have independent teams building separate validator software, such as Firedancer by JumpCrypto. This is because the possibility of having the same bug in different validator software is nearly zero, which prevents network-wide contagions that lead to downtimes.

    Solana
    Share. Facebook Twitter LinkedIn Email Reddit Telegram
    Damocles
    • Website
    • Twitter

    Damocles is a self-imposed web3 nomad with a background in journalism and education. Having served as a freelance writer for three years, he decided to focus on one area he loves: the blockchain industry. He enjoys the drama, controversies, and pleasant surprises of this growing arena, pumping him to write more daily. To this day, he still considers himself a newbie who is dead set on honing his skills and being more involved in web3 projects.

    Related Posts

    ChatGPT: How AI Can Potentially Impact the World’s Religions

    January 27, 2023

    Humans’ Quest to Decode Animal Languages Through AI

    January 25, 2023

    ChatGPT + Wolfram Alpha: A Super Powerful Assistant

    January 22, 2023

    AI-Powered Sexbots and the Risks They Could Trigger in the Future

    January 21, 2023
    Add A Comment

    Comments are closed.

    Latest Articles
    News

    Ex-Google SVP Launched AI-Powered Search Engine

    By Stark
    News

    BuzzFeed Plans to ‘Hire’ ChatGPT as Newest Content Creator

    By Damocles
    Feature

    ChatGPT: How AI Can Potentially Impact the World’s Religions

    By Damocles
    Metaroids
    Twitter
    • Home
    • About
    • Contact
    • Our Authors
    • Privacy Policy
    • Sitemap
    Privacy Policy and Terms of Services
    Copyright © 2023 - All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.