The developers of the zkSync second-level solution based on rollups received an unexpected “gift” for Catholic Christmas – the network went offline for 5 hours. Then they managed to restore the functionality of the blockchain.

The team had to postpone the celebration and investigate the incident. Yesterday, December 25, at 7:36 UTC, the zkSync team on the social network X reported that they were “fixing various problems” in the network. The developers said they are “actively working” on the problem.

It turned out that due to an error in the program code of one of the servers, “one of the automatic network security protocols” was launched. Moreover, the failure itself occurred at 5:50 am UTC. The solution was restored by 11:00 am. The developers emphasized that no user funds were at risk.

“The problem itself was not too significant – a relatively simple error related to updating states. They are calculated as part of the protocol, but the operator needs to send them to the first layer of the network. The operator itself also calculates the state data and there was a slight error in the way this calculation was done on the operator's side. Therefore, the states calculated by the protocol and the operator did not match, so the network switched to the most secure mode,” said the developers.

After the error was discovered and corrected, zkSync developers distributed the software update to all network nodes to ensure that the error did not recur. They apologized for the solution not working and promised to work on a faster response process during the holidays.

In March of this year, a large-scale failure occurred in another second-tier solution – Avalanche.