‘Like an Esport’: How OpenSea Secured Its Code With Code4rena’s Audit Contest

Traditional security audits just don’t cut it for smart contracts.

“The stakes are too high,” said Sockdrawermoney, one of many contributors building a decentralized auditing platform called Code4rena. The DAO has pioneered a new model designed by Scott Lewis and Zak Cole, where auditors compete to keep bugs out of live code.

And that’s vital for the smart contracts that power decentralized finance (DeFi). While an undiscovered software bug might cause a program to crash, or some other operational issue, a smart contract bug in a DeFi or asset exchange protocol could instantly lead to the loss of hundreds of millions of dollars worth of tokens.

A sea change

This was top of mind for OpenSea, the world’s largest NFT marketplace, earlier this year. In May, OpenSea introduced a new protocol called Seaport to handle its transactions and help reduce gas fees.

The plan was to migrate to Seaport from the well-established Wyvern protocol. But given the volumes of traffic and cryptocurrency flowing through OpenSea—it hit $10 billion in total volume last year—Seaport needed to be thoroughly battle tested before it took the plunge.

OpenSea contracted with a security firm to run an audit of Seaport. But for a move as big as the migration of its entire platform to a brand-new protocol, it wanted more protection.

“We wanted people from different backgrounds and thought processes to come and look at the smart contracts holistically,” an OpenSea spokesperson told Decrypt. “Particularly, we wanted the best of the best developers to look at Seaport—and many of those don’t necessarily work for auditing firms.”

Wisdom of the crowd

Even the top security firms can usually only spare one or two auditors to review a project for one or two weeks, which isn’t enough time or enough eyeballs to thoroughly analyze a smart contract protocol for lurking vulnerabilities.

Code4rena incentivizes a huge community of auditors to hunt for the rarest and highest value bugs, just like attackers. But unlike bug bounties, the primary goal of C4 is to keep bugs out of production code.

Which is why OpenSea decided to sponsor a two-week public contest with Code4rena to give the Seaport code a second scrubbing before the migration. The prize pool was $1 million.

Strength in numbers

Code4rena is a DAO run by contributors and governed by $ARENA token holders. Its approach is deceptively simple: More is better. Its model involves three main actors: sponsors, wardens and judges.

Sponsors like OpenSea create a prize pool to attract wardens to audit their project. Wardens dig into the code and uncover as many threats as possible. And independent judges, usually the most elite engineers in the C4 community, vet the findings and hand out rewards to the wardens based on their performance.

(The bug submission process is totally anonymous, but for sponsors with KYC, AML or other legal obligations, C4 also hosts invite-only private contests with a smaller number of certified wardens working under NDA.)

Better together

The beauty of a public C4 audit is that it’s competitive and anyone can participate, Sock said.

Wardens run the gamut, from hardcore security engineers to green developers trying to get more experience auditing smart contracts, to people who have already made a fortune in crypto and just want to have some fun hunting bugs.

“It lets new developers and researchers enter the auditing space at a level playing field,” OpenSea told Decrypt.

Finding a rare or seriously bad bug will earn a warden more money than a commonplace bug that’s been submitted by multiple wardens—but everyone who submits a valid bug gets paid, even if that bug has already been reported.

That’s very different from bug bounty programs that only reward the first person to find an exploit. If someone else comes across the same bug a few minutes later, they’re out of luck.

This fosters a sense of healthy competition. Although wardens are incentivized to find the higher-severity bugs, they also end up finding a lot of other bugs in the process, and they’re rewarded for their effort. Some wardens even work together in ad hoc teams.

“You can think of it almost like an esport,” Sock said.

A typical C4 audit will see over 50 wardens generate roughly 400 different bug submissions over the course of one or two weeks.

If a typical audit firm staffs 1-2 week audits with 1-2 auditors, that yields between 40 and 160 hours of code review. Based on wardens’ self-reported averages of time spent reviewing code, most Code4rena contests now see a minimum of 600 hours of code review, with some seeing 1,000 hours or more.

Found and fixed

During the review of OpenSea’s Seaport protocol, C4 wardens uncovered multiple issues that previously went unnoticed, including two that C4 judges deemed as high severity. One of those issues was found by eight different wardens and/or teams. The other was found by only one.

For example, multiple reviewers realized that in certain circumstances orders that had only been partially filled could be processed multiple times. OpenSea corrected the issue within Seaport by putting a safeguard in place so that the numerator and denominator could never exceed a certain specific value.

C4’s auditors also noticed that triggering two specific errors related to fulfillment aggregation one after the other would bypass checks for both. A potentially very bad situation was fixed by making a small modification to the error check.

Many of the bugs identified in the code by Code4rena’s wardens had been missed by others during multiple past reviews and audits.

In June, OpenSea was able to migrate its marketplace to Seaport with the confidence that all major issues had been found and dealt with, an OpenSea rep told Decrypt.

The future of work

But beyond the speed, efficiency and value that a C4 audit can bring to companies looking to secure their smart contract code, the model is also a case study in how the nature of work is changing.

Code4rena runs multiple audit contests every week and anyone can participate, said Christoph Michel (aka cmichel), one of the C4’s top wardens. Since February 2021 when cmichel competed in his first C4 contest, he’s audited code bases for around 100 projects and earned more than $1 million.

“You don’t need to ask for permission or pass a job interview,” cmichel said. “The only thing that counts is how well you can break the protocol.”

The flexibility is also appealing.

“If I’m busy with other work or want a break, I just don’t compete in that week’s contests,” cmichel said. “For me, this is the future of work.”