DeFiSafety Quality Certificate proposal

Ecosystem 🤝 Partnerships
1

DeFiSafety just released its review of PoolTogether v5 using our recently updated protocol review process (0.9). This is the third review we have done of PoolTogether. The first being in 2021.

DeFiSafety asks that PoolTogether supports us by purchasing a Quality Certificate. DeFiSafety’s reviews help the ecosystem in general and PoolTogether in particular. This latest review highlighted how PoolTogether’s website links do not reflect the increased role GenerationSoftware plays. Note, once you have updated the links please let us know and we will update the links in our report.

In this way and many others we add value to the system. Our new process requires a documentation of protocol monitoring (both on chain and front-end), dev responsiveness (you guys were really quick), better documentation on protocol admin addresses, signatores and their transaction signing policy. Over the next few months we will be asking these questions of every protocol in defi and poking them to improve their transparency by answering.

We have been doing this consistently for over three years writing nearly 600 reviews. We have consistently made defi better. There is now improved documentation on Oracle’s, specifically on frontrunning and flash loan vulnerabilities. We improved documentation on access controls for many protocols. Protocols that are not transparent or using good software processes consistently get bad scores. This helps DeFi users.

DeFiSafety wants to keep going but we need a reliable funding vehicle that keeps DeFiSafety independent. Quality certificates fill this need. We varied the price of the certificates with respect to the specific protocol. For PoolTogether we ask for 5K for a year. We are flexible on this price, but as these certificates pay our salaries we have to be reasonable.

I spoke on a PoolTogether community call about this a few months ago. The comment was that we should check for grants from other communities. We did, they cannot allow us the funding we need. We ask you to reconsider. DeFiSafety adds value both generally and specifically to PoolTogether, far in excess of 5k per year.

Quality certificates offer several advantages.

  1. It improves visibility of your protocol on our site, meaning it will never be hidden behind a paywall.
  1. A quality certificate badge on your site is a clearer way to indicate safety than a list of audits
  2. It funds DeFiSafety so we can continue to improve the quality standards of DeFi. A bit of funding from different protocol allows us to maintain our independence plus grow our team
  3. We will offer souldbound NFT’s for your wallet addresses so that wallets can indicate the safety before a user clicks Confirm.

Let us know your thoughts. Note a certificate is not mandatory, we have already published your report at no cost. Thanks for your time.

2

Hi Rex,

Thanks for the thoughtful write-up! DeFiSafety has been at it for a long time, so if you haven’t had revenue I understand the need to develop a business model.

To show the safety of our protocol to our users, we always direct them to our audits. Audits are deep and comprehensive; they cover a lot of ground. Although they don’t cover items like documentation quality, they cover the core protocol. Experts can read the audit reports and verify that the protocol does what we say it’s doing.

Audits are not only good for transparency, but they serve as a feedback loop to improve the protocol. An audit report is the result of a back-and-forth between security researchers and the protocol team, and the process makes the protocol more secure.

The DeFiSafety report has pointed out a few broken links, but otherwise hasn’t been useful as a tool or process to improve the protocol.

You say that our DeFiSafety score is a clearer way to indicate safety than our audits, and yet I’ve only ever heard of people referring to our audits. People don’t mention our DeFiSafety score, so I’m not sure how compelling it is to users.

If you are providing value to DeFi users, then they should provide value back to you! They are your customers, whether directly or indirectly. I think you need to target them.

In short, I don’t think your ask makes sense for us.

I do want to thank you for taking the time Rex: both you and your team. I wrote a proper response because I respect the work that you guys have put in.

2 Likes
3

Brendan, Thank you for taking the time to respond. Our report did not point out many weaknesses in PoolTogther. You are a great protocol in security and take advantage of short-term elements of lotteries which allows you to have immutable software with limited update capabilities. This immunizes your protocol from many of the threats that have been growing in the industry. In addition, I have immense respect for the very strong community you have built.

However, your response highlights some of the consistent weaknesses I have found that is stunting the growth of our industry. These weaknesses are narrow focus, silos and “Somebody Else’s Problem”. I come from the aerospace industry. This industry developed when it was very young a tremendously healthy culture to improve safety. I hope our industry will take inspiration with what aerospace did in the 20s and 30s (between the wars) to develop a global culture of improvement. We need it. Right now we are not in a good place.

When I talk about “narrow focus” I specifically focus on our priority we put on smart contract audits. When DeFi was young (in 2020) smart contract security deserved to be the primary focus of our industry. But we have matured and for many protocols their actual risk of a smart contract hack is quite small. Most security incidents today are around attacks on individuals (phishing attacks) leading to lost keys, weak development processes (leading to predictable hacks) or front end attacks. Security is more than just audits and this has to become central to our culture. When areospace was young, safety was not just building better airplanes, but better runway lights and radio aids. DeFi must have a broader culture of security if we are to grow.

In my experience and I deal with a lot of different protocols, each protocol lives in its own silo. Except for exchanging tokens, each protocol has limited communication and cooperation. Our industry has not created industry related groups. The DeFi ecosystem hates supporting “centralized” organizations to represent it globally. In this area our quest for “decentralization” works against us. In its young days aerospace creating organizations such as ARINC and GAMA. These industry run and funded organizations worked to improve safety overall globally. With our phobia of centralization we are hurting ourselves badly.

Let’s take an example of perceived safety. This directly affects PoolTogether. We all know that if you’re using MetaMask, and a Ledger properly with AAVE, Curve, Balancer Yearn and PoolTogether your actual risk of loss is extreamly low. However there is no organization to research and communicate this to the general public. The result is the perceived safety of DeFi transactions is very low when most transactions are very safe. This hurts PoolTogether greatly. The news industry has no incentive to communicate this, because sensationalized hacks drive readership. I am sorry to say it but the auditor community also has no incentive to communicate this. They are paid by the hour by the protocols. This bad perception helps their bottom line. We do not have an industry group to communicate this simple truth. It hurts our industry greatly.

We as an industry must find a way to work together better. We need to do this in order to thrive and the sooner we start the better. DeFiSafety is a small part of that solution. But the global problem of silos is the real problem.

Finally, I highlight your comment about the users of DeFi who should fund DeFiSafety. Guess what, the users of DeFi think everything should be free. Or they think the protocols should pay for it. Either way they consider it, as did you, that Quality in DeFi is perenially “Somebody Elses problem”.

I hope someone gets some value out of this essay. If anyone wants to try to improve things and sees DeFiSafety as any part of the solution, please reach out.