Audit Methodology.md

Audit Methodology

The Kaju Katli Approach

Notice the shape

1. We start from the bottom, approach the middle become large and then again become narrow down going to the top.
2. That means, I do not study the docs or try to get the larger picture about the protocol in the beginning.I completely avoid that.
   • Reasoning: Do not want my mind to be corrupted by more information than necessary at this point in time. This is the time to focus completely on the lower level details, the implementational issues that often spring up during other audits.
   • Some examples are:
     • check whether the target addresses of call and delegatecall are being validated or not
     • are the functions able to properly handle the weird ERC20 tokens.
     • Is the CEI pattern being followed everywhere.
     • Verifying the calculations used. Over time you start seeing patterns.
     • Price Oracle validations are completely done or not
   • Here, experience as a dev comes in handy. Think about the biases that you might have had as a developer and see where it could have snuck in.
   • Some examples are:
     • The default value hell. Eg. The first value in an enum would be the default state

       •  enum Status {
               Pending,
               Shipped,
               Accepted,
               Rejected,
               Canceled
           }
         

     • For any for loop you encounter, ask the three question:
       • What if i pass an empty array
       • What if i pass duplicate items in the array
       • What if i pass the default values in the array
     • Are the external functions having sufficient validation
     • Is the dev trusting someone that they should not
3. Now, having done that and marked all the problem issues. I determine whether those marked places are issues or not. So, now I am sure I do have issues, I do not yet categorise the severity.
4. Now, I get on to the part where I do not see code for some time.
   • I read the docs.
   • I study about the underlying financial concepts (necessary when the DeFi protocol is trying to try something new)
   • Because most of the times the new variation was tried out earlier and it was dropped because of some reason. Find that reason and ascertain that this protocol does away with that reason convincingly.
   • This is basically preparing you for a bit of threat modelling
5. Now when I understand what the protocol/code is actually meant to do, I re-read the contracts, but this time in a much more cohesive way. This time, the code should tell you a story and not seem like an implementation
6. The only difference between the docs and the code is that the code tells you a story in Solidity while the docs tell you a story in English.
7. Be good enough at both languages to understand where exactly the 2 stories do not match
8. This part of the audit is mostly doing threat modelling where I try and find faults with the basic underlying assumptions of the protocol logic. Good understanding of economics, game theory and the defi landscape in general is helpful here.
9. Evaluate situations like what happens if you get blocklisted, what happens if this token is depegged, what happens if a bank run happens against your token, what happens if the rate of withdrawal reaches such and such level.
10. Lastly, think of breaking the protocol not as a black hat hacker but as a competitor who is completely fine taking on a financial hit just so that they can grief the protocol. You'll be surprised at the number of things you find thinking this way.

Some really good-to-do steps (that I can't always get around doing)

1. If a contest for a streaming protocol is coming up, then go through the publicaly available audit reports of other streaming protocols.
2. Go through the tests, and get a very good idea of what particular areas are under-tested. These are the areas that require your special attention.
3. Even for the areas that are well tested, see if you increase the sophestication of fuzzing used. This often leads to finding some really cool edge-case bugs that aren't practically discoverable using just manual analysis.
4. [Slightly more extreme] Go through the audit reports of the external protocols that are interacting with your protocol and also see what changes have been made to that external protocol since the last audit.
5. Build out POCs. That is THE ULTIMATE challenge to your assumptions. You'll be surprised how many times you are humbled. This is reason why c4 has invite-only contests and sherlock has penalties on wrong submissions

General tips for a better future as an auditor

1. Prioritise & focus. You can do anything you put your mind to. But you cannot do everything and you have to put your mind at some place.
2. Expect to be humbled every now and then. Do not let your ego get in the way of taking risks. Remember what got you here.
3. There are two types of people when it comes to motivation. Ranveer Singh and ShahRukh Khan Figure out which type are you and optimize your mental state for that.
4. Discard believes that do not serve you. When that voice inside your head says, man, i am not good enough to find a bug in this already audited code, tell that inner voice to go fuck itself.
5. Truly embrace the beliefs that serve you. For example: If it is a very long and comprehensive implementation then sure there are bugs due to the sheer complexity of the code & if it is a very short straightforward implementation then sure there are bugs due to them missing out several edge-cases and/or scenarios.
6. Make it physically easy for you to work hard.
7. Find a community where you can hang out and nerd out on web3 security. This is a lonely profession, you need people to hang out with to remain sane.

Source :

• Rahul Saxena: https://www.youtube.com/watch?v=BDtbTCuJoOM&list=PLn17rrqArf_kin2jhT9447rprtBwwFcL4&index=1&t=6s