Looty postmortem, part 3

The actual postmortem this time

Establishing the timeline and facts of the situation is usually the boring part of the postmortem. This time it happened to be the most drama-packed part.

But lets get into the actual meat of a traditional postmortem. If you came here for funny jokes and pictures, this is not the post for you, move along.

Normally this would be written by someone in consultation with all the stakeholders, but that’s not really in the cards for this one. So I’ll do it from my point of view.

What went right

This is often the emptiest section, and this case is no exception. The only thing I can really put down in here is the fact that no one acted on the vulnerability earlier, which had been present for at least half a day.

And, I guess, the fact that the whitehack and subsequent return of assets went smoothly.

Where we got lucky

Also not much to put down here. If you squint a bit, maybe you can say that the more obvious and less problematic vulnerabilities coupled with the ensuing Twitter drama lead to the issue being detected via an impromptu audit, where it might not otherwise have been until something went catastrophically wrong.

If you want to include the S1 events, then maybe also lucky that some dumb asshole came forward to assist instead of just ignoring the issue or posting about it on Twitter for clout.

What went wrong

Is there a word limit on these things? I guess we’re about to find out. I’ll break this into sections and address them one by one.

Root cause - unsafe code in production

It seems almost tautological to have to say this, but the code was exploited because the code was exploitable. If the code was safe, none of this would have happened, end of story.

But why was the code unsafe?

1 - Lack of auditing

Obviously, an official audit would have caught these problems. And in hindsight, it does seem a little shortsighted not to get an audit to protect 200K of assets (twice!).

Not everyone can afford an audit. I’ve written a ton of Solidity but never paid for an audit myself.

But there are people out there who offer ‘audit lite’ services that are more affordable. I’m honestly not qualified to even offer that, but regardless they had three competent devs in chat and didn’t bother to get a sign-off that the code had been reviewed from even one of them.

It shows a fundamental disrespect for the concept of getting outside advice.

Sidebar: do you, the reader, need an audit lite?

Then let me direct you to independent security auditor 0xToshii, someone far more competent than I am at finding security vulnerabilities. I’m told they are available for smaller engagements (like Looty would have needed) or larger ones as necessary.

2 - Insufficient time

Pure speculation on my part, but between the short amount of time between being asked to test, and the launch, I’m guessing the tragic ‘develop to meet a schedule’ problem took place. I’m personally very familiar with that situation (‘we need to launch by this conference date!’).

If I make even small changes to Solidity code for Ferdy Flip, I AGONIZE over them for a whole day. I write tests. I test manually. I get someone to review.

It’s possible they were rushed even during development, but I can guarantee that they didn’t do sufficient testing on Fuji, otherwise some of these issues were literally impossible to not notice. A lack of time allocated to testing could explain that.

3 - Insufficient devs

On the other hand, it’s possible that they thought they they had enough time to dev it, and enough time to test it, and that their testing was sufficient. Clearly it was not.

It’s hard to say, having only had brief discussions with them, and viewing the end result. But there’s definitely some of this at play. It’s just too unlikely that they fucked things up again and again and again without some of it being their fault.

Like, come on. No one thought to review every public/external function for access controls? That’s just the low hanging fruit. The Chainlink problems should also have been obvious in my opinion. Lack of understanding of how ERC-721 approvals work, awful. There were other marginally harder exploits as well (I’ll leave finding them as an exercise to the Looty devs).

Some of the things they said pointed to them being extremely new to EVM. I’ve noticed that there are a set of tells that Solana developers have. One of them, for example, is a reluctance to verify their code, and the incorrect belief that unverified code cannot be exploited.

I have no idea why you would let novice devs (at a minimum, to EVM, not clear on their other work) write code protecting assets like this, unsupervised by an adult.

Contributing cause - being assholes

I probably made it clear in my previous articles already that I was unhappy with the Looty team, pretty much from the first interaction. It didn’t really get much better from there.

But, as you might be aware, I’m generally kind of a pushover. It took a lot for me to get to the point where I was strongly considering doing things I might regret. It’s actually kind of impressive that they managed to push me that far.

It probably goes without saying that this is a bad thing. I’m something of an asshole myself, but I hope that one day when someone comes to help me with something I’ve fucked up, that I don’t piss them off this badly.

But no, the real mistake here was getting in a pissing match with VirtualQuery on Twitter (and harassing people into trying to silence him). He has something of a devil-may-care attitude, so he was ready and willing to execute the whitehack.

Although the best practice is to whitehack at-risk funds immediately and deal with things afterwards, someone on better terms would probably have given them a heads up, or even inquired if they are prepared to quickly remove the at-risk funds.

Instead, the presumption was that they were jackasses that either wouldn’t believe it or wouldn’t act quickly enough, and either way didn’t really deserve a courtesy notification.

Action items

Normally this is where I would put work items to clean up after the incident, remediate the high level risks, and plan for longer term improvements in reliability.

But they can just fuck right off. I’m not their employee. Figure it out on your own.

Always willing to help a friend, or a friend of a friend, with advice or casual review. But this is the second time in the last few months I have regretted getting involved and helping strangers.

So my personal AI is to attempt to commit myself to not doing that again.