Looty postmortem, part 1

Because it's dead, get it?

I’m (unfortunately) very familiar with the process of writing and reviewing a technical postmortem; a document that discusses what went wrong, when, why, and what needs to be done to make sure it doesn’t happen again.

I’m going to go a bit more heavily into the timeline aspect of things, and although I’ll try to present events factually as I knew them, you should be aware that I am very annoyed over this whole thing and there’s probably some bias in this post.

Ancient backstory

Many years ago (December 2023) Deep Water Studios (DWS) was approached by someone looking to build what eventually became Looty. Although it seemed interesting to me, unfortunately due to life commitments and the holidays we didn’t have any capacity to take on a job of this size.

In fact, we generally never have capacity to take on extra work, we’ve always either got a full plate or are on vacation. Sorry to all the people who keep asking =(

Looty, the product

A lot of this post is going to be very down on Looty, so let me front load the good things I have to say about it.

Personally I think this product is a great idea. I’m not sure what the market is like for the business’s they hope to get to sponsor these events, but I imagine businesses wanting to use it. And I can imagine people liking to use it.

This is the classic example of ‘ideas are easy, execution is hard’. Making ‘a site that has lootboxes’ is easy-peasy (theoretically, I guess at this point). Getting people to sign up and give you money to run these mint/open events is a very hard, real world, business development problem.

Which is what makes this AvaLabs sponsored giveaway thing fucking genius, IMO. Everybody wins; Looty gets free exposure and testing, users get free shit, AvaLabs gets to reward communities and make people happy.

Not to beat a dead horse here, but running the Discord and Twitter and marketing and organizing partnerships is very difficult stuff that you can’t solve by just building good tech. And I do think they did a reasonably good job at it.

If someone had come to DWS and said ‘hey we want you to give away a bunch of shit on Ferdy Flip using loot boxes’, sure, we could have done that. But we definitely would not have done all this, the community stuff we would have not done as well, and we definitely would not have spun it into a business. Kudos to them for trying.

And also, I have to give them credit for the UX. It’s not DWS quality (unparalleled, never before seen, groundbreaking, etc) but it’s very good.

The first vulnerability

So as a niche micro celebrity on a downbad blockchain CT community, obviously I was eligible for one of the InspectXYZ keys (87th most popular person in a community of 139 people). So I got a free key to mint. And I minted it.

Immediately after I minted it, I thought to myself “wait, what happened, how was my key type picked, why didn’t I try to see if I could manipulate it”. And right away I found my first vulnerability.

I have to admit, I was a bit giddy. I had found my first critical vulnerability in a production (ish) product.

But before reporting, obviously I wanted to make sure it was legit. So I wrote a Forge script to simulate burning it.

Anyway, I was pretty satisfied that it was a legitimate issue, enough to report and (hopefully) get a bounty for. It’s pretty common to get paid a bounty when you report a critical security vulnerability right?

Suffice it to say I was a bit annoyed by this response, so I closed the ticket. I told some people about the conversation and they convinced me that For The Good Of The Ecosystem I should just report it regardless.

Fine. So I burned my friend’s key to prove that it works (with their approval) and went back to try and reopen the ticket with that evidence.

But I couldn’t; apparently they had the bot set up so that you could not open a ticket after you’d already opened one? I still had access to the ticket channel though, and the conversation had continued on without me.

This irritated me, and I went off to draft a Twitter post flaming the shit out of them. But again, someone intervened in time to convince me to disclose, and I did.

In the interests of keeping everything copacetic, I deleted my thread.

As a ‘reward’ for this I was given three keys, one of each type. Since they never reimbursed my friend for the key I burned to demonstrate, I sent him one of the keys.

They opted not to fix this issue (create a new non-vulnerable contract, airdrop the keys) and just went ahead with the mint. It seemed to have paid off, as far as I know no one else noticed the issue.

Side note - unverified contracts

The fact that the keys contract was unverified was a huge red flag to me, that made me want to dig deeper. I posted a thread of my thoughts on this topic here. Quick preview:

The second vulnerability

Circling back to my initial investigation, I was originally interested in figuring out how they were minting the keys. So I reviewed the decompile and I went to my trusted companion 0xSmitty with my findings.

Can you believe this piece of shit, exploiting something without telling me so I can exploit it as well?

Anyway, the trick here was that you can just keep requesting to mint keys, and then you can look at the data field of the request to tell what you were going to mint, and only submit the TX that has the key you want to get.

If this sounds familiar, it should, but we’ll get to that later.

The third vulnerability

We got an explanation for why the “burn anyone’s keys” thing was in there, and it was less than convincing.

I’m not sure why you would say something so patently false and easily disprovable. If this doesn’t sound like bullshit to you, you need to check your bullshit detector settings.

The saying goes “where there’s smoke, there’s fire”. So I dug into their website to find their ‘burn key for prize’ contract address, and then looked into that.

Pretty quickly it became obvious that they were using the same ‘backend provided proof’ as with the keys, with the same vulnerability.

So obviously I went and wrote a script to do this, although the backup I suggested to all my friends was the now infamous ‘refresh the page to hack Looty’ exploit. More on that later.

The fourth vulnerability (not really)

This is where the story gets really embarrassing (for me, it’s already embarrassing for Looty). But as always, I’m willing to document my failures as well as my successes.

I kept digging into the decompile, and the more I looked, it seemed like there was a more serious flaw in the code.

If this was correct, then it would be trivial to just drain rewards, e.g. every ERC20 in a single TX, any specific NFT we wanted in a single TX.

So we got into a chat room with them, and pushed them to investigate the code. Because if this was accurate, it was a huge security problem, obviously.

And I’m going to be really brutally honest here, I was kind of an asshole about it. Not entirely unjustified given how many fuckups we’ve encountered so far. But ultimately, I was wrong. The decompiler had incorrectly indicated what was going on, and the code was safe.

Man, it’s hard to describe how terrible you feel after you make a big deal out of something that turns out to be a nothing-burger. I believe at one point I said ‘you have to cancel the mint today’. Super embarrassing, crawl-into-a-hole-and-die bad.

So anyway, dispirited after the more serious issue haven been proven incorrect, I finished up by mentioning the issue with minting keys being deterministic and how people could just simulate the TX and reroll to get whatever they want, and that I didn’t think it was a good idea to do the mint that way.

I was told that they had implemented rate limiting on the backend to prevent that, and I accepted it at face value.

Back to that third vulnerability

So, as you may be aware, the supposed rate limiting did not exist. Rabby displayed the rewards when you tried to unbox, and many enterprising individuals did discover and use this to their advantage.

Obviously I told all my friends who had keys that I expected refreshing the page would let them get better results, and I personally did it as well. The two keys I had been rewarded with for reporting the initial bug turned into two Steady NFTs instead of two “$25 of $COQ” vouchers (actually worth $19) which were my initial rolls.

The people who were not aware, were obviously quite upset about this outcome. Even though I had warned that this would be a possibility, they Looty team was still denying that this was an issue in Discord for quite some time. When I saw that, I reached out privately to let them know.

The fifth vulnerability

After this debacle they created a ‘looty testing group’ chat room with myself and a few other people, with the promise that “people will be rewarded for assisting”. The goal here was mainly to make sure that people couldn’t simulate rewards; they swapped to having the backend submit the TX to do the unboxing.

Pretty quickly I found my first (non) issue; it was possible to get the backend to spit out an error message that contained the proof required to mint the box. This ended up not being exploitable because they had correctly added a safeguard so that only a trusted EOA could do the mint.

Then I found another issue; they weren’t validating the authentication on the web2 request matched the owner of the token being minted. Basically, it was trivial to request that the backend unbox anyone’s token, similar to how in the original bug I found that you could burn anyone’s token, except easier to do because you didn’t even need to submit a TX.

They rejected this report, making up some BS about how they planned to change everything (in the next day?) and that it wouldn’t be a viable exploit. They ended up not doing that and just adjusting it to do the check I suggested (I verified when it went live).

The payoff

As a ‘reward’ for this work, I received a platinum key. Since they supposedly had fixed it so that the keys got better rewards than originally (people were upset about their $300 key opening $19 of $COQ, go figure) I decided to open a box instead of selling it.

Big mistake ($42 of $COQ).

Regrets

So at this point, I had some regrets. I wasted a lot of my time, I didn’t make a lot of money, literally had to exploit a bug I reported to make anything.

I literally got paid in NFTs for doing serious, important, high value dev work. Avax NFTs. What am I doing with my life here.

Basically the only good thing that happened was that I managed to help some friends avoid getting shit rewards.

I decided to just let it go, learn from the experience, etc. But things continued into Season 2, apparently I am incapable of learning lessons from previous projects.

The end?

This ran a bit longer than I expected. Stay tuned for part 2, the ChainLink-ining.