A fan decompiled the original console versions of groundbreaking action-adventure titles Shenmue and Shenmue II, opening the door to new mods, ports to other platforms and more.
LemonHaze uploaded the decompilations to his GitHub today. (Edit: It appears to be private again as of right now, but it was public earlier this morning.) They include every region of Shenmue on Dreamcast and Shenmue II on Dreamcast and Xbox, both games on PC, the U.S. “Passport” demo, a couple beta releases and What’s Shenmue, a Japanese prerelease demo for Dreamcast. The complete list is as follows:
[DC] What’s Shenmue
[DC] Shenmue 1 JP
[DC] Shenmue 1 US
[DC] Shenmue 1 US Passport
[DC] Shenmue 1 EU
[DC] Shenmue 2 JP – GameJam Beta
[DC] Shenmue 2 JP
[DC] Shenmue 2 US – Beta
[DC] Shenmue 2 EU
[XB] Shenmue 2
[PC] Shenmue 1
[PC] Shenmue 2
A software engineer and security researcher by day, LemonHaze used IDA and Ghidra to decompile the titles. His work even helped make Ghidra better, as he found that it didn’t support a few instructions in the Dreamcast’s SuperH-4 processor, for which he submitted changes to Ghidra’s GitHub.
While so many retro gaming projects this year have leveraged LLM AI, LemonHaze did everything himself.
“I’ve spent quite a lot of time on it over many years, way before LLMs even existed,” he told SHIRO!.
“Quite a lot of the code itself is ‘baked’ in, meaning that AM2 essentially wrote code that generates small code for them and I’ve essentially reversed that process, so I can turn regular text into the same code again, so like I said earlier, I used a hell of a lot of custom scripts that I wrote specifically to name certain functions and apply certain type definitions across memory ranges and of course the usual type recovery and variable/field naming. ShenmueDK and ShenmueDKSharp/wudecon are used inside the decomp too, but they were released publicly in their bare forms years before LLMs too. in terms of writing code for actual use within the decomp, the answer is still no, I’m still experimenting with it basically even right now. the fact that the original code was written in pure C is something of a benefit that could potentially warrant a refactor to more modern approaches, but mostly the decomp is following what the original code does already, and I used heavy amounts of scripting to batch decompile certain functions for example years ago, but I’m definitely interested in a refactor that is based out of the decomp”
— LemonHaze
LemonHaze has a long history of digging through Shenmue games’ code. It began when Sega released a remaster of Shenmue I & II on modern platforms in 2018.
“Many people asked for model extractions for a while on forums and I always liked Shenmue, I had a Dreamcast when I was younger,” LemonHaze told SHIRO!. “But it didn’t really become something bigger until the Shenmue 3 announcement/re-releases, when I actually picked it up and in the first month I essentially went straight ahead to reversing it, because I wanted to produce modding tools again at that time and it seemed like the obvious target.”
He joined a Discord server called Shenmue Reverse Engineers and worked with others to unused features that the remaster’s developer D3T left in the code. After extracting models and fixing bugs over the course of a few weeks, most of the other fans dropped off, but LemonHaze kept plugging away.
“I essentially kept working on fixes, mods, all sorts of projects and within the following month I’d already maintained a reversing database that was nearly 50% complete,” he said. “Back then, reversing was a pretty intense thing, but the mastery was in scripting things, how well you could notice patterns which you could then automate solutions for was pretty much the goal, and that’s how wulinshu.com’s wiki become reality. I started maintaining bigger and bigger materials over all of the games and after about 5-6 months all of the actual game code was covered and researched and slowly I would keep spending time on making everything ‘neat.'”
That work led to him looking at the original versions of Shenmue and Shenmue II on Dreamcast. He recounted the story to SHIRO!:
“Well I wasn’t just reversing the PC versions at first, in fact due to the fact some people decades earlier had already kind of documented messing around with things on the Dreamcast, I was looking at the Dreamcast versions in particular quite a lot and especially trying to look at all of the differences between the regular releases, the beta versions, etc and that also got me involved with working with Comby Laurent to try to find as much beta content as possible in order to preserve it directly within the research efforts I was undertaking. Earlier on I noticed how there were clear similarities between them and picked up on a few hints that led me to reproduce a workflow that the original developers would have used in order to create new content, initially by using Lua, but eventually once I figured out the complete ABI that was used for events event_sdk and the compiler/toolchain that they used for the Dreamcast. This is another reason I chose intentionally not to bother with a byte-identical function matching approach, the actual algorithms, structures and overall architecture was pretty well understood, we just had to make sure we implemented them correctly. Toward the end of the process is when I started to look at some of the remaster stuff, things like how the d3t UI worked, the exact render pipeline, what stuff we knew was still present, but disabled, for d3t’s own modern remaster efforts, and trying to trace any other potential library usage that we hadn’t already identified.”
— LemonHaze
Shenmue began development on the Sega Saturn as an RPG set in the Virtua Fighter universe. During its lengthy development, it morphed into a unique IP and moved to the Dreamcast where it launched in Japan in late December 1999. It was localized for the West the following November.
Director Yu Suzuki and his team at Sega’s AM2 began developing a sequel right away. Shenmue II released on Dreamcast in Japan and Europe in September and November 2001, respectively. Its North American Dreamcast version was canceled so it could be ported to the Xbox where it released there in October 2002 followed by a European Xbox launch in March 2003. The North American Dreamcast version leaked onto the Internet five years ago.
Be the first to comment