Wash Emulation

I've added a new section to the site where I'm going to be publishing the various hardware tests I create while I'm working on WashingtonDC. I've been making Dreamcast hardware tests for various reasons ever since the beginning of WashingtonDC six years ago, but they've usually been simple "throwaway" tests that get discarded once they've served their purpose. Going forward I'm going to publish them here so that they're around in case I or somebody else needs them in the future.

So far there are two tests, one that I ported from DS back in 2019, and a newer one I just created in the past few days to test PowerVR2's backface culling. In both cases I've made the source code available. I'll also be providing binary downloads in the form of .cdi images, although I don't have one ready yet for the culling test because it's so new.

I really regret having listened to the doctor I mentioned in the last snickerblog post who said the thing growing in my leg was nothing to worry about. Basically, I had been sent to see somebody who is supposed to know what she's talking about because she has an M.D. in Oncology from Harvard University, she told me the large lump in my leg didn't need to be biopsied because it was obviously just a hematoma, and she monitored it for about a year during which time it steadily got smaller. Then over the course of the past couple of months (about the same timeframe as the coronavirus apocalypse albeit unrelated) I noticed it looked like my leg was getting bigger again.

Towards the end of May, I had a previously scheduled ultrasound appointment so the oncologist could monitor its progress, and that showed the growth had become significantly larger. The Oncologist was alarmed but still informed me over the phone that it would be "unheard of" for a tumor to shrink without treatment and then start growing again like mine had. Eventually a sudden bought of pain drove me to the emergency room at the hospital, where they biopsied it and confirmed that it is indeed a type of cancer called Synovial Sarcoma.

So unfortunately the situation is that I don't know how much longer I'm going to be alive. From what I gather this is definitely not the sort of cancer where you can afford to wait an entire year to begin treatment because it's incurable after it metastasizes, and even if it hasn't metastatized it still has a tendency to reoccur. I'm starting the first of 8 rounds of chemotherapy soon and I'm going to get the primary tumor removed surgically.

I wish this story had a real obvious moral like "don't listen to traditional herbalists instead of real doctors", but the person I was getting my advice from was actually supposed to be a real doctor and she still gave be dangerous advice that might have already killed me so IDK.

I don't expect to be able to work on WashingtonDC during my cancer treatment, and even if I survive this I don't know if I can keep working on a large-scale project alone knowing that I'll have a reduced life expectency so I've decided to abandon the project. I may come back to it if I survive, or I may leave it abandoned forever and get involved with a larger group project where I know my work won't all go to waste if I die.

WashingtonDC has been moved to the 3-clause BSD license to maximize the amount of ways its code can be useful to anybody who may need it. This is a feature-complete Dreamcast emulator which was going to be better than its competitors given just a few more years of dev work. I had plans and infrastructure to support all sorts of cool features like Windows CE games, a Vulkan renderer, NAOMI arcade emulation and even a Wii U homebrew port but unfortunately that will probably never happen now.

The relicensing has forced me to ditch support for .cdi homebrew images because that was GPL code I had copied over from lxdream. That's unfortunate but it shouldn't be too hard to code up a replacement since this isn't "real" emulation code; it's just code to load an image of a CD in a certain file format from the host computer's filesystem. The only other bit of WashingtonDC code I didn't write was the ADPCM decoding stuff in AICA; that had been copied from FFMPEG (LGPL) but I was able to replace it with equivalent code from MAME which is already under the 3-clause BSD license. Other than those two bits of code which have already been taken care of, I am the sole author and copyright holder of WashingtonDC so I have the authority to do this licensing unilaterally. Unlike other emulators which have made similar license changes, there are no assblasted formed contributors who claim to still hold copyrights on part of the codebase, so if you use WashingtonDC's code for your project you can be assurred it's completely legitimately available under the terms of the 3-clause BSD license.

Anyways, it now occurrs to be that it's been over a year since the last snickerblog post so I may as well go over some of the things which I've gotten accomplished in that time. I've never been a big fan of the way certain emudevs hype themselves up and make an /r/emulation post every time they take a shit, but perhaps in retrospect I should have done that maybe a little so people would know how much work I do/did.

• Siginificant JIT improvements, especially in dispatch - this boosted performance a LOT
• Microsoft Windows port - you can now build WashingtonDC with mingw-w64 or Microsoft Visual Studio and run it on Windows!
• various timing fixes - now you can finally go for broke on Street Fighter Alpha 3!
• SH4 IRQ multiplexing bug - this one mistake I made in January 2017 was causing dozens of games to hang because IRQs wouldn't get raised, but after three years it got fixed
• MMU emmulation (interpreter mode only) - allows you to boot dozens of Windows CE games in interpreter mode only. Fan favorites like Resident Evil 2!

Those are just the major things that took a lot of time, there were also smaller things too, and the things on that list were all multi-month endeavors. I was working on this emulator almost every day since I started on October 14, 2016.

Long story short: sometimes doctors over-estimate how serious something is, and it turns out that the combination tumor/blood-clot from the last post is actually something far less life-threatening than what was initially believed. So it looks like I'll be getting back to work on WashingtonDC sooner rather than later.

Might still be a bit more of a hiatus, though. Just because my leg isn't about to kill me anymore doesn't mean that my leg doesn't still hurt like a bitch. And then there's still that CAVE 1000 stuff.

Soooo, despite the optimistic tone of last week's post, it looks like I'm going to be forced to take a hiatus from WashingtonDC, which means that I won't be able to release in time for the 20th anniversary of 9/9/99. I found out over the weekend that I have a tumor growing in my leg, and a Deep Vein Thrombosis (DVT) that was formed because the tumor is compressing a blood vein. I don't know if the tumor is cancerous or not yet, but even if it's benign the DVT could still kill me by dislodging and causing a stroke. I'm in a lot of pain, and I'm not supposed to be sitting in office chairs so this means that for at least a month (and possibly longer) I will not be making any commits to WashingtonDC.

I'm not making this post because I'm looking for a pity-party or anything, I just want everybody to know that the reason why the project is about to go without updates for an extended period of time is not that it's abandoned, it's that I'm not able to work on it for medical reasons.

When I come back, my first order of business is going to be see if I can help out the Final Burn Neo folks with CAVE CV1000 emulation. This will further delay my return to WashingtonDC, but I found out they need somebody who is experienced with SH and I am such a person and this sounds like a fascinating project to be involved with.

I'm also going to be uploading to github a copy of WashingtonDC's SH4 and JIT code relicensed in a way which is compatible with FBN's license so somebody else can use it as a basis for SH3 emulation in case I don't survive. I don't think I'm going to die, but there is a tangible possibility that I will so I need to be prepared for that.

Look up "Puzzle! Mushihimetama" if you want to know why I'm fascinated by the cv1k; its a puzzle game where you help a cute anime character with large bouncing titties rescue beetles by matching up groups of colored rocks that you launch into the playing area with a plunger. I got to play it at 2018's California Extreme arcade convention, and it was my favorite game of the show. It's really cute.

Once again, the snickerblog is back after 6 months of inactivity. Luckily, WashingtonDC hasn't been nearly as dead, as I've been making a commitment to regular commits to the github. I'm generally the sort of person who prefers getting work done over talking about how much work I'm doing. Maybe I should be doing more to evangelize WashingtonDC, but ultimately I'm still afraid of doing too much advertising too early; my worst nightmare would be for it to be one of those vaporware emulators that makes a big splash on /r/emulation but ultimately never accomplishes anything.

WashingtonDC has made great strides in the past year towards becoming a viable Dreamcast emulator. A number of new games are booting, including both Sonic Adventures, Guilty Gear X, and more. Things are running a bit faster, to the point that some games can consistently run at fullspeed on my shiny new Intel i9 9900k CPU Those of you running slower computers need not fear - I intend for WashingtonDC to eventually be usable on potatoes, so I'm holding on to the 7 year-old AMD Bulldozer build which used to serve as my primary PC so I can use it for performance testing. Even more important, however, is that it finally has working audio! You can see below a video I recorded of City Escape from Sonic Adventure 2 running. That video's already a few weeks old, and I've already fixed several of the missing sound effects and boosted performance a bit.

I think WashingtonDC's finally approaching the point where it will be viable as a platform for playing Dreamcast games. In retrospect, the release deadline I came up with last year (11/27/2018, the 20th anniversary of Dreamcast's launch) was way off the mark, but I think I can have a release of sorts in time for the 20th anniversary of Dreamcast's North American launch on 9/9/2019. In the more immediate future I'm going to get to work on improving documentation and getting together a compatibility database so it's usable at that date.

The snickerblog itself is undergoing a rennovation as I try to migrate over from raw HTML to jekyll. This has been a bit of a mess and has resulted in it looking more disshevled than usual due to my lack of experience working with HTML, Ruby, and Jekyll. Once I have this under control I'll be able to update the Snickerblog more often. I already have a few posts planned out further detailing what's changed since the last snickerblog update in November. Going forward I want to improve communication and start to attract a userbase.

The snickerblog has also relocated to washemu.org, as you're probably already aware. This will be the new permanent home of the snickerblog and WashingtonDC's official website. I also have a shiny new snickerbockers@washemu.org e-mail address, thus breaking my dependence on GMail's stupid datamining operation. @sbockers on twitter is still my only point-of-contact for social networking.

There's an offical WashingtonDC Discord server now; you can find an invite to it at the top of this page. I'm somewhat abivalent about Discord's rapid rise to prominence as the internet's new favorite chatroom app; it's a fun place to shitpost and organize online gaming sessions, but it's also a completely closed system. I thought about opening up a channel on some IRC network somewhere instead, but I ultimately decided against it because my priority is to make it easier for people to follow the progress of WashingtonDC, not to try (and fail) to prop up a dying messaging service. Anyways, I don't expect Discord to last more than a few years. There have been dozens of other messaging services sitting atop that same throne over the past two decades, but nobody's ever managed to stay at the top for long (see AIM, MSN, Xfire, Skype, etc).

I'm going to be at Fanime-con in San Jose, California this weekend. This means that I probably won't be able to make any more commits to WashingtonDC for the next week. When I get back to work, my first priority will be to fix a texture caching bug which is causing a major performance hit in Guilty Gear X, and then after that I'm going to be looking into implementing audio channel attenuation in the AICA. I always try to force myself to get at least one commit a week in even when I really don't want to, so I hate being unable to get anything done this week, but I'll be back to my usual commit schedule next week starting on either Tuesday or Wednesday.

I've also been making a Jet Set Radio costume for Fanime, which I may or may not post pictures of depending on how self-conscious I feel…

Today's the big anniversary: Dreamcast was released in Japan twenty years ago on November 27, 1998. For some reason this makes me feel like I'm having a mid-life crisis, which is weird because I'm only 8 years older than the Dreamcast. I feel much better about Genesis turning 30 (that also happened this year!) since I wasn't even born until a couple years later. I guess there's something unnervinf about being able to remember hearing about Dreamcast back when it was the future instead of the distant past.

Unfortunately, I didn't manage to stick to my plan to have a release of WashingtonDC ready in time for the 20th anniversary. AICA turned out to be more complicated than I anticipated, and I'm just now getting the snickerbockers/aica_audio branch to a state where things are starting to work out. I've made some major progress since the last snickerblog update and as a result there are several more games booting in addition to the ones shown there. Right now that branch has emulation of the audio hardware, but no actual audio output This means that Dreamcast programs think they're playing audio, but in reality the samples they generate get discarded insted of being sent to an audio backend on the host machine. I'm going to clean up a few more things and then merge the branch to master before I get started on the audio backend. After that, I still need to work on getting the ARM7 fully integrated with the JIT, figuring out what the UI will be like and implementing VMUs before I can actually have an early release ready.

I've mentioned before in previous snickerblog posts that the reason why so many games don't even boot in WashingtonDC is that I've yet to implement the AICA audio hardware. I'm working on getting that done in the snickerbockers/aica_audio branch over on github. On that branch, I recently implemented sample timer interrupts which go to the ARM7, and that has unblocked at least three games. Jet Set Radio, Space Channel 5 and Skies of Arcadia are now in-game on the aica_audio branch. Additionally, the full SoulCalibur intro movie now plays. Previously, it would end prematurely after the camera goes over Mitsurugi's shoulder during the first shot and then go straight to the title screen.

There's still a lot of stuff unimplemented in this branch, and it doesn't actually play audio yet. I think this will get merged to master when there is working audio even if I don't have all the AICA hardware implemented yet.

Aerowings works now. This game necessitated the implementation of 32-bit ARGB paletted textures.

I just discovered that HotD2 is in-game. As with Zombie Revenge and Crazy Taxi 2, this is a game which worked the first time I tried it, meaning that it has probably been working for a while and I just didn't know it.

The House of the Dead 2 is a lightgun shooter. I don't have the lightgun implemented in yet but this game also lets you play with the controller by using the analog stick to move an on-screen cursor, and that does work in WashingtonDC.

Things have been going a little slower this month due to some IRL things getting in the way: a really tough project at work, Bay Area gaming/anime conventions, and most recently a crippling Monster Hunter World addiction. I think I'll have more time for WashingtonDC in September but even now I'm making sure to get some progress made every day even if it's something minor like fixing bugs in the remote gdb stub. My current focus is on the AICA, which is Dreamcast's audio system. I already have AICA's CPU, the ARM7 working but the actual audio hardware is mostly unimplemented. I suspect that most games which aren't working aren't working because the AICA implementation is incomplete.

Evolution is now in-game. This is a JRPG featuring a cast of cute characters and randomly-generated dungeons. It has some really memorable music, too (not that you'd know playing it on WashingtonDC…I still haven't implemented sound yet). Some of the enemies look like they belong in a PSX game, but the main party is pretty high-polygon and the lighting model this game uses looks amazing.

The technological improvement that made this game playable in WashingtonDC was the implementation of the PowerVR2's YUV420 converter. What this does is implement part of the MPEG/JPEG image compression in hardware. Specifically, it doubles the vertial chrominance sampling and interleaves the luminance and chrominance channels together into a format that PowerVR2 can sample from as a texture. Evolution uses this YUV converter to show an intro cinematic of Mag and Linear entering the dungeon at the beginning of the game. Unfortunately, the still movie doesn't actually get displayed becase the game sees that there's something wrong with the audio hardware (which there is, because I haven't implemented it yet) and gives up on showing it right after it begins. Nevertheless, I still had to get the YUV converter working so the game can proceed because the game does actually try to show movie before it realises there's no audio hardware.

BTW, one interesting fact about this game: it got ported to the Neo Geo Pocket Color, of all platforms. In the waning years of SEGA's hardware days, they gave up on trying to beat the Game Boy and threw their weight behind SNK's handheld platform instead. Neo Geo Pocket Color received this, a 2D Sonic game, and Puyo Puyo Tsu (and probably more SEGA stuff too, those three are just all I can name off the top of my head). I haven't had the pleasure of trying out the NGPC port of Evolution, but from what I've seen it does a great job of remaking Evolution's 3D graphics as old-school pixel art.

Looks like we got ourselves a twofer in today's snickerblog update! Crazy Taxi 2 and Zombie Revenge are both games which I tried to run for the first time in the past week and both of them got in-game without needing any changes to WashingtonDC, so probably they've both been working for a while now and I just didn't know it.

Both of these games remain Dreamcast exclusives to this day. Zombie Revenge is a beat-emu-up game set in the world of House of the Dead. Crazy Taxi 2 is largely similar to its predecessor, except it's set in New York City instead of San Francisco and there are some new game mechanics thrown in.

— Here's SoulCalibur running on the snickerbockers/soulcalibur branch. This game does some strange things with the PowerVR2's TAFIFO register, so to get it working I had to make some changes which might not be correct. One of those changes ended up breaking Crazy Taxi, so I need to do some more research and development before this can get merged into master.

Namco Museum has joined Crazy Taxi, Daytona USA, and Power Stone to become the fourth (known) game to get in-game in WashingtonDC. This game was previously unbootable because WashingtonDC was exposing a bug in the game itself. All of WashingtonDC's IRQs happen instantly, and it seems that was causing this game to miss a PowerVR2 interrupt and hang forever.

Now I have PowerVR2 IRQs running on a 1 millisecond delay, and as a result this game can boot and go in-game. Other IRQs are still happening instantly for now because I haven't bothered to get good measurements for how long they generally take on real hardware. I don't really even know if 1 millisecond is too long or too short for the PowerVR2 IRQs.

This game exposed some bugs in WashingtonDC that had to be fixed. There are two remaining problems with this game that need to be fixed. One of them is a soft hang that only happens with the JIT enabled. I think that's a self-modifying code problem, caused by the game not clearing out the instruction after overwriting program memory. The other bug involves the spinning logos at the beginning disappearing when they stop spinning. That's being caused by the screen getting cleared when it shouldn't have. I need to run some more hardware tests to figure out exactly what's supposed to be happening there, but it looks like I don't have the tile rendering implemented correctly.

There were a couple other games that had the same IRQ timing bug as Namco Museum. Since those are finally unblocked, I'm going to get some of them in-game as well so I have more test-cases before I move on to finishing AICA. The next game will probably be either Soul Calibur or Star Wars Episode I Racer.

This blog update is actually about a month and half late because I got this working way back in late May, but WashingtonDC is finally workin in Windows via Cygwin.

It runs really slow (which I think is because OpenGL might be using a software rasterizer under cygwin) but it does have a working x86_64 backend for the dynarec so I can use this as a starting point to gradually get a native Windows port running.

Going forward, I think the roadmap is going to be to get WashingtonDC to build in mingw and then later get it working under MSVC. The main roadblock is going to be that MSVC doesn't implement C11 and it doesn't really even implement C99 very well. I don't really know what the solution to this is going to be. I'm rather fond of C11, so if I can get things running fast enough in mingw then I just might never bother with MSVC. The dynarec is going to emit the same code regardless of what compiler it was built with itself, so I might find there is a negligible difference between MSVC and mingw for my purposes.

I still hope to have WashingtonDC ready for a release in time for the Dreamcast's 20th birthday in November. That only leaves me a little over four months to get it to a usable state, so I'm thinking Windows support might not be available in the first release. Build systems and GUIs tend to be the two biggest time-sinks in software development even though there's no reason why they should, so I'm worried that the effort needed to get the Windows port boostrapped might cause me to miss my target date.

Wow, it's already been five months since I wrote the post about regression testing! This site might not have changed much, but there's been a lot going on with WashingtonDC in that time. When I wrote that last update, WashingtonDC was barely rendering the Dreamcast's spiral-swirl bootup at a mere 8% speed (on my PC). Back then WashingtonDC only had a simple interpreter. Now it has a dynamic recompiler (sometimes called a dynarec) which crushes the spiral-swirl at an average speed of 140%. That's faster than a real Dreamcast!

In-game performance isn't quite as perfect as bootup performance; you can generally expect that to swing between 30% and 60%. I think the main bottleneck there is the FPU; all of its opcodes are implemented by calling into the interpreter's FPU opcode handlers. This brings in all the overhead of a function call, plus the overhead of not being able to hold stuff in registers for long periods of time, plus the overhead of having to branch based on whether the FPU is configured for single or double precision on every instruction. I imagine that can add up and turn into a bottleneck even though it's just one subset of the SH4's instruction-set.

Most importantly, I just got Daytona USA booting a couple nights ago! A lot of the textures are being rendered with the wrong colors, but Three-Seven Speedway still has its classic charm. I haven't gotten any new games working since Crazy Taxi started working way back in September (or was it October?), so I'm excited to finally have a third game in my testing set alongside Crazy Taxi and Power Stone.

Beyond that, there are a bunch of games which are booting, but can't make it ingame. Puyo Puyo~n shows a message in Japanese (which I think is it complaining about the VMU not being plugged in) before failing due to unimplemented functionality. Evolution: The World of Sacred Device bitches about the VMU not being plugged in and then flashes the logos of its developers before ultimately failing due to unipmlemented functionality. Shenmue and Jet Set Radio both display "Now Loading" screens before failing (due to unimplemented functionality, of course). Twinkle Star Sprites technically gets in-game, but the graphics are heavily corrupted.

I didn't quite get around to finishing the regression tests I was talking about in my last post; I don't think I've touched them since January. I need to keep my time balanced between auxiliary stuff like that and exciting stuff like making progress. Naturally, the boring auxiliary stuff gets way less attention compared to exciting things like a faster recompiler and booting new games. I'll get back around to it some day.

The primary thing that keeps WashingtonDC from booting most games is the ARM7 CPU. This is a relatively unadvanced 45MHz RISC machine which is wired into the Dreamcast's audio system (AICA). Games can load whatever code they want onto the ARM7 to control AICA. Most games will end up hanging indefinitely without this because the SH4 spins forever waiting for the program it just loaded onto ARM7 to respond. I get around this by manually hacking the AICA memory to show games what they want to see (so they think the ARM7 program is responding) but this is only feasible for a very small set of games, and if you actually want the audio to work then it's not feasible for any games at all.

I don't anticipate that it will be difficult to implement ARM7 emulation, but I do expect it will be difficult to keep this and the SH4 in sync with each other. I've been making a lot of infrastructure changes to support this and I still have more work to do before I can begin implementing the ARM7 in earnest.

After that the obvious next steps are to get the audio working, and to get WashingtonDC running at a smooth 60FPS on typical games. I want to have WashingtonDC ready for a release (insomuch as you can "release" something that's been sitting in a publicliy-accessible github repo since 2016) on 11/27/2018. That's the twentieth anniversary of Dreamcast's Japanese launch, so it's the perfect date to launch a Dreamcast emulator. That gives me 202 days, which is a tight schedule but I ought to be able to get WashingtonDC to a level of basic functionality by then.

I also intend to overhaul this site and have a real regularly-updated development blog going like other emulator projects. I might end up moving to a dedicated blogging platform or I might overhaul this webpage into something less spartan. Whatever I do, I probably won't get around to it until after I have the ARM7 working at the earliest. Ever since I got started way back in October 2016 I've been treating WashingtonDC like my hobby project, but going forward I'm going to have to start treating it as a product (albeit an open-source product that doesn't make any money).

To that end, I think I need a new name. "WashingtonDC" has always been a placeholder name which I chose because it has DC in it, and also because I was watching the 2016 Presidential Debates when I got started. I don't actually live in Washington DC or even Washington state so it feels a bit dishonest to keep calling it that. In my head, I tend to abbreviate it to "wash", so I'm thinking of rebranding WashingtonDC as washDC, with the logo being a wave (of water) curled over to vaguely resemble a Dreamcast logo. The other benefit of this is that I'll get better search-engine results if my emulator doesn't share a name with the capital of the United States (as somebody on /r/emulation astutely pointed out back in February).

One of the difficult things about starting my emulator was trying to test it without having anything to test against. During the early stages there wasn't any software I could boot, and I didn't even have any form of visual feedback from the emulator until I had been working on it for six months. By my count there are 236 opcodes in the SuperH-4 instructin set, and most of them get used during the boot process. If one of those opcodes is wrong, the emulated software might loop forever (best-case scenario), or it might call the wrong function, or it might write a garbage-value somewhere and corrupt its own state. In the latter two cases it's very difficult to figure out what went wrong because the root-cause of a bug could be very far removed from its symptons. Worse still, without any form of feedback from the emulated software I might not even know that something did go wrong.

The obvious solution is to write test programs that can detect when something is wrong, but in order for that to work I'd need a way to load them into the emulator and a way to get feedback from them. Loading them can be accomplished by writing a test program that replaces the firmware image, but then I still need a way to get feedback from them; there's no way to get feedback from emulated software until the emulator is functional enough to support the mechanisms that software would use to send feedback, and I needed test programs to help get the emulator to that state.

My solution to this was to make a couple of unit-test programs which would link against WashingtonDC's .o files, initialize some of the hardware, artificially inject sh4 code into the memory via an sh4 assembler library I wrote, execute that code, and then inspect the CPU's registers to see if they matched the intended final values. This resulted in three separate tests: sh4mem_test, sh4inst_test, sh4div_test, sh4tmu_test, and sh4asm_test (which tested the assembler library). These four tests helped root out several bugs so I could move WashingtonDC along.

Since they all needed to link against WashingtonDC as a library, they had to be refactored every time WashingtonDC's internal APIs changes. This was an especially arduous task in the case of sh4inst_test, which had swollen to be more than 10000 lines of code as I kept adding more and more testcases. Sometime after I got the firmware booting, I decided the work needed to keep them running was greater than the benefit I got from them so I let them die of bitrot, and eventually I deleted them from git.

Recently, I've been getting ready to make some major changes to the sh4 to support a JIT recompiler, and also to make the existing interpreter faster. This motivated me to bring back the unit_tests, only this time I have the infrastructure needed to build real test roms for the Dreamcast. My current approach is to have the roms do complicated operations and print the results to the serial port. The tests are designed so that if anything goes wrong the output will change. I have a couple perl scripts which launch WashingtonDC with the given test roms and compare the output to captures taken from the same roms running on a real Dreamcast. If there are any discrepancies then the perl script knows something is wrong and it raises an error.

This scheme sounds primitive but it works much better than the old unit_tests scheme did because it's actually comparing WashingtonDC against a real Dreamcast instead of comparing it against my estimations of what a real Dreamcast would have done. I'm just getting started but I've already found and fixed two bugs which the old sh4inst_test was missing. I was worried initially that this would turn out to be a pedantic waste of time since WashingtonDC can already run a few games and I didn't think there would be any basic instruction bugs which wouldn't prevent something complicated like Crazy Taxi from booting; this assumption could not have been more incorrect.

I think the best part of the new test roms is that since they're real Dreamcast programs, they'll never die of bitrot like the old unit_tests did. I can keep making whatever major architectural changes I want to WashingtonDC and that will never require a major refactor of the test roms. I don't really even need to keep the source code around (not that I would ever get rid of it).

WashingtonDC is an in-development SEGA Dreamcast emulator I started working on almost a year ago as a pet project. It's my first attempt at emulation development, and I'm quite pleased with the progress I've made so far. This is something I started last year as a pet project. It's the most complicated program I've ever worked on, and a year ago I didn't know if it was going to work out, or if it would turn out to be a waste of time.

On October 14 2016, I started with nothing and begin implementing some basic CPU opcode handlers and a simple memory device. I didn't have enough infrastructure at that point to run real Dreamcast programs, so I wrote a test function for each opcode which would send it a randomized input, execute an instruction, and check the output to see if it matches expectations. These test functions constituted more than half of WashingtonDC's source for the first few months, and ultimately grew to become a 10000+ line file before they outlived their usefulness. This testing system also included its own SH4 assembler so that I could specify the assembler directives in plain-text.

The assembler proved to be surprisingly difficult to implement, and I remember spending an entire weekend panicking over it because I had never written this much code before and I was worried it would come to dominate the entire project. In retrospect it wasn't that big of a deal because it only came out to approximately 3k-lines and got mostly written over the course of that single weekend, but at the time it felt like a major undertaking.

Eventually I had approximately half of the opcodes implemented, so I used the opcode handlers to implement a simple CPU interpreter. This interpreter would load a firmware dump (dc_bios.bin) and a flash dump (dc_flash.bin) from a file and begin decoding/executing the firmware one instruction at a time. It's designed to print an error-dump and exit every time it sees something it doesn't recognize so I know to go figure out what just failed and implement it. To make forward progress, I would run the firmware through the interpreter and observe the output when it failed. Usually the failure cause would be some unimplemented functionality, so I'd implement it and then re-run the firmware; it would then make it slightly further before failing again on some other thing that I would need to implement. The goal was to continue this cycle until I could boot the firmware.

Not all of the problems I encountered were related to unimplemented opcodes or unimplmented memory-mappings. Some of them were the result of mistakes I had made. Trying to debug a CPU interpreter when something goes wrong can be difficult when you don't even have access to the source code, so I decided to skip the firmware for now and make my emulator boot directly to a homebrew program (which it does by loading the program into memory where the firmware would have put it and pointing the SH4's program counter at the beginning). The Dreamcast has the best homebrew community of any classic game console, and part of that community is an open-source SDK called KallistiOS.

KallistiOS is a sort of lightweight OS kernel that provides high-level APIs for interacting with the Dreamcast hardware, as well cooperative multithreading, filesystems, networking and a host of other features. Having access to the source code made it a lot easier for me to debug issues caused by bugs in WashingtonDC. KallistiOS also gives me an easy way to run code on a real Dreamcast system so I can test for consistency between WashingtonDC and Dreamcast.

What's really great is that since KallistiOS' cross-compiler toolchain is based off of GNU, it generates standard ELF binaries complete with GDB-compatible debugger symbols. To leverage this, I wrote a remote GDB backend for WashingtonDC; it connects with GDB using GDB's remote debugging protocol over TCP via localhost, and it provides GDB with a way to interact with the state of the emulated Dreamcast. On the GDB-side of this connection, there's a normal GDB client (with SH4 support enabled), and I can control the execution of the program runnign inside of WashingtonDC's virtual machine just like it's running on my local machine; that includes breakpoints, watchpoints, single-stepping, memory-access, register access and even ELF symbol lookup (which the GDB client implements entirely independently from my GDB stub using the lower-level functionality my GDB stub provides it with).

Eventually WashingtonDC could boot through KallstiOS' initialization and get to the homebrew program's main function. I didn't have graphics at this time, so I implemented the SCIF, which is a UART on the SH4 which connects to the serial port on the back of the Dreamcast. KallistiOS sends its printf output to the SCIF so you can connect a null modem from your Dreamcast to your PC and view the output on your PC. My SCIF implementation sends that output over TCP to a telnet session so I can use telnet to watch the nessages KallistiOS prints when it boots.

The examples directory in KallistiOS' source includes several homebrew programs which demonstrate how to use KallistiOS. One of them is a port of the classic BSD text adventure, Colossal Cave Adventure which became the first game to work on WashingtonDC when I implemented the serial port.

With KallistiOS booting, I had a way to run my own programs in WashingtonDC or on a real Dreamcast in a high-level environment; this was a huge boon for testing and reverse-engineering. I kept moving forward a little bit at a time by getting more and more of KallistiOS' example programs to work. I got framebuffer graphics working, which allowed me to see visual output from some of KallistiOS' example programs such as this:

More importantly, this meant I had enough infrastructure to run the SEGA bootstrap program that is included in every Dreamcast game. This program is commonly referred to as IP.BIN within the context of homebrew development. It resides in the ISO9660 filesystem's first 32-kilobytes. When the Dreamcast firmware loads a game, it first runs some simple authentication tests whcih are designed to stop software pirates and homebrew developers from running unauthorized code off of CD-R discs (hackers figured out how to circumvent these checks a long time ago) and then it loads IP.BIN from the disc. IP.BIN's job is to draw a SEGA logo to the framebuffer along with the words "PRODUCED BY OR UNDER LICENSE FROM SEGA ENTERPRISED, LTD" and then load the game off of the disc.

It wasn't actually loading the game because I didn't have the GD-ROM code working yet, but it was drawing that logo. This was the first time I had a "real" Dreamcast program running in my emulator. At this point in time, the date was April 14. 2017 and I was just now starting to get visual feedback from WashingtonDC after 6 months of work.

I kept moving forward with remaining unimplemented features such as the GD-ROM drive, the PowerVR2 GPU (which does 3D graphics rendering), and the remaining unimplemented opcodes (which at this point were mostly just the SH4's floating-point unit). For these, KallstiOS was once again a huge help because I could use its source to understand what exactly programs expected these components to do. MAME's PowerVR2 code was also a huge help. After another couple of months I was running some of the KallistiOS demos that made use of the PowerVR2, such as this port of one of the classic NeHe OpenGL tutorials:

Eventually, I felt like I had enough infrastructure to go back to trying to boot the BIOS, so I gave it a try and this happened:

I immediately recognized this as a distorted version of the Dreamcast's "spiral swirl" bootup animation. After about a day spent fixing up the PowerVR2 code, I got this image which confirmed I was watching the firmware's bootup animation:

And after another day, I actually had a working spiral-swirl:

From there it wasn't long until I had the Real-Time Clock reset screen and the main firmware menu working. This was in mid-July, so it took me a total of 9 months to get to the point where I finally had something which I could honestly consider a working Dreamcast emulator.

Over the past three months, I've been working to get actual games booting. The first game I got working was Power Stone. I don't have the AICA (audio hardware) implemented yet, and Power Stone (along with most other games) doesn't boot without working audio hardware. Actually implementing the audio hardware will be time-consuming because it has its own dedicated CPU I have not yet implemented. I was feeling impatient after 9+ months of development, so instead I implemented a hack to fool it into thinking the audio hardware is present by editing the AICA's memory to show Power Stone what it wants to see (thus fooling it into thinking the program it loaded onto the AICA's ARM7 processor is talking back to it). This hack works, and it allows me to get in-game in both Power Stone and Crazy Taxi (and maybe other games, too).

A year ago I started this project not knowing if I would be able to create a working Dreamcast emulator. Now I have one and I look forward to turning it into a usable platform in terms of both compatibility and performance. Currently WashingtonDC can run two games, and it runs both of them at approximately 8% of the speed of a real Dreamcast on my humble 3.1GHz AMD Bulldozer. I hope by this time next year WashingtonDC can be a truly viable Dreamcast emulator, able to run a significant portion of the Dreamcast library at full-speed.

For the immediate future, my goals are to add the analog stick and analog triggers to my controller implementation (which currently only supports the digital buttons on the Dreamcast's gamepad) and then get started on a JIT compiler which can convert sh4 machine-code into x86_64 machine-code. With the JIT compiler implemented, WashingtonDC should be running siginificantly faster than it is now and it will be feasible to get the audio hardware working (if I got the audio hardware working now, it's doubtful I'd be able to hear anything due to the pitch-shifting from running significantly slower than a real Dreamcast).