Xbox 360 Emulator Research Project | Hacker News Reader

Xbox 360 Emulator Research Project

github.com

132 pointsstriking69 comments

I wonder if we'll see viable emulators for PS4/XboxOne sooner than we did for the previous generation given that they are both x86-based (and the PS4 using FreeBSD as well).

The original Xbox hasn't seen a lot of emulation activity either -- I found that strange. It feels like we might lose an entire generation of games to bitrot.

I am more worried about some small but amazing Flash games. Currently it is still possible to play them in the browser, but what about in 5 or 10 years. I image there might be some kind of emulator which wraps tje flash player and the game in a single exe so it can be played independent of a browser.

You could do just that with a modern.ie VM today. The biggest challenge there is ripping the flash and all its assets (and maybe patching it up to remove any dependencies on stale domains)

Check out Shumway - it can play SWFs in the browser without using Flash: http://mozilla.github.io/shumway/

If you have the SWF, you can run them in the Flash Projector, the standalone Player. What'll be problematic is if they preload resources from external locations.

There are a lot of third-party Flash implementations. If the official Flash client becomes unusable, one of these open-source implementations will emerge as the new de-facto standard. I don't think we have to worry much about Flash's survival.

You Gnash , Lightspark and Shumway.

You can download SWF files with unplug for Firefox and such.

Especially since the original Xbox was infamous for randomly dying. Thirty years from now there will still be working NES consoles, but will there be any Xbox ones?

The original front-loading NES is quite infamous for having a poorly designed cartridge mechanism that bends and loses contact with the game cart pins. This is why people were always blowing on games and jamming them back in the NES back in the day. The later top-loading NES fixed the problem but is kinda rare and doesn't have RCA video outputs.

I don't think the original Xbox had many reliability problems though, maybe you're thinking of the Xbox 360 and its red ring of death problem?

The NES cart connector can be repaired with some patience or replaced inexpensively.

A lot of sites selling new connectors claim they are made with more robust materials than the original connectors, but I'm not sure how true that is.

One of the biggest differences between the NES and much newer consoles like the Xbox is that the NES is, in comparison, extremely simple; it's been reverse-engineered down to the transistor level:

http://breaknes.com/ (If you are interested in advanced RE, I recommend learning Russian...)

http://www.qmtpro.com/~nes/chipimages/

In other words, even if there are no original physical NES or their parts around in 30 years, relatively speaking it would be trivial to perfectly emulate one in software or hardware (e.g. an FPGA), because the functionality of the entire system is known.

I'm fixing that: http://xqemu.com

What's required to test retail games / demos for the original XBox? I've done a reasonable amount of testing for Xenia because of easy access to demos. Can the discs be read in PC DVD players?

You have to use an Xbox console (or specially modified DVD drive) to dump your discs.

I'm tempted to bring my old Xbox out of retirement to play around with this. Have you considered a Show HN?

You may need to mod it to dump games.

Could you use (just) a DVD drive from an original XBOX?

This looks like a much better approach than cxbx. Thanks for the pointer!

Cheers to you and JayFoxRox.

Its a major misconception that emulators are easier for Xbox/PS4/Xbox one because they use x86. According to that logic it should be super easy to port something from windows to linux x86_64, but compiling the same program for x86_64 and ARM linux should be very hard. Turns out it is the complete opposite.

Here's what the developer of CBXB (a xbox original emulator) has to say about it, gives the reasons as to why it is to blatantly false: http://www.neogaf.com/forum/showpost.php?p=48088464&postcoun... (original is deleted)

That post is actually more meaningless speculation than accurate.

I don't see how the same logic applies: porting a source program to compile to something which runs on a new architecture seems pretty different to me than writing an efficient software to emulate another architecture for the purposes of running a program.

>t should be super easy to port something from windows to linux x86_64, but compiling the same program for x86_64 and ARM linux should be very hard. Turns out it is the complete opposite.

WineLib. And if not, just use the Wine as a binary loader and run your stuff.

There is nothing strange about it. The original Xbox used a custom NVidia chipset for GPU, sound, peripherals, network etc. The only semi off-the-shelf part was the Intel's CPU (the exact same CPU was not on the market but it seems like a different packaging of a regular Celeron). The system is very complex and all its documentation is very confidential, it was not given to the third parties and, by this time, I am afraid even NVidia does not have it readily available so you cannot hope somebody will break the NDA and release it. The same is true for the later generations: 360, PS3, PS4, X1 all have custom chipsets and SoC in the current gen. Having a known ISA from the point of view of writing an emulator is similar to having a known encoding when writing a translation program from an unknown language.

The Xbox gpu and apu are 'off-the-shelf' in the sense that they were re-used in nvidia retail chips. But you're right, that doesn't mean much in terms of public documentation.

So you do some reverse-engineering. On the whole it isn't much work than other consoles of that generation. The only reason there hasn't been any activity is because nobody (outside of the Xbox backwards compatibility team) sat down and did the work (because there's no incentive to)

>The Xbox gpu and apu are 'off-the-shelf' in the sense that they were re-used in nvidia retail chips.

Not really[1]. It was based on the GeFroce 3 architecture (NV20) but the actual chip had been quite different from the ones you could get on a retail card (NV2A). Even if it had been a straight off-the-shelf device it does not mean you know how it works. Look at the open-source drivers for the off-the-shelf video cards to see what is possible through reverse-engineering and what is not.

I cannot speak for other consoles of that generation, as I have not touched the GC but PS2 devkits came with nice printed manuals, describing every register and every command for almost every device (if memory serves there was not printed IOP manual but it was pretty much the PS1 chipset so its documentation had been obtainable too).

1. https://en.wikipedia.org/wiki/GeForce_3_series

The actual graphics core has only elementary differences from the retail chips [1]. The differences were mainly in integration (merging in a northbridge, tweaks for UMA, ...)

1. https://github.com/espes/xqemu/blob/xbox/hw/xbox/nv2a.c#L708

You can get very detailed ISA's for GPU's NVIDIA ones require an NDA (rubber stamped to some degree).

There have been quite a few Xbox 360 and Xbox SDK's appearing on Ebay, even some of the early PowerPC based Mac Soft SDK's for the Xbox 360 have appeared online so some one has them, too bad that people who buy them either collect them or in the case of the 360 dev box to create free XBLA gold accounts :P

I worked on a couple of Xbox games when the Xbox was around. You didn't get much concrete information about it even as part of the official docs. Plenty of stuff about the performance characteristics and memory layout for texture/vertex/index data, but nothing low level. I think there might have been enough exposed by D3D to let you bypass it entirely, at least in theory, but you'd have been on your own when it came to figuring out the ins and outs of what needed to go in the FIFO...

I vaguely recall somebody from MS saying on the Xbox developer newsgroups that providing the full information wasn't part of their deal with NVidia, and that was that.

Yes, Microsoft signed a pretty bad contract with NVidia so it could not publish much since the NV owned all the IP rights. When asked about the "push-buffer" layout and format MS would plainly tell "We won't prevent you from figuring it but we are prohibited from telling you directly". So everybody did just that, reversed the push-buffer to write their own APIs/debug tools.

That depends on reverse engineering the custom chips that they use in those systems.

The Amiga had custom chips, but had released the information on how they worked so Amiga emulators could be made. I doubt Sony or Microsoft would release the information.

Just having a X86 CPU doesn't always mean the system can be emulated. For example I got BeOS for X86 and there is no emulator that can run it that I know of. My only hope of running BeOS apps is Haiku.

The latest revisions of PCem can run BeOs I think.

Qemu can run BeOSPeMax with some tweaks. I tried it. You have to run in safe mode first.

This is way further along than I would have expected. There are some small games already playable [0]!

[0]: http://xenia.jp/compatibility/

Come help test demos from the marketplace - it's reasonably easy once you copy the content to USB. See https://github.com/xenia-project/game-compatibility/issues

This is really cool, the fact that MS was able to build an 360 emulator that worked for xbone was really a huge engineering feat. Will be interesting to follow xenia's progress.

They didn't though ;)

Yes they did :). This was their most recent announcement [1]. It is currently available only for the preview members but when it is finally released, it will be available to everyone for free as an update. The idea is to emulate 360's architecture in XOne and let users play not only their digital but also their disc based games.

[1] https://twitter.com/majornelson/status/639636383368376320

I have the preview and they are still porting a majority of the games over. Only a select few 360 games can be played on the Xbox one.

It's not that they're porting over games individually, it's that they have to get express permission from the publisher of each individual game. Something about the terms of the contract between the publishers and Microsoft being such that a user is only able to play the game on that specific platform (Xbox 360) unless the publisher says otherwise. One could somewhat cynically look at this and say that this is because publishers love to rerelease games for the new platforms; already we've seen a bunch of rereleases and "HD Editions" (even though last generation was also HD) for the PS4 and Xbox One.

Hmm sorry but they are porting games, once you insert the disk your Xbox One will download a huge chunk of data of the internet were talking 5GB+ for each game.

Microsoft also isn't hiding it: "The digital titles that you own and are part of the Back Compat game catalog will automatically show up in the “Ready to Install” section on your Xbox One. For disc-based games that are a part of the Back Compat game catalog, simply insert the disc and the console will begin downloading the game to your hard drive. After the game has finished downloading, you will still need to keep the game disc in the drive to play."

http://www.xbox.com/en-US/xbox-one/backward-compatibility

Sure they might be downloading the 360 version and then doing the emulation but it's much much more likely that they've actually simply ported the game.

If they have ported the game they've also ported a large part of the 360 dashboard - chat and the guide button look pretty much identical. The early info coming out around E3 seemed to suggest full system emulation.

Why then do you have to download every game even if you have the disk?

Because it comes with the emulator, compatibility modules and anything else specifically the game may need to be able to run in the emulator.

That's weird because the 360 emulation played it from the disk just fine ;)

It downloads the entire game for each disk game you put in, seems to me that the could've found a more efficient way to actually do that since you know both the 360 and the XboxOne support installing games from disk so they got the ability to create a disk image.

If all they needed to download is an emulator and a compatibility config file that would've been a much smaller download package..

they are not porting. microsoft has explicitly said they are emulating.

"We have to do packaging and validation work on each title to make it available through Xbox One backward compatibility," explains a Microsoft spokesperson.

Xbox One Backward Compatibility is an Xbox 360 emulator that runs on Xbox One and is used to play Xbox 360 games," says a Microsoft spokesperson in a statement to The Verge.

http://www.theverge.com/2015/6/15/8785955/microsoft-xbox-one...

they LIED, not the first time, not even the first time this month

They are not porting, they have built an emulator that runs the 360 OS and then they boot the games inside of that.

>Delving deeper, Spencer explained exactly how the emulator packages the Xbox 360 games, and how it compares to Xbox 360's emulation of original Xbox games.

>"You download a kind of manifest of wrapper for the 360 game, so we can say 'hey, this is actually Banjo, or this is Mass Effect. The emulator runs exactly the same for all the games.

">I was around when we did the original Xbox [backwards compatibility] for Xbox 360 where we had a shim for every game and it just didn't scale very well. This is actually the same emulator running for all of the games. Different games do different things, as we're rolling them out we'll say 'oh maybe we have to tweak the emulator.' But in the end, the emulator is emulating the 360, so it's for everybody."

>Asked about whether Microsoft would require permission from game publishers to adjust game code, Spencer clarified it would not be interfering with code.

>"The bits are not touched," he said. "There's some caveats, and as always I like to be as transparent as I can be on this: Kinect games won't work from the 360, because translating between the Kinect sensors is almost impossible."

http://www.gamespot.com/articles/revealed-how-xbox-one-can-p...

I also remember watching a video where they talked about it, it had some more details. I can't remember what it's called though and I couldn't find it with a cursory search.

This is incorrect. Although, they do need to get permission from the original publisher, they are porting the games. How can you tell? Just have a look at the filename of the file downloaded. It has x64 attached to it.

this is incorrect. according to microsoft they are running emulation.

Behind the scenes, Microsoft has built a full Xbox 360 emulator for its Xbox One console. "Xbox One Backward Compatibility is an Xbox 360 emulator that runs on Xbox One and is used to play Xbox 360 games," says a Microsoft spokesperson in a statement to The Verge.

http://www.theverge.com/2015/6/15/8785955/microsoft-xbox-one...

the package they are sending when you insert the disc is a container that contains the xbox emulator, any media assets, the game and compatibility stuff that is needed.

You keep quoting that same Verge quote of a spokesman. Do you have a more technical or independent source? Spokespeople are rarely accurate on a technical level.

Do you have any source that they're actually recompiling everything?

For example running original Xbox games on Xbox 360 also required a download of a binary which was an emulation / compatibility wrapper around the binary on the DVD that shimmed out and hotpatched code so the games worked properly.

The download sizes reported by other commenters seem to suggest a bit more than simple shims.

The download sizes do suggest you're downloading a whole game, or at least most of one. This does not, however, imply that the game is ported -- just that they pull the game from the server rather than the disc. This makes sense even if they're emulating, because most Xbox 360 games will have received a lot of updates compared to what's on the disc, and Microsoft have probably only tested the emulation against one specific version.

its also certainly possible that part of the emulation requirements include xbox 360 patch levels, an emulated file system for updates and everything else to be together in a single container.

Nobody will go on record saying this. But it's not a straight emulator. Some of the other suggestions in this thread suggest others know whats up as well.

I believe that the xbox one doesn't emulate for backwards compatibility, instead it downloads a new version of the executable that was built for x86 and uses the game content from the original disks.

nope, the xbox one emulates the xbox 360 OS -- and the games themselves run inside the xbox 360 OS[1]

for the game, it's like running on a normal 360. and that's why things like multiplayer work. [2]

[1] http://www.engadget.com/2015/06/18/xbox-one-backward-compati...

[2] http://www.xbox.com/en-US/xbox-one/backward-compatibility

They probably haven't. What you have to download from the xbox store when you insert an xbox 360 game into the console is likely a version of the game compiled for xbox 360. Whether they are creating it through a cross-compile or by translating the assembly code is unknown atm.

Is this compatible with the Open Emu app?

No, but if it can run as an x86-64 OS X app then it can be ported without too much work.

Porting to an OE core is kind of invasive since it replaces the entire "app" part of an emulator including the event loop, sound output and windowing. So OpenGL renderers also have to be capable of rendering to a framebuffer and such. Also, the requirement for x86-64 is reasonable for OS X but excludes some Windows-based emu codebases, notably PCSX2.

why is it that we always need so much more CPU and GPU power to emulate an older, fixed, lower power requiring console?

http://arstechnica.com/gaming/2011/08/accuracy-takes-power-o...

This might be a good place to start, it's a long read but pretty interesting.

Note that in some ways, more modern consoles may actually be easier (or at least have different problems) - there are a whole bunch of abstraction layers that stop programmers from doing weird cycle-accurate tricks, and they don't have the development time for it anyway.

Modern may be "easier" because they are being built on top of hardware that is similar to what is found in modern consumer pc systems. That's no to say they are nessessarily the same but being x86 based means they have many of the same instruction sets.

Because afaik emulators, especially of older consoles, have to mimick in software every aspect of the original hardware of the console. This obviously takes a lot more computing power than just what the original hardware needed to run the games. I don't know how this situation has changed, however, as the latest generation of consoles are x86 based.

It's not so much the software that they need to mimic but more so the hardware. They have to emulate the hardware to be able to run the instructions on a different cpu/gpu/audio architecture than what the game was built for. That's why it takes a much more powerful system to emulate a much weaker system.

Reply on news.ycombinator.com