Arcan-FE 0.2 Released

[demeth]

Homepage

... actually happened two days ago, but forgot my credentials to this place and apparently the "lost password" part of the site is broken.

Anyhow, the latest major release is out, with the major change being support for libretro cores so internal launch works for quite a few emulators in windows now The planned focus for the next release will be in improving display of vector games, but since I lack a proper vector cab or even a vectrex (shameful, I know) I'm looking for people with access to that along with a pretty modern (linux preferred) computer with a decent GPU interested in testing and tweaking.

[demeth]

Nothing interesting happening tonight anyhow so started playing around with a vector display mode for 0.2.1.

Regular hijacked mame output:


Gaussian blur + linear upscale:


Gaussian blur + linear upscale + additive blend to backdrop + CRT (curvature, halation, phosphor- distortion):


Other than the glow in img#2 being a bit over the top, the biggest downside is that the glow is uniform based solely on the current frame,
rather than a weighted blend from a series of frames (glow fading according to previous movement). Got that sortof running but the $50 GPU is struggling ..

[Gray_Area]

Over the top?  No, I'd say it's getting there.

What do you mean by 'hijacked mame output?   And are you altering MAME, or using an 'overlay' feature within your fe?

[Howard_Casto]

If you are going to do things with the vector graphics in mame, I would reccomend you just modify mame's source instead. 

With the recent addition of shaders into mame's source it's very easy to overlay graphical effects on a game now.  The vector stuff really needs some attention, so if you are willing to do it, then I would say alter mame's code directly.

[demeth]

Howard: I have little to no interest in all the work that'd be needed to get that one through and accepted (and I want it working with more emulators than mame), it'd be quite radical changes to the rendering pipe to get everything in place -- along with the headache of dealing with HLSL/FX and friends, no thank you -- but hey, the project is open source so feel free --

here's the script that I use as theme (some API changes from the 0.2.0 branch)
http://sourceforge.net/p/arcanfe/code/282/tree//trunk/themes/vectortest/vectortest.lua?force=True

Gray_Area: It's sortof like what more exotic game- cheating tools, FRAPS and others do -- inject a library into the target process, intercept calls to functions that are useful (here, the SDL PollEvent queue, GL_SwapBuffers and a few others), add input events to the local input queue, copy image buffer back across a shared memory segment, those kinds of things.)

[demeth]

Got the glow-trails working, a bit too many variables (# frames trail, weight for each frame, blurkernel resolution in relation to the original image source, horizontal / vertical bias factor for the gaussian blur, blur vs source image blend weights, ...) that requires tuning but all the heavy work is done.

[demeth]

gotten a bit further in actually integrating the vector rendering experiments:

[Hoopz]

Wow, that's pretty cool.   

[headkaze]

I don't understand what I'm seeing here. Are you overlaying options in MAME or is this some sort of theme inside the FE?

[demeth]

Headkaze:
[Long explanation ahead, but I have run out of any short ones]

There are basically three ways you could design a frontend for games and emulators. The first is by far the easiest (the basics takes a few hours) and by far the most common. The frontend (FE) just acting as a launcher that boils down to setting a bunch of environment parameters and tell the OS to start a new program in this environment. The problems here is that you lack control of most of the inputs and outputs and have to resort to a collection of specialized tools to account for these shortcomings (glovepie, etc.).

The second approach (takes a few days or weeks depending on how detailed control you want) is to have an interface/API that exposes these features and embed (so called static or dynamic link, e.g dll files) that into the running FE process. A lot of game engines are designed this way, along with some way of connecting this to user scripts. 'Libretro' (http://www.libretro.org) is a project that defines such an interface suitable for emulators for a bunch of 2D systems, and has created specialized version of a ton of emulators -- ripping out OS specific junk for graphics, input and game-loading and replacing with this interface.  Retroarch, Retrocopy works this way. One of several  problems here is that bugs in the emulator can propagate and take the frontend down, or bugs in the frontend can affect the emulator and that it's hard to separate the FE and the emulator without restarting the entire FE, which creates all sorts of problems.

The third approach (now we're talking months getting high-frequency, high-bandwidth, low-latency data like video/audio/... right) is taking advantage of the fact that certain tasks (drawing stuff, playing music, reading joysticks) can only ever be achieved in a limited set of ways. Combine this fact with the first approach and use the same tricks that debuggers and other tools have to partially take control over another program and you can trick the emulator into thinking it's working like normal, redirect the video/audio and add keyboard/joystick/whatever events to the event-loop.  There's lots of caveats and details to account for here however, but the benefits that can be found is the reason why malware, bots, cheats etc. prefer to use it -- but even more legitimate applications like steam.

Arcan has all three approaches built-in -- with a hybrid of the second/third as the preferred approach, third as a fallback and first ("external launch") as a last resort. The scripts that make out the theme decides which one to actually use and enable/disable features accordingly. The hybrid- approach is that the program is actually split into two parts, the main app works a lot like a scriptable game engine, and then the frameserver. The frameserver is a sandboxed process that gets assigned a role when created (decoding snapshots, recording/streaming video or running a libretro core). This means that you can have many concurrent frameserver sessions running and the frontend can continue working should one suffer from a malfunction.

The different display modes won't work with external launch. About 4 seconds into the video, you can see mame being "activated" in the third launch mode.  The Vector display in particular, works best in the "third" mode but everything except the "Line Width / Point Size" parts work in the hybrid mode using the FBA libretro core. The way it works (depending on which effects that are activated) is that a frame is stored in a series of numbered buffers (the glow trail) that are changed and re-ordered every so often (the "trail step" part). Depending on the number of buffers that you want, a shader is generated that mixes these frames together into a lower resolution image based on a weight determined by how old the frame is ("trail falloff"). This image is then blurred first vertically, then horizontally and finally scaled to the dimension of the game and mixed together in with the latest frame and optionally -- backdrop, display and overlay fed into the CRT step.

[headkaze]

For the 3rd method I'm assuming you're doing an "API hijack" of the DirectX dll's by attaching a dll to the MAME process and passing calls through your own code? I believe MAME uses LoadLibrary for this so I assume it would be the first call you hijack? If so that's quite a task to do that for all the different types of emulators and different versions of DirectX. I'm guessing Libretro is an attempt to standardize an interface so this type of thing is easier to achieve?

I've done a similar thing for hijacking RawInput (which is what MAME uses for keyboard input) but I believe it would be alot more work to do this for Direct3D as you have to create dummy functions for all the API calls and pass them onto the real dll. It also requires that you attach a 32 or 64 bit dll based on the bitness of the target process. I'm guessing this is what you're referring to when you say "lots of caveats and details to account for". I'm also wondering if the benefits are worth it? How often do you need to change input or rendering settings for and emulator? Most people set them once and that's it.

Things like global keyboard hooks work fine for blocking and injecting keyboard input for most emulators and without the need to attach to its process but what is the use of that unless you know the default keys? For blocking and injecting keys for RawInput the only way I know is API hijacking which I consider a "hack". Same goes for hijacking DirectX or OpenGL for that matter unless you replace the dll's themselves (I believe the SF4 Keyboard Patch replaces the XInput dll for hijacking XInput calls for example)

It's impressive work but I do have to wonder if it's worth the effort. Of course unifying input would be great but most emulators deal with input in their own way with their own defaults so you would need to address each emulator separately to support them all.

Either way I'm impressed by what you've achieved so far even if it's not really practical at this stage or until there is standardization across all emulators.

[Gray_Area]

Looks cool. Is the vector flicker adjustable?  (Though apparently a common phenomenon, I don't know as I remember seeing it. My Vectrex doesn't do it. So in AAE, for example, I turn it off.)

[Howard_Casto]

For the 3rd method I'm assuming you're doing an "API hijack" of the DirectX dll's by attaching a dll to the MAME process and passing calls through your own code? I believe MAME uses LoadLibrary for this so I assume it would be the first call you hijack? If so that's quite a task to do that for all the different types of emulators and different versions of DirectX. I'm guessing Libretro is an attempt to standardize an interface so this type of thing is easier to achieve?

I've done a similar thing for hijacking RawInput (which is what MAME uses for keyboard input) but I believe it would be alot more work to do this for Direct3D as you have to create dummy functions for all the API calls and pass them onto the real dll. It also requires that you attach a 32 or 64 bit dll based on the bitness of the target process. I'm guessing this is what you're referring to when you say "lots of caveats and details to account for". I'm also wondering if the benefits are worth it? How often do you need to change input or rendering settings for and emulator? Most people set them once and that's it.

Things like global keyboard hooks work fine for blocking and injecting keyboard input for most emulators and without the need to attach to its process but what is the use of that unless you know the default keys? For blocking and injecting keys for RawInput the only way I know is API hijacking which I consider a "hack". Same goes for hijacking DirectX or OpenGL for that matter unless you replace the dll's themselves (I believe the SF4 Keyboard Patch replaces the XInput dll for hijacking XInput calls for example)

It's impressive work but I do have to wonder if it's worth the effort. Of course unifying input would be great but most emulators deal with input in their own way with their own defaults so you would need to address each emulator separately to support them all.

Either way I'm impressed by what you've achieved so far even if it's not really practical at this stage or until there is standardization across all emulators.

Yeah I hate to see such termendous effort wasted, which is why I suggested submitting mame source instead.  I've hijacked the display once or twice using an external dll (not mine, I found it) but I deemed it not worth the effort because mame does one or two rendering things so strangely that any additions I did would only work in mame and not in anything else.  You'll find that most emulators, even ones that you would think would be similar, do the exact same things in totally different ways.   I couldn't see a system like this truely working unless you coded special cases for every single emulator out there. 

And it seems like it would be well worth the effort until you realise:

1.  All emulators have an external cfg file, so you could just "translate" the input settings from mame into the input settings of the emulator in question, even at runtime.  This means you don't have to hook the inputs and you've essentially cut your workload in half.

2.  There is NO standardization in terms of video rendering for emulators. 

3.  There gradually IS a standardization being added to emulators in terms of screen effects.  Since mame got hlsl, other emu devs have been looking into it.  I forget the name, but there is a hlsl filter(s) created for one of the open source console emulators that's gradually being added to every console emulator out there.  Think of the old Eagle screen filters in terms of it's popularity.  So a more cost-effective method of adding effects is to either submit shader fx files to mame/mess or to this other hlsl engine.

Again not knocking the work you are doing or anything, it is amazingly impressive.  I just hate to see somebody toil away for hours making nails by hand when they are right in front of a hardware store. 

[demeth]

Looks cool. Is the vector flicker adjustable?  (Though apparently a common phenomenon, I don't know as I remember seeing it. My Vectrex doesn't do it. So in AAE, for example, I turn it off.)

The flicker is actually the hackish flicker setting in mame, I just happened to have it turned on ;p
The underlying shader system exposes the logical clock of the engine (@40Hz) and fractions (for interpolations) and that alone can be added to the blending to get a controllable flicker that would imitate the beam but I havn't done so yet (it's pretty low on my list of priorities ).

[demeth]

For the 3rd method I'm assuming you're doing an "API hijack" of the DirectX dll's by attaching a dll to the MAME process and passing calls through your own code?
I believe MAME uses LoadLibrary for this so I assume it would be the first call you hijack? If so that's quite a task to do that for all the different types of emulators and different versions of DirectX. I'm guessing Libretro is an attempt to standardize an interface so this type of thing is easier to achieve?

That's the easiest way, along with the CreateRemoteThread class of functions -- run as a "process parasite" (read the latest issue of phrack for that) and you can even memcpy over parts of the target's renderingcode with your own position independent code. I never ported the hijack stuff to windows, just fooled around to see which tricks actually worked against WoW Warden and friends. Heck, even writing the hijack part as a buffer overflow exploit worked surprisingly well ;p

The libretro approach is cleaner, and covers a decent enough set of emulators already and that's why I won't put in the hours to port and test the hijack- lib for windows, I don't use that thing outside VMWare or Wine ever. What the libretro guys have done is defining a decent enough API for graphics/audio output, input devices and rom-loading. Then they've added patches for a whole bunch of emulators (think they're up to 20 or so emulators by now) so that they use this API instead of whatever system-specific stuff that was in place, and made sure that these compile on a whole bunch of platforms (wii, xbox, xbox360, ps3, linux, osx, ...).

I've done a similar thing for hijacking RawInput (which is what MAME uses for keyboard input) but I believe it would be alot more work to do this for Direct3D as you have to create dummy functions for all the API calls and pass them onto the real dll.

Nine times out of ten, the emulators interactions with the system are so simple that there's only a handful of functions you'll ever need to redirect and know or care about, the rest you can leave in place. There are however open projects that have already done this for quickly reversing game engines, grabbing / replacing textures / shaders etc.

It also requires that you attach a 32 or 64 bit dll based on the bitness of the target process. I'm guessing this is what you're referring to when you say "lots of caveats and details to account for".

Patching the threading library to stop others from interfering while you do your dirty work might be necessary, and, if you're poking around in commercial stuff -- telling whatever DRM junk there is to look the other way for a few seconds. Also, it's one thing getting a hundred or so I/O events into a process -- grabbing 70-100MB/s within strict deadlines (50 or 60Hz, evenly distributed with as low a mean deviation from the ideal as possible etc.) takes you on a deeper tour into performance-tuning land :-)

I'm also wondering if the benefits are worth it? How often do you need to change input or rendering settings for and emulator? Most people set them once and that's it.

Depends on what you want to do. Look at the http://sourceforge.net/p/arcanfe/wiki/Roadmap%2C%20Future%20Changes%2C%20Nifty%20Ideas/ I'm not particularly interested in the "omg pirate romz" sortof thing, there's hyperspin for that. I'm way more curious about weird, creative and non-obvious ways of using emulators (or well, any full-virtualization, but that's a different story) but to get there, I need a good and stable baseline.

Things like global keyboard hooks work fine for blocking and injecting keyboard input for most emulators and without the need to attach to its process but what is the use of that unless you know the default keys?

Write a static translation table between namespaces (heck, there's a ---smurfy--- 5-minute script hidden in there somewhere that does this for arcan<->mame, far from accurate with every usecase, but enough to cover the basics), query the user, write an odd little USB device driver -- there's quite a few options ;-) IDT hooking and friends tend to raise certain alarms, but that's what you get in CreateProcessEx hell ;-p

[demeth]

Yeah I hate to see such termendous effort wasted, which is why I suggested submitting mame source instead.  I've hijacked the display once or twice using an external dll (not mine, I found it) but I deemed it not worth the effort because mame does one or two rendering things so strangely that any additions I did would only work in mame and not in anything else.  You'll find that most emulators, even ones that you would think would be similar, do the exact same things in totally different ways.   I couldn't see a system like this truely working unless you coded special cases for every single emulator out there. 

And it seems like it would be well worth the effort until you realise:

1.  All emulators have an external cfg file, so you could just "translate" the input settings from mame into the input settings of the emulator in question, even at runtime.  This means you don't have to hook the inputs and you've essentially cut your workload in half.

2.  There is NO standardization in terms of video rendering for emulators. 

3.  There gradually IS a standardization being added to emulators in terms of screen effects.  Since mame got hlsl, other emu devs have been looking into it.  I forget the name, but there is a hlsl filter(s) created for one of the open source console emulators that's gradually being added to every console emulator out there.  Think of the old Eagle screen filters in terms of it's popularity.  So a more cost-effective method of adding effects is to either submit shader fx files to mame/mess or to this other hlsl engine.

Again not knocking the work you are doing or anything, it is amazingly impressive.  I just hate to see somebody toil away for hours making nails by hand when they are right in front of a hardware store. 

There's no "totally different things". Every emulator that outputs 2D has to squeeze out a buffer of video data that somehow needs to get drawn to the display at, preferably, a fixed rate. Great, there's a very limited amount of system interfaces you're allowed to do that on. That's a vantage point to monitor. From there you can trace back to the point where this buffer was prepared, switch the pointers to a shared memory page and now that little buffer is yours. Heck, I even tested just looking for mmap (MapViewOfFile) calls for suspicious buffer-sizes that matched that of, width * height * bpp and returned a slightly different address, and that also worked surprisingly well -- "you flip the projectors, the movie keeps right on going, and no one in the audiance has any idea".

2. Again, look into libretro. The coverage is way more to call it a "standard" without the likes of IEEE budging in. They have complete shader packs that cover all of those features mentioned, and hit a nice bunch of platforms. The HLSL shader topic is a full-on flamewar and THOSE are useless time-wasters -- my brief motivation HLSL/FX isn't portable, and the FX business will be left to rot as DX progress, it's too limited and dumb for what you want in AAA engines.

So, this thing is a derivative of a larger project of mine that I've successfully depended on for incoming. The entire "frontend" business is essentially a perverted testcase to increase coverage surface and shake out the weird systemic bugs, so was the windows/OSX ports -- and something to fiddle with when there's nothing good on TV or on yet another work- related trip. What I look for is ways to expand and test the ideas behind the API (the scriptable interface side) and in the process kill four or five birds with one stone.

The vector experiments have been an enjoyable thing for a few nights, and a warm- up round for a much needed GPU programming refresher course. It also tests most features needed for the 3D- engine bits (dynamic shadows / reflections / glows). Among code I haven't cleaned up or integrated, there's working ways of hierarchically linking several FE instances together, locally or over a network -- (hook up several cabs, separate display for marquees etc.) -- almost everything is in place for android tables, to the point of using it as a wii-U style input/display device connected to the FE.

If you're making a shed, you might be satisfied with poorly galvanized hardware store ones. I'm not building a shed.

[Howard_Casto]

If you're making a shed, you might be satisfied with poorly galvanized hardware store ones. I'm not building a shed.

YOU are building a shed by putting up the walls first and hoping that you can somehow jack them up later on to get the floor joists under them and then somehow pour a foundation with a completed shed on top.  Myself, I'd just rather do it the right way. 

[Gray_Area]

So much (seemingly to me) intelligent information given, and I'm sitting wondering who's clever.

[demeth]

What I thought was 'obvious' (such a dangerous word) from the details of my previous posts, the way I do things here are both necessary and sufficient for my needs, thus it is the --right-- way, perhaps not from your context but this is my code-base.

The vector stuff is a toy I threw in there because I thought it was quick and fun to do, a neat test-case for some really hairy edge-cases that just happened to expose bugs I'd otherwise have a hard-time finding. It turns out that by tuning the parameters, lots of other non-vector games started looking more appealing to my eyes. My beliefs of what should and should not be done in MAME aside, as my contributions there won't go further than donating to the DU, integrating the same thing into MAME would be considerably more difficult and, for a portable solution, require a lot of restructuring that is unlikely to be accepted in, and that's not MAME bashing, few devs in any serious project would accept it.

Forgot to reply to this point:

1.  All emulators have an external cfg file, so you could just "translate" the input settings from mame into the input settings of the emulator in question, even at runtime.  This means you don't have to hook the inputs and you've essentially cut your workload in half.

Some things are appropriate to do statically, some things dynamically. Look in the codebase what's hiding behind the input configuration dialogs (resources/scripts/keyconf_mame.lua).

Yeah I hate to see such termendous effort wasted, which is why I suggested submitting mame source instead.  I've hijacked the display once or twice using an external dll (not mine, I found it) but I deemed it not worth the effort because mame does one or two rendering things so strangely that any additions I did would only work in mame and not in anything else. You'll find that most emulators, even ones that you would think would be similar, do the exact same things in totally different ways.   I couldn't see a system like this truly working unless you coded special cases for every single emulator out there. 

Then get new prescription glasses -- without changing a line of code, I had this working in a handful of games, along with the 5 or so emulators I've tried (ScummVM, supermodel, mess, ...) the 0.1.4 features video,   shows it working for ScummVM around the 3 minute mark. The big caveat is PCI bandwidth as resolution and/or framerate increase.