It is Windows 10 x64, but I thought it came with all the older directX libraries? I tried installing the redist anyway, but it didn't help. It only fails for the system resolutions - the other gmame created resolutions work fine, and the regular d3d9 version works fine with the same config. I've tried running as Admin, setting compatibility, etc... Sometimes I just get the quarter screen in the upper left, and other times I get a more verbose message on stderr like:
SwitchRes: [tempest] (1) vertical (640x480@60.000000)->(856x480@60.000000)
Unable to create the Direct3D device (8876086C)
Unable to initialize Direct3D 9
Fatal error: Unable to complete window creation
Researching this a bit further I've seen bug reports against older versions of mame related to disabling Aero (which I have done), and missing dx libraries. Running it with -window works actually, but then it just runs in a window which is not terribly useful to me :-) I've patched my v186 to change the galaxian/frogger behavior, and votrax sounds, so it is not vanilla, but I don't think those changes should affect anything. I'll build a v190 version just with d3d9ex and see if anything changes.....
[ update ] I installed the directXredist from 2010 and built version 190 with the patch and same behavior.