I had a revelation "why not look up the part #, duh!"
So I typed "HP 5064-0289" into my search engine of choice and came up with the term "PCI TERMINATOR"
Now I no friggin' clue what the heck a "PCI TERMINATOR" is, but it's one step closer to figguring out why this thing should not be removed and yet the computer is still working just fine without it...