So I went ahead and kept it. It's fast, runs cool, has been bulletproof in pretty much every game I've run on it so far. I think that's what's important.
While now I'm thinking maybe the annoying sound was just GTX 970 coil whine afterall (and really, when I'm gaming, I don't notice anything, so I'll live), I did go ahead and replace the 500W Delta power supply. It's marketed as "Silence defined," so it seemed made for me, right?
Of course now essentially I'm flushing away the Antec system warranty. Maybe I'll regret it, but I've never gotten warranty service done on store-bought PCs in 23 years, and generally have been upgrading them (more often video cards, hard drives etc.) soon after getting them, over that time.
After some research and remembering my good experience with an Antech PSU in 2007 (only previous experience doing a replacement), I went with the Antech Edge 750:http://store.antec.com/edge-series/edge-750.html#
*While it has these cool "anti-vibration" rubber gaskets (red and black colors included), I had to skip it -- just not quite enough room (as the Antec manual warned). The PSU has the same slot dimension as the 500W Delta, just a bit longer. The case has a little side lip to support the PSU so it's not "dangling by the mounting screws."
As weird as the drive bracket may be, having such a roomy case and small motherboard actually made the PSU removal and installation relatively easy. I won't call it a "snap," but I did it Sunday afternoon and really wasn't anything scary.
Just been 8 years since I've done it before.
No, I won't get the Tidy Cable Routing of the Year Award.
I may eventually try to tidy it up a bit; mostly I just wanted to keep the cables away from the fans. As does the Corsair I was considering, the Antec uses flat black modular cables rather than round/cylindrical cables. The GPU cable had dual six-pin connectors on one cable, and the SATA cable had three connectors. So it's not so much I have too many cables, I just didn't have ties to tidy up the spaghetti.
Replacing the PSU was pretty satisfying, whether it was really necessary or not. Everything booted up and worked fine afterwards. I think maybe next time, whenever that is, I'll finally Build My Own. I guess I've been most intimidated by motherboard-installation, and the thermal paste that was required (maybe still is) for CPU installation, and the PSU. It doesn't look like the latest CPUs requiring messing with paste (or maybe they still do), and certainly a motherboard like this one essentially just screws in with 4 screws. And of course the PSU is not brain surgery. I did find it useful to stick Post-It notes labeling some of the cables on the old PSU before I removed it.