Audio switches almost always introduce noise. Whether that bothers you or not will depend heavily on how picky you are. If you're not then it's a good suggestion.
Yeah, for a long time I used the Andrea Audio Controller MC-100, fwiw:
http://www.andreaelectronics.com/buy/ProductDesc/MC100.htmNowadays I use a cordless Plantronics rechargeable headset, which, alas, won't work with the switch.