is this one of those shows where you have to wear a skirt to watch it?

Lord, I hope not. My wife's fanatical about it. Watches each new episode 2-3 times, bought the first two seasons on DVD (doubling the size of our library), bought a book about it...and extracted a promise from me to watch it with her.
I made her the same deal that she gave me for The Walking Dead -- I'll watch 3 episodes, and if I'm not hooked she'll never mention it to me again. She loves shows about the British class system, and especially the waning days of the aristocracy. I like historical dramas, but don't have any special affinity for England or for that particular time.