I like this idea very very much. Have you ever read the book Phantom, by Susan Kay? There's a scene in there like that, where the main character is given a young slave girl to use, but she's so afraid of him that he can't bring himself to do it and gives her back. Every time I read that part I'm like nooooooooo (especially since they killed the girl as a result of her lack of cooperation).
I would love to read a story with that theme. Maybe I'll even write one.