Here are some reasons why sometimes, men say no to sex.