The point of downvotes is to indicate that something isn't appropriate or relevant to the sub, it's not a "dislike button" and shouldn't be hurting people's feelings if used correctly.