I don't think that people should open their door just to not send the wrong message. That's bullshit. If you open your door and they notice something that you shouldn't have, then they are legally allowed to arrest you for it -- and I don't really care if it means that the person would HAVE something to hide, but seriously, you're not always aware that you have something illegal because you don't know all the laws and privacy is still a right. People just giving it away all the time makes it harder for others to keep this right.
Now, if the police officers don't have a warrant, I'd say a possible way to deal with it would be to ask for their names and state your intentions clearly. I've heard people say that you can call the police station to make sure they really are police officers sent to you specifically for a reason -- you could then clarify that you are not interested in whatever their business is or, if you are but don't want the bother right now, try to make an appointment for you to go and visit the police station. Surely that could work? And even if the cops are insistent, if you talk to someone from the station who gives an okay, they'll probably call the cops back.
At least in my mind this sounds like a probable course of action.