Prominent phallus signs often indicate a house of ill repute, with an arrow pointing upstairs to where the action was, but sometimes they were just a shopkeeper's good luck sign.
Phallus signs, despite their connotations, were never used to mark brothels. 