Beyond the removal of contamination, the wash breaks down the surface tension that naturally builds up over time. Surface tension resists paint adhesion. If the contamination and surface tension is not washed off the coating may look funny and have premature failure.
Two big reasons why folks hesitate to use a high pressure wash on their house: concern of water getting inside and the stories told of inexperienced operators. Yes windows do leak sometimes around the sash. We require someone be inside the house to watch for leaks and have a towel on hand. The bigger hesitation is with the person doing the washing. Our company requires that person have experience and pass a competency test before running this equipment.