Dust particles that are not removed will interfere with the signal’s path as the light passes between the two mated fibers. The light from the outgoing ferrule could be absorbed (“insertion loss”) or reflected back into the fiber (“back reflection”) as the signal comes into contact with dust particles. There are also times when the dust particles can interfere in the mating process causing small air gaps. That slight gap can causing a diffuse reflection effect where the signal is reflected in multiple angles which spikes insertion loss. All of this can be seen in a quick test with your OTDR.
Important point here: dust can be locked on to an end-face by static. For the best cleaning results, use the Sticklers® cleaning fluid in addition to your lint-free wipe or cleaning swab, because the cleaning fluid will dissipate the static and allow your cleaning tool to do a better job.