What you experience is a relatively common phenomenon in Southern Greece. When south winds are blowing, a high concentration of sand in the atmosphere coming from the Sahara Desert is transported to Greece and falls on the ground with the raindrops.
This occurs mainly in the southern regions of the country, like Santorini but sometimes it may even cross the northern borders!
But remember that what we consider mud may be beneficial for the sea. The Mediterranean Sea is a partly closed sea, so the iron ions contained in the sand are valuable for marine biology!