Fungi spores, like bacteria, are everywhere. I suspect in time, our AP systems would get colonized by something that blew in on the wind, or came in on our hands after we were weeding the dirt garden, for example. As far as fungi getting washed off, when I dig into the gravel bed, the stones are coated with a thin layer of decomposed poo. If that is staying there, I suspect the fungi would find a nice home and stay, too.
Perhaps it would be good to add a nice scoop of rich soil when you start off a growbed. That way you could get a jump on establishing the fungus. It starts sounding like a Witch's Brew:
By the Waxing Moon, dance in a circle and wash the gravel. Put the gravel in the growbed with Filter sludge from the aquarium, a shovelful of rich garden dirt from the garden, and a double handful of compost worms from the bin. Don't forget to add the hair of frog and eye of newt.
(It's an adaptation of Macbeth Act IV scene I, the band's Neo-celtic pagan hippie-ish, and they use instruments like the hurdygurdy, didgeridoo, bodrhan, lots of flutes etc.)