I leave them be until all the leaves have shrivelled completely. Then I remove those leaves at the p-stem, leaving the sheath intact.
I only add shredded oak leaves and pine needles that happen to mbe mixed in. I added these very thick this year (5"-6") and will try to ad 1' next year. The crawlers drag this material into the clay very quickly so more is better.
Some are in sandy soil, some in clay. The sandy soil is only 4"- 2' deep with clay underneath.
And today...it's hard to see because it's in deep shade and it's very red/purple.