The other reasons they might need replacement is the adhesives used in the spars (those lines across the panel) can weaken over time and you'll develop buzzes in the panel.
ML has been steadily improving the quality of the adhesives used, but as you can imagine, it's hard to test for longevity, so some models (mostly older) will develop buzzing due to spar glue failure.
Additionally, as Roberto mentioned, not only will dust and other airborne pollutants break down the metallic bond with the Mylar, if they adhere to the Mylar, they physically increase the mass of the diaphragm and reduce its efficiency and change its frequency response characteristics.
I’ve measured a roughly 3db drop off in High Frequency response over a period of six years (on 12 year old speakers) with my units. And that’s after taking really good care of them and vacuuming every other month.
So yes, after a decade or more, replacing the panels is generally a good idea. The harsher the environment (dust, smoke, sun, etc.), the more important it is.