I've always believed feeding an active woofer an already amplified to speaker level signal is sub-optimal. You are absolutely right. No matter how vanishingly low the distortion of your main power amp, your are further cascading it with the woofer driver and its distortion. Some audio gurus say this is the way to go, since you want the characteristics of your main amp matched to the woofer. I say this is nonsense. The advantage, though, is that the balance between the woofer and panel is baked in by the engineers so you don't have to worry your pretty little head over it.
As for the bass "speed", you are asking the wrong questions. Yes, a low frequency signal has a low rate of change. There is no such thing as a "fast" woofer. What you really want is a woofer free from peaks and "hangover". An ESL panel is highly damped by the air, so hangover is practically nonexistent. When paired with an underdamped woofer, it is perceived as poor panel/woofer integration. Martin Logan has come a really long way with their hybrid systems in that regard. I was listening to Ascents (not "i") for a few years and the woofer that came with it I considered mediocre at best. The newer active woofers allow negative feedback from the woofer itself. If the woofer cone can start and stop on a dime, which this "servo" arrangement enables, I suppose you could call that "fast". To some extent, you even get some feedback correction from the room.
Regarding further throughput delay through the woofer amplifier after already being delayed by the main amp: technically yes, but throughput delay of an audio amplifier is inconsequential. A signal could go through the amps a hundred times in the time either the woofer cone or panel is supposed to think about starting to move.
Unless you are up to doing some design work--at the very least figuring out how to match the levels-- I would stick with what you have. It is good enough for all practical purposes. The folks here discussing active woofers are mostly talking about converting from passive. You already have 99.9% of the advantage of an active woofer. The guys home-brewing active crossover (often with a favorite Crown amp featuring built-in crossover) are not getting servo control.
In looking up the Vantage on the ML Web site (they have produced quite a few models over the years) I was shocked to see that when I landed on the home page there was not an ESL in sight! This is a recent development. It was not so when rumors of ML phasing out ESL's was first a thread topic. Who knows?