Sounds like the speakers aren’t correctly (quickly) communicating what they’re playing to your controller. The first thing I’d try is a network refresh, by unplugging all your Sonos devices from power, then rebooting your router. Wait a couple of minutes before plugging back in your Sonos devices.
Or, this can be avoided by setting up reserved IP addresses for your Sonos devices in your router’s software, if that is available to you, both do exactly the same thing.
If that doesn’t work, I’d recreate the issue, submit a system diagnostic, and call Sonos Support to discuss it.
There may be information included in the diagnostic that will help Sonos pinpoint the issue and help you find a solution.
When you speak directly to the phone folks, they have tools at their disposal that will allow them to give you advice specific to your network and Sonos system.