If it’s a soundbar, @buzz ‘s suggestion is a great one. It does seem like ducking, you may want to turn off voice recognition completely, or including the soundbar in any ‘room’ defined in either Amazon or Google’s software.
Or, you could recreate the issue, and submit a system diagnostic within 10 minutes, and eventually call Sonos Support to discuss it. Don’t post the resulting diagnostic number here, Sonos’ lawyers get sensitive about GDPR, I suspect.
There may be information included in the diagnostic that will help Sonos pinpoint the issue and help you find a solution.
When you speak directly to the Support staff, they have tools at their disposal that will allow them to give you advice specific to your network and Sonos system.