You likely would need to turn off bravia sync each time you want to use the TV speakers.  Bravia sync is essentially communicating with the Beam, and when it recognizes the speaker there, it assumes you intend to send audio to the Beam every time.

Thanks. I don't think I can turn Bravia sync off. I have sony bravia kdl-55nx723. This is so annoying.

The user manual isn’t very helpful on this.  Your remote has a ‘SYNC Menu’ button, I would try that and see if you have the option of turning it off.   If not there, I would look for audio settings, as that’s a common place for it.

Out of curiosity, why do you want to use your TV speakers?

No there is no turn off sync. 

Late nights beam makes to much noise. Turning it off on TV will make family members turn it on when ever they want. Using Sonos app is not an option.

Have you tried turning on Night Sound (the moon icon) on the Beam to make it less “noisy” at night?

Yes I tried night mode but then it's only me that can turn it off.