Sonos making thousands of SMB3 connections to Synology

  • 27 January 2022
  • 66 replies
  • 2534 views


Show first post
This topic has been closed for further comments. You can use the search bar to find a similar topic, or create a new one by clicking Create Topic at the top of the page.

66 replies

Userlevel 1
Badge +1

Definitely not fixed in 14.2.

I think I’ve been lucky so far and haven't seen the hundreds of connections yet.

I still see random SMB1 connections to the NAS and will need to keep pushing support to find out why that is doing that.

Also, if you remove the music library, no connections should be made to the NAS. I’m still seeing the speakers making connections which means that the software is saving the username and password even though you are deleting the service that was using it. This needs to be fixed as well. If you delete the music library which contains the username and password to connect to the share, those credentials should no longer be available and so the speakers should not be able to successfully connect to the NAS.

Confirmed that it’s not fixed in 14.2. I’ve reached out to support again to find out if it was supposed to be fixed. I also let them know about the continued SMB1 connections being attempted.

Userlevel 1
Badge +1

In my case I don’t have any stereo pairs, but now I’m seeing the SMB connections staying open again. Currently up to 238 connections since yesterday. If I don’t remove the NAS from the music library, this will continue to grow until the speakers stop responding on the network. Support tries to blame the wi-fi for this behavior, but it’s the number of open connections to the NAS that is consuming onboard RAM of the speaker and this causes the speaker software to stop responding. The high number of connections to the NAS is the first symptom. The second is when the http://<ip of speaker>:1400/xml/device_description.xml page stops responding. My home automation system uses these pages to query the speakers for services and I typically see the speakers drop offline just before they start crashing. The third symptom is that the speakers will either show up as ZP100 speakers and are marked incompatible, or they simply stop showing up in the mobile app.

Userlevel 1
Badge +1

In my case I have a theory that it may be two controllers creating connections back and forth, when played together. I have two stereo pairs in each room, but I usually play the rooms together.  When I see large numbers of connections I see both controllers from my two stereo pairs. But when things are well behaved I see only one ip sharing 5 connections. Also when it works properly with a single ip the album art is much quicker to load and I don’t get blanks when I scroll through the queue. If your system has two or more pairs that are linked to play together, this could be the root cause.

 

Userlevel 1
Badge +1

I just re-added my music library and I can see the associated product making thousands of connections to the NAS. On the bright side, there’s only one active connection that is showing, so the speaker appears to be opening and closing the SMB3 connection for every song on the NAS, but so far they aren’t staying open. Not sure why the speaker has to keep closing and opening new connections. I’ll keep an eye on it and see if they start to stay open.

Userlevel 1
Badge +1

root@ix4-300d:/opt# smbstatus | grep Music | grep Fri
Music        2043    192.168.1.113 Fri Feb 25 05:46:00 PM 2022 EST  -            -
Music        1316    192.168.1.113 Fri Feb 25 05:37:49 PM 2022 EST  -            -
Music        9053    192.168.1.113 Fri Feb 25 07:12:24 PM 2022 EST  -            -
Music        8574    192.168.1.113 Fri Feb 25 07:07:14 PM 2022 EST  -            -
Music        9014    192.168.1.113 Fri Feb 25 07:12:16 PM 2022 EST  -            -
Music        1307    192.168.1.113 Fri Feb 25 05:37:47 PM 2022 EST  -            -
Music        2010    192.168.1.113 Fri Feb 25 05:45:53 PM 2022 EST  -            -
Music        9064    192.168.1.113 Fri Feb 25 07:12:26 PM 2022 EST  -            -
Music        1287    192.168.1.113 Fri Feb 25 05:37:43 PM 2022 EST  -            -
Music        2012    192.168.1.113 Fri Feb 25 05:45:54 PM 2022 EST  -            -
Music        9051    192.168.1.113 Fri Feb 25 07:12:24 PM 2022 EST  -            -
Music        1312    192.168.1.113 Fri Feb 25 05:37:48 PM 2022 EST  -            -
Music        9049    192.168.1.113 Fri Feb 25 07:12:23 PM 2022 EST  -            -
Music        9055    192.168.1.113 Fri Feb 25 07:12:25 PM 2022 EST  -            -
Music        1318    192.168.1.113 Fri Feb 25 05:37:49 PM 2022 EST  -            -
Music        1314    192.168.1.113 Fri Feb 25 05:37:48 PM 2022 EST  -            -
Music        9047    192.168.1.113 Fri Feb 25 07:12:23 PM 2022 EST  -            -
Music        2005    192.168.1.113 Fri Feb 25 05:45:51 PM 2022 EST  -            -
Music        1282    192.168.1.113 Fri Feb 25 05:37:42 PM 2022 EST  -            -
Music        2039    192.168.1.113 Fri Feb 25 05:45:58 PM 2022 EST  -            -
Music        9028    192.168.1.113 Fri Feb 25 07:12:20 PM 2022 EST  -            -
Music        2041    192.168.1.113 Fri Feb 25 05:45:59 PM 2022 EST  -            -
Music        1305    192.168.1.113 Fri Feb 25 05:37:46 PM 2022 EST  -            -
Music        1303    192.168.1.113 Fri Feb 25 05:37:46 PM 2022 EST  -            -
Music        2037    192.168.1.113 Fri Feb 25 05:45:57 PM 2022 EST  -            -

 

So what happens is during a “session” the controller opens the connections. Then if you don’t use the Sonos, or go back later and change something, it opens an entire new set of connections. I’ve only ever seen SMB3_02 protocol used by the Sonos but I’m not looking very carefully. At least the orphaned connections are 3.02.  I have 75 from Thursday just sitting there doing nothing. The idea of clearing the music library is interesting, but I loath doing that because with this new bug I’ve had scan failures so I risk losing everything (I have 20000 songs on the nas).

 

 

Userlevel 1
Badge +1

Definitely not fixed in 14.2.

I think I’ve been lucky so far and haven't seen the hundreds of connections yet.

I still see random SMB1 connections to the NAS and will need to keep pushing support to find out why that is doing that.

Also, if you remove the music library, no connections should be made to the NAS. I’m still seeing the speakers making connections which means that the software is saving the username and password even though you are deleting the service that was using it. This needs to be fixed as well. If you delete the music library which contains the username and password to connect to the share, those credentials should no longer be available and so the speakers should not be able to successfully connect to the NAS.

Userlevel 1
Badge +1

Definitely not fixed in 14.2.

Userlevel 1
Badge +1

BTW, I don’t believe I’ve noticed this behavior until sometime around the release of 13.4 back in November. I had the music library enabled since Halloween to play songs on the Sonos speakers and it was working fine until about Thanksgiving. That’s when I noticed my NAS starting to hang and go offline and I opened a ticket with Synology and that’s when we discovered the thousands of connections from the Sonos speakers.

My old NAS had 2GB of RAM and would lock up after a day or 2. I recently replaced that older NAS with one that has 8GB of RAM and it lasts a lot longer before becoming unresponsive. Most of the time, the Sonos speakers will start to crash first before the NAS does since the speakers don’t have enough RAM to maintain the high number of SMB connections to the NAS.

Userlevel 1
Badge +1

If I switch the Sonos to a dhcp reservation and put that in “hosts allow”, I still have to contend with some other non-Sonos devices connecting to the same shares. I don’t see anyway to limit the connection count per ip address, at least in the smb.conf. I could set max connections really low but don’t want to limit other well-behaved clients.

 

I have several SMB shares on my NAS as well and several Windows clients are connecting to them. My experience has been that each of these Windows machines only has one connection open to the NAS at a time. I’m assuming that “well behaved” clients only make a single connection to the SMB share and so limiting SMB to a single connection per IP shouldn’t break those clients. I agree with you though, we shouldn’t have to enable such strict measures just for the Sonos speakers.

Userlevel 1
Badge +1

Update 14.2 was released yesterday. I’ve re-enabled my music library to my NAS and will monitor for the next day or so to see if the connections continue.

Something else I’ve noticed, is that if you have any playlists created that contain music from your NAS, even if you don’t have the music library feature enabled, Sonos will still make SMB connections to the NAS. I’m assuming it’s to grab artwork for the songs in the playlist. I’ve also noticed that Sonos uses SMB1 connections first and then falls back to SMB3 which seems backwards to me. It should try the more secure method first and only fallback to SMB1 if it can’t connect via SMB3.

A slight correction to the above, Sonos doesn’t consistently make SMB1 connections. I just tested browsing playlists after adding the music library back, and only see SMB3 connections. I’m not sure what actions are triggering the SMB1 connections to the NAS at this time. I can just see that there are failed attempts from the speakers because I have SMB1 disabled.

Userlevel 1
Badge +1

If I switch the Sonos to a dhcp reservation and put that in “hosts allow”, I still have to contend with some other non-Sonos devices connecting to the same shares. I don’t see anyway to limit the connection count per ip address, at least in the smb.conf. I could set max connections really low but don’t want to limit other well-behaved clients.

 

Userlevel 1
Badge +1

What I’ve done to help the problem is to set “deadtime” parameter in the [global] section of the NAS smb.conf to around 3 days. This will time out the orphaned connections -- I run with about 200 connections now since there are always going to be some dead ones  -- there should normally only be 5 or so connections I think.  You could drop deadtime lower but then if you have other devices or windows workstations using the NAS they may have to reopen connections.

 

The other parameter you could set is the max connections setting in the sections for the specific shares. I set mine to 500 which gives me enough room, considering the extent of this bug :-)

 

Some NASs may do this already, and others may have a UI to change these values, but my old NAS doesn’t so I need to edit the smb.conf

 

 

 

 

On Synology, there is a setting to allow only one SMB connection per IP address. This definitely prevents the issue from happening, however, when you go to play songs from the NAS, the initial song will fail to play, but then subsequent songs will play without issue. If development can’t figure out how to keep the speakers from making hundreds of calls to the NAS, I may just end up turning on the single connection option on the Synology and be done with it.

I also think the number of connections to the NAS may be related to the number of songs you have in your library. I’m thinking the more songs you have, the more connections that are made and so might affect people differently depending upon the number of songs in the library.

Userlevel 1
Badge +1

Update 14.2 was released yesterday. I’ve re-enabled my music library to my NAS and will monitor for the next day or so to see if the connections continue.

Something else I’ve noticed, is that if you have any playlists created that contain music from your NAS, even if you don’t have the music library feature enabled, Sonos will still make SMB connections to the NAS. I’m assuming it’s to grab artwork for the songs in the playlist. I’ve also noticed that Sonos uses SMB1 connections first and then falls back to SMB3 which seems backwards to me. It should try the more secure method first and only fallback to SMB1 if it can’t connect via SMB3.

Userlevel 7
Badge +23

Have you tried a 3rd party app? Such as SonoPhone or mine for Windows?

Userlevel 1
Badge +1

I had noticed exactly the same thing here:

 

 

 

 

Basically, over time,  the newer firmware leaves orphaned connections to the NAS that are not cleaned up. Eventually you will run out of connections or memory.  Scrolling around and fiddling with the S2 app seems to make the problem worse.

 

I tried to work with support but they were not helpful. I’m pretty amazed that nobody else has complained much about this problem, since it can overwhelm the NAS. I guess maybe people don’t update the firmware that much - I never had the problem until December when I was forced to update due to purchasing an addition. Also its possible that some NAS’s are set up to timeout connections (see below...)

 

What I’ve done to help the problem is to set “deadtime” parameter in the [global] section of the NAS smb.conf to around 3 days. This will time out the orphaned connections -- I run with about 200 connections now since there are always going to be some dead ones  -- there should normally only be 5 or so connections I think.  You could drop deadtime lower but then if you have other devices or windows workstations using the NAS they may have to reopen connections.

 

The other parameter you could set is the max connections setting in the sections for the specific shares. I set mine to 500 which gives me enough room, considering the extent of this bug :-)

 

Some NASs may do this already, and others may have a UI to change these values, but my old NAS doesn’t so I need to edit the smb.conf

 

My main music library is on a single share called Music. If I look at the connections now I have 186 connections!

 

root@ix4-300d:/opt# /usr/local/samba/bin/smbstatus | grep Music | wc -l
186

It would be much higher (until memory ran out) if I didn’t set deadtime.

 

 

You can clearly see I have connections left over from Sunday (which are about to be cleaned up since they are almost 3 days old)

root@ix4-300d:/opt# /usr/local/samba/bin/smbstatus | grep Sun
Music        30681   192.168.1.113 Sun Feb 20 06:26:48 PM 2022 EST  -            -
Music        27294   192.168.1.139 Sun Feb 20 05:42:33 PM 2022 EST  -            -
Music        24291   192.168.1.139 Sun Feb 20 05:10:20 PM 2022 EST  -            -
Music        29298   192.168.1.168 Sun Feb 20 06:10:22 PM 2022 EST  -            -

…. (and many more).

These are all SMB3 connections btw, such as:

1688    nobody       users        192.168.1.139 (ipv4:192.168.1.139:33922)  SMB3_02           -                    -

 

 

Again, I’m not sure how most people cope with this level of connections. Maybe the problem only expresses itself  in certain configurations. Maybe nobody uses the music library anymore…..  I couldn’t convince support to figure it out with me :-)

 

 

 

 

 

Userlevel 1
Badge +1

Jut received word from support that they believe they have the root cause identified and are working on a fix. I’ll let you know once the fix is out and if it resolves the issue.