Sonos making thousands of SMB3 connections to Synology

  • 27 January 2022
  • 66 replies
  • 2483 views

Userlevel 1
Badge +1

Not sure if anyone else is experiencing issues with Sonos Music Library and the Synology NAS becoming unresponsive or very sluggish, or with random speakers disappearing from the S2 app.

 

I recently enabled the Music Library feature in Sonos (before the 13.4.1 update in Dec 2021 which introduced SMB3 support). At that time, I configured my Synology NAS to use SMB1 to allow the connection to work. I was on DSM Version: 7.0.1-42218 and running S2 13.3.2. Somewhere around 11/25/21, my Synology started going offline and become unresponsive. I was running a DS1517+ with 2GB of RAM and noticed that the SMB process was eating a lot more RAM than usual. Also, started seeing thousands of connections from the Sonos speakers, multiple times for each speaker. My Sonos updated to 13.4, but the issue still remained.

I opened a case with Synology and they determined that the Sonos speakers were opening hundreds of connections per speaker and leaving the connections open. This was slowly draining the memory until the NAS would become unresponsive.

On 12/7/21, S2 updated to 13.4.1 which introduced the SMB3 support. I turned off SMB1 on the Synology NAS and still saw the hundreds and thousands of connections. I would also see the speakers attempt an SMB1 connection still which was denied by the speakers. For testing, we tried leaving SMB1 on, but still saw the large amount of connections from the Sonos speakers.

I ended up switching to a newer DS1621+ with 8GB of RAM and the issue still continued to happen, but took several days instead of just one. At one point, the number of active connections from Sonos was near 4000.

 

After switching to the DS1621+ with more memory, the issues started getting worse on the Sonos side. Several speakers would start to show up as ZP100 (would be various speakers and not always the same ones). This marked these speakers as incompatible with the S2 app and so I couldn’t do anything relating to the music library (like remove it) until the affected speakers were rebooted. If you tried going to http://<ip of speaker>:1400/xml/device_information.xml nothing would show up, or just the XML banner stating “This XML file does not appear to have any style information associated with it. The document tree is shown below.” The speakers were acting similar to the old NAS where they would start to become unresponsive. I can only guess that with the hundreds of open connections to the NAS, the memory utilization would get very high and the speaker would stop responding correctly.

 

After rebooting the speaker that showed up as a ZP100 you could then hit the XML page and the speaker showed up correctly as a Sonos One. I would also get some speakers that would just completely drop from the application, but after a reboot, they’d show up just fine.. In all cases, my wireless controller showed all speakers connected on the network and they all responded to pings. After a reboot of all the speakers, they all showed up correctly in the S2 app and could all play music from the Synology. Also, after a reboot of the speaker, all the connections would slowly drop on the NAS and memory usage would return to normal.
 

The whole system would run fine for a couple of days and then the connections would start appearing on the NAS at various times (not just at the 2am re-indexing time). I also noticed that whenever I had the S2 app up and when I would explore a Sonos playlist that contained music from the NAS, I would get SMB1 failures in the NAS logs for the associated speaker. I believe that the code is still set to use SMB1 for some functions and is getting blocked by the NAS..

 

If I disable the Music Library, all the speakers stay connected to the wifi network and I can stream music with no issues for several days. The memory on the NAS remains at normal levels and doesn’t climb over time. I still see SMB1 connections whenever I pull up the S2 app as long as the Sonos playlists exist that point to music on the NAS, even if the Muisc Library is disabled. I still think the Sonos code is trying to use SMB1 to update something either with album art or the track icons and is having trouble. If I browse to the album directly under Music Library, the album art does show correctly. It just seems to be the track icons in the playlist has trouble populating. These playlists were created by selecting various tracks in the Music Library and adding them to the Sonos playlist.

 

I have tickets open with both Synology and Sonos and currently waiting to hear back from Sonos after giving them countless diagnostics and logs. It seems that the diagnostic logs don’t show the connections to the NAS and so support doesn’t have any visibility into the problem other than looking at the thousands of connections in the Synology logs that I’m providing them.


This topic has been closed for further comments. You can use the search bar to find a similar topic, or create a new one by clicking Create Topic at the top of the page.

66 replies

Userlevel 7
Badge +22

You really need to call and talk with Sonos, they are most likely to find your problem.

Userlevel 1
Badge +1

I called them months ago and they asked me to upload logs which I did. Never heard back from them. I think OP also said he called and gave logs. Considering the issue has been open for about a year now, I disagree with your premise.

Userlevel 1
Badge +1

Still seeing the same issue on my system. 362 open connections on my DS214 from three different speakers not playing anything.

S2-System is on 14.14.

SMB-Service on Syno is 4.10.18-0327

Please help...

 

I think the easiest workaround at the moment is to set deadtime on the smb.conf to 4320 which is 3 days, and also make sure that the “max connections” setting for your music shares is >= 400. Not sure how to do that on a Synology but those are pretty common adjustments.

 

If you want to live life dangerously you could also try this in the global, but I don’t think it will help much.

socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=524288 SO_SNDBUF=524288

 

 

 

Userlevel 1
Badge +1

I’m also still seeing the issue on S2 14.14 with SMB on Syno 4.10.18-0434.

 

At some point, I have to reboot all my speakers. There is a workaround on Synology where you can block multiple connection attempts by the same IP address under File Services->SMB->Advanced Settings->Others “Disable multiple connections from the same IP address”. This keeps the syno from running out of memory and helps to keep the speakers from running out of memory. I’ve noticed when enabling this setting that sometimes you get an error when first playing a song from the Syno, but trying it again works. With this setting, the speakers stay online for much longer before they start having problems.

Userlevel 1
Badge +1

If you set the “deadtime” parameter in smb.conf to close out the orphaned connections after 3 days,  the speakers will not need to reboot. I’ve never rebooted my speakers. Average use for 5 connected sonos clients is about 200 connections for a rolling 3 day average.  Amazed Sonos doesn’t seem to do anything about the issue though.

Userlevel 7
Badge +22

Sonos seems to prioritize problems by the number of folks complaining to them so unless a bunch of folks with your hardware/problem complain it isn’t likely to top the list.

It would be an interesting experiment to see how your setup would respond to using an SMB gateway between the NAS and your Sonos. I’d say use a Raspberry Pi but they are about impossible to find and priced way too high due to the chip shortage. Maybe another SMB capable system instead? Same basic setup.

SMB v1 Gateway  https://stan-miller.livejournal.com/357.html

Userlevel 1
Badge +1

I’ve run an SMB gateway my pi to the NAS but it didn’t work. I tried various versions of samba.

Userlevel 7
Badge +22

That is sounding more and more like a NAS issue. 

While playing with the Pi as a gateway I connected it to several different data sources from a Windows PC to Linux systems and with different protocols with no issues.

Userlevel 1
Badge +1

Which version of samba are you using and can you send the global config section?

Userlevel 7
Badge +22

Samba 4.9.5-Debian that ships with the  5.10.103-v7+ Raspberrian I have running and whatever OpenSuse 15.3 used, haven’t tried it on 15.4.

Userlevel 1
Badge +1

4.13.13 ships with bullseye

root@rpi4:~# lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 11 (bullseye)
Release:        11
Codename:       bullseye
root@rpi4:~# samba -V
Version 4.13.13-Debian

 

I had downloaded the source and built the latest 4.16 a few months ago, and had the same problem. Went back to older containerized versions and same problem. Given that my NAS is a 10 year old lenovo ix4 that has never had any problems with leaving open files with any other application, including the 13.x version of the sonos fw, I’m inclined to believe that this is a sonos problem. My library is 20,000 songs though, so it may be quantity. Not ruling out there is some global config that could be set to fix it though.

 

 

Userlevel 7
Badge +22

I really should update that Pi but since it just sits in the rack quietly running I hate to disturb it. I swapped power bricks a couple months ago so this is a low number.

pi@pi-3b-2:~ $ uptime
 17:05:56 up 65 days,  4:33,  2 users,  load average: 0.23, 0.40, 0.35

pi@pi-3b-2:~ $ lsb_release -a
No LSB modules are available.
Distributor ID:    Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:    10
Codename:    buster
 

Userlevel 1
Badge +1

For my particular use case, I have various songs on my NAS that I then create sonos playlists and then these are triggered by home automation. Rather than connect to the root music share that has many thousands of songs, I now have a new folder for sonos that I put the songs I need to play on the speakers and nothing else. Connecting to this sonos only share seems to have helped since there are now just a dozen songs or less at any one time.

If you’re wanting to have access to your entire library and you can’t modify the smb.conf file to close stale connections, you can go into the Synology Control Panel, File Services, SMB, Advanced Settings, Others, and then check the box next to “Disable multiple connections from the same IP address”. This keeps the connections from getting out of control, but does introduce an issue where the first song of the playlist won’t play and you have to hit play a second time before the music will start playing.

Userlevel 1
Badge +1

Yeah this was the original problem whereby the max connections was too low, and then you wind up skipping songs if you are just doing a shuffle.  But the bigger issue when you set max connections to a low value is that the music library scan fails, so I was having a problem updating it until I set the maxconnections to a higher number like 500.  A deadtime of 3 days and maxcon 500 seems to avoid problems at least for me.

 

Userlevel 1
Badge +1

I’d prefer to use your settings billfor since doing the single connection breaks things.

What are the settings in your smb.conf file that you’re using? I’ll see if there’s a way to modify this on the Synology so that it persists across a reboot. Typically, you aren’t supposed to modify config files from the CLI, but there’s no GUI settings to update the deadtime and max connections.

Userlevel 1
Badge +1

In the [global] section ,

deadtime=4320

max smbd processes = 500

 

and in the section for the specific share,

max connections = 400

 

On my old NAS, even though the max connections is technically per share, my NAS autogenerates the smb.conf from another config, so I had to make all my max connections 400.

 

Also in [global] you could try adding:

socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=524288 SO_SNDBUF=524288

 

But I don’t think it makes a difference and it might cause other issues, so best to leave it alone unless you want to experiment.