Question

Help us understand Sonos diagnostics

  • 5 November 2015
  • 16 replies
  • 3676 views

Userlevel 3
Badge +8
I would expect that if Sonos owners with some technical knowledge would benefit from being able to run and interpret Sonos diagnostics information. The nice thing about Sonos is that there is a wealth of information available, but very little documentation on how to interpret it.

Basic instructions on how to access the diagnostics are:

1. From a web browser on your network, enter the following URL: http://xxx.xxx.xx.xxx:1400/support/review, where xxx.xxx.xx.xxx is the IP address of one of your Sonos components (e.g. a Player or the Boost). You should see a short list of links to your Sonos components, plus a link to the Network Matrix. Note: if the links do not appear, click Refresh in your browser. For some reason, the links may not appear the first time you access the URL.

2. Click on one of the links to a Sonos component. A long list of diagnostic links will appear.

3. Scroll down until you see the link /proc/auth_rincon/phyerr and open the link. This link shows a historical log of performance information for the device. A screen shot, plus some questions, are shown below.

4. Scroll upwards until you see the link /opt/log/anacapa.trace and open the link. This list of log entries is where you will see events associated with your activity. If, for example, you are losing a connection to a music service, there will be a log entry here.

5. Finally, open the link http://xxx.xxx.xx.xxx:1400/status/perf. This will show several areas of the Sonos buffer. The CHSNK Fill Level seems to be the important one for diagnosing connection issues.

Here is an example of the Phyerr screen: https://dl.dropboxusercontent.com/u/78476446/Sonos%20Forum%20Posts/Sonos%20Boost%20Phyerr.JPG

Questions for anyone who knows:

The Phyerr screen shows six columns of information, 10, 100, 1000, etc., with numbers in the columns. My understanding is that the numbers always add up to 121 horizontally, and that the number 121 in column 1000 is an "error free" reading. When numbers appear in other columns, this is an indication of undesirable interference on the Sonos network. Questions: are values to the left (columns 100, 10) better or worse than values to the right (5000, 10000)? What do the columns actually mean? What is an "acceptable" level of entries outside of the 1000 column, and what represents interference that is likely to cause music streaming issues?

Here is an example of the Network Matrix screen: https://dl.dropboxusercontent.com/u/78476446/Sonos%20Forum%20Posts/Sonos%20Network%20Matrix.JPG

Questions:

What is the meaning of "OFDM Weak Signal Level" and "OFDM ANI Level"? What values of these statistics are considered "good", "bad" , or "marginal"? What are acceptable Noise Floor levels? What is the meaning of the colors green, yellow, orange and red (not shown)?

Here is an example of the buffer screen: https://dl.dropboxusercontent.com/u/78476446/Sonos%20Forum%20Posts/Sonos%20Buffer.JPG

Questions:

There are seven columns with headings 0 through 100. What is the meaning of these columns? In the example, the 40 and 60 columns are populated until you get to the bottom of the scree, and there is an entry where all columns to the left have values. In my experience, the line fully populated with values indicates a problem with the buffer, and is typically associated with music streaming errors such as "Connection Lost" and "Insufficient Network Bandwidth". Please explain what the fully-populated line signifies, and what might be causing such a condition.

In my opinion, there could be interest in a "Sonos User Guide to Interpreting Diagnostic Information". I would be willing to develop such a guide, but I need someone to work with who can provide guidance. Any volunteers?

This topic has been closed for further comments. You can use the search bar to find a similar topic, or create a new one by clicking Create Topic at the top of the page.

16 replies

The Phyerr screen shows six columns of information, 10, 100, 1000, etc., with numbers in the columns. My understanding is that the numbers always add up to 121 horizontally, and that the number 121 in column 1000 is an "error free" reading.

I don't know where that idea came from. Figures in the 1000 column represent the number of 500ms samples where the error count is between 100 and 1000. Numbers in the other columns work the same way. Those in the leftmost column are the number of 500ms samples with an error count of up to 10.

What is an "acceptable" level of entries outside of the 1000 column, and what represents interference that is likely to cause music streaming issues?

Larger numbers in the last two columns are not good, but whether it causes dropouts depends on other factors too: signal strengths, bandwidth demand and playout buffer depth being the most obvious.

What is the meaning of "OFDM Weak Signal Level" and "OFDM ANI Level"?

They're coping strategies in the face of ambient noise. Later wireless cards use ANI Level.

What values of these statistics are considered "good", "bad" , or "marginal"?

OFDM Weak Signal Level of 5 is best. OFDM ANI Level of 0 is best. The latter can go up to 9. However the system can still work perfectly. Again, it all depends on signal strengths and link demands.

What are acceptable Noise Floor levels? What is the meaning of the colors green, yellow, orange and red (not shown)?

'Acceptable' is one which doesn't cause problems. The larger the negative number the better, The colours indicate how good the conditions look.

There are seven columns with headings 0 through 100. What is the meaning of these columns?

Measurements of buffer fill percentages, hence "fill level".

In my experience, the line fully populated with values indicates a problem with the buffer, and is typically associated with music streaming errors such as "Connection Lost" and "Insufficient Network Bandwidth".

The left column is a timestamp. Figures in the percentage fill columns are measurements of, well, fill level. Each line represents 30 seconds. The most recent is at the top.

If a stream drops you'd expect to see the most recent line showing samples in the =0 column, because the playout buffer starved. Similarly you'll see figures in =0 during the 30s measurement period when the stream first starts up.

In my opinion, there could be interest in a "Sonos User Guide to Interpreting Diagnostic Information". I would be willing to develop such a guide, but I need someone to work with who can provide guidance. Any volunteers?

Err, no. The diagnostics are for internal use only, are undocumented (though we've figured out a few things) and can often change when the software is updated.
Userlevel 3
Badge +8
Thank you for taking the time to try and answer my questions. Unfortunately, while my wish is to have less ambiguous answers, your "it depends" answers have left me unsatisfied.

The reality of my situation is that I continue to have dropped connections with the Deezer music service, and my efforts to get to the root cause by working with Sonos technical support have not been successful. So, I continue to look at diagnostics, hoping to find clues to what is going on. I have followed the recommendations that have been given, isolating Sonos components from other wireless devices, adding a Boost, changing wireless channels, etc., but nothing has eliminated the Deezer issues. At this point, I don't know what else to do.
Do you have issues only with Deezer?

Do you have issues only with internet sourced music - or do you also have issues playing music from a wired to router NAS?
Unfortunately, while my wish is to have less ambiguous answers, your "it depends" answers have left me unsatisfied.
Well, sorry about that, but wireless communications are subject to the laws of physics and every situation is different.

The reality of my situation is that I continue to have dropped connections with the Deezer music service, and my efforts to get to the root cause by working with Sonos technical support have not been successful. So, I continue to look at diagnostics, hoping to find clues to what is going on. I have followed the recommendations that have been given, isolating Sonos components from other wireless devices, adding a Boost, changing wireless channels, etc., but nothing has eliminated the Deezer issues. At this point, I don't know what else to do.

I've never encountered a single online service which didn't throw errors from time to time. The internet is inherently an unreliable network transport, although there are of course layer upon layer of protocols on top to guarantee error-free end-to-end transmission. However, such error-free delivery isn't always consistent with timely delivery, which is why real-time streams can occasionally glitch.

If Deezer is the only source which gives problems, then I would say the odds are that either the originating servers or the internet path from them to you have issues periodically. If other sources misbehave too, then the problems are likely to be in your local network. It sounds like Sonos Support have gone through your diagnostics with a fine-tooth comb and if they can't find anything amiss then I very much doubt that we or you will spot the answer.

A Deezer dropout caused at the server end can leave a characteristic error record in the anacapa.trace log. What do the messages actually say?
Userlevel 3
Badge +8
@Kumar: I listen to Pandora, Beats, and Deezer streaming services, and I stream Apple Lossless files from a NAS connected directly to my router. I only experience errors with the Deezer service, which of course is streaming at a higher bitrate than the other on-line services. The errors are of two types, either "the connection was lost" or "the file is improperly coded". When I receive these errors, the streaming service resumes after a several second pause.

@Ratty: Yes, the anacapa.trace log has a record of the errors. Here is a recent log entry:

[Sun Jan 9 06:01:47 2000] Transport error ERROR_LOST_CONNECTION for account type 43015, URI: x-sonos-http:tr-flac%3a532170.flac?sid=168&flags=8224&sn=4, friendly name: On The Bound (Album Version), share/server: Deezer, path: x-sonos-http:tr-flac%3a532170.flac?sid=168&flags=8224&sn=4, extra info: , http: 200, framer: flac
[Sun Jan 9 06:01:49 2000] large sync error triggered resync
[Sun Jan 9 06:01:49 2000] error was 18702 ms behind; cpu usage was 0.3%

Does it provide any insight into what might have caused the lost connection, other than a failure to keep the buffer full?

As I mentioned to Kumar, I don't get errors on any of the other streaming service. I get the errors while streaming from Deezer EVERY session. If I listen for an extended period of time, I may experience several lost connections.

I am tempted to sign up for the Tidal service, since their bit rate is the same, just to see if the higher bit rate is causing an issue. BTW, I have a 300Mbs internet service from Time Warner, so connection speed is unlikely to be the cause.
The problem looks to be on the Deezer side, and exactly what it is will not be visible to Sonos support in their diagnostic tools except to know that there is a problem receiving music streams. Since your NAS streaming of lossless files isn't a problem, there isn't any issue with your Sonos installation even for high bit rate files.

Can you also play Deezer on your computer? If so, you ought to see music interruptions there as well, which would further demonstrate that your troubles with Deezer are not caused by anything, including Sonos, in your home.

Tidal must have a limited feature free service that should allow you to see if the problem is with your internet pipe. If Tidal plays fine, Deezer is the only possible culprit.
It doesn't necessarily follow that deezer will also glitch if played through the computer, we already know that spotify have dedicated Sonos servers that can have issues even when the rest of the service is operating normally, deezer may be exactly the same.
@Ratty: Yes, the anacapa.trace log has a record of the errors. Here is a recent log entry:

[Sun Jan 9 06:01:47 2000] Transport error ERROR_LOST_CONNECTION for account type 43015, URI: x-sonos-http:tr-flac%3a532170.flac?sid=168&flags=8224&sn=4, friendly name: On The Bound (Album Version), share/server: Deezer, path: x-sonos-http:tr-flac%3a532170.flac?sid=168&flags=8224&sn=4, extra info: , http: 200, framer: flac
[Sun Jan 9 06:01:49 2000] large sync error triggered resync
[Sun Jan 9 06:01:49 2000] error was 18702 ms behind; cpu usage was 0.3%

Does it provide any insight into what might have caused the lost connection, other than a failure to keep the buffer full?

As I believe I've told you before, playout buffer exhaustion is a symptom not a cause.

In my experience an ERROR_LOST_CONNECTION / http: 200 seems to be associated with a connection drop at the Deezer end, i.e. your local access point. This would be whoever Deezer use in your area for content distribution.
Userlevel 3
Badge +8
Thanks, Ratty. I guess I'll just have to live with the occasional lost connections. I do intend to try Tidal as well, and keep whichever service proves to be more reliable.
I do intend to try Tidal as well, and keep whichever service proves to be more reliable.
I haven't tried TIDAL in 8 months. At the time I discontinued my subscription on account of .... dropped connections. YMMV of course. Let us know how you get on.
Having checked out Apple Music and using the only service available in India through Sonos - Rdio - I found little to choose from and see them as commodities. Differences in the native apps largely get eliminated in the Sonos controller.

I wonder now if this dropped connections issue is a differentiator across the various services?
Hi, Does anyone know what time period the data displayed in the Network Matrix chart refers to ? For example does it display the average of readings taken over the last 60 mins or is it a snapshot?
Thanks !
Signal strength figures appear to average over about 30 seconds. ANI and other left-column figures change over a longer timeframe.
Hi. Error messages often flash up on the Controller screen but disappear before they can be read. In which log can I find a record of messages such as "Network connection speed insufficient to maintain buffer playback”. For example, should they be in the anacapa.trace file?

Thanks
If it happens to be open at the time the Desktop Controller keeps a limited history at Help/Error Log.

Otherwise there would definitely be evidence in anacapa.trace. The timestamps aren't terribly useful, unless you correct them to wall-clock time by reference to /bin/date which contains the system time corresponding to Now.

If there's a stereo pair, look in the log for the left one. If players are grouped, look in the log for the group coordinator (the first room in the list, to which others were added).
Thanks Ratty !