Music Library Compilation Issue under new 16.3 Firmware

If you’re going to drool, do it properly and go Naim … 

HA!  Good one, Ian.  I just looked up NAIM on Crutchfield and, if I bought that, my wife would neuter me. Again! $$$$  She’s pretty miffed that my Fives are only a bit over a year old and has to listen to me vent over the “troubled” app.

Same problem here. Just when I actually thought my Sonos system is back to performing well it broke with this multiple album problem. I have (30) identical  albums with Christmas songs and all of them are empty…

Have you been naughty this year? 

Not sure about US pricing, but in the UK the Naim mu-so 2 speakers aren’t that eye watering for the smaller Qb Cube in comparison.

Linn and their Openhome system on the other hand 

Sonos - for people who use local libraries this is a pretty important issue and for there still to be no ETA when you broke it fixing something else is pretty poor. It’s not even mentioned in your Trello board… How did it pass the most basic regression test, or does it just show there isn’t any?

@Corry P 
I totally agree with @Ian_S. This issue was created with a firmware update many weeks ago and it would be good to know that Sonos intend to fix the issue in a timely manner.
Being that it is missing from the Trello Board can you please confirm that this issue is going to be fixed or do we have to head down the path of playing around with the metadata for our music libraries to try and work around this issue - which wouldn’t be a small undertaking for those of us that have large music libraries.

Ross.

Likewise - surely this is an easy fix. 

The trello board is titled as app improvement and bug fixing, so I doubt there will be anything firmware specific showing up on it. As ever the firmware side is like a black hole with no information or release notes.

@rosswells127 I’d be very reluctant to retag things because what’s the odds if/when it is fixed you’ll need to redo it all again?

@Ian_S agree, have to wonder what the test suite covers. Surely they are using the existing test suite and extending/changing tests as the firmware changes. Automated tests for something as basic as metadata checks when making calls to the firmware catalogue server should already exist.

@ninjabob in theory, but it depends what other code changes it was rolled into and whether undoing the metadata extraction/catalogue server code change means those changes will break.

My feeling is there are other issues within the firmware affecting even more people that are taking higher priority.

Hi @sigh 

This issue will be getting addressed - I recommend that you do not alter your metadata (unless you are willing to change it back again too).

You are correct - it is not related to the new app, so no app update will address it, and it’s not showing on the trello board as a result. This will take a player firmware update to address - work is being done, but I cannot give you an ETA presently.

I hope this helps.

Aside from the compilation album problem (which basically means that I have to spend many minutes to find anything in my music library), there is also a dire need for an alphabetical index on the right side fo the screen like the one we had before the update, which lets you jump to albums (or artists, or genres, etc.) that start with a certain letter.  This is enormously useful for those of us who have extensive music libraries.  I can’t imagine that this is complicated to add to the app, since it’s just a minor indexing feature.  There seems to be no assurance that we will ever get this feature back.  Unless we do, eventually I’ll just junk my whole system and try something else (at the cost of a small fortune that I’ve spent over the years buying Sonos equipment). 

Hi @Vanallen 

Thanks for the feedback - the alphabet shortcut has been asked for so much since the introduction of the new app that I would be shocked if it was not added back in.

Is the compilation indexing issue a bigger problem than we originally thought? 

If I look at my library, I have 77 compilation albums (I don’t include greatest hits for an artist in these) that I have the album artist set to ‘Various Artists’ for. If I do a find into a line count, I appear to have about 2255 tracks, so an average of around 30 tracks per compilation. Some will be more (2+CD’s) some less, single CD’s. I’m sure occasionally you’ll get 2 tracks on one compilation by the same artist. 

If I use the desktop Sonos controller to look under Various Artists, the list is huge, and randomly selecting a fake album entry it appears to include most if not all the compilation It came from but in an odd track order. 

So the worst case scenario is 77 albums X 30 tracks X 30 tracks again, which is a colossal 69300 index entries. 

As I understand it, the local library index max as limited to 65000. So with around 11,000 ‘proper’ tracks in addition to the broken compilations, I must presumably be under 65,000 which I’d think is quite likely as I’m sure it’s not as bad as the worst case, but I have no way to easily traverse my Sonos index to get actual figures. 

So, I’m curious how many compilation albums others have on here compared to overall library size. 

I used this command on my Mac to get the info I wanted… from the Compilations directory created by Apple Music (was iTunes...)

find . -type f -name “*.m??” | wc -l

And I used m?? as I have either m4a or mp3 files, so that finds those. If you use wav or flac, you may need to use multiple finds and add the results or a more esoteric search param. 

Ease of finding depends how you arrange your local library of course. 

I do have non-compilations that suffer from the broken index, but they’re rarely as extreme as a proper compilation album, so I’m assuming I can ignore them. However, if you have lots of jazz/classical with many artists that also may start significantly bloating your indexes. 

My final question I suppose here would be is the 65,000 an actual list limit and the amount of memory used will depend on the metadata associated with each of a max of 65,000 entries, or is there a fixed memory limit and 65,000 is a rough estimate of the maximum based on that? So will speakers with more memory handle this better than those with much less? Or does it make no difference as the memory area for an index is fixed across all speakers? 

I won‘t get too deep into that and stress my mind as Sonos is aware of the issue and probably it will be fixed by one of the next firmware updates. 

Yep. I also found this when I updated the Android app on my phone, so I went to my PC to use the software on that, had to do an update and exactly the same. All ‘Various Artist’ compilation albums are now listed as single track albums using the artistes names.

I’ve using the workaround by lists the albums using the ‘Folders’ option in ‘Music Library’; I’ve also resurrected an old Windows tablet that I’m now using as my ‘portable’ controller, as the app on this is the same as the PC app.

I’ve deleted the Android app as it was just taking up storage capacity on my phone.

Will I recommend Sonos to anyone else at the moment ? No, I wouldn’t inflict this product range on my worst enemy, and it breaks my heart to say that, because, up to a few months ago, I LOVED this product range.

If it has been implemented in a similar way to upnp content directory services then probably not.

The metadata extraction which will be stored in data files and is used to return data to the queries. The app then uses the returned data to organise into a display. I’d expect the backing data store like almost every lightweight upnp/dlna server is SQLite based.

The upnp content directory service has a defined schema and api for browse, search, pagination, metadata to return etc. Sonos have extended it I think as the old app has the option to choose an album artist display mode, which I don’t  think is part of the upnp spec.

There only needs to be the single data store for any view, query/search the app implements. There are examples of the SonosApi automatically exported from the devices floating around the Internet.

This view from my AssetUpnp server of album by release is just a query to the same data files used to provide the album artist, album, artist views, it will request the year and album title metadata and probably a sort order as part of the query.

Hard track limit, but could be less based on memory required for metadata according to Sonos.

https://support.sonos.com/en-gb/article/missing-tracks-or-albums-in-sonos-music-library

A Sonos music library has a 65,000 track limit. The limit can be reached by either track count, or by memory usage. The memory limit can be reached prior to 65,000 tracks if there is a large amount of metadata. For example, classical music tracks tend to have a large amount of metadata and can use up the available memory before the 65,000 track limit is reached.

Suggests it’s a fixed memory limit to me as there is no mention of different memory limits for different groups of devices like there is with say WiFi.

Ian_S:

 Is it true that when the metadata in my library is read it looks at the at the metadata as I structured it using tagging software?  The reason I ask is because most of my library is classical music which you stated has a large amount of metadata.  The way I tag classical music is with the following tags:  Artist, Album, Track No., Date, Album Artist.  That’s it.  So would that help to minimize the memory used when the library is read?

Depends ow much data you put in those fields! If you have the whole orchestra listed under artist, then it could be large. I’m guessing you don’t though. 

The other thing that seems to impact on it is the overall file path for each track which appears to be stored. There’s another thread where reducing that helped someone keep their library within limits. There may well be Sonos specific stuff generated too that we have no control over. 

The metadata extraction which will be stored in data files and is used to return data to the queries. The app then uses the returned data to organise into a display. I’d expect the backing data store like almost every lightweight upnp/dlna server is SQLite based.

I guess we can only hope that that is how they are storing the metadata they need in the format they require. On the flip side, if they haven’t screwed the data, why is it taking so long to fix what would then just be some poorly written queries? Which if they are returning 1000’s of rows when it should be 10’s won’t be helping with CPU or RAM usage on speakers, esp. older ones. 

I do think they could do with re-instating the metric for how much memory your metadata is using either absolute or as a % of what’s available in the About your system output. Would certainly help. 

Ian_S:

 My Artist & Album Artist tags are the last name of the composer.  All other tags in purchased downloads, other than the ones I use, are removed.  I use DbPoweramp for tagging.

Updated to 16.3.3 today. Rescanned the library, to be sure. Still no fix to this. Disappointing naturally. A low priority or difficult to put right? 

Imo more of a low priority. 
Thanks for reporting. 

 What was the priority then?  What did they fix?  To me the glaring flaw was the compilation issue.  Definitely did not improve speed.  Just opened an Apple Music playlist I created, tapped on a track a ways down the playlist.  It took 10 seconds to start playing.  Nuts!

Well I was stupidly optimistic that the new Firmware update would resolve this issue. Silly me!
I struggle to understand why things are taking sooooo long to fix. I reported this issue in mid July.

Sonos, this bug was CREATED in Firmware update 16.3.0 and you are now up to 16.3.3. A lot of your users are affected by this issue and being a Firmware bug it affects ALL controllers including 3rd party ones so we have no viable workaround. Can you therefore please move this up your priority list. @Corry P 

Ross.

@Eejay60 

Was it a firmware only update or also one of the app? Just asking because for me in Germany there’s no update at all yet. Firmware 16.3.1 is shown and no update available. 
But let’s wait until tomorrow.