iTunes Library Manager

Darren
For some reason my transformation rules have stopped working. I added some new tracks to iTunes and assumed the app would index them since it looks for closure of iTunes.
This did not happen so I did it manually and still it did not find them (and they are in the correct place).
So I thought something had become corrupted so I deleted the destination iTunes Library file and also the two cached csv's and started from scratch and it found no files at all. I have rebooted and tried again and still not found anything.
I am not entirely sure which version I am running since there is no About menu, but its dated 7 Jan 22.48. I think this is the version after the one I used previously which worked!
Gordon

Hi,

Late last night I posted version 2.60.1 which is the first version to have a setup wizard. I discovered a bug this morning where the configurations files were being saved to the last used directory rather than the application directory. This has been fixed in version 2.60.2 which I have just posted.

On my Windows 7 PC I have a minor issue when it is installed to C:\Program Files\Darren Link\iTunes Library Manager. After installing, the application does not have the necessary permissions to write the configuration files to this directory. If I install the application to C:\iTunes Library Manager everything works as expected.

When the application is uninstalled via the control panel the configurations files are left installed. This should aid upgrading to a later version of the application without losing your configuration files.

If I change the permissions of directory C:\Program Files\Darren Link\iTunes Library Manager so that "Users (Darren-PC\Users)" has full control, the application is able to create and update the configuration files.

Hopefully I will have a fix for this installation issue in the near future. I have tested the installer on Windows XP and the above permission problem does not occur under this operating system.

Darren

All,

I have posted a new version (2.59.2) tonight which has better error handling whilst processing the tracks within the source XML file. If an error occurs processing a track the application should now report details about the error and also give details about the track it failed on.

Gordon,

Glad you are no longer getting the earlier error whilst the application was processing the tracks. The above error handling would have been useful if you were continuing to get that earlier error.

It's excellent to here that you now have the application working for your system. You are the first person to report success with my app, which is music to my ears. :):) :) I hope you find the application as useful as I do going forward. It's good to hear it working fully with FLAC files as the majority of my use/testing has been with Apple Lossless files. (I did create 1 FLAC file and did 1 test but not extensive testing)

Before I developed the application I was frustrated about playlists not importing because I had too many iTunes playlists for the available Sonos memory. I also didn't want to maintain playlists in both a lossey (mp3) and lossless (Apple Lossless) iTunes library. Now I only need to create playlists in my MP3 iTunes library (the only one I have) in order to get them onto my Sonos system pointing to the Apple Lossless tracks.

Have you tried changing any of the settings under Edit Settings? Now that you have the processing working for your needs you should be able to enable the following features:

Minimize to system tray
Minimize to system tray on startup
Minimize to system tray on close
Listen for closure of iTunes
Sonos Indexer Enabled.

After you have enabled these settings the application can run in the backgroud continually (if you wish) and kick in when iTunes closes. I recommend creating a shortcut to the "iTunes Library Manager.exe" and placing it in your All Programs -> Startup folder. The application will then start when you start windows and will be minimized to the system tray. If the notification messages become annoying you can supress them under Edit -> Settings (Surpress all system tray notifications)

Now onto your issues with window sizes. I haven't encountered any issues with the 'Suggest track match' window. Perhaps you could email me a screenshot to illustrate what is happening? The same applies to the 'Analyse Library' function. I am surprised you are having these problems as the layout looks fine on my Windows 7 machine (although 32 bit). Are you using a non default Windows 7 skin perhaps?

Here is a screenshot of the 'Suggest track match' window on my machine:

http://www.darrenlink.talktalk.net/ituneslibrarymanager/images/non_exact_match_found.png

and also with it resized:

http://www.darrenlink.talktalk.net/ituneslibrarymanager/images/non_exact_match_found_resized.png

Also, here is a screenshot of the 'Analyse Library' function on my machine:

http://www.darrenlink.talktalk.net/ituneslibrarymanager/images/analyse_library.png

Can you see what the differences are? Screenshots from your system would be very helpful. (See my profile page for my email address)

Many thanks for having the patience to set up your files in the necesary structure and for persisting with my app. Your input on your reported issues would be greatly appreciated. Also feel free to suggest improvements to the application.

Thanks again for your patience and feedback.

Darren

Gordon,

Apologies for the delay in my response. I am now back at work following Christmas and the new year so many apolgies if I am a little shlower at responding.

I had a go at repointing an iTunes playlist to tracks on a non indexed NAS share a while ago but I did not get the desired results. If you do this you will find that when you add the playlist to the queue of tracks the filenames are displayed in the queue rather than the track details from the tags. (The tag details are not stored in the Sonos memory as the tracks have not been indexed). If you can put up with tracks only being available from playlists and don't mind the filenames being displayed in the queue you can use this approach to get round the 60,000 track limit. In your case it sounds like you only have 35,000 tracks so you should be able to index the highest quality version of each one without any problems.

Unfortunately my application cannot control which tracks are indexed by the Sonos indexer. It can only control what playlists are written to the customised "iTunes Library.xml" file and where each track points to. The paths to tracks can be to an indexed or non indexed share but point the tracks to indexed files wherever possible.

The only way I know of ensuring that only 1 file per track is indexed is to ensure that only 1 file per track is within the share (or shares) that is indexed by Sonos. Each file should be in the highest quality that you have available. I am not aware of any way of telling the Sonos indexer to pick the highest quality track if you have multiple versions of a track in shares indexed by Sonos.

In my setup I have my entire library as MP3's on my local PC. The iTunes library is also stored on my local PC and points to these MP3's. It is this library that I use to sync with my ipod. This is the best approach if you have a laptop as the MP3's will be available whereever your laptop goes, and you will be able to build playlists for your Sonos system when you are away from home (or indeed on your ipod)

My nas share that is indexed by Sonos (\\nas\sonos) is basically a copy of the local MP3's with any MP3's replaced with the Apple Lossless equivalent where possible. The folder structure for my local MP3's is identical on my NAS. Folders beneath Z:\Music (could be C:\Music or any other letter) on my local PC are indentical under \\nas\sonos\Music with mp3's replaced with Apple Lossless files where possible. It is important to note the word replaced. Each directory on my NAS does not contain files in both formats. Rather only 1 file in the highest available quality.

My application takes the local "iTunes Library.xml" file and changes each track definition to point to the equivalent file on my NAS based upon the rules defined within the application. The modified XML file is then written to \\nas\sonos\iTunes\Darren\iTunes Library.xml.

Regarding the problem you had with my application crashing. It sounds like it is crashing on one particular track. To debug (on your PC) I would need to add logging to the application and identify which track it was failing on and why. If you could email me a zipped copy of your "iTunes Library.xml" file I could take a look at this file for any tracks that could cause my application to crash and may be able to debug in my environment. I am keen to eliminate as many bugs as possible from my application. If you could note the track number that it fails on that would be useful too.

I hope that clears up any confusion.

Darren

Darren
Thanks for your response to my query. Apologies for not getting back to you sooner. Have been away over New Year.

An update on where I am up to. I decided to split my shares on my NAS drive so that I now have my FLAC and MP3 shares on different shares. I had read on the forum that the Sonos would not index duplicates if they had the same tags, so I was hoping that by indexing the flac share followed by the mp3 share on the SONOS it would ignore the mp3 duplicates and only pick up the mp3only tracks. Not so. Even though I can see just one record under say Artist/Track on the SONOS controller under Folder they are both there and a check on the status summary screen reveals I have far more records than I should have in the index! SONOS support have confirmed that they are both picked up becuase they are different (one is mp3 and one is flac). I have blown the 65,000 limit (I have about 35k UNIQUE tracks at present).

Unfortunately I have also had problems with your app with the mp3 to flac transformation feature. It kept giving me an Unknown Windows Application error (I tried it on Vista and Windows 7) at the same record number and aborting, so I gave up and decided to create the seperate share on the NAS for the FLACS.

So how do I get just the mp3only tracks into SONOS. I am now thinking - if I create a smart playlist in itunes for my mp3only tracks (I can do this based on a specific user defined tag field that I have already created). I could then use your app to just import that playlist into SONOS.

My question is. Will the SONOS recognise the mp3 tracks in the imported playlist (even though the tracks have not been directly indexed - I will only index the FLAC share) and will these mp3only tracks appear under the Artist/Title/Genre sections on the SONOS or will they just appear under the imported playlist?

Regards
Gordon

Joel,

In response to the following:


Noted and will be checked but that said, likely not the case because the problem was a doubling up in iTunes itself which, of course, resulted in a doubling up in SONOS...


Do you have any tracks still in your iTunes library that are still doubled up?
I appreciate your agreement that it is not my application that is doing this. If you have any duplicates still, right click on each track in iTunes and choose Get Info. Take a look at the first tab for each track and note the path to each track. I am 99% confident they will be different paths. I have tried adding the same file to my iTunes library multiple times by dragging the file into my library. iTunes doesn't report any errors but simply ignores the file if it is already in the library. I therefore supect the duplicates are two different files. The fact that the tracks also appear twice in your Sonos library also points to two physical files for the same track existing in your share. One thing you could do is try renaming one of Apple Lossless files on your NAS share that is appearing twice in your Sonos index (and then don't reindex). If you try playing both tracks one may play and the other may report a missing file.

A note to Sonos. I don't think there is a function for finding out the path to a track file within the index. This feature is a must. Please add it or correct me if I have not seen this feature. Your should be able to right click on a track within the Sonos desktop controller and do a 'Get Info' just like in iTunes. The Get Info screen should show all the index information about the track and the path to the track that Sonos is using to play the file.

One way of getting the same physical file into your iTunes library twice, would be to add it in from the network path as well as from a mapped drive path. iTunes is likely to consider \\nas\AppleLossless\Track1.m4a and S:\Track1.m4a as two different files (where S is mapped to your \\nas\AppleLossless share). You should take care to ensure you always add tracks to iTunes from the same path. I recommend adding the tracks from a mapped network drive mapped to your \\nas\AppleLossless share.

You could also use my application to investigate the issue. If you use the Analyse Library function for the source library path and double click on the count of m4a files found you will see a list of all of the paths to your tracks. If you see a mixture of different path types that is your problem.

That's my brain dump over. I hope that helps.

Darren

Joel,

Thanks for your detailed explanation of your setup. It has helped a lot.

Please can you explain exactly what you mean by "doubled the albums in iTunes". If you actually go into iTunes and type the album name in the search bar at the top do you see the list of tracks but each track doubled up? If this is the case, this has nothing to do with my application. Right click on the track details header within iTunes and tick the Kind option to see the file type for each track. Also you can right click on each doubled up track and choose get info to view the path to each track.

If you have a playlist for an album in iTunes and it is the playlist that appears twice within Sonos, I think I know what the problem may be. Check your appless Lossless share where your iTunes files are stored and make sure there are not multiple XML files. If you are writting the destination library xml file to the Apple Losless share the Sonos indexer will pick up both the orginal iTunes XML file and also the modified file written to your NAS by my application. My application currently prevents the source and destination library paths from being the same to prevent the original iTunes XML file from being overwritten. An enhancement to my application maybe to allow the paths to be the same, but to rename the original (source) XML file to "iTunes Library Original yyyymmddhhmmss.xml.bak" for example before the destination XML file is written to the same path as the original XML file. The Sonos indexer will not pick up the .bak file as it is only looking for iTunes XML files with the .xml extension. I do not have the problem of multiple XML files being on my Sonos share because my MP3 library (and all of the track files) are stored on my local PC. Only the XML file (destination file) pointing to my lossless tracks is stored (written by my app) to my Sonos share. If you have the source and destination iTunes Library.xml files in your appless lossless share move the original file created by iTunes to your desktop and then re-index. The Sonos indexer will then only find the XML file created by my application. It is safe to move the original XML file as iTunes will recreate it next time iTunes is opened and re-closed. The xml file is essentially created by iTunes from the master iTunes .itl file in the same directory as the xml file.

Another thought regarding the XML file. At some point iTunes changed the filename of the XML file from "iTunes Music Library.xml" to "iTunes Library.xml". I have heard reports that when this happened the original "iTunes Music Library.xml" file was left in place so that 2 xml files were found by the Sonos indexer. This also results in a doubling up of playlists. If you have both files it is safe to remove the original "iTunes Music Library.xml". It should have an old modified date.

I hope all of the above helps.

Darren

Joel,

As far as the doubling up goes please note:

1. All my lossless files and all my compressed files are stored on my NAS drive BUT under two different shares (i.e. one share is called AppleLossless, the other share is called MP3).

Are you indexing both of these shares within Sonos and do you have separate "iTunes Library.xml" files under each share? If you are only indexing the AppleLossless share in Sonos what do you do about tracks that you only have in a compressed/lossey format? For the albums in AppleLossless format do you have all of the tracks in MP3 format under the MP3 share also? If both shares are indexed, Sonos will pick up both versions of the same track. For this reason my share on my NAS is simply called Sonos and only contains one file per track in the highest quality format available. I don't have different share names for different filetypes. Each track is also on my local PC in the compressed format (mainly mp3) for use with my iPod and these tracks are not indexed by Sonos.

2. I am certain that only one lossless track exists for each album, that only one lossless track appears in the SONOS library, etc.

3. Perhaps to be clearer I was using iTLU separately for the lossless and lossy libraries. That is:

a) I would rip the CD in AppleLossless to the AppleLossless share drive using dBpoweramp.

b) I would convert the AppleLoss files to the MP3 files on the MP3 share drive using dBpoweramp's batch converter

c) I would then use iTLU separately on each share to add the files to the iTune library but, in each of the shares, the files would be doubled (i.e. not physically doubled on the drive but doubled within iTunes because the resulting iTL after running the application listed each file/song twice).

d) Thus, I stopped using the application but am very happy to give the new version a go...I will simply make a copy of my iTL before running it.

I have never used ITLU (iTunes Library Updater I assume?) but you have answered one of my earlier questions. For a single track you have an MP3 version under your MP3 share and an AppleLossless version under your AppleLossless share. If you have MP3 downloads (with no Apple Lossless equivalent) I presume you are placing them in the MP3 share? If you are indexing both shares any tracks in both formats will be imported into Sonos twice regardless of whatever iTunes libraries you have in these shares. The Sonos indexer will index all tracks it finds rather than just the tracks defined within your iTunes libraries.

Do you have an Apple Lossless iTunes Library on your Apple Lossless share and an MP3 iTunes Library on your MP3 share? Are tracks that you only have in MP3 format in both shares and libraries?

It sounds like the ITLU application is adding both MP3 and AppleLossless copies of each track to the same library. Showing the Kind column within iTunes will confirm this.

Not usre I understand the above bit regarding but will play with things when I give it a go...really, at least for now, most interested in using iTLU to:

a) Add lossless songs to my lossless iTunes library that have been ripped by dBpoweramp

b) After a) above automatically i) convert the above added songs to lossy formats and ii) add said songs to my lossy iTune library...a *HUGE* added bonus would be the ability to carry over ratings in the conversion process from lossless to lossy.

c) After a) above automatically re-index my SONOS library

I can't advise on how to use ITLU as I'm not the author of that application and have never used it. I can advise on how to structure your music for use with my application however. I only have a single iTunes library for my lossey tracks (mainly mp3's stored on my local pc and not indexed by Sonos) and use my app to generate the "iTunes Library.xml" file pointing to the mainly lossless tracks on my Sonos share that are indexed.

Got it...sounds (pun intended) good!

Again, much thanks...hope you had a Merry Christmas and a good satrt to the new year...will connect with you over the next few days when I have a new CD to rip and have used the latest version...

Joel

I had a great Christmas thanks. I hope yours was good too and you had a great overseas trip. If you have MSN it would be good to chat on there to clear up your understanding of my application and my understanding of how your music is currently organised. I really should do a tutorial about structuring music for lossey and lossless setups. Essentially you need to keep the lossey and lossless files independent from each other and only index the lossless files + any tracks that you don't have in a lossless format. If you keep MP3's and your master lossey library pointing to those mp3's on a laptop those tracks and library are available on your travels. If you store the mp3's on your NAS they are not available when you leave the house.

In Summary my recommended approach is:
MP3 = Best for portability and ipods so store on your local pc/laptop and do not index in your Sonos system.

Apple Lossless = Best for use just when you are at home using your Sonos system so store on a NAS share along with any tracks that you do not have a lossless version of. Index only these tracks within your Sonos system and ensure that there is only one version of each track available to Sonos.

iTunes = Have one master iTunes library on your local PC pointing to the local MP3's, and use this library with your iPod.

Then use my app to repoint local mp3 links in your master lossey library to the same mp3's or higher quality equivalents on your nas share indexed by Sonos.

Keeping the folder structures the same on your local PC and NAS helps to keep the mapping rules simple.

Speak soon

Darren

Gordon,

If you are indexing your //NAS/Media share and you have a track in both MP3 and FLAC formats (in the MP3 and FLAC subdirectories) the Sonos indexer will pick up both files resulting in duplicates within Sonos. This will be the case no matter what "iTunes Library.xml" file is found and indexed. This is because the "iTunes Library.xml" is used to import playlists, rather than to control which tracks are indexed by the Sonos indexer.

For this reason I keep all of my lossey tracks (mainly MP3's) on my local PC and my lossless tracks (Apple Lossless) on my NAS. I have many tracks that I only have a lossey copy of and these are stored both on my local PC and on my NAS (keeping the folder structures the same). I always ensure that I only have the highest quality version of each tracK within my NAS share that is indexed by Sonos (and no other lower quality versions of the same track available to the Sonos indexer)

Here is what I recommend (but please read and fully understand before you do anything)

If you are confident that all of your tracks within iTunes point to M:\Music\MP3 (where M is mapped to \\NAS\Media) you could copy your NAS MP3 folder (//NAS/Media/Music/MP3) to a local folder such as C:\Local Media\Music\MP3 and then make C:\Local Media a share on your PC. This share could then be mapped as an M network drive on your local PC (instead of pointing to your NAS). The paths to all of your MP3's within iTunes will not need updating but they will be pointing to your local PC instead of your NAS. After doing this you would be able to remove any MP3's from //NAS/Media/Music/MP3 that are also available in FLAC format under //NAS/Media/Music/FLAC. (i.e remove the lower quality duplicates from the share indexed by Sonos)

If you have multiple transformation rules within your configuration it will attempt each rule in turn until it finds a destination track to point to. If all rules are applied and no equivalent track is found then the application changes the path within the XML file to "No equivalent found"

In your scenario I would set up the following transformation rules

Rule 1
~~~~~~
Souce Media Type: MPEG Audio File (mp3)
Source Path Find: M:\Music
Path Replace: \\NAS\Media\Music
Destination Media Type: FLAC - Free Lossles Audio Codec (flac)

Rule 2 (If the equivalent FLAC file is not found via Rule 1 this rule will be used to find the MP3 on your NAS)
~~~~~~
Souce Media Type: MPEG Audio File (mp3)
Source Path Find: M:\Music
Path Replace: \\NAS\Media\Music
Destination Media Type: MPEG Audio File (mp3)

If you change the rules I recommend deleting the "Mapping Cache File" and my program will recreate it from the new rules.

Finally your last question. The "iTunes Library.xml" file can be on any share that is indexed by Sonos but you must remember that the share that it is stored on must be available when you re-index Sonos. I therefore think that the best place for your "iTunes Library.xml" is on your NAS. The Sonos indexer will import multiple "iTunes Library.xml" files. These can be stored in the same share, or they can be in different shares. In your scenario I would set the Destination Library Path to: \\nas\Media\iTunes\Gordon\iTunes Library.xml or \\nas\Media\Music\iTunes\Gordon\iTunes Library.xml depending on your preference). If another nember of your family wishes to have the MP3's on their laptop or PC they could be mirrored from your local PC to their laptop retaining the same M:\Music access that iTunes is using to find the MP3's on your PC. If the paths to the MP3's are the same on a second PC, you can simply copy the 'Source Library' iTunes Library.xml file that is generated from iTunes to the secondary PC and then import this file into a blank iTunes library. Also if you wish, you could use my program to generate a modified copy of your "iTunes Library.xml" file that doesn't contain any playlists or user related data such a ratings and play counts. This is what I did for my girlfriends laptop where my local MP3's are mirrored to.

I know that the above trick of creating a 'Music/Media Partition' share and mapping it as a network drive on the same PC works. I have this setup on my PC to save re-partitioning my drive and assigning a drive letter to the physical partition that my Music is stored on. In the past I had a Z drive physical partition for storing all of my music (Z:\Music) but when I got a new hard drive I did not want to restrict the size of the partition for my Music so I used the share/mapped network drive trick so that I would not need to repoint iTunes to a different drive and location. I don't think there is any performance limitation in doing this as the network drive is pointing to the same PC (anyone, please correct me if I am wrong)

I have probably given you more info than you need, but if you have any questions please ask before you make any changes to your current setup. It is always best to be 100% sure before you make any changes. It is not straightforward maintaining lossy and lossless copies of each track but I hope my application makes the process easier. If you have lots of playlists you should find the feature for limiting which playlists are written to the Destination Library XML file useful. Also after you have done any restruturing you can use the "Analyse Library" feature to check that there are no broken links to any tracks. You can use the "Analyse Library" feature on any source or destination library XML file, or you can specify a different "iTunes Library.xml" file altogether via "Library -> Analyse iTunes Library XML file".

If you have a laptop it makes sense to store your MP3's and iTunes library files on that laptop so that the library and music can be used on your travels. Any playlists that you make on your travels can be imported into your Sonos system on your return, but they will be pointing to the lossless equivalents (for each track where a lossless copy is available).

Any questions, please ask. I understand that my application is not the easiest thing to understand.

Darren

I have just posted version 2.58 (http://www.darrenlink.talktalk.net/ituneslibrarymanager/download/2.58.0/iTunesLibraryManager.zip) which has the ability to kick off the Sonos indexer once the destination "iTunes Library.xml" file has been written to your NAS. By default this Sonos functionality is turned off, but once it is enabled (Under Edit Settings) and the application has been restarted you will see an additional option under each iTunes library configuration tab to start the Sonos indexer once the destination "iTunes Library.xml" has been written.

Credit goes out to Steve Eisner, whose C# code I have adapted to kick off the Sonos indexer. Once the Sonos functionality is enabled the number of detected Zones will be shown in the bottom left hand corner of the application. If no Sonos zones are detected the application will fail to kick off the Sonos indexer. (You will need to allow Windows to connect to your network for it to detect your Zone Players) There is an additional button to kick off the Sonos indexer whenever you wish. This button is disabled until at least one Sonos zone is detected.

I have noticed that after kicking off the Sonos indexer via my app, the desktop controller does not show that the system is busy indexing the music. The original wireless controllers (CR 100) show that the system is indexing after a short delay.

Feedback on the reliability of the Sonos indexer will be greatly appreciated. Under development tonight I noticed on occasions that the application failed to detect all my Zones but I have not experienced this problem with the release version that I have posted.

I hope you like the update. Any questions or problems please ask and I will do my best to help.

Regards

Darren

Darren - thanks very much for your hard work on this app. I still must download it, but I very much look forward to trying it out, as I am also off work for the next 10 days or so!

Joel,

I have tomorrow afternoon off work for Christmas Eve, so after I've finished wrapping all the presents I intend to post the latest version of the application on my web site. The version on my web site is quite old now. The latest version has been tidied up quite considerably, has an option so that the program can kick in and transform your library file as soon as iTunes is closed and the ability to analyse your iTunes library XML files for stats and broken links.

I am off work until the 4th January so I'm hoping to do further developments during this time. Feedback and suggestions for improvements would be useful. I should be around a fair bit over Christmas so feel free to catch me on MSN for a chat about the app. (My MSN address is on my Sonos forums profile page as previously mentioned)

Darren

Darren:

Fantastic news noting that I look forward to trying / using the new version...the one problem that I did have with the old version was that at random times -- and I could never figure this out -- it doubled the albums in iTunes meaning that as a backup I was always made a copy of the iTunes library before running the applications...that said, perhaps making a default copy before running it would be a beneficial safety feature...any thoughts?

Thanks,

Joel