Music Library Cleanup and Merging - Revision history

DocGyver: /* Find Duplicates */

2020-04-21T13:43:21Z

Find Duplicates

← Older revision		Revision as of 13:43, 21 April 2020
Line 45:		Line 45:
	*Using the following command line or similar create a playlist of duplicates		*Using the following command line or similar create a playlist of duplicates
	<pre>		<pre>
	egrep -f <(cut -d"," ~~-f1 duplicate_check.csv~~ \| egrep -v '^$' \| sort \| uniq -d) ~~duplicate_check.csv~~ \| sed -e 's/.*Media\\Music/M:/' > ~~abc~~.m3u		infile=acoust_id.csv ; egrep -f <(cut -f1 "$infile" \| egrep -v '^$' \| sort \| uniq -d) "$infile" \| sed -e 's/.*Media\\Music/M:/' > "${infile}-dupes.m3u"
	</pre>		</pre>
	*Load the m3u playlist into Mp3Tag.<br/>This may take some time possibly even longer than loading the original folders. Not sure why but just be patient. My very first time was a greater than 8k list that took several minutes.		*Load the m3u playlist into Mp3Tag.<br/>This may take some time possibly even longer than loading the original folders. Not sure why but just be patient. My very first time was a greater than 8k list that took several minutes.

DocGyver: /* Tools */

2015-01-26T11:23:12Z

Tools

← Older revision		Revision as of 11:23, 26 January 2015
Line 84:		Line 84:
	* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.		* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.
	* [http://xrecode.com/ xrecode$$] is a tool for recoding to different formats that supports moving of tags.		* [http://xrecode.com/ xrecode$$] is a tool for recoding to different formats that supports moving of tags.
			* [http://www.mediahuman.com/audio-converter/ AudioConverter] is a free tool for recoding to different formats.
			* [http://www.mediahuman.com/lyrics-finder/ Lyrics Finder] is a free tool for adding Lyrics tags.

DocGyver: /* AcoustID Tag */

2014-12-30T02:08:47Z

AcoustID Tag

← Older revision		Revision as of 02:08, 30 December 2014
Line 27:		Line 27:
	**\222		**\222

	==~~AcoustID Tag~~==		==Find Duplicates==
	*Use Jaikoz to add AcoustID Fingerprint and, if it exists, ID on all files.<br/>Even if you do not purchase Jaikoz, the 20 file per session limit does not apply for adding the AcoustID value.		*Use Jaikoz to add AcoustID Fingerprint and, if it exists, ID on all files.
	Having added this value you will be able to search for potential duplicates.		<br/>Even if you do not purchase Jaikoz, the 20 file per session limit does not apply for adding the AcoustID value.
			<br/>Having added this value you will be able to search for potential duplicates.

	*Create csv/text file with AcoustID ID and full filename to be able to search for duplicates		*Create csv/text file with AcoustID ID and full filename to be able to search for duplicates
			**Open Mp3Tag
			**Open your main library folder (File:Change Directory...)
			**For each folder you are merging with your library do File:Add Directory...<br/>When complete Mp3Tag will have a list of all files in your current library and any new files you are adding
			**Export a CSV file with AcoustID and Full path. (File:Export...)<br/>The following script should be used for the export.
			<pre>
			$filename(csv,utf-8)AcoustID ID Filename
			$loop(%_filename_ext%)%AcoustID ID% %_path%
			$loopend()
			</pre>
			**The resulting CSV (actually Tab not Comma)
	*Using the following command line or similar create a playlist of duplicates		*Using the following command line or similar create a playlist of duplicates
	<pre>		<pre>
	egrep -f <(cut -d"," -f1 duplicate_check.csv \| egrep -v '^$' \| sort \| uniq -d) duplicate_check.csv \| sed -e 's/.*Media\\Music/M:/' > abc.m3u		egrep -f <(cut -d"," -f1 duplicate_check.csv \| egrep -v '^$' \| sort \| uniq -d) duplicate_check.csv \| sed -e 's/.*Media\\Music/M:/' > abc.m3u
	</pre>		</pre>
			*Load the m3u playlist into Mp3Tag.<br/>This may take some time possibly even longer than loading the original folders. Not sure why but just be patient. My very first time was a greater than 8k list that took several minutes.
			===General Strategy to remove duplicates===
			*Sort by Artist (not Album Artist) and make sure you have an artist for most rows. Not absolutely necessary but very helpful for filtering.
			*Use Filter (F3 or View:Filter) to pare off a group of files. e.g. %artist% MATCHES ZZ Top
			*Look for whole album duplicates first by sorting on path name, selecting an album from the "import" folder then sorting by either album name (if it is already set) or by AcoustID which will put the two album copies next to each other.

	==Tagging==		==Tagging==

DocGyver: /* Process */

2014-12-28T13:58:38Z

Process

@@ Line 26: / Line 26: @@
 **\242\245
 **\222
 ==Tagging==

DocGyver: /* Tools */

2014-12-26T21:58:48Z

Tools

← Older revision		Revision as of 21:58, 26 December 2014
Line 54:		Line 54:

	=Tools=		=Tools=
			* [http://www.mp3tag.de/en/ MP3Tag] is for bulk changing of tags and file renaming. It can look up tags online but Picard tends to be better.
			* MusicBrainz [https://picard.musicbrainz.org/ picard] for tagging based on audio fingerprint.
	* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.		* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.
	* [http://xrecode.com/ xrecode$$] is a tool for recoding to different formats that supports moving of tags.		* [http://xrecode.com/ xrecode$$] is a tool for recoding to different formats that supports moving of tags.

DocGyver: /* Process */

2013-07-01T17:59:37Z

Process

← Older revision		Revision as of 17:59, 1 July 2013
Line 1:		Line 1:
	=Process=		=Process=
	*Find all files with ~~incorrect~~ extensions		==File Name issues==
	**find ~~Artist~~ ! -type d -regex "[^.]+"<br/>replace "~~Artist~~" with directory to search. It cannot contain or be "." due to the regex used.		*Find all files with missing extensions
			**find <folder> ! -type d -regex "[^.]+"<br/>replace "<folder>" with directory to search. It cannot contain or be "." due to the regex used. This will find files with no "." in their name.

			*Find all files with MP3 and make them mp3 (confirm options for "rename" they change from OS to OS)
			*find <folder> -name ".MP3" -exec rename MP3 mp3 "{}" \;

	*Find files that are DRM protected and remove them or the protection		*Find files that are DRM protected and remove them or the protection
	<pre>		<pre>
Line 10:		Line 15:
	fi		fi
	</pre>		</pre>

	*Find files with special or unicode characters and fix		*Find files with special or unicode characters and fix
	<pre>		<pre>
	find . -name "[$(/bin/echo -e '\201')-$(/bin/echo -e '\377')]" \| tee /tmp/filelist.log		find . -name "[$(/bin/echo -e '\201')-$(/bin/echo -e '\377')]" \| tee /tmp/filelist.log
	</pre>		</pre>
			*Examples
	**\302\264 becomes apostrophe		**\302\264 becomes apostrophe
	**\342\200\231		**\342\200\231
Line 20:		Line 27:
	**\222		**\222

			==Tagging==
			*Save "contextual" meta data like folders<br/>
			Folders may represent compilations or playlists which were created artificially due to a lack of support by the library/player/etc. This contextual information should be saved prior to any file renaming and possibly before any tag corrections.
			*Find full and mostly complete albums and verify tags
			*Rename these tracks to new folder structure
			**OST/Various Artists
			***X:\_Stage\pass1\V\Various Artists\%album%\$num(%track%,2) - %artist% - %title%
			**Single Artist Albums
			***X:\_Stage\pass1\$left(%artist%,1)\%artist%\%album%\$num(%track%,2) - %title%
			*Incomplete Albums/"singles"
			***X:\_Stage\pass1\$left(%artist%,1)\%artist%\_Singles\%title%

	*Convert ~~remaining all non-MP3 files~~ to MP3 with same quality and transfer tags		==File Format/codec/bitrate Issues==
			*Convert wma to MP3 with same quality and transfer tags
	**Load all tracks into MP3TAG and sort first by bitrate and then by encoding.		**Load all tracks into MP3TAG and sort first by bitrate and then by encoding.
	**Go to the bottom and select all wma with bitrate set (missing bitrate most likely means a damaged file)		**Go to the bottom and select all wma with bitrate set (missing bitrate most likely means a damaged file)

172.29.1.1: /* Process */

2010-07-20T18:49:56Z

Process

← Older revision		Revision as of 18:49, 20 July 2010
Line 22:		Line 22:

	*Convert remaining all non-MP3 files to MP3 with same quality and transfer tags		*Convert remaining all non-MP3 files to MP3 with same quality and transfer tags
			**Load all tracks into MP3TAG and sort first by bitrate and then by encoding.
			**Go to the bottom and select all wma with bitrate set (missing bitrate most likely means a damaged file)
			**Open Xrecode
			**Drag from MP3TAG to Xrecode all selected tracks
			**Select mp3 output
			**Select encoding rate
			**(optional) Select tracks in MP3TAG by bitrate and choose encoding rate in Xrecode based on source
	*Use MP3Tag to normalize track numbers to 2 digits.<br/>This will help in fixing tags using Jaikoz in a later step		*Use MP3Tag to normalize track numbers to 2 digits.<br/>This will help in fixing tags using Jaikoz in a later step
	*Open Folder in Jaikoz		*Open Folder in Jaikoz
	**sort by track number then subfolder		**sort by track number then subfolder
	**open		**open

	=Tools=		=Tools=
	* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.		* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.
	* [http://xrecode.com/ xrecode$$] is a tool for recoding to different formats that supports moving of tags.		* [http://xrecode.com/ xrecode$$] is a tool for recoding to different formats that supports moving of tags.

DocGyver at 02:21, 21 March 2010

2010-03-21T02:21:26Z

@@ Line 10: / Line 10: @@
 fi
 </pre>
 *Convert remaining all non-MP3 files to MP3 with same quality and transfer tags

24.30.94.116: /* Tools */

2009-07-10T19:25:06Z

Tools

← Older revision		Revision as of 19:25, 10 July 2009
Line 18:		Line 18:
	=Tools=		=Tools=
	* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.		* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.
			* [http://xrecode.com/ xrecode$$] is a tool for recoding to different formats that supports moving of tags.

24.30.94.116 at 19:23, 10 July 2009

2009-07-10T19:23:51Z

← Older revision		Revision as of 19:23, 10 July 2009
Line 1:		Line 1:
	= Process =		=Process=
	*Find all files with incorrect extensions		*Find all files with incorrect extensions
	**find Artist ! -type d -regex "[^.]+"<br/>replace "Artist" with directory to search. It cannot contain or be "." due to the regex used.		**find Artist ! -type d -regex "[^.]+"<br/>replace "Artist" with directory to search. It cannot contain or be "." due to the regex used.
Line 16:		Line 16:
	**sort by track number then subfolder		**sort by track number then subfolder
	**open		**open
	= Tools =		=Tools=
	* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.		* [http://en.wikipedia.org/wiki/Replay_Gain Replay Gain] is a technology for audio level normalization.