Argument List too Long

[Kevin Krieser]

On Jun 1, 2010, at 6:06 PM, Sanjay Chakraborty wrote:

> xargs man page is helpful -- thank you I was missing this one. However
> patch may help and that I will be doing this weekend. Let me see that
> sort out the issue.
> 
> On Tue, Jun 1, 2010 at 12:17 PM, cliff here <c4ifford at gmail.com> wrote:
>> Look in the man pages for Xargs, there are quite a few examples in there on
>> how to move and delete file when the 'Argument list is too long'
>> 
>> On Mon, May 31, 2010 at 2:26 PM, Sanjay Chakraborty <sanjaychakrab at gmail.com
>>> wrote:
>> 
>>> I have a script and that run every month, In the script it has a mv
>>> command and that moves about 35000 files from one directory to other
>>> directory. In one system it is working but in other system I am
>>> getting "Argument List too Long" error message.
>>> 
>>> mv* ../$directory2/ cannot work.
>>> 
>>> 
>>> I can try with "find $directory -type f -name '*' -exec mv {}
>>> $directory2/. \;"  but I am not sure it will work and it will take
>>> time to do the testing through script modification.
>>> 
>>>  It is running RHEL 5.1, patching may solve the problem. But I cannot
>>> patch this server soon.
>>> 
>>> Any one can help about this ?
>>> 
>>> --
>>> Regards.
>>> Sanjay Chakraborty
>>> 
>>> --
>>> redhat-list mailing list
>>> unsubscribe mailto:redhat-list-request at redhat.com?subject=unsubscribe
>>> https://www.redhat.com/mailman/listinfo/redhat-list
>>> 
>> 

find is probably the appropriate tool.  Use -exec command {} ... \; or feed it to xargs, depending on what you want to do.

Reading the man pages for xargs, there is an option to place the arguments read from standard input into a specified location in the command, rather than at the end, which would be needed in your mv example.  It is probably best too to use the -print0 of find, and -0 option of xargs, to handle spaces within filenames.  This was suggested to me the last time this issue came up on a list I follow.

Another issue with find is if you want to descend into the subdirectories.  your mv * ../directory2 
is not the same as find . -exec mv {} ../directory2 \;
since the find command would descend into subdirectories, and move the individual files.  But if there are directories, your mv would have moved the directory itself to the new directory, and it's containing files.  Also, as a non root user, mv * would not move files starting with ., such as .profile.  So maybe:
find . -name '*' -maxdepth 1 -exec mv {} ../directory2 \;
or use the -print0 option, and the appropriate options to xargs like -0 and -I.

The xargs version is more efficient, but the -exec one should work too.