注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

notes

@-@

 
 
 

日志

 
 
 
 

大批量移动文件  

2009-10-30 17:30:56|  分类: UNIX |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |

I created two directories, "source" and "target" and created 20,000 empty text files like so:

mkdir source
mkdir target
for i in `seq 1 20000`; do touch source/$i.txt; done

Doing any of these:

mv source/* target/
cp source/* target/
rm source/*
ls source/*

will result in errors like this:

-bash: /bin/mv: Argument list too long
-bash: /bin/cp: Argument list too long
-bash: /bin/rm: Argument list too long
-bash: /bin/ls: Argument list too long

To workaround the problem you can use xargs or find, or a combination of the two.

Using the "ls" command, you could do this to move all the files from source into target:

ls -1 source | xargs -i mv source/{} target/

The -i flag in xargs tells it to use string replacements and {} is the placeholder. You cannot use a pattern for ls such as "ls *.txt" because you'll get the arugment list too long error before the data can be piped through to xargs. In order to do pattern matching you have to use the find command.

Using find to do the same move from source to target, you could do this:

find source/ -name "*.txt" -exec mv {} target \;

or this, using xargs instead of the -exec flag:

find source/ -name "*.txt" | xargs -i mv {} target/

In the "find" examples above you could omit the de>-name *.txtde> part to move everything, and/or add further pattern matches such as de>-type fde> to only match files.

All three examples above take roughly the same amount of time to run although I found in testing the find ... -exec method to be slightly faster than find ... xargs

The find method is better than the ls method because it allows you to specify filename patterns and can give you a lot more control over what you are copying, moving, deleting etc.

As a final set of examples, you could do the following to then delete the files from the target directory:

ls -1 target | xargs -i rm target/{}

or

find target/ -name "*.txt" -exec rm {} \;

or

find target/ -name "*.txt" | xargs -i rm {}

Having to delete/move/etc files in this way is trickier and more time consuming than being able to simply do something like rm -f source/* but sometimes this is the only way to do it if there are too many files matching your pattern.

  评论这张
 
阅读(1039)| 评论(8)
推荐 转载

历史上的今天

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2017