If you’ve used Perl at all you are probably familiar with the simple oneliner to do a search and replace on a given string:
perl -p -i -e 's/oldstring/newstring/g' *
This will replace all occurrences of oldstring with newstring in all of the files in your current directory. Being able to do this quickly and easily is absolutely awesome, but what if you want to do this recursively across the current directory and all directories below that? I’m sure that there are plenty of ways to do this, the one below is the method that seems the easiest:
perl -p -i -e 's/oldstring/newstring/g' `find ./ -name *.html`
If you wanted to be more precise you could replace find with grep and only perform the search and replace on files that you know already contain oldstring, like this:
perl -p -i -e 's/oldstring/newstring/g' `grep -ril oldstring *`
This is one of those things that I don’t have to do very often, but when I do, this Perl oneliner is a real life saver.
No related posts.
wouldn’t it be better to pipe the “find” output to xargs and pass this to the perl expression?
Couple problems. First, you have to quote ‘*.html’ or the shell will expand it and find will barf. Also, if there are spaces in any of the file names that find returns, perl will barf. So, as Klaus suggests, you should use
find . -name '*.html' -print0 | xargs -0 perl -pi -e 's/oldstring/newstring/g'Klaus is right because the find clause might produce enough output that it will reach the limit for line length (input too long). Remarkably, this happens frequently enough. I remember having to debug somebody’s perl script that was written to search through 750gb of text files. One day, the number just managed to hit the max and it errored out with “input too long” messages.
find . | xargs perl -p -i -e ‘s/something/else/g’
However, for smaller directories, I really like your second solution, grepping good matches out first. You should run some time trials to see if it’s a speed win to have grep do the detection and then only perl the ones you need. I bet there’s a break-even where less than some percentage of relevant files will give you a speed advantage.
-Josh-
Pingback: Michael’s Tech Blog » Blog Archive » Perl Oneliner: Recursive Search and Replace
Just one more way to do:
find ./ -exec perl -p -i -e ‘s/faltoo_string/aur_bhi_faltoo_string/g’ {} \;
Pingback: PrestonLee.com
Hey guys, be careful with your symlinks! I used Nics method from post number 2, and it converted my symlink files to real files. That ended up breaking some stuff for me.
Using grep to select the files is a performance killer : It means that the files matching the regexp will be scanned twice : Once by grep and once by perl. If the file is big, it’s a problem.
Not selecting the files that will be changed is a problem: Files that do not match will change (ie time, inode). Not good for keeping timestamps of unchanged files (think for example source files: make will rebuild the objects!)
/me is wondering if perl has an option “do not replace the file by a new one if it did not change, just keep it”.
I tried all these examples to no avail using Solaris9, just trying to find all files in filesystem with *.zip and change it to *.ZIP.
# perl -v
This is perl, v5.6.1 built for sun4-solaris-64int
(with 48 registered patches, see perl -V for more detail)
# find /usr/local/tmp/aaa -name “*.zip” -exec perl -p -i -e ‘s/zip/ZIP/g’ {} \;
# ls -l /usr/local/tmp/aaa
total 6
-rw-r–r– 1 carlos staff 49 Feb 22 09:35 9_Recommended.zip
-rw-r–r– 1 carlos staff 49 Feb 22 09:35 DLG_V11_2.zip
-rw-r–r– 1 carlos staff 49 Feb 22 09:35 tzupdater-1_0_1.zip
#
Single quotes doesn’t work too
# find /usr/local/tmp/aaa -name ‘*.zip’ -exec perl -p -i -e ‘s/zip/ZIP/g’ {} \;
# ls -l /usr/local/tmp/aaa
total 6
-rw-r–r– 1 carlos staff 49 Feb 22 09:37 9_Recommended.zip
-rw-r–r– 1 carlos staff 49 Feb 22 09:37 DLG_V11_2.zip
-rw-r–r– 1 carlos staff 49 Feb 22 09:37 tzupdater-1_0_1.zip
Carlos, perl -pi -e replaces the CONTENTS of the file. You are trying to RENAME the file. Two very different things. See if you have the rename command; it is a perl script and will let you rename a file using regex substitution.
Hey There,
For “not replacing” files that already exist, I use this with that perl one-liner when I’m piping filenames to it – it’s not built into the options, but is pretty simple to type once
perl -p -i -e ‘s/ThisThing/ThatThing/; rename $_, unless -e $_;’
It doesn’t work “out of context” because $_ requires a value, but you can use the rename half of my version of the one-liner to insert in the previous code to prevent against overwriting
Great post
, Mike
hello all
I execute this:
# perl -p -w -e ‘s/(?:\n)(.*)//g’ test1.txt
And get this:
Bareword found where operator expected at -e line 1, near “s/(?:\n)(.*)</collectioninfo”
Unquoted string “collectioninfo” may clash with future reserved word at -e line 1.
syntax error at -e line 1, near “s/(?:\n)(.*)</collectioninfo”
Execution of -e aborted due to compilation errors.
Any help ?
Thx !!
Hey There,
That error almost always has to do with unmatched quotation marks. Probably, the easiest way to figure out what’s wrong with it would be to turn your one-liner into a simple script. You’ll get waaaaaay too much information, but the answer will probably be in there.
If not, check test1.txt for any unmatched double quotes, single quotes, backticks, etc and that may lead you to the answer, as well.
Best wishes,
Mike
I should note, that, with the script, you’ll at least get a line number that probably won’t be 1
Thanks a bunch! This saves me a lot of time on a daily basis as I’ve started using this in a script. Just thought I’d come let you know it helped. Although you need to add “quotes” around your find target for this to work properly.
The second oneliner worked like a charm, just what I was looking for. Thanks!!
I love perl, but for these use-cases sed would have been more than sufficient?
sed -i ‘s/oldstring/newstring/g’ *
Same w/ the other examples …
Cheers,
Andrej
sed, as shipped with many OS installs, doesn’t work with escaped characters, so it can’t be used for many cases.