{"id":66161,"date":"2022-08-02T09:01:00","date_gmt":"2022-08-02T09:01:00","guid":{"rendered":"https:\/\/www.cryptocabaret.com\/?p=66161"},"modified":"2022-08-02T09:01:00","modified_gmt":"2022-08-02T09:01:00","slug":"how-i-use-the-linux-sed-command-to-automate-file-edits","status":"publish","type":"post","link":"https:\/\/www.cryptocabaret.com\/?p=66161","title":{"rendered":"How I use the Linux sed command to automate file edits"},"content":{"rendered":"

How I use the Linux sed command to automate file edits<\/span>
\nJim Hall<\/a><\/span>
\nTue, 08\/02\/2022 – 03:00<\/span><\/p>\n

\n
Register<\/a> or Login<\/a> to like<\/div>\n
Register<\/a> or Login<\/a> to like<\/div>\n<\/div>\n
\n

When I use the Linux command line, whether I’m writing a new program on my desktop computer or managing a website on my web server, I often need to process text files. Linux provides powerful tools that I leverage to get my work done. I frequently use sed<\/code>, an editor that can modify text according to a pattern.<\/p>\n

sed<\/code> stands for stream editor<\/em>, and it edits text in a file and prints the results. One way to use sed<\/code> is to identify several occurrences of one string in a file and replace them with a different string. You can use sed<\/code> to process text files to a seemingly endless degree, but I’d like to share a few ways I use sed<\/code> to help me manage files.<\/p>\n

Search and replace text in a file on Linux<\/h2>\n

To use sed<\/code>, you need to use a regular expression<\/em>. A regular expression is a set of special characters that define a pattern. My most frequent example of using sed<\/code> is replacing text in a file. The syntax for replacing text looks like this: s\/originaltext\/newtext\/<\/code>. The s<\/code> tells sed to perform text replacement or swap occurrences of text. Provide the original text and new text between slashes.<\/p>\n

This syntax will only replace the first occurrence of originaltext<\/code> on each line. To replace every occurrence, even if the original text appears more than once on a line, append g<\/code> to the end of the expression. Here is an example: s\/originaltext\/newtext\/g<\/code>.<\/p>\n

To use this with sed<\/code>, specify this regular expression with the -e<\/code> option:<\/p>\n

\n$ <\/span>sed<\/span> -e<\/span> 's\/originaltext\/newtext\/g'<\/span><\/code><\/span><\/pre>\n
For example, let’s say I have a Makefile for a program called game<\/strong>, which simulates Conway’s Game of Life:<\/p>\n
\n
.PHONY: all run clean

\nall: game

\ngame: game.o
\n\u00a0 \u00a0 \u00a0 \u00a0 $(CC) $(CFLAGS) -o game game.o $(LDFLAGS)

\nrun: game
\n\u00a0 \u00a0 \u00a0 \u00a0 .\/game

\nclean:
\n\u00a0 \u00a0 \u00a0 \u00a0 $(RM) *~
\n\u00a0 \u00a0 \u00a0 \u00a0 $(RM) *.o
\n\u00a0 \u00a0 \u00a0 \u00a0 $(RM) game<\/div><\/div><\/pre>\n
The name game<\/strong> isn’t very descriptive, so I might choose to rename it life<\/strong>. Renaming the game.c<\/code> source file to life.c<\/code> is easy enough, but now I need to modify the Makefile to use the new name. I can use sed<\/code> to change every occurrence of game<\/strong> to life<\/strong>:<\/p>\n
\n
$ sed<\/span> -e<\/span> 's\/game\/life\/g'<\/span> Makefile
\n.PHONY: all run clean

\nall: life

\nlife: life.o
\n\u00a0 \u00a0 \u00a0 \u00a0 $(<\/span>CC)<\/span> $(<\/span>CFLAGS)<\/span> -o<\/span> life life.o $(<\/span>LDFLAGS)<\/span>

\nrun: life
\n\u00a0 \u00a0 \u00a0 \u00a0 .\/<\/span>life

\nclean:
\n\u00a0 \u00a0 \u00a0 \u00a0 $(<\/span>RM)<\/span> *<\/span>~
\n\u00a0 \u00a0 \u00a0 \u00a0 $(<\/span>RM)<\/span> *<\/span>.o
\n\u00a0 \u00a0 \u00a0 \u00a0 $(<\/span>RM)<\/span> life<\/div><\/div><\/pre>\n
This prints the sed<\/code> output to the screen, which is a good way to check if the text replacement will do what you want. To make these changes to the Makefile, first, make a backup of the file, then run sed<\/code> and save the output to the original filename:<\/p>\n
\n
$ cp<\/span> Makefile Makefile.old
\n$ sed<\/span> -e<\/span> 's\/game\/life\/g'<\/span> Makefile.old ><\/span> Makefile<\/div><\/div><\/pre>\n
If you are confident that your changes are exactly what you want, use the -i<\/code> or --in-place<\/code> option to edit the file in place. However, I recommend adding a backup filename suffix like --in-place=.old<\/code> to save a copy of the original file in case you need to restore it later. It looks like this:<\/p>\n
\n
$ sed<\/span> --in-place<\/span>=.old -e<\/span> 's\/game\/life\/g'<\/span> Makefile
\n$ ls<\/span> Makefile*<\/span>
\nMakefile \u00a0Makefile.old<\/div><\/div><\/pre>\n<\/p>\n
\n
More Linux resources<\/div>\n
\n
Linux commands cheat sheet<\/a><\/div>\n
Advanced Linux commands cheat sheet<\/a><\/div>\n
Free online course: RHEL technical overview<\/a><\/div>\n
Linux networking cheat sheet<\/a><\/div>\n
SELinux cheat sheet<\/a><\/div>\n
Linux common commands cheat sheet<\/a><\/div>\n
What are Linux containers?<\/a><\/div>\n
Our latest Linux articles<\/a><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
Quoting files with sed on Linux<\/h2>\n
You can use other features of regular expressions to match specific instances of text. For example, you might need to replace text that occurs at the start of a line. With sed<\/code>, you can match the beginning of a line with ^<\/strong>, the caret character.<\/p>\n
One way I use “start of line” in replacing text is when I need to quote a file in an email. Let’s say I want to share my Makefile in an email, but I don’t want to include it as a file attachment. Instead, I prefer to “quote” the file in the body of an email, using ><\/strong> before each line. I can use the following sed<\/code> command to print out an edited version to my terminal, which I can copy and paste into a new email:<\/p>\n
\n
$ sed<\/span> -e<\/span> 's\/^\/>\/'<\/span> Makefile
><\/span>.PHONY: all run clean
><\/span>
><\/span>all: life
><\/span>
><\/span>life: life.o
><\/span> \u00a0 \u00a0 \u00a0 $(<\/span>CC)<\/span> $(<\/span>CFLAGS)<\/span> -o<\/span> life life.o $(<\/span>LDFLAGS)<\/span>
><\/span>
><\/span>run: life
><\/span> \u00a0 \u00a0 \u00a0 .\/<\/span>life
><\/span>
><\/span>clean:
><\/span> \u00a0 \u00a0 \u00a0 $(<\/span>RM)<\/span> *<\/span>~
><\/span> \u00a0 \u00a0 \u00a0 $(<\/span>RM)<\/span> *<\/span>.o
><\/span> \u00a0 \u00a0 \u00a0 $(<\/span>RM)<\/span> life<\/div><\/div><\/pre>\n
The s\/^\/>\/<\/code> regular expression matches the start of each line (^<\/strong>) and places a ><\/strong> there. Effectively, this starts each line with the ><\/strong> symbol.<\/p>\n
The tabs might not show up correctly in an email, but I can replace all tabs in the Makefile with a few spaces by adding another regular expression:<\/p>\n
\n
$ sed<\/span> -e<\/span> 's\/^\/>\/'<\/span> -e<\/span> 's\/t\/ \u00a0\/g'<\/span> Makefile
><\/span>.PHONY: all run clean
><\/span>
><\/span>all: life
><\/span>
><\/span>life: life.o
><\/span> \u00a0$(<\/span>CC)<\/span> $(<\/span>CFLAGS)<\/span> -o<\/span> life life.o $(<\/span>LDFLAGS)<\/span>
><\/span>
><\/span>run: life
><\/span> \u00a0.\/<\/span>life
><\/span>
><\/span>clean:
><\/span> \u00a0$(<\/span>RM)<\/span> *<\/span>~
><\/span> \u00a0$(<\/span>RM)<\/span> *<\/span>.o
><\/span> \u00a0$(<\/span>RM)<\/span> life<\/div><\/div><\/pre>\n
The t<\/code> indicates a literal tab, so s\/t\/ \/g <\/code>tells sed to replace all tabs in the input with two spaces in the output.<\/p>\n
If you need to apply lots of edits to files, you can save your -e<\/code> commands in a file and use -f<\/code> to tell sed<\/code> to use that file as a “script.” This approach is especially useful if you need to make the same edits frequently. I might have prepared the Makefile for quoting in email using a script file called quotemail.sed<\/code>:<\/p>\n
\n
$ cat<\/span> quotemail.sed
\ns\/<\/span>^\/>\/<\/span>
\ns\/<\/span>t\/<\/span> \u00a0\/<\/span>g
\n$ sed<\/span> -f<\/span> quotemail.sed Makefile
><\/span>.PHONY: all run clean
><\/span>
><\/span>all: life
><\/span>
><\/span>life: life.o
><\/span> \u00a0$(<\/span>CC)<\/span> $(<\/span>CFLAGS)<\/span> -o<\/span> life life.o $(<\/span>LDFLAGS)<\/span>
><\/span>
><\/span>run: life
><\/span> \u00a0.\/<\/span>life
><\/span>
><\/span>clean:
><\/span> \u00a0$(<\/span>RM)<\/span> *<\/span>~
><\/span> \u00a0$(<\/span>RM)<\/span> *<\/span>.o
><\/span> \u00a0$(<\/span>RM)<\/span> life<\/div><\/div><\/pre>\n
Learn to work with sed on Linux<\/h2>\n
sed<\/code> is a great tool to keep in your Linux command-line toolkit. Explore the sed<\/code> manual page and learn more about how to use it. Type man sed<\/code> at the command line to get complete documentation about the different command line options and how to use sed<\/code> to process text files.<\/p>\n<\/div>\n
\n
Here are some tips and tricks to automating file edits from the Linux command line.<\/p>\n<\/div>\n
\n
\n
  \"computer<\/div>\n
Image by: <\/span><\/p>\n
Opensource.com<\/p>\n<\/div>\n<\/article>\n<\/div>\n
\n
Linux<\/a><\/div>\n<\/p><\/div>\n
What to read next<\/div>\n

\n        \"Creative<\/a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.<\/div>\n
\n
\n
Register<\/a> or Login<\/a> to post a comment.<\/div>\n<\/p><\/div>\n<\/section>\n
Powered by WPeMatico<\/a><\/small><\/p>\n","protected":false},"excerpt":{"rendered":"
How I use the Linux sed command to automate file edits Jim Hall Tue, 08\/02\/2022 – 03:00 Register or Login to like Register or Login to like When I use the Linux command line, whether I’m writing a new program on my desktop computer or managing a website on my web server, I often need […]<\/p>\n","protected":false},"author":1,"featured_media":66162,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[307],"tags":[],"class_list":["post-66161","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-open-source"],"_links":{"self":[{"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/posts\/66161","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=66161"}],"version-history":[{"count":0,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/posts\/66161\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/media\/66162"}],"wp:attachment":[{"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=66161"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=66161"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=66161"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}