Asked  6 Months ago    Answers:  5   Viewed   35 times

Using awk or sed how can I select lines which are occurring between two different marker patterns? There may be multiple sections marked with these patterns.

For example: Suppose the file contains:

abc
def1
ghi1
jkl1
mno
abc
def2
ghi2
jkl2
mno
pqr
stu

And the starting pattern is abc and ending pattern is mno So, I need the output as:

def1
ghi1
jkl1
def2
ghi2
jkl2

I am using sed to match the pattern once:

sed -e '1,/abc/d' -e '/mno/,$d' <FILE>

Is there any way in sed or awk to do it repeatedly until the end of file?

 Answers

53

Use awk with a flag to trigger the print when necessary:

$ awk '/abc/{flag=1;next}/mno/{flag=0}flag' file
def1
ghi1
jkl1
def2
ghi2
jkl2

How does this work?

  • /abc/ matches lines having this text, as well as /mno/ does.
  • /abc/{flag=1;next} sets the flag when the text abc is found. Then, it skips the line.
  • /mno/{flag=0} unsets the flag when the text mno is found.
  • The final flag is a pattern with the default action, which is to print $0: if flag is equal 1 the line is printed.

For a more detailed description and examples, together with cases when the patterns are either shown or not, see How to select lines between two patterns?.

Tuesday, June 1, 2021
 
Hilmi
answered 6 Months ago
62

Print lines between PAT1 and PAT2

$ awk '/PAT1/,/PAT2/' file
PAT1
3    - first block
4
PAT2
PAT1
7    - second block
PAT2
PAT1
10    - third block

Or, using variables:

awk '/PAT1/{flag=1} flag; /PAT2/{flag=0}' file

How does this work?

  • /PAT1/ matches lines having this text, as well as /PAT2/ does.
  • /PAT1/{flag=1} sets the flag when the text PAT1 is found in a line.
  • /PAT2/{flag=0} unsets the flag when the text PAT2 is found in a line.
  • flag is a pattern with the default action, which is to print $0: if flag is equal 1 the line is printed. This way, it will print all those lines occurring from the time PAT1 occurs and up to the next PAT2 is seen. This will also print the lines from the last match of PAT1 up to the end of the file.

Print lines between PAT1 and PAT2 - not including PAT1 and PAT2

$ awk '/PAT1/{flag=1; next} /PAT2/{flag=0} flag' file
3    - first block
4
7    - second block
10    - third block

This uses next to skip the line that contains PAT1 in order to avoid this being printed.

This call to next can be dropped by reshuffling the blocks: awk '/PAT2/{flag=0} flag; /PAT1/{flag=1}' file.

Print lines between PAT1 and PAT2 - including PAT1

$ awk '/PAT1/{flag=1} /PAT2/{flag=0} flag' file
PAT1
3    - first block
4
PAT1
7    - second block
PAT1
10    - third block

By placing flag at the very end, it triggers the action that was set on either PAT1 or PAT2: to print on PAT1, not to print on PAT2.

Print lines between PAT1 and PAT2 - including PAT2

$ awk 'flag; /PAT1/{flag=1} /PAT2/{flag=0}' file
3    - first block
4
PAT2
7    - second block
PAT2
10    - third block

By placing flag at the very beginning, it triggers the action that was set previously and hence print the closing pattern but not the starting one.

Print lines between PAT1 and PAT2 - excluding lines from the last PAT1 to the end of file if no other PAT2 occurs

This is based on a solution by Ed Morton.

awk 'flag{
        if (/PAT2/)
           {printf "%s", buf; flag=0; buf=""}
        else
            buf = buf $0 ORS
     }
     /PAT1/ {flag=1}' file

As a one-liner:

$ awk 'flag{ if (/PAT2/){printf "%s", buf; flag=0; buf=""} else buf = buf $0 ORS}; /PAT1/{flag=1}' file
3    - first block
4
7    - second block

# note the lack of third block, since no other PAT2 happens after it

This keeps all the selected lines in a buffer that gets populated from the moment PAT1 is found. Then, it keeps being filled with the following lines until PAT2 is found. In that point, it prints the stored content and empties the buffer.

Tuesday, June 1, 2021
 
barden
answered 6 Months ago
22

I'll have a go at this.

To delete 5 lines after a pattern (including the line with the pattern):

sed -e '/pattern/,+5d' file.txt

To delete 5 lines after a pattern (excluding the line with the pattern):

sed -e '/pattern/{n;N;N;N;N;d}' file.txt
Wednesday, June 2, 2021
 
commonpike
answered 6 Months ago
67

You can do something like this:

 mDb.query(MY_TABLE, null, DATE_COL + " BETWEEN ? AND ?", new String[] {
                minDate + " 00:00:00", maxDate + " 23:59:59" }, null, null, null, null);

minDate and maxDate, which are date strings, make up your range. This query will get all rows from MY_TABLE which fall between this range.

Thursday, August 5, 2021
 
Jens
answered 4 Months ago
14

Here's a awk script that works (I put it on multiple lines and made it a bit more verbose so you can see what's going on):

#!/bin/bash

awk 'BEGIN{FS="[():,]"}
/client_id/ {
cid="no_client_id"
for (i=1; i<NF; i++) {
    if ($i == "client_id") {
        cid = $(i+1)
    } else if ($i == "id") {
        id = $(i+1);
        print cid OFS id;
    }
 }
}' input_file_name

Output:

03 204
03 491
03 29
04 209
04 301
05 20

Explanation:

  • awk 'BEGIN{FS="[():,]"}: invoke awk, use ( ) : and , as delimiters to separate your fields
  • /client_id/ {: Only do the following for the lines that contain client_id:
  • for (i=1; i<NF; i++) {: iterate through the fields on each line one field at a time
  • if ($i == "client_id") { cid = $(i+1) }: if the field we are currently on is client_id, then its value is the next field in order.
  • else if ($i == "id") { id = $(i+1); print cid OFS id;}: otherwise if the field we are currently on is id, then print the client_id : id pair onto stdout
  • input_file_name: supply the name of your input file as first argument to the awk script.
Wednesday, September 22, 2021
 
Elijah W. Gagne
answered 2 Months ago
Only authorized users can answer the question. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :
 
Share