Grab a section from a large file between two known substrings using sed?

I have a pretty large XML file without line-breaks.

It's so big it's slow to open and operate on in Emacs or other text editor. But I just want to extract a shortish section of it between two known substrings.

I don't care about preserving the XML structure, I just want a chunk of characters.

This ought to be a one-liner in sed, no?

Any idea how to do this? I tried adapting but it doesn't seem to work when I pipe my file into it. (It works on toy examples, but I'm thinking that my file may be too big.)

3 Answers

With GNU grep:

With frompattern and topattern in output:

grep -o 'frompattern.*topattern' file.xml 

Without frompattern and topattern in output:

grep -Po 'frompattern\K.*(?=topattern)' file.xml 

Well, usually it's easy to do with sed. But it's ALWAYS easy to do it with awk:

awk '/frompattern/,/topattern/' your.xml > chunk.xml 

Here the two patterns are regular expressions (as would be with sed). If it discourages you for any reason, you can use simple strings, if you know where they are:

awk '$x=="fromstring",$y=="tostring"' your.xml > chunk.xml 

Here x and y are the field positions of the strings you want to be the barrier signs. (More can be done with tiny effort.)

I use a command called pv (Pipe viewer) - once installed you can see the progress of the command and how long to complete. great for large files

For Mac - (for others go to ostechnix.com)

brew install pv 

my example is using sed.

 pv bigFile.txt | sed -n '/^FIRST PART OF STRING/,/^LAST PART OF STRING `/p' > output.txt 

If you dont use pv you can just use anything else to echo the file

eg. cat or echo