Batch script to parse a text file

[dsd17]

We have a backup software that creates log files every time a backup is run. We want to email a portion of that log file to each customer so that they can see if their backup was successful or failed each day. I found a program to email out the text files, but now I need something that will pull the data out of the log file. We don't want our clients seeing 10k lines of items that are backed up. They just need to see the 5 lines that matter to them. Does anyone know how to write this or have a program they use to parse information?

This is an example of what the text portion of the log files would look like that we want to extract:

[11-Jun-2007:10:58:01] BACKUP SUMMARY:
[11-Jun-2007:10:58:01] Total no. of volumes processed: 1
[11-Jun-2007:10:58:01] Total no. of files processed: 2
[11-Jun-2007:10:58:01] Total no. of files skipped: 0
[11-Jun-2007:10:58:01] Total size of compressed files: 0.0001MB (0.08KB)
[11-Jun-2007:10:58:01] Total size of uncompressed files: 1.7510MB (1793.00KB)
[11-Jun-2007:10:58:01] Total upload time: 00:00:01


Above all of this is each file that was backed up. Once it is extracted, we want to put it in an email and send it out.

 

[nweaver]

if you are parsing text, then perl > *

and you can parse the text, send the email, flush the toilet, and pet the dog...all in under 3 lines if you are good

 

[dsd17]

Guess I need to learn Perl...

The program we're using to check for new files and email them out needs to read a batch file in order to parse out the data. That's why I was hoping this could be accomplished with a batch file.

 

[jamesave]

perl. use the grep function (require regular expressions)

 

[Descartes]

It's very easy with regular expressions. You could use any of the common regex tools, but Perl is probably the best choice since it has very complete support of all regex features. I came up with the following expression very quickly:

(\[.*?\] BACKUP SUMMARY.*Total upload time: \d{2}:\d{2}:\d{2}

I say quickly, because it's not optimized. It just matches what you say you want. You could throw that into a Perl script like what I have attached to this post. It's quick, dirty, but it works. That might get you started, or perhaps it confuses you more.

[edit]The code attachment doesn't like me for some reason. I don't want to mess with it, so I'll simply quote it here:

# A little Perl "slurp" action
undef $/;
# read in everything from stdin
# WARNING: Careful with large files
$_ = <>;

# Dirty regex to extrapolate what you said you needed
/(\[.*?\] BACKUP SUMMARY.*Total upload time: \d{2}:\d{2}:\d{2})/s;

# Now we'll just print it. You might need to add
# line feeds, etc. to fit your purposes
print $1;

[/edit]

 

[GoatMonkey]

How do you know which section of that log file is for each customer? There doesn't seem to be anything unique in that log that I see.

Also, you can do regular expressions in pretty much any major language out there today. What languages do you already know?

 

[Nothinman]

While I love perl you should probably look at VBScript if you're stuck on Windows since it's already installed on all of your Windows boxes.

 

[engineereeyore]

Lex/Flex is your friend. Lex/Flex is a freeware program used in compilers to parse files looking for specific tokens or regular expressions. Pretty easy to use and free. My parser of choice, but then I've very familiar with it. Perl is a good choice as well.

 

[dsd17]

Originally posted by: GoatMonkey
How do you know which section of that log file is for each customer? There doesn't seem to be anything unique in that log that I see.

Also, you can do regular expressions in pretty much any major language out there today. What languages do you already know?

There are specific folders for each customer, but each log file is structured the same way. I only pasted the portion that I need stripped. The rest of it is all the files that were backed up.

As far as programming languages I'm more towards web design. HTML, PHP, stuff like that. I haven't done anything with Perl or VB.

Originally posted by: Nothinman
While I love perl you should probably look at VBScript if you're stuck on Windows since it's already installed on all of your Windows boxes.

yes, this program runs on Windows... if it were for linux or unix I could use tail and be done with it...

 

[nweaver]

Originally posted by: dsd17

Originally posted by: GoatMonkey
How do you know which section of that log file is for each customer? There doesn't seem to be anything unique in that log that I see.

Also, you can do regular expressions in pretty much any major language out there today. What languages do you already know?

There are specific folders for each customer, but each log file is structured the same way. I only pasted the portion that I need stripped. The rest of it is all the files that were backed up.

As far as programming languages I'm more towards web design. HTML, PHP, stuff like that. I haven't done anything with Perl or VB.

Originally posted by: Nothinman
While I love perl you should probably look at VBScript if you're stuck on Windows since it's already installed on all of your Windows boxes.

yes, this program runs on Windows... if it were for linux or unix I could use tail and be done with it...

Windows 2000 or 2003 server resource kits have a "tail" utility (not named tail, but I renamed it to tail).

 

[Daverino]

Another vote for Perl. This could probably be done in only a few lines. If you're on Windows, though, you'd be looking at ActiveState.

 

[SoulAssassin]

In a completely different direction what about a simple post backup script? What software are you running? In Netbackup (it doesn't appear you're running NBU) it runs a post backup script which is passed several variables including the outcome of the backup. If what you're running will do the same, with a very simple if statement you can probably accomplish what you want.

 

[Ken g6]

That doesn't look very "grep"-able. Is there some line designating that this backup came from customer X?

If you just want the lines last written to file F, try the *nix shell script commands:

tail -6 F | head -5

You could do it in Perl, too, but the above would be faster. (This is generally true, where "it" is anything, and "the above" is some combination of C programs.)