A few years ago I took over webmaster duties for a very active community forum. About a month into the job the unthinkable happened, we lost the table that contained all of the posts!

I had backups (7 day rolling system) but the backups were a good 24 hours old and the server kept dropping lines while it was chugging away at restoring the database table.

I needed to go into the sql file and pull out all of the lines for the post table but I had one very large problem; The sql file was 524MB and my computer only had 128MB of RAM!

I tried to cajole my computer into opening the file but it was simply too big. Rather then curse my poor under-powered computer I got an idea.

Since I knew the structure of the sql commands I could write a program to open the file, line-by-line, and take the lines I needed and put them in a new file.

Now, it is important to note that I was unaware of the program Grep at the time. If I had known of Grep back then I would have just used that…

So, using some very simple C programming language functions I built a program that would search any text file for a given search string and output it all to a new file. Even though the post table itself was still huge, the shear number of sub-forums allowed me to break it down into files for each sub-forum.

Now I had about 124 sql files, one for each area of the forums. It was a simple process to load each one up onto the server and load the table data back into the mySQL server. Then I had only to a do a quick check of the number of items in the table compared to the number of lines in the file to be sure all the statements were executed.

I give the following code to you to use, modify, etc… I ask only that you follow my Creative Commons Attribution-ShareAlike 2.5 License.

It would be a very easy modification to change this program to search an entire directory of text files for a given string. I imagine some of those “spy on your kids” program do just that…

Please note that this was thrown together in about 5 minutes and is not optimized nor does it have any safegaurds written into it.

#include <fstream>
#include <iostream>
#include <string.h>
#include <stdio.h>

using namespace std;
const int MAX_LEN = 102400 ;
int main(int argc, char *argv[])

{
    int i = 0;
    if(argc!=3)

    {
        // give info on how to run program
        // Creative Commons Attribution-ShareAlike 2.5 License
        printf("sqlquery copyright 2005 Stephen De Chellis \n");
        printf("sqlquery takes 2 arguments\n\n");

        printf("sqlquery [filename] [search string]\n\n");
        printf("[filename]      = name of file to open for searching\n");
        printf("[search string] = string to search the file for\n\n");

        printf("sqlquery will search the file line by line and store the results\n");
        printf("in a new file named [search string].sql\n");
        return 0;

    }
    char line[MAX_LEN];
    char filename[256];

    char filename2[256];
    i = 0;
    strcpy(filename2,argv[2]);

    strcat(filename2,".sql");
    strcpy(filename,argv[1]);    

    ifstream infile;

    ofstream outfile;
    infile.open (filename, ifstream::in);

    if(!infile)
    {
        printf("unable to find file: %s",filename);

        return 0;
    }
    outfile.open (filename2, ofstream::out);

    if(!outfile)
    {
        printf("unable to create file: %s",filename2);

        return 0;
    }
    printf("\n working");
    do

    {
        infile.getline(line,MAX_LEN);

        if(strstr(line,argv[2])!=NULL)

        {
            outfile << line << endl;
            i++;

            printf(".");
            strcpy(line,"");
        }

    }while(infile.eof()==0);
    printf("Found %i matches",i);

    infile.close();
    outfile.close();
    return i;

}