Awk is a very useful tool for extracting data from text files, all you have to do is tell it which columns you want to extract and you're usually done. However, what if you didn't want to hardcode the column number and needed the ability to define it dynamically? That gets a little be more complicated but luckily there's a way to solve that requirement with bash and awk.

To demonstrate the requirement, lets say we had a data file with 5 columns like so:


To start lets say you knew you wanted to extract the first and third column. Then your awk command would look something like this:




That is very straight forward and works, but lets say now that for various reasons you didn't know which columns you needed to extract ahead of time (or wanted to make your script more flexible). The obvious thing to do here is to define which columns you were interested in as a script variable like so...


How do you use these with awk though? For example the following: `awk '{printf "%s\t%s\n", $$MY_COL1, $$MY_COL2}' < data_file.txt` will fail with this error:


The answer is to use awk's -v parameter to assign variables in awk to tell it which column numbers we're interested in. So assuming the above variables have been assigned with the column numbers we want, the following script will do the trick:


Note how the $1 and $3 in the original awk program are now $c1 and $c2 respectively and both of c1 and c2 variables are assigned values from the shell script variables using the -v parameter.

So now you can redefine which columns are extracted by changing the shell script variable while leaving the awk command alone.

-i