Use the Past to Conquer the Future - A How-To on BASH History Substitution

Use The Past To Conquer The Future - History Substitution

Since I had to get really close with bash recently and found myself typing the same things over and over again, I decided to open the amazing toolbox called “History Substitution” and was amazed by how much you can do with it. It’s a feature built into bash that allows you to get commands and arguments from your history.
In this How-To I hope to be able to share this amazement with you! So let’s go!


1. Introduction

A long time ago, the Grandmaster who taught me Linux brought a small detail to my attention.
It went a little something like this:

Me: apt update
$ apt update: permission denied
Me: sudo apt upda …
Him: wait! I shall teach you …
$ sudo !!
sudo apt update

Suffice to say, my mind was blown!

After learning this, I used it as often as I could, not realizing that this rabbit hole went even deeper than I imagined.


2. Down the hole we go

In our daily life with bash we often come across situations like this:

less /this/path/is/really/long/and/annoying/randomfile

Now if we’re done looking at the file and want to open it in vi we have to:

press the up arrow -> pos1 -> del/del/del/del -> vi

which is ok but still a pain in the butt if we have to do it more than once in our entire lifetime.
Instead with history subsitution we can just do:

vi !!$

So how does this work?
As you may have noticed by now, to start a history substitution command you begin with two exclamation points.
Given only two consecutive exclamation points returns the whole last command.
If we use history substitution with other options we can always omit the second exclamation point.

When used with a dollar sign it returns only the last argument.
The other way around, an exclamation point followed by a circumflex (this thing -> ^ ) takes only the first argument.
Let’s look at some examples with “echo a b c” as the last command used:

!! -> echo a b c
echo !! -> echo echo a b c
echo !$ -> echo c
echo !^ -> echo a
echo !* -> echo a b c

The first command above would also execute it immediately.
Notice that the two exclamations points also get the command and the one with ^ ignores it.
To get all arguments but not the command, the asterisk is used.
Additionally you can get specific arguments using a colon, in this context it’s called the word designator:

echo !:0 -> echo echo
echo !:2 -> echo b
echo !:2-3 -> echo b c
echo !:2* -> echo b c
echo !:1 !:3 -> echo a c

But let’s say that you had to run a different command in the meantime and the things you need are not in the last command anymore. This is also easily done by telling it “how long ago” the command was used.

$ echo a b c
$ echo d e f
!-1 -> echo d e f
!-2 -> echo a b c

This is very helpful when using them in quick succession but nigh impossible to keep track of over longer sessions. In those cases it’s a lot easier to use the built-in string function.

$ uname -r
$ uname -a
$ echo a b c
echo !un -> echo uname -a
echo !un:* -> echo -a
echo !?nam -> echo uname -a
echo !?me -r? -> echo uname -r

As we can see, only giving it the beginning of the command it automatically looks for the first occurence (going upwards) in the history and returns it. Using the question marks we can look for parts inside the command to further narrow it down and filter alike commands.

Now onto paths!

Another great feature built into this, is the path handling.
It requires some getting used to but is also very straightforward.
let’s assume we have this path:

/home/user/folder/file.txt

This is what we can do:

Get the path up to file:
echo !:h -> /home/user/folder

Get the filename plus extension:
echo !:t -> file.txt

Get the path up to file extension:
echo !:r -> /home/user/folder/file

Get only file extension:
echo !:e -> .txt

Get only filename:
echo !:t:r -> file

Substitution!

We can also substitute certain strings with other strings.
For example if we have /path/file1 and we want to change file1 to file2 we do:

!:s/file1/file2

But this only works on the first occurence found!
If we want to do this globally, so for every occurence of file1, we use:

!:gs/file1/file2

Example:

$ cp /path/file1 /path/folder/file1
!:gs/file1/file2 -> cp /path/file2 /path/folder/file2

If you want to repeat a successful substitution in a different context then you don’t need to write it again.
You can just use:

!:& -> for single occurence substitution
!:g& -> for global substitution


3. Summary

Summary/Cheat-Sheet

!! -> execute last command
!$ -> return last argument
!^ -> return first argument
!* -> return all arguments
!:n -> return the string on nth position
!:n-x -> return position n to x
!:n* -> return all arguments starting with n
!n -> execute command with history number n
!-n -> exectue command that was run n commands back
!?str -> execute first command (going up) that matches str
!?str? -> execute first command (going up) that contains str
!:h -> return path up to bas filename
!:t -> return only base filename
!:r -> return path up to extension
!:e -> return only extension
!:s/str1/str2 -> substitute first occurence of str1 with str2
!:gs/str1/str2 -> substitute all occurences of str1 with str2
!:& -> repeat last successful substitution
!:g& -> repeat last successful substitution and make it global
!:p -> don’t execute, print only


4. Extras

  • If you missed it in the summary, you can only print the return value of the history substitution without executing it by using !:p
  • Of course you can connect all of these to do some crazy things!
  • To use this feature in bash scripts, supply it with set -H
  • To run a command and not have it in the history, just type a space in front of it! This works for any command, not just history substitution.

And that’s all! If you have any questions or found a mistake just leave a comment below.

I sincerely hope you were able to learn something new today and if you already knew about it then I hope I was able to refresh your memory! :smiley:

Happy Hacking!

13 Likes

That’s a very useful feature that i’ve not heard about. Thank you! I’m now wondering if it could be used maliciously though :thinking: Maybe poisoning the bash_history with evil commands? idk

4 Likes

Another feature it could be good to point out is that you can use ctrl-r to search the command history.
https://www.gnu.org/software/bash/manual/html_node/Searching.html

2 Likes

Oh man, I’ve been using the linux command line for like 10 years and I never knew about these shortcuts.

Thank you so much!

2 Likes

I’m glad to hear you enjoyed it!
Your malicious use idea intrigued me @hunter so I tried to strace the history substitution, as a starting point, and to my surprise it didn’t work!
Maybe it’s also something to do with how I traced it!?
I used:
stty; cat | strace bash > /dev/null
But I always get "!!" - command not found
Anyone know why this could be? It doesn’t really make sense to me since it’s a shell keyword and built into bash…

EDIT: it works when attaching to an already active bash process… Not quite sure why though, maybe I’ll make a follow up post exploring this in more detail

Interestingly, zsh does the same thing, but replaces expressions not when enter is hit, but when the expression is done, wich means for zsh that there is a space after it.
For this reason, this example doesn’t work :

As echo !?me -r? is replaced by echo uname -a before we can type -r.
A simple fix is to use echo !?-r? instead.

This topic was automatically closed after 30 days. New replies are no longer allowed.