Stupid Nix Tricks: SSH Multiplexing

At this point, it should basically be “Stupid SSH tricks™”, but whatever.

Prerequisites: Basic understanding of *nix and SSH

Scenario

You have a server behind a firewalled network and you need to connect. You could just do something like ssh first-server and from there ssh internal-server. Not too bad, right?

What if there’s a whole subnet of servers that you might wanna reach? What if you don’t wanna SSH twice? What if you’re using something that wants to execute from your local machine but can’t really handle the manual two step SSH jump (such as Ansible)

This is where SSH Multiplexing comes in very handy

Solution, part 1: Pivot box

As you probably expected by now, we’re going to be digging into our ssh config file, so go ahead and pop an editor open to ~/.ssh/config

The first thing we want to do is setup our “gateway” or “bastion” or “jump” host. Whatever you want to call it. Doesn’t matter to me. Here’s an excerpt of mine with a breakdown of each line in the config

host jump
  User fraq
  Hostname foo.bar.baz
  ControlMaster auto
  ControlPath ~/.ssh/proxy-%r@%h:%p
  ControlPersist 5m

host jump defines a friendly name in SSH for your machine. Note that this isn’t in /etc/hosts, so you can’t ping jump, but you can ssh jump. When you do use ssh jump, ssh opens up the config file, looks for a host entry that matches, and then uses that config to build the connection. Nothing we haven’t covered already in previous chapters.

User fraq is the username we use to connect to jump

Hostname foo.bar.baz means that we connect to a server called foo.bar.baz in DNS. This can also be an IP address.

These next three are the ones you need for multiplexing:
ControlMaster auto tells the ssh connection that you want to listen for control connections (used for multiplexing)

ControlPath tells ssh where to put the control socket for the connections and how to format the file names. Per the OpenSSH cookbook:

The combination %r, %h and %p stand for the remote user name, the remote host and the remote host’s port. The control sockets should be given unique names.

ControlPersist tells ssh how long to leave the master open.

Solution, part 2: Target box

Now that your jump host is setup to accept multiplexed SSH connections, it’s time to tell the SSH how to connect to your target box through the jump box.

Host target
    ProxyCommand ssh -W %h:%p jump
    User fraq
    Hostname 10.10.10.10

As you can see, it looks very similar to a standard config file, except for the ProxyCommand option. This tells the ssh connection to execute that command first as a prerequisite for connecting to the target machine. In our case, we’re saying that we want to ssh -W through the machine known to ssh as jump.

Conclusion

With this in place, you can now use jump as a pivot to get into internal networks that it can access but you cannot, all without having to ssh twice manually. This means scripts or tools that invoke ssh can be used locally on your machine as well.

And yes, this can be chained basically indefinitely, so you can create some pretty long and crazy proxy connections.

Also, I very intentionally skipped over the in-depth explanation of ssh -W and the older ssh nc implementation. The whole purpose of this post was just to setup ssh multiplexing.

References

https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing

10 Likes

This is cool and all. But I think you’re complicating this massively.

Here is an example of a jump box I use in my ssh config.

Host myhost
    Hostname 10.10.10.10
    Port 4040 
    User pry0cc
    ProxyJump [email protected]

Simply putting a host inside ProxyJump does the same thing.

5 Likes

“Complicating” all depends on your use case.

I intentionally omitted the CLI version of this, but as you’re probably aware, anything you do on the CLI can be done in the ssh config and vice versa. For me, I always ssh in certain boxes via jump hosts, so I want to remove the -J whatevermiddlehost from my command and just ssh foo.

Beyond that, there’s a very good reason to setup and use this via the config file: SSH config files can accept regex! If you want to execute a command against a whole subnet (ie, Ansible plays) then you can setup a host entry like *.mydomain or 10.10.*, so in one fell swoop you setup proxycommands for a whole bunch of servers.

3 Likes

so, ssh -J is valid and good and useful if you’re doing a one-off. As you know I always recommend that if you do something twice, script/automate/remove the manual interaction.

2 Likes

Well, that code snippet ^ is an ssh config file. Not a command line switch.

1 Like

It should also be noted that ProxyCommand and ProxyJump are not the same. ProxyCommand is present on most but not all older SSH versions and allows for arbitrary commands attached to SSH (like nc), whereas ProxyJump is shorthand for sshing through multiple machines.

1 Like

Isn’t that what you want to achieve? Use a machine to connect through? Or am I mistaken?

2 Likes

This is a basic example of connecting through machines, yeah. If you wanna use this to do additional fun stuff, that’s up to you.

1 Like

Your “Stupid Nix Tricks” have become a favorite of mine fraq…thank you.

Also, I understand and appreciate pry0cc’s contribution to this post.

It should be noted that where exploitation is concerned, there could be any number of variables that could force improvisation of a concept or skill.

Stupid snags happen all the time on engagements, and sometimes you don’t have time to fully appreciate the cause of a snag until you are drafting your report (or even later when the client’s IT department let you in on what could have occurred); you just keep shifting the methods by which you’d achieve your next step.

Thus, I appreciate fraq’s introduction of this concept and both your and pry0cc’s methods of execution.

I also appreciate the measures by which any work here on the Big0 is open to review and fair, polite criticism and discourse…so thank you both for being an example of this as well.

There are no sacred cows in this community, only cows and BBQ.

5 Likes

I’m gonna have to remember that :wink:

1 Like

fraq-

I am glad you enjoyed the quip brother; it was in no way an equivalent exchange for the knowledge you (and pry0cc) shared.

It was a spontaneous flourish on my part; as well as being an apex skill of my trade, Nix Administration reminds of me of the older generations of Unix Administrators (far too many are retiring or being pushed into retirement ) who projected a tangible aura of arcane, strange power.

I was managed/mentored by one grey beard who got into Systems Administration of huge industrial/energy sector enterprise environments to “ease into retirement” .

His career had started as a programmer in the 1960’s: he designed and programmed microprocessors for submarines .

The company provided no tools or resources other then the microprocessors themselves (back then software bundling as a service was still going strong thanks to companies likeIBM).

He had to program not just the microprocessors, but also almost all of his tools…mostly in Haskell…

I think your post and the back and forth with pyr0cc (which those Unix sorcerers were also prone to) reminded me of those times.

I think my mind got weird with those words as I realized that this community is building the future facsimiles of those mages.

4 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.