This reworks a previously unmerged PR to clean up the code a little and adds a test for it.
These tests are more or less parity with the rspec-system tests, though dependent on a few patches that still have to make it into beaker and beaker-rspec