Prefer command-line directory to last visited directory

If you want to use the last visited directory you should omit the
directory from the command line.

Fixes #1132.