commandlinefu.com

I just rediscovered this excellent site. Well worth the occasional browse:

• Commandlinefu - main page
• Commandlinefu_byvote - sorted by vote count

There are some real gems in there! For example:

> python -m SimpleHTTPServer
Serve current directory tree at http://$HOSTNAME:8000/

Useful for tiding up your workspace whilst keeping jobs running:

> disown -a && exit
Close shell keeping all subprocess running

I do love a bit of process substitution:

> diff <(sort file1) <(sort file2)
diff two unsorted files without creating temporary files

Handy:

> rm !(*.foo|*.bar|*.baz)
Delete all files in a folder that don't match a certain file extension

And a "I should have thought of this; it's so obvious now!" trick:

> some_very_long_and_complex_command # label

Easy and fast access to often executed commands that are very long
and complex. When using reverse-i-search you have to type some 
part of the command that you want to retrieve. However, if the
command is very complex it might be difficult to recall the parts
that will uniquely identify this command. Using the above trick
it's possible to label your commands and access them easily by 
pressing ^R and typing the label (should be short and descriptive).

R: formatting numbers for output

I often produce tables of numbers with a lot of significant digits after the decimal point. It's confusing to look at 20 sig [fd]igs, especially in a large table of results, so I tend to format my output to make it more concise and easier to read.

The function 'format' is pretty good for this. Note that the 'format' call returns a character vector (which is fine if you're only going to write the number to file or the console).

Here's a simple example just using a randomly generated number:

> num <- rnorm(1, mean=10)
> num
[1] 10.24339

We call format with digits=4 (show 4 significant digits) and nsmall=4 (display at least 4 digits after the decimal - for real/complex numbers in non-scientific format):

> format(num, digits=4, nsmall=4)
[1] "10.2434"

You can see that the format command rounds the numbers. This uses the IEC 60559 standard - 'go to the even digit'. So 0.5 is rounded to 0 and 1.5 is rounded to 2...

Of course, if you're used to sprintf style commands then you can also use the sprintf function for this:

> sprintf("%.4f", num)
[1] "10.2434"

Simple parallel processing with xargs

We've all been there - looking in a nicely tidied up directory, full of archived data - hundreds of lovely data files all gzipped or (better) bzip2'ed; but now you want to use them and you have to uncompress them all... "if only I could use all n CPUs on my local machine to do this!": enter 'xargs'!

It's as simple as:

> ls *.bz2 | xargs -n 1 -P 6 bunzip2

This will set off bunzip2 on all bz2 files in the current directory.

The '-n 1' flag tells xargs to only provide one argument (file) per command line; the '-P 6' tells xargs how many concurrent processes to run.

hg over ssh

I'm working with a repo that I can't access using http, so it's just as well that Mercurial works just fine and dandy over ssh as well - you just have to do a little bit of tweaking to the basic configuration to get things working smoothly.

Add the following to the client .hgrc:

[ui]
remotecmd=<path_to_hg>

Where <path_to_hg> is the path to the hg executable on the main repo machine.

This was from StackOverflow: cloning-a-mercurial-repository-over-ssh

Now, I'm working very remotely from the main repo (6000 miles) so it is worth using compression. Just add the following to the [ui] section of .hgrc:

ssh = ssh -C

You could also configure ssh to use compression by default (see details in link below).

This page has a good overview of using ssh with mercurial: collaborating-with-other-people

bash while loops and pipes

In bash, if you pipe into a while loop, the while loop is run in a subshell. This means that you're going to be very disappointed if you were hoping to capture data/variables within the loop.

The work around is to not have the pipe there - which is possible through process substitution.

e.g.

echo -e "one\ntwo\nthree" | \
 while read name;
  do val=$name;
  echo $val;
 done;
echo $val

This outputs:

one
two 
three

Notice that the final value "three" is not printed twice as 'val' no longer contains a value.

And the work around:

while read name; 
 do val=$name;
 echo $val;
done < <(echo -e "one\ntwo\nthree")
echo $val

which now outputs:

one
two
three
three

Success! (or, if you're a "Bill and Ted's Bogus Journey" fan - Station!).

Mercurial file patterns

I wanted to add all the scripts in a directory tree to the local Mercurial repository. I was going to do something like this:

> find . -name \*.sh | xargs hg add

Which is nice enough (find and xargs go well together), but you can also do it using just Mercurial using patterns. e.g.

> hg add 'glob:**.sh'

The two most useful patterns (for me) are:

• '*' match any text in the current directory only
• '**' match anything in the entire tree

The patterns are much richer than this though; see hg help patterns for more on what's available (including regexes).

Removing VMWare Player - blank grey dialogue box

I've just spent way too long trying to upgrade VMWare Player on my laptop.  The issue was that, when I tried to uninstall the incumbent version, I was presented with an unhelpful blank grey dialogue box...  Notice the lack of, well, anything.

The box didn't go away , even after a couple of hours waiting -  left it because I was wondering if something was being unpacked in the background.  This kind of information is frustratingly difficult for me to get at on a windows machine...  Eventually, I had to use Task Manager to kill it.  I went through a few iterations of this trying out a few different ways of uninstalling or running the newer installer (even setting different default browsers since the contents of the box turned out to be HTML and I thought it might be a compatibility issue - but computer says "no"). 


I looked around for solutions & came across the following, which worked for me:


http://superuser.com/questions/245424/vmware-workstation-install-problem


The most pertinent advice was:

• To uninstall any old version, go to C:\Windows\Installer
• Add the "Authors" column and sort by it
• One of the .msi files with have a "VMware" author
• Double-click it and follow through with the uninstall steps

After uninstalling the older VMWare Player using this method, I was then able to install the latest version and get playing with my brand spanking new ACE image.  Success!