commandliners

In Linux, the seq command is pretty useful in some scripts, because it prints a sequence of numbers:

$ seq 1 5
1
2
3
4
5

It is usually used in for loops:

for i in `seq 1 5`;
do
    ...
done

But this command is not included in BSD-like OSes. It is contained in the sh-utils, so one option is downloading and compiling it. But I prefer using the commands that come by default with the OS, for portability. In the BSD case, I found the jot command that prints sequential or random data. The following example shows the seq behaviour with jot.

$ jot 5
1
2
3
4
5

Other example where jot shows the integers between 13 and 17: the first argument (5) represents how many values will be shown. The second argument (13) is the number to start counting at:

$ jot 5 13
13
14
15
16
17

You’ve got a bunch of files you want to reorder (that is rename so that they appear in a specific order) but do not want to really rename them (their original names are relevant). You may want to create a set of links to each of them with the appropriate ’001′, ’002′ term before. Here is a script for doing exactly this.
Use as follows (if named ‘mapnames.sh’):

$ mapnames.sh digits file1 [file2 file3 ...]

will use digits digits for the leading numeral (adding '0' before) and will link the files in the given order.
Here is the code:

#!/bin/sh
length=$1
shift
count=1
for name in $@
do
    zeroes=`printf "%0${length}d" $count`
    if [ -e "$zeroes$count.$name" ] ; then
        echo "$zeroes$count.$name exists, left unnamed."
    else
        ln -s "$name" "$zeroes$count.$name"
    fi
    i=$(($i+1));
done

This question came up on the UNIX bboard at sdf. Use with caution. Rafacas did a rather important code optimization...

Both rafacas and pfortuny are going next September to Barcelona to the No cON Name conference, to talk about The Sibyl, of which we have already written here.

If any of you is interested in IT Security and can make it to Barcelona in September (14-17), it would be nice to meet you there.

vidcontrol is a nice utility that among other things allows one to capture the state of a video buffer and dump it to a file. The format of this file is specified in the man page as it is not a regular image file. For this reason one needs a specific tool to convert it to something one can play with. These commands are scr2txt (/usr/ports/textproc/scr2txt) and scr2png (/usr/ports/graphics/scr2png).
They can then be combined using one single line:

$ vidcontrol -p < /dev/ttyv0 | scr2txt > /tmp/screenshot.txt

The -p flag instructs vidcontrol to get the current content of the corresponding video buffer. scr2txt reads a vidcontrol dump file from standard input and writes a txt equivalent file to standard output. scr2png works in a similar fashion for png files.

These are the contents of the screenshot.txt file:

$ cat /tmp/screenshot.txt
[n0str0m0@beastie ~]$ who am i
n0str0m0         ttyv0    Apr  5 18:59
[n0str0m0@beastie ~]$                                                           

<some blank lines here>

Enjoy!

I am a Mathematician and usually write more LaTeX than C but programming has been an active hobby since I owned my first Spectrum back in the eighties. What I am commenting in this post has been plaguing me for years and I have just learnt it (from laziness, no more): using pattern rules in makefiles. Caveat: I do not know whether this works for non-GNU Makes or not.

I had already read a lot of pro makefiles, which contained endless funny codes like $@, $%, $< etc. Today I have learnt the meaning of some of these.

I am supervising a project which needs compiling the same code with different versions of the same library. For the tests, we (this pupil of mine and I) had a makefile including the following jewel:

classic: *.c
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0a.c -o case0a_cl $(CL_LIB)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0b.c -o case0b_cl $(CL_LIB)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0c.c -o case0c_cl $(CL_LIB)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0d.c -o case0d_cl $(CL_LIB)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0e.c -o case0e_cl $(CL_LIB)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0f.c -o case0f_cl $(CL_LIB)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0g.c -o case0g_cl $(CL_LIB)

classic_no_fft: *.c
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0a.c -o case0a_cl_no_fft $(CL_LIB_NOFFT)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0b.c -o case0b_cl_no_fft $(CL_LIB_NOFFT)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0c.c -o case0c_cl_no_fft $(CL_LIB_NOFFT)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0d.c -o case0d_cl_no_fft $(CL_LIB_NOFFT)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0e.c -o case0e_cl_no_fft $(CL_LIB_NOFFT)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0f.c -o case0f_cl_no_fft $(CL_LIB_NOFFT)
        $(CC) -I$(CL_DIR) -L$(CL_DIR) case0g.c -o case0g_cl_no_fft $(CL_LIB_NOFFT)

(Yes, no way). That is what I was thinking while writing those lines; as a matter of fact, there are six other sections (profiling on/off, and a different version of the library with profiling as well on/off… you get the picture). But it worked, so why taking the trouble? Because one of our leading passions is learning. You can simplify all the stuff above with Make pattern rules. The simplest one (the one I used) is:

%_anything : %_whatever.c ; recipe

where _anything and _whatever are to be substituted by the needed text and recipe is the Make rule to build %_anything. In the above case, I just had to write:

%_cl : %.c ; $(CC) -I$(CL_DIR) -L$(CL_DIR) $< -o $@_cl $(CL_LIB)

%_th : %.c ; $(CC) -I$(CL_DIR) -L$(CL_DIR) $<.c -o $@_no_fft $(CL_LIB_NOFFT)

where there appear, in the recipe, the two funny codes $@ and $<. These are easy to understand: they are substituted inside the recipe by the part of the destination name corresponding to the % (this is the $@) pattern and the full name of the source (this is the $<) in each case for each of the targets. In the example, the targets are specified as

classic: case0a_cl case0b_cl case0c_cl case0d_cl case0e_cl case0f_cl case0g_cl

threads: case0a_cl case0b_th case0c_th case0d_th case0e_th case0f_th case0g_th

so that when one types

~$ make classic

The target case0a_cl, which falls into the rule %_cl : %.c, will be updated if case0a.c (the source corresponding to case0a_cl has no _cl in the name) has been modified since the last compilation. The recipe will be expanded to

 %_cl : %.c ; $(CC) -I$(CL_DIR) -L$(CL_DIR) case0a.c -o case0a_cl $(CL_LIB)

I am assuming the reader is familiar with Make’s variables (the $(CC) etc…).

So that I ended up with the following makefile (rather shorter, to be sure, than the original one):

CL_DIR=../../../mapm_4.9.5aa/
TH_DIR=../../
CC=gcc -g
CCP=gcc -g -pg
CL_LIB=-lm -lmapm
TH_LIB=-lm -lmapm -lpthread
CL_LIB_NOFFT=-lm -lmapm_no_fft -lpthread
TH_LIB_NOFFT=-lm -lmapm_no_fft -lpthread

%_cl : %.c ; $(CC) -I$(CL_DIR) -L$(CL_DIR) $< -o $@ $(CL_LIB)

%_th : %.c ; $(CC) -I$(TH_DIR) -L$(TH_DIR) $< -o $@ $(TH_LIB)

%_cl_no_fft : %.c ; $(CC) -I$(CL_DIR) -L$(CL_DIR) $< -o $@ $(CL_LIB_NOFFT)

%_th_no_fft: %.c ; $(CC) -I$(TH_DIR) -L$(TH_DIR) $< -o $@ $(TH_LIB_NOFFT)

default: all

classic: case0a_cl case0b_cl case0c_cl case0d_cl case0e_cl case0f_cl case0g_cl

classic_no_fft: case0a_cl_no_fft case0b_cl_no_fft case0c_cl_no_fft \
		case0d_cl_no_fft case0e_cl_no_fft case0f_cl_no_fft case0g_cl_no_fft

threads: case0a_th case0b_th case0c_th case0d_th case0e_th case0f_th case0g_th

threads_no_fft: case0a_th_no_fft case0b_th_no_fft case0c_th_no_fft \
		case0d_th_no_fft case0e_th_no_fft case0f_th_no_fft case0g_th_no_fft

all: classic threads classic_no_fft threads_no_fft

test: all
	./run_all.sh

clean:
	rm -f *~ *th *cl *_no_fft

I reckon all those rather awful lists might be shortened somehow, but it will probably take me a couple of years to learn.

I live in Spain. I work at Academia. This is a problem.

My School’s computers are plagued with Windows (XP!), my pupils most certainly use Windows (I bet it is 7) and I use (obviously) an MBPro with Snow Leopard. Even more, my School’s servers are Linux. This gives rise to latin-1, UTF-8 collisions.

While writing some text-only documents for use with MATLAB (another problem, by the way) I realized I was going to need to convert them from my Mac’s UTF-8 to Windows latin-1 every time I edited them. Time for a makefile. I am not commenting this though, only the rather short elisp code I found here. Say you have a file named convert.el containing:

;; Read the file with no character code conversion.
;; Assume crlf represents end-of-line.
(let ((coding-system-for-read \'utf-8))
    (insert-file-contents "name_of_input_file.m"))
(let ((coding-system-for-write \'latin-1))
    (write-file "name_of_output_file.m"))

Running emacs in batch mode allows you to perform the conversion between UTF-8 and latin-1 automatically:

~$ emacs -batch -l convert.el

Will convert name_of_input_file.m from UTF-8 into name_of_output_file.m which will be in latin-1.

If you mix this with some Perl (or even with some more elisp, but I am not comfortable with the latter), you can achieve conversion enlightenment with a simple makefile. I did. (Which automatically implies I did not, or does it?).

In my actual job I have a laptop with Windows 7 installed on it. The last time I used Windows on a daily basis was four years ago and it was Windows XP. The terminal (cmd.exe) was… well, that was not a real terminal neither a real shell.

The first days using Windows 7 I had a problem (which I will explain it later) and while I was installing cygwin to run some commands I found Powershell. I have to admit that Windows now has a real shell and it seems pretty powerful. It can be installed in Windows XP and Vista too.

The big difference between Powershell and any unix shell is that instead of the unix “all is a file” philosophy in Powershell one has to use objects.

The problem I had was I opened a Microsoft OneNote document I had received (I do not know why people do not like plain text files and have to use special programs to take notes, but that is another problem) and it created a file in every single directory of my G: partition. Each time I changed to a directory, there it was, a file with the extension onetoc2.

I looked for the find command in Powershell and I found Get-ChildItem (its alias is gci):

PS G:\> gci . -recurse -include *.onetoc2

The above command returns all the files with the extension onetoc2 in the G: partition. I was curious to know how many files OneNote had created so I run:

PS G:\> (gci . -recurse -include *.onetoc2).count
4228

I swear I only opened a single OneNote file a workmate sent me.

Finally to delete all the mysterious generated files I run:

PS G:\> gci . -recurse -include *.onetoc2 | ri

ri is the alias of the Remove-Item command.

I think this time Microsoft has provided sysadmins with a useful tool.

The other day, this colleague of mine asked me how he could use svn blame on the same file he was editing without leaving his vim session. This is a mapping I wrote for that purpose.

:map <F3> :sp %.tmp_svnblame \| :r! svn blame $(basename % .tmp_svnblame)<CR>

Enjoy!

limits is a FreeBSD’s base system utility for displaying and setting system resources limits. To some extent it is equivalent to the limit and ulimit commands. It can set limits for filesize, coredumpsize, maxproc, memoryuse and other parameters. Convenience modifiers like k, m or g for kilobytes, megabytes and gigabytes or s, m, h for seconds, minutes and hours can be used in all cases.

limits can be used to achieve three different goals:

• Set resource limits.
  For example:

  $ limits -t 1s ls -R /

  launches the ls -R command with a cputime limit of 1 second. In general, this is the output that will be generated:

  ...
  ...
  ...
  Killed: 9

  If we want to know what happened, we can have a look at the system message buffer:

  $ dmesg
  pid 1651 (ls), uid 1001, was killed: exceeded maximum CPU limit

• Show resource limits
  If we just want some information about resource limits, we can invoke the command this way:

  $ limits
  Resource limits (current):
    cputime                    infinity secs
    filesize                      infinity kB
    datasize                   33554432 kB
    stacksize                  524288 kB
    coredumpsize          infinity kB
    memoryuse              infinity kB
    memorylocked          infinity kB
    maxprocesses          5547
    openfiles                   11095
    sbsize                       infinity bytes
    vmemoryuse            infinity kB
    pseudo-terminals     infinity
    swapuse                   infinity kB

  Or if we are interested in one particular resource:

  $ limits -c
  Resource limits (current):
    coredumpsize         infinity kB 

• Report resource limits in a way suitable for feeding a shell.
  This is pretty smart. Sometimes you want to set some limits on the invoking shell but simply calling limits will set the resource limit for the forked shell. If you don’t know what I am talking about, have a look at the fork-exec model for spawning processes in UNIX systems. limits transforms the resource limit command to the appropriate command for the invoking shell using either ulimit or limit depending on the shell. The limit command knows the following shells: sh, csh, bash, tcsh, ksh, pdksh and rc). Invoking limits with the -e flag results in the following output:

  $ limits -e -t 10s
  ulimit -t 10;

  It makes sense, since I am using bash. We can use this output together with the eval built-in command to apply the limit to the current shell:

  $ eval `limits -e -t 10s`
  $ limits -t
  Resource limits (current):
    cputime                    10 secs

That’s all for today.

Enjoy!