commandliners

In my actual job I have a laptop with Windows 7 installed on it. The last time I used Windows on a daily basis was four years ago and it was Windows XP. The terminal (cmd.exe) was… well, that was not a real terminal neither a real shell.

The first days using Windows 7 I had a problem (which I will explain it later) and while I was installing cygwin to run some commands I found Powershell. I have to admit that Windows now has a real shell and it seems pretty powerful. It can be installed in Windows XP and Vista too.

The big difference between Powershell and any unix shell is that instead of the unix “all is a file” philosophy in Powershell one has to use objects.

The problem I had was I opened a Microsoft OneNote document I had received (I do not know why people do not like plain text files and have to use special programs to take notes, but that is another problem) and it created a file in every single directory of my G: partition. Each time I changed to a directory, there it was, a file with the extension onetoc2.

I looked for the find command in Powershell and I found Get-ChildItem (its alias is gci):

PS G:\> gci . -recurse -include *.onetoc2

The above command returns all the files with the extension onetoc2 in the G: partition. I was curious to know how many files OneNote had created so I run:

PS G:\> (gci . -recurse -include *.onetoc2).count
4228

I swear I only opened a single OneNote file a workmate sent me.

Finally to delete all the mysterious generated files I run:

PS G:\> gci . -recurse -include *.onetoc2 | ri

ri is the alias of the Remove-Item command.

I think this time Microsoft has provided sysadmins with a useful tool.

The other day, this colleague of mine asked me how he could use svn blame on the same file he was editing without leaving his vim session. This is a mapping I wrote for that purpose.

:map <F3> :sp %.tmp_svnblame \| :r! svn blame $(basename % .tmp_svnblame)<CR>

Enjoy!

limits is a FreeBSD’s base system utility for displaying and setting system resources limits. To some extent it is equivalent to the limit and ulimit commands. It can set limits for filesize, coredumpsize, maxproc, memoryuse and other parameters. Convenience modifiers like k, m or g for kilobytes, megabytes and gigabytes or s, m, h for seconds, minutes and hours can be used in all cases.

limits can be used to achieve three different goals:

• Set resource limits.
  For example:

  $ limits -t 1s ls -R /

  launches the ls -R command with a cputime limit of 1 second. In general, this is the output that will be generated:

  ...
  ...
  ...
  Killed: 9

  If we want to know what happened, we can have a look at the system message buffer:

  $ dmesg
  pid 1651 (ls), uid 1001, was killed: exceeded maximum CPU limit

• Show resource limits
  If we just want some information about resource limits, we can invoke the command this way:

  $ limits
  Resource limits (current):
    cputime                    infinity secs
    filesize                      infinity kB
    datasize                   33554432 kB
    stacksize                  524288 kB
    coredumpsize          infinity kB
    memoryuse              infinity kB
    memorylocked          infinity kB
    maxprocesses          5547
    openfiles                   11095
    sbsize                       infinity bytes
    vmemoryuse            infinity kB
    pseudo-terminals     infinity
    swapuse                   infinity kB

  Or if we are interested in one particular resource:

  $ limits -c
  Resource limits (current):
    coredumpsize         infinity kB 

• Report resource limits in a way suitable for feeding a shell.
  This is pretty smart. Sometimes you want to set some limits on the invoking shell but simply calling limits will set the resource limit for the forked shell. If you don’t know what I am talking about, have a look at the fork-exec model for spawning processes in UNIX systems. limits transforms the resource limit command to the appropriate command for the invoking shell using either ulimit or limit depending on the shell. The limit command knows the following shells: sh, csh, bash, tcsh, ksh, pdksh and rc). Invoking limits with the -e flag results in the following output:

  $ limits -e -t 10s
  ulimit -t 10;

  It makes sense, since I am using bash. We can use this output together with the eval built-in command to apply the limit to the current shell:

  $ eval `limits -e -t 10s`
  $ limits -t
  Resource limits (current):
    cputime                    10 secs

That’s all for today.

Enjoy!

The savannah.gnu.org compromise in December 2010 is another example of the weakness of system passwords: there will always be someone using a brute-force attackable key. And that is the weakest link.

I just want to point out to all our readers out there that the Sibyl is precisely an idea to implement a secure way to store hashes of passwords and prevent brute-force attacks. It is not computationally cheap or even the simplest of setups, but security has a price. At least, it is, but for the hardware part, which depends on your implementation, ‘gratis’ and BSD-licenced.

Hope you like it.

You have probably seen something like this in other editors. Macros (or “complex repetitions” as Vim’s help puts it) is a way to repeat complex commands sequences. In order to get the most out of this feature, you need to master movement and insertion commands.

The work cycle is the following: start recording, execute commands, stop recording. Then, you are ready to execute the whole sequence of commands you have recorded. Let us see this in more detail. To start recording, we use the “q” command followed by a register name in which the sequence of commands will be stored.

qa

At that moment, you will see the following message at the bottom of the editor:

recording

From now on, everything you type is stored in the “a” register. Type “w” and then “q” again to stop recording. Now if you type :registers you can see the contents of the “a” register. It contains the commands you typed (the “w” command). In fact, you can use the register as a normal one and paste its contents: "ap will paste “w” in your document.

Vim can interpret the contents of the register as commands too. In order to do this, use the @ command followed by the register name.
So, if you type:

@a

The cursor moves one word(w) forward.

Let us see a more useful example. Imagine you have some code looking like:

if (var1 == 0) or (var2 == 0) or (var3 == 0) or (var4 == 0) or (var5 == 0)

and you realize that you have to filter every variable using a function before you can do the comparison. Macros come in quite handy. The logic is the following: we go to the beginning of the line and start recording. Then we move until the next “(“, type the name of the function (let us call it “filter”) and a new parenthesis. Then we move to the end of the word and close the parenthesis. And the process repeats. So, once we are at the beginning of the line, we press “q” and then, the following: “f(afilter(<ESC>wea)”. Commands are in boldface.

With this sequence, we “filtered” var1, and now we can filter the second one by typing @a. And the third one… or even better, we can use repetition and type:4@a

We can now revisit one of the examples we worked on when we studied maps. Let us imagine we are in this situation:

stdio.h
unistd.h
stdlib.h
errno.h
sys/types.h
sys/param.h

We need to wrap those lines into <> characters and add #include at the beginning of the line. What we have to do for every line is the following: Insert at the beginning of the line the text “#include <”. Then add “>” to match the previous symbol wrapping the file name. Finally, we need to move down to the next line. This is the sequence: qaI#include <<ESC>A>jq
Typing 5@a modifies the rest of the lines.

So far we have used lower case register names. If you remember from a previous issue, if you use an uppercase letter, the content of the register is not replaced but added. So if we do the following:

qbwq

and then

qBjkq

the result of the :registers commands shows that the content of the “b” register is “wjk”

So far we have worked only in insert and command modes. Of course command line mode is also available for macros. Let us see an example:
Place the cursor at the first #include line.

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/param.h>

First we type “qa” to start recording. Then, we type V/^$:!sort. This goes into Visual mode, selects everything until the next empty line and then goes into command line mode and executes the external sort command. Now, you have something like this:

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/param.h>
#include <sys/types.h>
#include <unistd.h>

Every time you add more include lines, you can go to the first one and execute the @a macro.

Conclusion
Macros are really useful to save time. You can have many of them stored in the many registers available. Since you can go from insert to command mode and back, they are extremely powerful and flexible. Give them a try!

VIM SHEET (IX)

• q Start / Stop recording
• @ Execute macro

Maps are a way to create an association between a set of key strokes and a set of actions. They are really powerful. However, in this post, I will not explain them in the deepest detail. If you need further information, you will need a good Vim manual. As it happens with abbreviations, maps can be used regardless of the mode you are in Vim or they can be restricted to a certain operation mode. They work the same way in every mode. Type :help :map to know which map commands work in which mode.

First off,

:map
...
...
<Esc>OM       <CR>
<Esc>Ol       ,
<Esc>Ok       +
<Esc>Om       -
<Esc>Oj       *
<Esc>Oo       :
<F2>          :sp ~/TODO<CR>Go* <Esc><Up><Right><CR>
<xHome>       <Home>
<xEnd>        <End>
<S-xF4>       <S-F4>
<S-xF3>       <S-F3>
<S-xF2>       <S-F2>
<S-xF1>       <S-F1>
<xF4>         <F4>
<xF3>         <F3>

shows a list of available maps

Let us start with a small example:

:map <F2> :r!ls ~/<CR>

As you can see, maps are created very much like abbreviations.
When you type F2, Vim executes the associated sequence for that key. It is important to use <CR> when you are using :commands. Otherwise, you will have to type ENTER yourself. The map above reads the content of the ls ~/ command, i.e. it lists the $HOME directory.

When you work with maps, you are not limited to commands. You can tell Vim to “simulate” certain keystrokes. For example, you might want to execute something like “After this, I would like to move the cursor to the left”. This can be done with the “<>” notation (:help <> for more information). The following map illustrates this concept:

:map <F2> ifor(;;) {<CR>}<Up><Right><Right><Right>

Now, when pressing F2, the following text is inserted (the underscore represents the position of the cursor):

for(_;;) {
}

Notice you are still in insert mode. We could have achieved something similar with the following map:

:map <F2> ifor(;;) {<CR>}<ESC>kf;i

I personally prefer the latter. In my opinion, working in command mode is cleaner.

The following map is quite useful:

:map <F4> I#include <<ESC>A>

Let us analyze what that sequence does. First, it goes into command mode and places the cursor at the beginning of the line in insert mode. Then, it inserts the “#include <” string. Finally, it adds “>” at the end of the line.

Assuming you have a text like the following:

stdio.h

Placing the cursor on that line (at any column) and pressing F4 rewrites the line as follows:

#include <stdio.h>

Maps can deal with buffers, windows, tabs (from Vim 7 onwards) and such. Here is a small example. Suppose you have a TODO file in your $HOME directory. From time to time, while you are working, you want to add a new entry. Sounds reasonable, right?. Imagine we have a TODO file that looks like this:

* Stuff
* Things

We could easily open the file and make it ready to write a new entry with a simple map:

:map <F2> :sp ~/TODO<CR>Go* <Esc>a

Let us explain what is going on here: We split the window horizontally, editing the TODO file at the same time (:sp ~/TODO). Then the cursor is placed at the end of the file. We open a new empty line and write a “* ” (Go* ). Finally, we leave the cursor in insert mode. This way we just have to type the thing we want to remember and then we can :close the top window.

Another simple example:

:map <F6> :w %.bakcup

This saves the current buffer with the same name plus the “.backup” extension. The % is a special variable which contains the name of the current file.

Finally, if you want to delete a map because it is no longer useful, you can :unmap it.

Conclusion
Maps are a very powerful tool. You can do almost anything you may need, no matter how complex it is. There are a lot of interesting macros on the Internet. I recommend you to search the web!

VIM SHEET (VIII)

• map Map a key to a set of commands / List available maps
• unmap Unmap key

The Sibyl is a project invented and implemented by Pedro (pfortuny) and me (rafacas). Although I have to admit that it was Pedro’s idea.

It started with the goal of secure storage of the shadow file and, in general, of any database of secret authentication tokens (think of passwords -actually hashes of passwords- of users of a Web-based service, for example). We consider it addresses the main concern on those databases: dictionary attacks and rainbow tables, which have become available at negligible cost: there is a cloud-based service for doing dictionary attacks on a WPA key.

Our approach for storing shadow files is to use a separate server for checking the correctness of the password introduced by the user, taking advantage of asymmetric key encryption.

Instead of keeping the hash (as crypt(3) does, or SHA1) of the password in the shadow file, we store an OAEP RSA-cyphertext of the hashed password (using a public encryption key) and, each time the user tries to log in, ask via TCP someone (the owner of the private key) if another OAEP-encryption of the hash of the password issued by the logging user matches the stored token. That is: use an”oracle” to check if the user has entered the correct password or not. This “oracle”, which is a standalone machine, is the Sibyl and the device we use to implement it is a Bifferboard.

The details are on the web of the Sibyl project. We hope you like it and use it.

Welcome to a new issue of this short Vim tutorial!

As we saw in previous posts, Vim is the perfect tool if you want to save time in your daily work. It helps you in the write-compile-debug cycle, it indents and autoindents code, it is extremely powerful for searching, replacing and many other frequent tasks.

In this post I will introduce an interesting feature meant to make you type less: abbreviations.

Abbreviations are, as their name indicates, a way to create an association between a short set of characters and a piece of text (or a command).

Abbreviations can be used in insert, replace or command mode. The keywords used to work with abbreviations are all the same, but adding a proper prefix if necessary (i for insert mode, c for command mode and ! for both).

iab        Create abbreviation for theinsert mode
cab        Create abbreviation for the command mode

Recently, I had to write a dockbook template using perl and the Mason library and found myself typing continuously the following thing:

$data->[0]->{}

and between the curly braces the hash key. After writing that twice I got bored of doing it (yep, I am a lazy guy) so I created the following abbreviation:

iab dh $data->[0]->{!cursor!}:call search('!cursor!', 'b')cf!

It associates the “dh” characters with the $data->[0]->{!cursor!}:call search(‘!cursor!’, ‘b’)cf! string. This string writes “$data->[0]->{}” and places the cursor between the curly braces (do not worry about the “call search” stuff, the example could have been iab dh $data->[0]->{} ) By the way, abbreviations are not recursive, so you will never end up stuck in an infinite loop.

Since writing a tag-like document is a real pain, I decided to abbreviate some of the common tags I was using the most, so I created the following abbreviations:

iab fp <formalpara><CR><LF></formalpara>
iab p <para></para>
iab il <itemizedlist><CR><LF></itemizedlist>
iab li <listitem></listitem>
iab e <entry><CR><LF></entry>
iab t <title></title>
iab it <informaltable><CR><LF></informaltable>
iab r <row><CR><LF></row>

<CR> and <LF> have the common meanings (Carriage Return and Line Feed respectively). They were very convenient. You can copy them to your .vimrc file.

But what if you do not want an abbreviation to be expanded?

Suppose we have the following abbreviation:

iab to Total

and we want to write the following sentence:

"I went to Paris"

As soon as you type the space after “to”, the abbreviation will be expanded and you will end up with:

"I went Total "

To avoid this problem, press Ctrl-V after the problematic abbreviation and then continue typing normally.

To disable an abbreviation permanently you can use “unabbreviate”. For instance:

:unabbreviate dh

“abbreviate” lists the current abbreviations. E.g:

i  t             Total
i  b             back

The first column shows the mode in which the abbreviate is available, the second shows the abbreviation and the third one the text it expands to.

Abbreviations are a convenient way of saving some (or possibly a lot of) work, specially if it is repetitive.

Vim Sheet(VII)

• [i/c/!]ab Create an abbreviation in the specified mode
• unabbreviate Deletes an abbreviation
• abbreviate List available abbreviations