shell

Installing YouCompleteMe on FreeBSD

Recently, we had a discussion at work about editors and IDEs. And yes, I think Vim is not only a great editor but also a good IDE with the proper plugins installed.

There are several nice plugins out there, but I’m going to talk about YouCompleteMe. YouCompleteMe (in short, YCM), is a completion engine for Vim that uses libclang as a backend to support great C/C++ semantic completion. Unlike other pure Vim plugins, YCM needs a component that has to be compiled before you get semantic completion. Although it is not complicated, I’ll show you the steps to successfully install YCM in your FreeBSD:

  1. Install libclang. Since version 10.0, FreeBSD ships clang as a compiler, but not libclang itself. I installed clang35 (sudo pkg install clang35 or sudo portinstall clang35) which installs the library.
  2. Install vundle, the plugin manager.
  3. Use vundle to install YCM as explained in its site. It’s really simple
  4. Install all the needed libraries: sudo pkg install boost-libs boost-python-libs
  5. By default, YCM doesn’t find the boost libraries, so we need to help it a little: export LD_LIBRARY_PATH=/usr/local/llvm35/lib/
  6. Go to .vim/bundle/YouCompleteMe and execute ./install.sh --clang-completer --system-libclang --system-boost. After this, you’re almost done
  7. Copy the .ycm_extra_conf.py file to your home directory and tweak it at your own discretion. I can’t help you much here since your configuration will depend on the project and the language you are using.

Enjoy!

shell

Handling rc.conf(5) with sysrc

FreeBSD uses the traditional BSD boot system as opposed to the system used in Linux, i.e SysV.

This means that in FreeBSD, we have a directory called /etc/rc.d with a bunch of scripts that bring our system up and running. Alongside these scripts, there is a file used to control some configuration variables used at boot time:/etc/rc.conf

Since it is a very sensitive file, an unfortunate mistake while editing it can break havok if our system does not boot the next time. To avoid these kind of situations, there is a tool called sysrc. sysrc provides a way to edit rc.conf(5) safely and effectively.

In order to install the tool, just type:

% portinstall sysrc

or

% cd /usr/ports/sysutils/sysrc && make install

The tool is written completely in sh so it does not need any external dependencies and it does not require compiling. With a simple invocation, it reports the current configuration of our system:

$ sysrc -a
allscreens_flags: MODE_32
background_dhclient: YES
dbus_enable: YES
devfs_system_ruleset: localrules
dumpdev: /dev/ada0p3
hald_enable: YES
hostname: hammer
ifconfig_re0: DHCP
keymap: spanish.iso.acc.kbd
linux_enable: YES
moused_enable: YES
nisdomainname: NO
powerd_enable: YES
saver: daemon
sendmail_enable: NO
sendmail_msp_queue_enable: NO
sendmail_outbound_enable: NO
sendmail_submit_enable: NO
sshd_enable: NO

Changing the value of a variable is easy (and safe):

$ sysrc linux_enable=NO
linux_enable: YES -> NO
$ sysrc linux_enable
linux_enable: NO

Once we know about the existence of this utility it becomes indispensable in our installations of FreeBSD.

Enjoy!

shell

Mirroring a site

I recently had to mirror a website to make a backup. I found this very useful (though not perfect)

wget -mk www.website.com

Hope it helps

shell

PKGng: The new package manager of FreeBSD

So what’s PKGng?

pkg is the new package manager of FreeBSD. Packages in FreeBSD used to be managed using several different commands, some of them with overlapping functionality (pkg_delete, pkg_deinstall). The whole system was functional but inefficient. It was also slow due to the underlying infrastructure. Everyone who has ever used pkg_info knows what I am talking about. The new system is much faster due in part to the use of an SQLite database.

Some time ago, the task of re-thinking the package manager was taken over by a group FreeBSD developers. Finally, with the upcoming of the 9.1-RELEASE version, pkg was included in the base system. pkg provides both a way for managing binary packages (including upgrades) and a library so it should be easy to write frontends for the system. Installation is straightforward:

[root@beastie /usr/home/fernape]# pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg please wait
Installing pkg-1.0.11... done
If you are upgrading from the old package format, first run:

  # pkg2ng
usage: pkg [-v] [-d] [-j |-c ]  []

Global options supported:
	-d             Increment debug level
	-j             Execute pkg(1) inside a jail(8)
	-c             Execute pkg(1) inside a chroot(8)
	-v             Display pkg(1) version

Commands supported:
	add            Registers a package and installs it on the system
	audit          Reports vulnerable packages
	autoremove     Removes orphan packages
	backup         Backs-up and restores the local package database
	check          Checks for missing dependencies and database consistency
	clean          Cleans old packages from the cache
	create         Creates software package distributions
	delete         Deletes packages from the database and the system
	fetch          Fetches packages from a remote repository
	help           Displays help information
	info           Displays information about installed packages
	install        Installs packages from remote package repositories
	query          Queries information about installed packages
	register       Registers a package into the local database
	remove         Deletes packages from the database and the system
	repo           Creates a package repository catalogue
	rquery         Queries information in repository catalogues
	search         Performs a search of package repository catalogues
	set            Modifies information about packages in the local database
	shell          Opens a debug shell
	shlib          Displays which packages link against a specific shared library
	stats          Displays package database statistics
	update         Updates package repository catalogues
	updating       Displays UPDATING information for a package
	upgrade        Performs upgrades of packaged software distributions
	version        Displays the versions of installed packages
	which          Displays which package installed a specific file

For more information on the different commands see 'pkg help '.

First off, we need to convert the old packages to the new format.
The pkg2ng tool performs this task:

[root@beastie /usr/home/fernape]# pkg2ng 
Creating backup pkg_info(1) database directory in /var/db/pkg.bak.
Installing GraphicsMagick-1.1.15_5,1... done
Installing ImageMagick-6.8.0.7_1... done
Installing ORBit2-2.14.19... done
Installing OpenSP-1.5.2_2... done


...
...
Conversion has been successfully finished
Your old packages database has been moved to: /var/db/pkg.bak.

In FreeBSD, everything is a package. Even if we compile our software using the ports collection, the last stage before install is to create a package for the software. In order to instruct our system to use PKGng as the new package management tool, we have to edit /etc/make.conf and add the following line:

WITH_PKGNG= yes

pkg provides a useful query system for obtaining information about packages. With it, you can use certain defined placeholders that will be substituted properly. For instance:

$ pkg query "%n was installed from %o on %t" bash
bash was installed from shells/bash on 1369848469

$ pkg query "%n dependency:\n\t%dn" bash
bash dependency:
	libiconv
bash dependency:
	gettext

$ pkg query "%n version %v takes %sh of disk space" firefox
firefox version 20.0,1 takes 55 MB of disk space

For more information, see pkg’s man page.

pkg is the present and the future of FreeBSD packages. Due to its modular design (most of the functionality is encapsulated in a library) it should be easy to write frontends for it (see this and this)

The new system is much faster than the old one and a lot of thought was put on it during development. Try it now!

Enjoy!

shell

Limit running time to a program

There is no obvious way to specify that a program should run for no longer than x seconds. The following stackoverflow question solves it in a very perlish-way. Define a helper function doalarm:

$ doalarm () { perl -e 'alarm shift; exec @ARGV' "$@"; } # define a helper function

And you are done.

$ doalarm 300 ./my_prog

will run for at most 300 seconds.

shell

Spell checking

aspell is a command-line spell checker. It is a replacement for the older ispell. It can be used to manage dictionaries, to check a complete file, or words typed in your terminal, among other uses.

We can invoke aspell this way:

$ aspell -a --lang=en
@(#) International Ispell Version 3.1.20 (but really Aspell 0.60.3)

This leaves us with a prompt in which we can type a word. For instance, if we type:

@(#) International Ispell Version 3.1.20 (but really Aspell 0.60.3)
hello
*

The asterisk indicates that aspell found the word in the dictionary. On the contrary, if we type:

@(#) International Ispell Version 3.1.20 (but really Aspell 0.60.3)
rescoe
& rescoe 15 0: Roscoe, rescue, rescuer, Rosco, resow, resole, rescale, rescued, rescues, fresco, recce, Reece, rescan, Roscoe's, Rosco's

We are presented with a list of options.

One of the nice features of aspell is that it can check a complete file. This way, it interactively asks the user to take an action for every misspelled word. For example:

$ cat test.txt
Simple text to check
the use of aspell from inside Vim
The following werd was misspelled

If we ran:

$ aspell --lang=en -c test.txt

we are presented with the following screen:


Simple text to check
the use of aspell from inside Vim
The following werd was misspelled


1) Aspell                                                                        6) Aspell's
2) asp ell                                                                       7) Ascella
3) asp-ell                                                                       8) spill
4) Ispell                                                                        9) Ispell's
5) spell
i) Ignore                                                                        I) Ignore all
r) Replace                                                                       R) Replace all
a) Add                                                                           l) Add Lower
b) Abort                                                                         x) Exit


? 

This is very interesting as we can edit a file and check the spelling, but it would be nicer if we could spell check our text directly from our favorite editor…

You saw this coming, didn’t you? :)

Spell checking from inside Vim

While editing the same test.txt file with Vim, we can type the following:

:!aspell --lang=en -c %

This opens the aspell prompt which enables us to correct the misspelled words. Once we are done, Vim notifies us of the modification of the file:

W11: Warning: File "test.txt" has changed since editing started
See ":help W11" for more info.
[O]K, (L)oad File: 

Typing L returns to Vim with the corrected file.

If we want to check a single word, we can type the following:

:!echo <cword> \| aspell pipe --lang=en<CR>

<cword> expands to the word under the cursor. Then we pipe this word to aspell for spell checking. This is somehow long to write, so it could be a good idea to define a macro:

:map <F4> <Esc>:!echo <cword> \| aspell pipe --lang=en<CR>

Enjoy!

automated, scripts, shell

Use of find with regular expressions

I found myself needing to write a clean: rule in a makefile which would wipe out all the auxiliary files generated by latex (among other things). Moreover, the project has some subdirectories in which there might be more of those files.
Apart from finding files by date, size, modification time… the find utility can use regular expressions. I am not delving a lot into this (what kind of expressions, etc.), I just wanted to point out two things:

  • That it comes in quite handy.
  • That the regex must match the complete file name as reported by find.

The second item is worth noticing. If you need to find, say all the files in this directory structure having an x, you cannot use

$ find . -regex 'x'

which, if it were a perl regex would match anything containing an x (assuming the quotation marks stand for the regex delimiter). What you need is to match the whole string, taking into account that any find in . starts with './'. Hence, find the files in this directory structure ‘having an x in its name’ is written

$ find . -regex './.*x.*'

which means exactly: find all the files matching ‘start with dot-slash, then anything, an x and anything’, where anything can be of length 0.

The clean: in my makefile reads now:

clean:
    find -E . -regex "\./.*.(aux|log|blg|bbl|~)" -exec rm -f '{}' \;

The -E option means ‘use modern regular expressions’, which are the ones everybody uses nowadays. I prefer using ‘\.‘ for “real dots,” it looks cleaner to me.

It has been quite a while, has it not?

shell

When you think everything is lost…

I have seen people many times doing something like this:

$cat binary_file


The problem with the above is that usually you end up with a prompt that looks like this:

�g`�g���i`�i��k`�kX@l`(l  �


Whatever you type, you get garbage in your terminal. In this case, what most people do is to despair and close the terminal, losing the information contained in it.

reset is a command that gets the terminal back to its normal state. Sometimes everything is so mangled that there is no echo from the terminal. Don’t panic, just type reset and press ENTER.

If ENTER does not work, try to replace it by Ctrl-J

If none of the above works… you can fallback to your initial idea and close the terminal.

Enjoy!

shell

utf-8 blues in snow leopard with mutt and vim

Yes, it looks like a true spam page. But it is the only way to describe my problem. I was trying to set up mutt (yes, I am a bit fed up with Mail.app and its inability to be controlled with the keyboard alone in any sensible way). Terminal.app seems to work OK with utf-8. However, the included vim does not. Any time you type an accented char and delete it, the text gets mangled (you know, typical off-by-one cursor position). The solution for vim is incredibly stupid: open up vim and write :set encoding=utf-8 (or do that at any time while in command mode ESC).

For mutt, things are more or less according to the manual, but you must know where to look for. I ended up with the following two lines in my .muttrc:

set &charset ?charset
set charset="utf-8"

but I am uncertain whether the first line of the two is necessary.

YMMV, but that works for me.

shell

Make colored output

I found a nice script in stackoverflow about how to color the output of make in bash. It works by filtering the output and looking for certain expressions to highlight.

Placing this function definition in your .bashrc file defines a bash make function which gets executed instead of the command. This way we can shadow the real make command and create our own custom version.

make()
{
   ccred=$(echo -e "33[1;31m")
   ccyellow=$(echo -e "33[1;33m")
   ccend=$(echo -e "33[0m")
   /usr/bin/make "$@" 2>&1 | sed -e "s/[Ee]rror:/$ccred&$ccend/g" -e "s/[Ww]arning:/$ccyellow&$ccend/g"
   return ${PIPESTATUS[0]}
}

Of course, you can use this technique to wrap other commands as well.

Enjoy!