Monday, February 16, 2009

Introduction to Vim for Programmers

Vim editor:

It has 3 modes:

Ex mode <----> Command mode <-----> Insert mode.

You can never go from Insert to Ex mode directly or vice versa. You always
first need to go to command mode. Enter command mode by typing Esc and enter Ex mode by typing :.

For cursor movement:

k
h l
j
Cursor movement commands: {motion} commands:
G - Last line in file
gg - First line in file
% - Jump to matching brace (()}{[])
m{1etter} - example ma to mark a line. There can be at
max 26 marks in the file.
`{mark} - example `a to jump to the named mark.
`` - Jump to the last line where you jumped or searched from.
/expr - forward search where expr = is a regex.
?expr - backward search.
n - repeat last search in the same direction.
N - repeat last search in the opposite direction.
w - move cursor forward by one word.
b - move cursor backward by one word.
Some more cursor movement commands:
Cntl + F - move one screen forward.
Cntl + B - move one screen backward.
f{char} - position cursor at the matching {char} to the right.
t{char} - position cursor just before the matching {char} to the right.
F{char} - position cursor at the matching {char} to the left.
T{char} - position cursor just before the matching {char} to the left.


Basic insert commands:
i - enter insert mode.
a - enter insert mode after character under cursor.
I - enter insert mode starting before first non blank character on the line.
A - enter insert mode starting after last non blank character on the line.
o - enter insert mode starting on a new line after current cursor line.
O - enter insert mode starting on a new line before current cursor line.


Change commands:
C - delete everything from cursor to end of line.
c{motion} - delete everything from cursor position as determined by the motion (one of the cursor movement commands).
Example:
c5{h}
- deletes 5 chars to the left (as h is for
left movement).
c2w - deletes 2 words and puts in insert mode.
S - Substitute/change current line entirely. Delete all text on current line and put in insert mode.


Patterns for using change commands:
2cw - change 2 words.
cta - delete everything upto next occurrence of letter a.
5cta - delete everything upto 5th occurrence of letter a.
5S - delete current line and subsequent 4 lines.


Miscellaneous commands:
u - undo
Cntl + R - redo
Cntl + N/Cntl + P - autocompletion with text that already exists in the in the document.
. - repeat the last change.
zt - redraw the screen placing current line to top of screen.
zz - redraw the screen placing current line to the center of screen.
zb - redraw the screen placing current line to the bottom of screen.

Delete, cut and paste commands:
D - Delete from current cursor position to the end of line.
d{motion} - delete from current curson position to the position determined by {motion} command. Example,
d5w - delete next 5 words.
dn - delete till next matching search pattern.
d`a - delete till marked line named a.
y{motion} - yank or copy from current cursor position to the position
determined by {motion} command. Example,
y5w - copy next 5 words.
yG - copy till the end of file.
p - put the character in registry/clipboard after current character under cursor.
P - put the character in registry/clipboard before current character under cursor.
dfa - delete until character first character a from current cursor position.
dta - delete upto character first character a from current cursor position.
yy4p - copy current line and paste it 4 times.
dG - delete everything from current line till end of file.
y50G - copy everything from current line till line 50.
d5l - delete 5 characters to the right (l is for right motion).
5dd - delete 5 lines including current one.


Ex mode:
Ex mode commands are of the form:
:[firstline][,lastline]command


Example:
:25,30d - delete lines from 25 to 30.
Line numbers are optional.
If no line number is specified, command applies to current line.
If only one line number is specified, command applies to that line.
To affect a range of lines, we need to specify both firstline and lastline.

Shortcuts for specifying line numbers in Ex mode:
. - current line number. Example- :.,50d - delete from current to line 50.
$ - last line number in file.
% - entire file - same as specifying 1,$.
`a - line number of bookmarked line a.
/expr/ - next line that matches expr.
?expr? - previous line that matches expr.
\/ - next line that matches the most recent expr.
\? - prev line that matches the most recent expr.
\& - next line that matches the most recent substitution.


Examples:
:/Begin/,/End/d - deletes lines from first line that has word Begin till
last line which has word End.

10,40w bkup.dat - saves the lines 10 to 40 in a file bkup.dat.

Essential Ex Commands:
:w {filename} - save the current buffer to filename.
:q - quite the current buffer.
:x - same as :wq - quite and save unsaved data.
:e {filename} - open named file for editing in a new buffer.
:d - delete current line or range of lines
:map {a} {b} - remap the keys used in command mode.
:set {arguments} - configure vim (mostly to be used in .vimrc)
:help - vim help.


Search and Replace:
:s/search/replacement/flags


If search or replacement string contains / character then either we can
escape it with a \ or use a different delimiter (like #) other than / for
eg:
:s#/usr/share/file1#/usr/share/file2/#


Usage examples:
  1. first occurrence single line :s/bash/perl/
  2. all occurrences single line :s/bash/perl/g
  3. first occurrence between line range: :23,100s/bash/perl/
  4. all occurrences between line range: :23,100s/bash/perl/g
  5. first occurrence in whole text: :%s/bash/perl/
  6. all occurrences whole text: :%s/bash/perl/g
Code browsing:
Run ctags as follows in the source directory and it generates an index file
called tags of source files in current directory and its subdirectories.

ctags -R

Now in vim we can use each source file as a web page with each symbol
(variable, function) having hyperlinks, such that, if you do a cntl + ] on
the current symbol then it will take you to the definition of the symbol.

Following are commands for browsing code in vim:
Cntl + ] - jump to the tag under cursor (ie go to definition of function,
class, declaration of variable).
Cntl + T - Return to current tag from most recent jump off point.
:pop - Same as Cntl + T except that we can specify count to go back
multiple levels.
:tn - Jump to next match (when tag produces more than one match in
case of C++ overloaded functions).
:tp - Jump to previous match (in case of multiple matches).
:ts name - Shows a list of matching tags you can select from.
:tags - Show current tag stack with one line for each tag.

Lastly, to build c/c++ code:
:make {args} - invokes the make program. We can change the make program in to say ant by using Ex mode (or in .vimrc) as
:set makeprg=ant

One nice .vimrc can be found at:
http://research.iiit.ac.in/~masatran/config/.vimrc

Use
wget http://research.iiit.ac.in/~masatran/config/.vimrc
to get it.


Sunday, February 15, 2009

A short introduction to GNU make

A make rule looks like the following:

target: prerequisite
commands

Example:

.PHONY: all
all: program1 program2
program1: a.o b.o
gcc -o program1 a.o b.o
program2: c.o d.o
gcc -o program2 c.o d.o


This tells make not to search for a file named all and to assume that this target
is always obsolete.


Defining variables:

Variables in a make file are like in an ant build script but they
can only be set once unlike ant properties and dont change value.

FLAGS = first
all:
@echo FLAGS=$(FLAGS)

FLAGS = second
other:
@echo FLAGS=$(FLAGS)


The FLAGS variable is assigned the last value which is set in the build file
viz. second and so even if we run all target, the value echoed for FLAGS
will be second.
* So, a variable should be set just once in the make file. *

VAR = value
VAR := value


The := form allows variables to reference themselves without recursion.

VAR = value
# Wrong! Causes infinite recursion
VAR = $(VAR) more

# Okay, the := prevents recursion
VAR := $(VAR) more


# A GNU extension that does the same thing
VAR = value
VAR += more

make automatically removes leading and trailing white space from your
variable values when you use the traditional syntax.

TRAD= Hello World
all:
@echo "$(TRAD)" -- removes the spaces in the o/p


If we want to preserve the spaces:

TRAD=$() Hello World $()


we can use the built-in empty variables $() to mark the beginning and end.

GNU make comes with many implicit rules to do almost everything you need.


.c.o:
$(COMPILE.c) $(OUTPUT_OPTION) $<

where,
COMPILE.c = $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c
CC = gcc

This says, “If you see a target with an .o extension, and there is no explicit
rulefor it, look for a file with the same base and the .c extension. If you find
it, run these commands.”

We can checkout such implicit rules and built-in variables by:
make -p

CFLAGS and others are not defined but make allows referencing not defined
variables and will replace them with empty strings.

We can set CFLAGS= -g to enable debugging.

Common variables used in implicit rules:
CC = gcc - C Compiler
CXX = g++ - C++ compiler.
CFLAGS = none - flags passed to C compiler.
CXXFLAGS = none - flags passed to C++ compiler.
CPPFLAGS = none - flags passed to C Pre Processor. Typical flags include -I, -D and -U.



Default suffix rules to create object code:
.c = $(CC) -c $(CPPFLAGS) $(CFLAGS)
.cpp, .cc, .C = $(CXX) -c $(CPPFLAGS) $(CXXFLAGS)
.s = $(AS) $(ASFLAGS) -- for assembley code.


Automatic Variables:
Their values can change in the context they are used. Regular variables can
also contain automatic variables.

Example:
build_C_program=$(CC) -o $@ $^

program1: mod1.o mod2.o
$(build_C_program)
program2: mod3.o mod4.o
$(build_C_program)


Note: implicit rule will be used to create the pre-requisites modx.o:

.c.o:
$(COMPILE.c) $(OUTPUT_OPTION) $<


then modx.o will be fed to the build_C_program to generate the programx
executables.

Useful automatic variables in GNU make:
  1. $@ = target filename
  2. $^ = all pre-requisites with duplicate names removed.
  3. $+ = all pre-requisites with duplicate names.
  4. $< = name of first pre-requisite in the rule.
  5. $? = name of pre-requisite newer than target.
  6. $* = base name of target file (without extension).


Manipulating variables with functions:

CSRCS=foo.c
CXXSRCS=foo.cpp
OBJS=$(CSRCS:.c=.o) $(CXXSRCS:.cpp=.o)


or use:
OBJS:=$(addsuffix .o, $(basename $(SRCS)))


shell function:
CURRENT_TIME=$(shell date +%T)
something:
@echo started $(CURRENT_TIME)


Other functions provided by GNU make are:
  • $(subst from, to, text)
  • $(patsubst from-pattern, to-pattern, filenames)
  • $(strip string) -- removes the whitespace chars.
  • $(findstring match, string)
  • $(filter patterns, filenames) -- returns files that match one or more space
  • separated patterns
  • $(sort text) - lexical order sort
  • $(word n, text) - nth word in text
  • $(error message) - fails the build after printing the message.


User can define custom functions which can be called as:
myfunction = @echo $(1) $(2)
all:
$(call myfunction,hello,world)


Conditionals:
ifeq, ifneq
ifdef, ifndef

ifeq ($(shell uname -o),GNU/Linux)
CPPFLAGS += -DLINUX
endif


Note: The text that appears inside the conditional clause may be any valid make lines, but conditionals may not be used inside the command section of rules or inside variable/function definitions.

To ensure that A_VITAL_VARIABLE is defined or else the build has to fail:

ifndef (A_VITAL_VARIABLE)
$(error A_VITAL_VARIABLE is not set)
endif
Generally, projects will have a Makefile for each project source directory.

A sample makefile:
The below makefile resides in the bin/ directory and compiles C and C++ sources in src/ directory. The executables are created in bin/ directory.
.PHONY: all


VPATH=../src
CFLAGS += -Wall -Wformat-nonliteral -Wformat-y2k
build_C_program=$(CC) $(CFLAGS) -Wstrict-prototypes -o test $^
build_CXX_program=$(CXX) $(CXXFLAGS) -Weffc++ -o hello $^
CSRCS=test.c
CXXSRCS=hello.cpp
COBJS=$(CSRCS: .c=.o)
CPPOBJS=$(CXXSRCS: .cpp=.o)
CURRENT_TIME=$(shell date +%T)

ifdef (debug)
CFLAGS += -g
CXXFLAGS += -g
else
CFLAGS += -O2
CXXFLAGS += -O2
endif


all: empty.so ccode cppcode
@echo started $(CURRENT_TIME);

ccode: $(COBJS)
@echo Building C code; \
$(build_C_program)

cppcode: $(CPPOBJS)
@echo Building CPP code;\
$(build_CXX_program)

empty.so: empty.c
$(CC) -shared -fpic -o $@ $^

run: all
LD_LIBRARY_PATH=../src ./test

clean:
rm -f empty.so $(COBJS) $(CPPOBJS)

cleanall: clean
rm -f $(basename $(COBJS) $(CPPOBJS))

Friday, February 13, 2009

VNCServer Configuration

In the ~/.vnc/xstartup we can configure vnc session to start in gnome or kde, as shown below:
#!/bin/sh
# Uncomment the following two lines for normal desktop:
# unset SESSION_MANAGER
# exec /etc/X11/xinit/xinitrc

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &            
/usr/dt/bin/dtwm &
#startkde &

unset DBUS_SESSION_BUS_ADDRESS; gnome-session &
Then start vnc server as:
$ vncserver 
This will tell you the hostname:sessionnum and set the password:

$ vncpasswd
Optionally you can change the screen size for your session as:
vncserver :displaynum -geometry 1920x1080
Now you are all set to connect with your favorite vnc client (i use ultravnc).

For XFCE desktop change the xstartup file to:
#!/bin/sh
# Uncomment the following two lines for normal desktop:
 unset SESSION_MANAGER
 exec /etc/X11/xinit/xinitrc

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &             

x-window-manager &
startxface4 &
exec /usr/bin/xfce4-session

Monday, February 09, 2009

Port Forwarding : Accessing your home computer from work

In order to access the home ubuntu desktop from work, following is what i needed to do:
  1. Installed open-ssh demon on ubuntu with: sudo apt-get install open-ssh
  2. Tested that i could use a windows box on the home lan to access the sshd using putty.
  3. Once sshd is setup, followed the instructions at http://www.portforward.com/help/pfprogression.htm, which is broadly the following:
  • First we need to setup static/manual IP on the host computer to which we want to forward the port(s) to. Mine was setup to obtain IP address using DHCP so i first changed that to manual.
  • Then login to the router (mine is an Air-link AR670W) and enable Port Forwarding for the SSH application port 22 for both TCP/UDP protocols and provided the static IP of my ubuntu desktop.
  • Saved the configuration and router reboots after that.


Then i noted down the ISP assigned public IP to my router's external interface and used that IP (external IP of my router) from a windows box to putty to. Since the SSH port 22 on router is configured to be forwarded to my ubuntu desktop, so i was allowed my way to the ubuntu desktop.

http://www.portforward.com/routers.htm has a listing of several routers and the documentation on how to configure that router for port forwarding.

Also checkout how to setup a easy to remember domain name for your router so that you dont have to remember your router IP (http://www.techthrob.com/tech/dyndns.php). This way even if your router is assigned a new external IP by your ISP, the ddclient demon will automatically update the mapping of your domain id to the new IP. The dyndns.com site has a script generator which can generate the ddclient.conf file for you. Use the generator and it will work without fail. You can find it here.

To configure ubuntu 8.10 for static IP, just use the System > Preferences > Network Configuration UI and select Manual for Method in IPv4 Settings tab and provide your static IP address (say 192.168.1.100), the netmask and the gateway (router ip - like 192.168.1.1). Also you may provide DNS servers (check your router's status page for knowing what DNS servers to use - it will automatically have got it from your ISP).

Popular micro services patterns

Here are some popular Microservice design patterns that a programmer should know: Service Registry  pattern provides a  central location  fo...