Monday, April 27, 2020

Setting up VSCode for Go Programming

  • Idea behind Golang is to minimize the number of lines of code.
  • Primary use of Go is to write server-side programs or command line programs.
  • Creators of Golang - Ken Thomson, Rob Pike and Robert Griesemer
  • Modern standard library
  • Compile to single native binary - no runtime required.
  • Has garbage collection built in
  • Simple, strongly-typed, procedural language with object oriented features - very few language features have been added since 1.0 release
  • Built-in concurrency
  • Fast to compile and run quickly
  • Some popular Go projects:
    • K8s
    • Docker
    • Prometheus
To setup an IDE for Go - Visual Studio Code is a good choice.
  • Setup VS Code for Go development by installing the “Go” Extension from Microsoft
  • Also install the Go packages by:
  • Cntl + Shift+ P or View > Command Pallet
  • Enter: Go: Install/Update Tools
  • Select All and Click OK.
  • Go to Settings for Go Extensions - Look up “Infer GoPath” setting and enable/check it.
Now you should be able to use the IDE to edit and compile/run go code.
Also watch this screencast https://www.youtube.com/watch?v=XCsL89YtqCs on writing, building, installing and testing Go code.
Set GOPATH environment variable to a directory where your go code will reside.
For example: ~/go directory.
export GOPATH=~/go
Under ~/go create 3 directories that go requires:
src, pkg and bin
At any point look for the go documentation as:
> go doc fmt.Println

Following is a hello world program that will go in ~/go/src directory.

package main
import "fmt"
func main() {
    fmt.Println("Hello, World")
}

In VS Code launch the Terminal (Cntl + Shift + `) or Terminal > New Terminal.
To build the program:
> go build hello.go
> ./hello
To directly run the program and not build it first:
> go run hello.go

Tuesday, April 21, 2020

Creating a docker swarm cluster with virtualbox

On the host system:

1. Install virtualbox:
sudo apt install virtualbox

2. Install docker:
sudo apt install docker.io

Add user to docker group:
sudo groupadd docker
sudo usermod -aG docker $USER

Either manually create 3 VMs (1 manager and 2 workers) or use docker-machine.

Note: With Secure UEFI boot enabled in BIOS installation of virtualbox requires us to generate a MOD key protected by password. When installation completes we are required to reboot and then enroll the MOD key by entering the same passphrase for the key.


Approach 1 - Manually create 3 VMs  (1 manager and 2 workers) - Recommended
Networking for the 3 VMs should meet the following requirements:
1. We should be able to ssh to each of the ubuntu VMs from host system.
2. We should be able to access internet from ubuntu VMs. This is required for the docker registry access.
3. Lastly, each VM should be able to reach the IP address of other nodes in the cluster.

If installing ubuntu VMs also install docker -- this can be selected during OS install .. but you will still need to add your user to docker group post installation.

To setup networking for the VMs:

1. Virtual Box > Host Network Manager > create an adapter (it will be named vboxnet0 by default) say with the following config:
DHCP - Enabled (though this is not best way but generally the IP allocated to a VM wont change and it saves the effort of manually configuring the static ips on each VM)

Refer - https://www.tecmint.com/network-between-guest-vm-and-host-virtualbox/

Configure Adapter Manually :
IPv4 address: 192.168.56.1 -- this becomes the gateway address
IPv4 Network Mask: 255.255.255.0
Leave the IPv6 settings to default

On DHCP tab:
Server Address: 192.168.56.100 -- the DHCP server
Server Mask: 255.255.255.0
Lower Address Bound: 192.168.56.101 -- first VM's ip address can begin with this.
Upper Address Bound: 192.168.56.254

2.  Select each ubuntu VM > Settings > Network > create 2 Network Adapters.
Network Adapter 1 ->
  • Check the option: “Enable Network Adapter” to turn it on.
  • In the field Attached to: select Host-only Adapter
  • Then select the Name of the network: vboxnet0
Network Adapter 2:
  • Check the option: “Enable Network Adapter” to activate it.
  • In the field Attached to: select NAT
For ease of being able to ssh to VMs by name:
Add the following entries to /etc/hosts (change to your preferred names) on the host system:

192.168.56.101  manager
192.168.56.102  worker1
192.168.56.103  worker2

Also to enable ssh without password:
create ~/.ssh directory and a file under it called authorized_keys
mkdir -p ~/.ssh
echo "[public key]" >> ~/.ssh/authorized_keys

Now you can ssh to the VMs without having to type in the password. For e.g.:
ssh [user]@manager 

Once on the VM, try doing nslookup www.google.com and verify the internet connection is working.
Also verify you are able to ping the other 2 node ips.

Approach 2 - Use docker-machine
To create via docker-machine use the command though i prefer the above approach as it gives more flexibility to choose our preferred OS version to install instead of having to live with the version that comes with docker-machine. Also with docker machine approach you end up running one additional VM which runs docker. On a Linux host we dont need that as docker engine can run natively on the host system rather than requiring a separate docker host VM.

docker-machine create -d virtualbox default

The above command will create a VM with name "default" that will be used as host for running the docker engine and additionally will create 3 nodes for swarm cluster.

To Setup Swarm mode:
On Manager node:

$ docker swarm init --advertise-addr 192.168.56.101:2377 --listen-addr 192.168.56.101:237

where, 192.168.56.101 is this manager node's ip address.

We only have one manager which also is the leader. Docker swarm supports having more than one manager nodes but there can be only one leader among them at any given time. We run the swarm init command on the leader node and all other manager nodes will then join the swarm cluster using the manager token.

On 2 Workers:
Run swarm join command for worker nodes to join the swarm cluster.

$ docker swarm join --token SWMTKN-1-4azauf7ujxp711zzwb1ihfluqjy5om6pa4zlicmm7pq3l0svcp-d2bgy5mtsl0w82n99suvb1uu6 192.168.56.101:2377 --advertise-addr 192.168.56.102:2377 --listen-addr 192.168.56.102:2377

The above command will be given in o/p of swarm init.

To get the above commands again anytime following commands can be used on the manager node to get it:

docker swarm join-token manager
docker swarm join-token worker

Running docker info will show there is 1 manager and 3 nodes in swarm cluster and that swarm is active. It will also show if the current node is a manager or not.

To deploy a test app (an example from nigel poulton's course on pluralsight.com).
docker service create --name psight1 -p 8080:8080 --replicas 5 nigelpoulton/pluralsight-docker-ci

docker service ps psight1

The above command will show which node is each replica running on.

$ docker service ps psight1

ID                  NAME                IMAGE                                       NODE                DESIRED STATE       CURRENT STATE            ERROR               PORTS
jk55y1btc0mo        psight1.1           nigelpoulton/pluralsight-docker-ci:latest   worker2             Running             Running 40 seconds ago                       
3tlq4zb5s78m        psight1.2           nigelpoulton/pluralsight-docker-ci:latest   worker1             Running             Running 40 seconds ago                       
tonkv7bmjei5        psight1.3           nigelpoulton/pluralsight-docker-ci:latest   worker2             Running             Running 41 seconds ago                       
znoek7dwq6x5        psight1.4           nigelpoulton/pluralsight-docker-ci:latest   manager             Running             Running 41 seconds ago                       
teg2amzw2f73        psight1.5           nigelpoulton/pluralsight-docker-ci:latest   worker1             Running             Running 40 seconds ago                     

That's it! Now we have a working swarm cluster that we can use to deploy apps on.

Using pyenv for managing multiple python versions

Reference: https://realpython.com/intro-to-pyenv

Working on projects that require different python versions... pyenv can help manage the multiple python versions for each project by installing multiple python versions locally or even makes working with virtualenv easier (no need to activate and deactivate every time when we switch into and out of a project).

Install pyenv on ubuntu:

$ curl https://pyenv.run | bash

Add these lines to .bashrc or .zshrc

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

Now restart your current shell:

$ exec $SHELL

Using pyenv:

pyenv installs all python versions under ~/.pyenv.

$ pyenv install --list | grep " 3\.[678]"

$ pyenv install -v 3.8.2

The above command will install python version 3.8.2

➜  ~ pyenv versions                       
* system (set by /home/rwatsh/.pyenv/version)
  3.8.2
➜  ~ pyenv which python
/usr/bin/python

As seen above, currently the system's python is active.
To change to the new python we installed:

➜  ~ pyenv global 3.8.2

➜  ~ pyenv which python
/home/rwatsh/.pyenv/versions/3.8.2/bin/python

➜  ~ pyenv versions 
  system
* 3.8.2 (set by /home/rwatsh/.pyenv/version)

➜  ~ python3 -V     
Python 3.8.2


The pyenv which python command shows that python 3.8.2 is installed under pyenv root directory ($HOME/.pyenv). Pyenv builds python from source and installs it under its root directory. 

Creating virtualenv via pyenv:

➜  pyenv virtualenv 3.8.2 myproj
pyenv-virtualenv: `/home/rwatsh/.pyenv/versions/myproj' already exists.

Then in your project directory run:

➜  pyenv local myproj           
(myproj) ➜  myproj 

This will use the virtualenv under your local project directory. It creates a .python-version file to track the virtualenv being used in the project directory.

(myproj) ➜  cat .python-version 
myproj

(myproj) ➜  pyenv which python 
/home/rwatsh/.pyenv/versions/myproj/bin/python


Since we added eval "$(pyenv virtualenv-init -)" to run in your shell virtualenv associated with the project directory will automatically be activated when we cd to that directory and deactivated when we move out of it.

We can also manually activate or deactivate as:
$ pyenv activate 
$ pyenv deactivate

Thus, with pyenv we can:

  • Install multiple versions of Python
  • Switch between the installed versions
  • Use virtual environments with pyenv
  • Activate different Python versions and virtual environments automatically

Sunday, April 19, 2020

Using Sublime Text 3


I had purchased the license of Sublime Text 3 couple years ago when i was using it on my Mac. But there were other options available on Mac then and for whatever reason i did not end up using Sublime Text on Mac as much as i had expected to. The good thing about this editor is it is cross-platform and so is its license (which was a pleasant surprise). When i moved to using Ubuntu 19.10 last year i could get all my programming IDEs on it but i still needed some light-weight text editor for taking quick notes and writing quick programs and scripts. I tried out Sublime Text 3 and my old license that was not getting any use and it worked like a charm. I can now use my Sublime Text for that purpose and it is serving me well.
I like this cheat sheet (which by the way is the first one a google search for a cheat sheet for Sublime Text will return) - https://www.makeuseof.com/tag/sublime-text-keyboard-shortcuts-cheat-sheet/

Popular micro services patterns

Here are some popular Microservice design patterns that a programmer should know: Service Registry  pattern provides a  central location  fo...