MongoDB for Java Developer Course

I successfully completed MongoDB for Java Developer course on 12/09/2015.

The course completion certificate is at: http://university.mongodb.com/course_completion/24540ba216594c40ac7f3686d79a1046
(this pdf file will be stored in MongoDB GridFS).

I obtained 77% grade and grade for being certified was 65%.

The course spanned over 3 months from October to December 2015 and following was the syllabus:

Week1 - Introduction - Overview, Design Goals, the Mongo Shell, JSON Intro, Installing Tools, Overview of Blog Project. Maven, Spark and Freemarker Intro
Week 2 - CRUD - Mongo Shell, Query Operators, Update Operators and a Few Commands
Week 3 - Schema Design - Patterns, Case Studies & Tradeoffs
Week 4 - Using Indexes, Monitoring And Understanding Performance. Performance In Sharded Environments
Week 5 - Aggregation Framework - Goals, The Use Of The Pipeline, Comparison With SQL Facilities.
Week 6 - Application Engineering - Drivers, Impact Of Replication And Sharding On Design And Development.
Week 7 - Case Studies - Interview with Jon Hoffman, foursquare and Interview with Ryan Bubunksi, codecademy
Week 8 - Final Exam 

I am glad to have invested some time to do this nicely made course by MongoDB and it is a good tool to have under your belt as a developer. I did 3 projects of my MSSE 1st semester using MongoDB and hosted it on a virtual box VM, docker container, AWS EC2 instance and of course my Mac and it was always an easy to install, very reliable DB. The programming interface using Morphia (document object mapper) is very intuitive and simple. I did struggle a bit to keep the entity objects same for both backend modules (REST endpoint and DAO and between REST endpoint and UI) because of the _id being a BSON type but later found a simpler solution (see my Morphia and Dropwizard intergration post). Doing this course gave me good confidence in the use of the DB for my projects.

Understanding Docker

LXC (Linux Container)

2 unique traits of containers are:

1. Namespace - is for: ls -al /proc/1341/ns - isilates
1. Filesystem and mount points (mnt)
2. Process space (pid)
3. Network (net)
1. Network devices
2. Routes
3. Firewall Rules
4. Hostname (uts)
5. Inter process communication (ipc)

2. Control Groups - cat /proc/cgroups
1. Manages what portion of the resources each container can consume
1. Certain cpus
2. Etc - check notes

/container --- is where the containers live.

Under each container:

1. Separate namespace for containers - Rootfs - represents the / (root) file system
1. Which has /bin, /boot etc.
2. /container/mycontainer1/rootfs/tmp/newfile - is seen under container mycontainer1 as /tmp/newfile
3. So host can copy a file between 2 containers.
4. Within a container (once logged in) we can only go upto / and cannot cd .. Beyond it. So one container cannot see files of another container.
5. Separate namespaces for processes -
1. Each lxc-start process will spawn lxc-init process within container and it will have pid=1 within the container but outside on the host it will not.
6. Network namespace -

7. System or OS containers vs Application containers:
1. OS/System containers - They have full OS and close to VM. Generally long running containers. (lxc-start -n o17cont1
2. App containers - they are temporary, execute one command or process and dies, they are given access to parts of the host's root file system. Similar to JVM sandbox, OSGI etc. (lxc-execute 0n myappcont -- ps -A) after executing the specified command or app  it will come back to
8. If container messes up the kernel it can impact the host OS but in VM the kernel is not impacted.

Docker - packages all dependencies required by an application and packages it together and puts it into a container.

1. Docker engine - uses docker containers (similar to LXC application containers)
2. Docker hub registry - SaaS operated by Docker Inc, contains docker images, users must register, archives of hundreds of ready made container images and community can upload their images.

Busybox - is most useful unix commands all packaged as one application so instead of ps you need to type busybox ps.

Docker - install on oracle linux - https://docs.oracle.com/cd/E37670_01/E37355/html/section_kfy_f2z_fp.html

Magnum - container as a service (openstack)

Adv of container -

1. Quick startup
2. Deterministic packaging - on any target host
3. Portability - just like JVM (all application which work within docker can be installed on any OS with docker container support)
4. Much smaller than VM image

Disadv of container -

1. Less secure than VM
2. Less isolation

1. Concept: Docker uses lxc (linux containers) and so it is native to linux. On PC or MAC, we need to install docker toolbox which uses Oracle Virtualbox to install a small Linux VM and uses that to create docker containers.
2. PODA - Package once deploy anywhere.
3. Docker helps in packaging an application and all its dependencies into one image. The image creation is defined in a dockerfile (http://docs.docker.com/engine/reference/builder/ ). Docker can parse this file and build an image. This file can include other existing images. Docker will first try to find the images locally and if not found then it searches in the remote docker registry.
4. Docker has 3 main stages:
1. Build  - builds a dockerfile to create an image (images are all versioned and have a SHA id like git commit id).
2. Run - runs an image to create a container
3. Ship - posts the image to a repository (push/pull).
5. Important to understand is - that docker container should run only one service. Anytime we have multiple JVM processes being started within our application, think of separating those into multiple containers so each process has its own container. The containers can then be linked.
6. Docker composer - is a tool to build applications comprising several containers. A typical example is:
1. Run 4 instances of mongodb containers - say where 3 are part of a replica set and 1 is to run the mongos router.
2. Run 2 instances of application server - say weblogic or wildfly.
3. Run 1 instance of apache http proxy load balancer fronting the application servers.
4. Run 2 instances of web server hosting the web UI.
5. Run 1 instance of apache http proxy load balancer fronting the web server.
7. The above set of containers can be defined within a docker composer yaml file or a set of yaml definition files. SO when we say docker-composer up -d it will create the required images and run the containers.

8. Docker hub:

You can login using:

$ docker login

User: xyz

Password: abc

Email: myself@me.com

9. Install docker toolbox (for Mac or PC only). It installs docker-machine which can be used to create a TinyCore Linux VM where docker daemon will listen on port 2375 and provide REST service and the docker client will be a REST cli client to the docker daemon process in the docker host VM. Docker toolbox uses Oracle VirtualBox to create docker host linux VM.
10. Docker commands:
1. Dokcer build --tag   sjsucohort6/esp . --- will build an image.
2. Docker push  [image]-- can push the image to docker hub.
3. Docker pull  [image]-- image is analogous to a commit id
4. Before issuing docker pull you must be logged in to docker hub (same as github).
5. Docker pull will get the [image]:latest (the latest version)

11. Open a shell and type:
1. Docker-machine create --driver virtualbox --virtualbox-memory 8096 default -> this will create a Linux VM named default using virtualbox. On MAC, the VM is created in ~/.docker/machine/machines/default.
2. Once the VM is created we can ssh to it by name (default in this case).
3. Docker-machine ls -> should show the new machine as running
4. Docker-machine env default -> will give the environment settings to export
5. Eval "$(docker-machine env default)"
12. Docker-machine ssh default --> this will log you in to the docker host VM.

Docker pull mongo

docker run -d --name mongodb mongo -- starts mongodb

Docker ps

$ docker run -it --link mongodb:mongo --name mongoclient mongo sh -- create a new container mongoclient to run mongo client. The above will use mongo image to launch a client but not run mongodb instead will bring the shell prompt and now we can connect to the mongo host.

# cat /etc/hosts

# mongo --host=mongo

$ docker build -f ./Dockerfile --tag sjsucohort6/esp .

$ docker run -it --link mongodb:mongo --name studentreg sjsucohort6/esp

When we create a link this way, docker will inject environment variables named MONGO_PORT_27017_TCP_ADDR (will give the source container's ip address) and MONGO_PORT_27017_TCP_PORT (will give the port exposed by the container) into the container so the application can use these to connect to the DB service in the parent container.

In Docker 1.9.0 and later, docker has virtual networking support between containers so all we need to know is the container name to connect to and docker can resolve the named container to the ip address of the container.

$sudo docker exec -i -t loving_heisenberg bash
The above command will launch bash shell in the image loving_heisenberg.

OpenStack cloud service project demo

The project code is at - https://github.com/sjsucohort6/openstack

Documentation - OpenStack Cloud Service Project.pptx and OpenStack Cloud Service.docx