OSGi Overview

What is OSGi?

OSGi (formerly Open Services Gateway initiative) refers to a Java based services platform that allows for remote management of a system, as well as adding life cycle management of internal components (bundles), allowing the dynamic install, delete, and updating of these components with minimal impact to the system. OSGi allows for the quick release and incorporation or interconnected components into an active system, and have them interact with no interruption to the active system.

The prime use of the technology seems geared at the ability to quickly replace bundles in an active system.

The OSGi alliance specification like EJB and servlet specification, defines two things.

1. A set of services an OSGi container must implement

2. A contract between the container and the application

 There are number of open source OSGi container implementations such as

• Equinox
• Knoplerfish
• Apache Felix

Example

Imagine a risk calculation system that is attempting to handle all risk calculations for a bank. The system is designed with modularity in mind, and to be active 24/7. We implement a central node to handle risk calculation requests, and each instrument's calculation component is a separate module.
OSGi would facilitate this by implementing the central component and its many hooks as a bundle, followed by the implementation of each of the individual instrument risk calculators as services that implement a risk calculation interface. This would allow for these services to be added to the running system as they become available, register with the Service Registry, and thus allow the central node to find them as it receives requests to calculate those instruments. The version that is contained in the bundles allows OSGi to track which is the latest bundle, stop the current one and replace it with this later version.
Furthermore by having the calculation libraries as their own bundle it would allow for a quick update of those internal libraries without impacting the rest of the system.
OSGi allows the central node to request a particular risk calculator given a known string query and parameters, and OSGi would handle the returning of the appropriate calculator, or NULL if there is non available. It would allow for the rapid delivery of such a system, and its pre-defined modularity. It would also allow for updates of new code to be quickly accomplished. It also forces components of the system to be able to handle the case of a portion of the system being unavailable.

OSGi technology is the dynamic module system for Java.  Java provides the portability that is required to support products on many different platforms. The OSGi technology provides the standardized primitives that allow applications to be constructed from small, reusable and collaborative components. These components can be composed into an application and deployed.

The OSGi Service Platform provides the functions to change the composition dynamically on the device of a variety of networks, without requiring restarts. To minimize the coupling, as well as make these couplings managed, the OSGi technology provides a service-oriented architecture that enables these components to dynamically discover each other for collaboration. The OSGi Alliance has developed many standard component interfaces for common functions like HTTP servers, configuration, logging, security, user administration, XML and many more. Plug-compatible implementations of these components can be obtained from different vendors with different optimizations and costs. However, service interfaces can also be developed on a proprietary basis.

Apache Karaf is a small OSGi-based runtime which provides a lightweight container onto which various components and applications can be deployed.

Features:

1.    Hot deployment: Karaf supports hot deployment of OSGi bundles by monitoring jar files inside the $home/deploy directory. 

2.    Dynamic configuration: Services are usually configured through the ConfigurationAdmin OSGi service. Such configuration can be defined in Karaf using property files inside the $home/etc directory. These configurations are monitored and changes on the properties files will be propagated to the services.

3.    Logging System: using a centralized logging back end supported by Log4J

4.    Provisioning: Provisioning of libraries or applications can be done through a number of different ways, by which they will be downloaded locally, installed and started.

5.    Native OS integration: Karaf can be integrated into your own Operating System as a service

6.    Extensible Shell console: Karaf features a nice text console where you can manage the services, install new applications or libraries and manage their state. This shell is easily extensible by deploying new commands dynamically along with new features or applications.

7.    Remote access: use any SSH client to connect to Karaf and issue commands in the console

8.    Security framework based on JAAS

9.    Managing instances: Karaf provides simple commands for managing multiple instances. You can easily create, delete, start and stop instances of Karaf through the console.

Architecture

Frameworks that implement the OSGi standard provide an enviorment for the modularization of applications into small bundles. These bundles refer to tightly-coupled, dynamically loadable collections of classes, jars, and configurations. In other words, the OSGi standard looks to reduce applications into smaller components known as bundles. The primary requirement of these bundles is that they are entirely self-contained, so that they can be swapped in and out of the application as is necessary.

• Bundles - Bundles are the OSGi components made by the developers.
• Services - The services layer connects bundles in a dynamic way by offering a publish-find-bind model for plain old Java objects.
• Life-Cycle - The API to install, start, stop, update, and uninstall bundles.
• Modules - The layer that defines how a bundle can import and export code.
• Security - The layer that handles the security aspects.
• Execution Environment - Defines what methods and classes are available in a specific platform.

http://www.osgi.org/Technology/WhatIsOSGi

In Java terms, a bundle is a plain old JAR file. However, where in standard Java everything in a JAR is completely visible to all other JARs, OSGi hides everything in that JAR unless explicitly exported. A bundle that wants to use another JAR must explicitly import the parts it needs. By default, there is no sharing.

Though the code hiding and explicit sharing provides many benefits (for example, allowing multiple versions of the same library being used in a single VM), the code sharing was only there to support OSGi services model. The services model is about bundles that collaborate.

Bundles are deployed on an OSGi framework, the bundle runtime environment. This is not a container like Java Application Servers. It is a collaborative environment. Bundles run in the same VM and can actually share code. The framework uses the explicit imports and exports to wire up the bundles so they do not have to concern themselves with class loading. Another contrast with the application servers is that the management of the framework is standardized. A simple API allows bundles to install, start, stop, and update other bundles, as well as enumerating the bundles and their service usage. This API has been used by manymanagement agents to control OSGi frameworks. 

Normalized Message Router

The Normalized Message Router (NMR) is a general-purpose message bus used for communication between bundles in the OSGi container.
It's modeled after the Normalized Message Router (NMR) defined in the Java Business Integration (JBI) specification.

Role of OSGi in a SOA Runtime

1.       Pluggability – extend runtime with additional functionality – adding services, containers etc.

2.       Isolation – control packages exposed and consumed

3.       Dynamism – bundles have lifecycle independent of VM

4.       Dependency Management – dependency between services

Bundles can act as both simple libraries to be used and referred to by other bundles, or they can have an Activator, that will perform an action when it is started or stopped. Having an activator allows the creation of services, by registering objects into the Service Registry under a given interface. It also allows to define further parameters / properties to narrow down the case of collisions between service names.

//Simple activator

package osgi;

import org.osgi.framework.BundleActivator;

import org.osgi.framework.BundleContext;

public class Activator implements BundleActivator {

    private BundleContext context;

    public void start(BundleContext context) throws Exception {

        log ("Starting: Hello World");

        this.context = context;

    }//end start

    public void stop(BundleContext context) throws Exception {

        log ("Stopping: Goodbye World");

        this.context = null;

    }//end stop

}

For more examples: http://www.knopflerfish.org/tutorials/osgi_tutorial.pdf 

Life Cycle

The life-cycle layer controls the activation and deactivation of bundles. It is responsible for starting, stopping, installing, and updating bundles, and making sure that other bundles that have dependencies are notified. This layer is also in charge of handling the class loading and the dependencies between bundles.

 Application life cycle management is implemented via APIs that allow for remote downloading of management policies. The service registry allows bundles to detect the addition of new services, or the removal of services, and adapt accordingly.

Security

The security layer is based on the standard Java 2 security model. It is a permission based model based by location and by signer. OSGi provides real time management of the security configuration of an application, and provides APIs for admin permission, and conditional admin permission. The way this works is as follows: An Operator grants Networking rights to Company X on their devices. Now every bundle that Company X signs and deploys can use Networking on the Operator's devices. Furthermore a specific bundle can be granted permission to handle the life-cycle of bundles deployed by Company X.

Modules

Modules define collections of bundles, and their interactions. Including how bundles import and export code. By default bundles do not share code, so in their manifests we need to define how they import / export different parts of the codebase to function. The module layer handles those interactions via the manifest files of each bundle.

Sample Manifest File

//Sample Manifest File Bundle-Name: Hello World Bundle-SymbolicName: com.rf.osgi Bundle-Description: A Hello World bundle Bundle-ManifestVersion: 2 Bundle-Version: 1.0.0 Bundle-Activator: com.rf.andres.osgi.Activator Export-Package: com.rf.andres.osgi.helloworld;version="1.0.0" Import-Package: org.osgi.framework;version="1.3.0"

Given that bundles define what exactly they export through their manifest, it is possible to restrict what classes inside a JAR are published for use by other bundles. Further, utilizing the Modules aspect of the architecture, it allows for multiple version of the JAR to coexist happily inside the system without wrecking total havoc.

Apache Karaf

Apache Karaf is a small OSGi based runtime which provides a lightweight container onto which various components and applications can be deployed.

Felix is just the OSGi core runtime. Karaf provides a "distribution" based on Felix by adding other features such as a console, an SSH remoting mechanism, a file deployer and more.

In this diagram of the Karaf architecture, Felix (or other OSGi implementation - currently Equinox is also supported) is the OSGi box, the other boxes are the features added by Karaf:

Based on:
http://en.wikipedia.org/wiki/OSGi 

Using Apache Shiro for JSF 2.0 Web Application Session Management

• Configure Shiro filter as the first filter in web.xml:

        
  
  webapp.CdiEnvironmentLoaderListener
 
 
  ShiroFilter
  org.apache.shiro.web.servlet.ShiroFilter
 

 
  ShiroFilter
  /*
  REQUEST
  FORWARD
  INCLUDE
  ERROR
 

public class CustomRealm extends AuthenticatingRealm {

  private CredentialsMatcher credentialsMatcher;

  public String getName() {
    return "myRealm";
  }

  public boolean supports(AuthenticationToken token) {
    return true;
  }

  public CredentialsMatcher getCredentialsMatcher() {
    return credentialsMatcher;
  }

  public void setCredentialsMatcher(CredentialsMatcher credentialsMatcher) {
    this.credentialsMatcher = credentialsMatcher;
  }

  @Override
  protected AuthenticationInfo doGetAuthenticationInfo(AuthenticationToken token) throws AuthenticationException {
    // we can safely cast to a UsernamePasswordToken here, because this class
    // 'supports' UsernamePasswordToken
    // objects. See the Realm.supports() method if your application will use a
    // different type of token.
    UsernamePasswordToken upToken = (UsernamePasswordToken) token;
    return new SimpleAuthenticationInfo(upToken.getUsername(), upToken.getPassword(), getName());
  }
}

public class CustomAuthenticator extends AbstractAuthenticator {
  @Override
  protected AuthenticationInfo doAuthenticate(AuthenticationToken token) throws AuthenticationException {
    // perform custom authentication - lookup DB or invoke session EJB for auth
  }
}

import java.io.IOException;

import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;

import org.apache.shiro.web.filter.authc.UserFilter;

public class FacesAjaxAwareUserFilter extends UserFilter {

    private static final String FACES_REDIRECT_XML = ""
            + "";

    @Override
    protected void redirectToLogin(ServletRequest req, ServletResponse res) throws IOException {
        HttpServletRequest request = (HttpServletRequest) req;

        if ("partial/ajax".equals(request.getHeader("Faces-Request"))) {
            res.setContentType("text/xml");
            res.setCharacterEncoding("UTF-8");
            res.getWriter().printf(FACES_REDIRECT_XML, request.getContextPath() + getLoginUrl());
        }
        else {
            super.redirectToLogin(req, res);
        }
    }

}

public class CdiEnvironmentLoaderListener extends EnvironmentLoaderListener {

  CustomRealm customRealm = null;

  @Override
  protected WebEnvironment createEnvironment(ServletContext context) {
    WebEnvironment environment = super.createEnvironment(context);
    customRealm = new CustomRealm();

    RealmSecurityManager rsm = (RealmSecurityManager) environment.getSecurityManager();

    /*-HashedCredentialsMatcher matcher = new HashedCredentialsMatcher();
    matcher.setHashAlgorithmName(Sha512Hash.ALGORITHM_NAME);*/
    AllowAllCredentialsMatcher matcher = new AllowAllCredentialsMatcher();
    
    customRealm.setCredentialsMatcher(matcher);

    rsm.setRealm(customRealm);

    ((DefaultWebEnvironment) environment).setSecurityManager(rsm);

    return environment;
  }
}

@Named
@SessionScoped
public class LoginBean implements Serializable {
 private String username;
 private String password;
private static Factory factory = null;

  @PostConstruct
  public void init() {
    if (factory == null) {
      factory = new IniSecurityManagerFactory("classpath:shiro.ini");
      SecurityManager securityManager = factory.getInstance();
      SecurityUtils.setSecurityManager(securityManager);
    }
  }

 public String login() {
  UsernamePasswordToken token = new UsernamePasswordToken(username, password);
    token.setRememberMe(true);

    // Submit the principals and credentials
    Subject currentUser = SecurityUtils.getSubject();

   try {
     FacesContext fc = FacesContext.getCurrentInstance();
      ExternalContext externalContext = fc.getExternalContext();
      HttpServletRequest request = (HttpServletRequest) externalContext.getRequest();
      /*
       * Enable the session creation only during login.
       */
      request.setAttribute(DefaultSubjectContext.SESSION_CREATION_ENABLED, Boolean.TRUE);

      currentUser.login(token);
      return "SUCCESS";
    } catch (AuthenticationException | IOException e) {
      return "FAILED";
    }
   }

  public void logout() {
ExternalContext externalContext = FacesContext.getCurrentInstance().getExternalContext();
    try {

      // Invalidate HTTP session.
      Subject currentUser = SecurityUtils.getSubject();
      if (currentUser != null) {
          try {
          currentUser.logout();
        } catch (Exception e) {
          
        }
        externalContext.invalidateSession();
      }
    } catch (Exception ex) {
      
    } finally {
      try {
        externalContext.redirect("/login.xhtml");
      } catch (IOException e) {
        
      }
    }
  }
 }
}

Login
    
        
            
            
                
            
            

            
            
                
            
            

            
            
            

            
            
                
            
            
        
    

Lastly, the shiro.ini:

[main]
user = webapp.FacesAjaxAwareUserFilter
user.loginUrl = /login.xhtml

# Configure The EhCacheManager
cacheManager = org.apache.shiro.cache.ehcache.EhCacheManager
cacheManager.cacheManagerConfigFile = classpath:ehcache.xml

# Configure the above CacheManager on Shiro's SecurityManager
# to use it for all of Shiro's caching needs:
securityManager.cacheManager = $cacheManager

# Auth
myRealm = webapp.CustomRealm
myRealmCredentialsMatcher = org.apache.shiro.authc.credential.AllowAllCredentialsMatcher
myRealm.credentialsMatcher = $myRealmCredentialsMatcher
securityManager.realms = $myRealm

authenticator = webapp.CustomAuthenticator
securityManager.authenticator = $authenticator

#Remember Me
rememberMe = org.apache.shiro.web.mgt.CookieRememberMeManager
securityManager.rememberMeManager = $rememberMe

[urls]
/ = user
/login.xhtml = user

/javax.faces.resource/** = noSessionCreation, anon
/images/** = noSessionCreation, anon
/js/**= noSessionCreation, anon
/css/** = noSessionCreation, anon
/** =  user

Understanding Apache Shiro

Excerpted from: www.infoq.com/articles/apache-shiro

What is Apache Shiro?

Apache Shiro (pronounced “shee-roh”, the Japanese word for ‘castle’) is a powerful and easy-to-use Java security framework that performs authentication, authorization, cryptography, and session management and can be used to secure any application - from the command line applications, mobile applications to the largest web and enterprise applications. 

Shiro provides the application security API to perform: 

• Authentication - proving user identity, often called user ‘login’.
• Authorization - access control
• Cryptography - protecting or hiding data from prying eyes - simplifies JCA usage.
• Session Management - per-user time-sensitive state

I only used Shiro for session management as for our application we already had an existing authentication/authorization framework in place. 

Shiro framework was created in 2003 (so its already 10+ years old at the time of this writing). 

Why Shiro came into being?

1. Shiro was created to overcome the shortcomings of JAAS (Java Authentication and Authorization Service). 

JAAS was heavily tied to Virtual Machine-level security concerns, for example, determining if a class should be allowed to be loaded in the JVM. As an application developer, I cared more about what an application end-user could do rather than what my code could do inside the JVM.

2. Shiro was also created to provide a clean, container-agnostic session mechanism (unlike the HttpSession that requires web container, or EJB Stateful Session Bean which requires EJB container).

Benefits of using Apache Shiro in your project

1. Easy to use framework
2. Flexible - you can use Shiro for only session management and by pass its other features if you already have say authentication/authorization custom framework in place. Also Shiro can work in web, EJB and IoC container or standalone Java application.
3. Web capabilities - Shiro security can be configured for REST based web services quickly and very intuitively by defining URL mappings in shiro.ini.
4. Pluggable - Shiro integrates easily with other frameworks like Spring, Grails, Vaadin, Wicket etc.
5. Supported - Both open source community and commercial support available. It is also top level Apache project.
6. Wide Adoption - Shiro is used in several open source frameworks like Spring, Grails etc. and is widely used by companies of all sizes.

Core Concepts of Shiro

• Subject - represents the current user

import org.apache.shiro.subject.Subject;
import org.apache.shiro.SecurityUtils;
...
Subject currentUser = SecurityUtils.getSubject();

Now you can do almost everything you’d want to do with Shiro for the current user, such as login, logout, access their session, execute authorization checks, and more.

• Security Manager - While the Subject represents security operations for the current user, the SecurityManager manages security operations for all users. It is the heart of Shiro’s architecture and acts as a sort of ‘umbrella’ object that references many internally nested security components that form an object graph. However, once the SecurityManager and its internal object graph is configured, it is usually left alone and application developers spend almost all of their time with the Subject API.

There is almost always a single SecurityManager instance per application. It is essentially an application singleton. SecurityManager (and its associated object graph) can be configured in several ways including a text based INI configuration. 

Configuring Shiro with INI

[main]
cm = org.apache.shiro.authc.credential.HashedCredentialsMatcher
cm.hashAlgorithm = SHA-512
cm.hashIterations = 1024
# Base64 encoding (less text):
cm.storedCredentialsHexEncoded = false

iniRealm.credentialsMatcher = $cm

[users]
jdoe = TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJpcyByZWFzb2
asmith = IHNpbmd1bGFyIHBhc3Npb24gZnJvbSBvdGhlciBhbXNoZWQsIG5vdCB

There are two INI sections: [main] and [users].

The [main] section is where you configure the SecurityManager object and/or any objects (like Realms) used by the SecurityManager.

The [users] section is where you can specify a static list of user accounts - convenient for simple applications or when testing.

Loading shiro.ini Configuration File

import org.apache.shiro.SecurityUtils;
import org.apache.shiro.config.IniSecurityManagerFactory;
import org.apache.shiro.mgt.SecurityManager;
import org.apache.shiro.util.Factory;

...

//1. Load the INI configuration
Factory factory =
new IniSecurityManagerFactory("classpath:shiro.ini");

//2. Create the SecurityManager
SecurityManager securityManager = factory.getInstance();

//3. Make it accessible
SecurityUtils.setSecurityManager(securityManager);

• Realm - A Realm acts as the ‘bridge’ or ‘connector’ between Shiro and your application’s security data. That is, when it comes time to actually interact with security-related data like user accounts to perform authentication (login) and authorization (access control), Shiro looks up many of these things from one or more Realms configured for an application.

In this sense a Realm is essentially a security-specific DAO: it encapsulates connection details for data sources and makes the associated data available to Shiro as needed. When configuring Shiro, you must specify at least one Realm to use for authentication and/or authorization. 

Shiro provides out-of-the-box Realms to connect to a number of security data sources (aka directories) such as LDAP, relational databases (JDBC), text configuration sources like INI and properties files, and more. You can plug-in your own Realm implementations to represent custom data sources if the default Realms do not meet your needs. 

Example realm configuration snippet to connect to LDAP user data store

[main]
ldapRealm = org.apache.shiro.realm.ldap.JndiLdapRealm
ldapRealm.userDnTemplate = uid={0},ou=users,dc=mycompany,dc=com
ldapRealm.contextFactory.url = ldap://ldapHost:389
ldapRealm.contextFactory.authenticationMechanism = DIGEST-MD5 

Authentication

 This is typically a three-step process.

1. Collect the user’s identifying information, called principals (e.g. username), and supporting proof of identity, called credentials (e.g. password)
2. Submit the principals and credentials to the system.
3. If the submitted credentials match what the system expects for that user identity (principal), the user is considered authenticated. If they don’t match, the user is not considered authenticated.

Shiro has a Subject-centric API - almost everything you care to do with Shiro at runtime is achieved by interacting with the currently executing Subject. So, to login a Subject, you simply call its login method, passing an AuthenticationToken instance that represents the submitted principals and credentials (in this case, a username and password). 

Subject Login

//1. Acquire submitted principals and credentials:
AuthenticationToken token =new UsernamePasswordToken(username, password);



//2. Get the current Subject:
Subject currentUser = SecurityUtils.getSubject();

//3. Login:
try {
    currentUser.login(token);
} catch (IncorrectCredentialsException ice) {
} catch (LockedAccountException lae) {
} catch (AuthenticationException ae) {
}

Authorization

Perform access control by employing concepts such as roles and permissions.

Role Check

if ( subject.hasRole(“administrator”) ) {
    //show the ‘Create User’ button
} else {
    //grey-out the button?
} 

Permission Check

if ( subject.isPermitted(“user:create”) ) {
    //show the ‘Create User’ button
} else {
    //grey-out the button?
} 

Finally, just as with authentication, the above calls eventually make their way to the SecurityManager, which will consult one or more Realms to make the access control decisions. This allows a Realm to respond to both authentication and authorization operations as necessary.

Session Management

Container agnostic session management: Application developers who wish to use sessions are no longer forced to use Servlet or EJB containers if they don’t need them otherwise. Or, if using these containers, developers now have the option of using a unified and consistent session API in any tier, instead of servlet or EJB-specific mechanisms.

Shiro’s architecture allows for pluggable Session data stores, such as enterprise caches, relational databases, NoSQL systems and more. This means that you can configure session clustering once and it will work the same way regardless of your deployment environment - Tomcat, Jetty, JEE Server or standalone application. There is no need to reconfigure your app based on how you deploy your application.

Another benefit of Shiro’s sessions is session data can be shared across client technologies if desired. For example, a Swing desktop client can participate in the same web application session if desired - useful if the end-user is using both simultaneously. 

Subject’s Session

Session session = subject.getSession();
Session session = subject.getSession(boolean create);

The methods are identical in concept to the HttpServletRequest API. The first method will return the Subject’s existing Session, or if there isn’t one yet, it will create a new one and return it. The second method accepts a boolean argument that determines whether or not a new Session will be created if it does not yet exist. Once you acquire the Subject’s Session, you can use it almost identically to an HttpSession.

 Session methods

Session session = subject.getSession();

session.getAttribute(“key”, someValue);
Date start = session.getStartTimestamp();
Date timestamp = session.getLastAccessTime();
session.setTimeout(millis);

Shiro Web Support

ShiroFilter in web.xml

    org.apache.shiro.web.env.EnvironmentLoaderListener


...


    ShiroFilter
    org.apache.shiro.web.servlet.ShiroFilter



    ShiroFilter
    /*
    REQUEST 
    FORWARD 
    INCLUDE 
    ERROR

Once configured, the Shiro Filter will filter every request and ensure the request-specific Subject is accessible during the request. And because it filters every request, you can perform security-specific logic to ensure only requests that meet certain criteria are allowed through.

URL-specific filter chains

Shiro supports security-specific filter rules through its innovative URL filter chaining capability. It allows you to specify ad-hoc filter chains for any matching URL pattern. 

Path-specific Filter Chains

[urls]
/assets/** = anon
/user/signup = anon
/user/** = user
/rpc/rest/** = perms[rpc:invoke], authc
/** = authc

For each line, the values on the left of the equals sign represent a context-relative web application path. The values on the right define a Filter chain - an ordered, comma delimited list of Servlet filters to execute for the given path. Each filter is a normal Servlet Filter, but the filter names you see above (anon, user, perms, authc) are special security-related filters that Shiro provides out-of-the-box. You can mix and match these security filters to create a very custom security experience. You can also specify any other existing Servlet Filter you may have.

How much nicer is this compared to using web.xml, where you define a block of filters and then a separate disconnected block of filter patterns? Using Shiro’s approach, it is much easier to see exactly the filter chain that is executed for a given matching path. 

If you wanted to, you could define only the Shiro Filter in web.xml and define all of your other filters and filter chains in shiro.ini for a much more succinct and easy to understand filter chain definition mechanism than web.xml. Even if you didn’t use any of Shiro’s security features, this one small convenience alone can make Shiro worth using.

Web Session Management

For web applications, Shiro defaults its session infrastructure to use the existing Servlet Container sessions. That is, when you call the methods subject.getSession() and subject.getSession(boolean) Shiro will return Session instances backed by the Servlet Container’s HttpSession instance. The beauty of this approach is that business-tier code that calls subject.getSession() interacts with a Shiro Session instance - it has no ‘knowledge’ that it is working with a web-based HttpSession object. This is a very good thing when maintaining clean separation across architectural tiers.

If you’ve enabled Shiro’s native session management in a web application because you need Shiro’s enterprise session features (like container-independent clustering), you of course want the HttpServletRequest.getSession() and HttpSession API to work with the ‘native’ sessions and not the servlet container sessions.

Basic Introduction to Apache Hadoop

Introduction to NoSQL by Martin Fowler

JMockit by Example

public class DBManager {
 public String retrieveAccountHolderName(int accountId) {
  String accountHolderName = null;

  // connect to db
  // retrieve the Account Holder Name

  return accountHolderName;
 }

 public static String getConnectionString() {
  return "ORIGINAL";
 }
}

public class Bank {
 DBManager dbManager = new DBManager();

 public String processAccount(int accountID) {

  // Some other code goes here

  String accountHolderName = dbManager
    .retrieveAccountHolderName(accountID);

  // some more processing code

  return accountHolderName;
 }

 public String getConnection() throws Exception {
  return "Connection";
  // some thing here might throw an exception
 }

 public String makeConnection() {
  // some connection related code
  // goes here

  // call to static method
  String conStr = DBManager.getConnectionString();

  // If the connection String
  // is anything other than
  // ORIGINAL return FAIL
  if (conStr.equals("ORIGINAL"))
   return "SUCCESS";
  else
   return "FAIL";
 }
}

import static org.junit.Assert.assertEquals;
import mockit.Expectations;
import mockit.Mocked;
import mockit.NonStrictExpectations;

import org.junit.Test;

public class BankTest {
 @Mocked
 DBManager dbManager; // variables declared here are mocked by default

 @Test
 public void testProcessAccount() {
  Bank bank = new Bank();

  // Define the Expectations block here
  new Expectations() {
   {
    dbManager.retrieveAccountHolderName(10);
    returns("Abhi");
   }
  };

  String name = bank.processAccount(10);

  assertEquals("Account holder Name for A/C id 10 is 'Abhi' ", "Abhi",
    name);
 }

 @Test
 public void testMakeConnection() {

  new NonStrictExpectations() {
   {
    DBManager.getConnectionString();
    returns("DUPLICATE");
   }
  };

  Bank bank = new Bank();
  String status = bank.makeConnection();

  assertEquals("Status is FAIL", "FAIL", status);
 }

 @Test(expected = Exception.class)
 public void testGetConnection() throws Exception {
  final Bank bank = new Bank();
  new Expectations(bank) {
   {
    bank.getConnection();
    result = new Exception();
   }
  };
  bank.getConnection();
 }

}

The Internal Architecture of JVM

Excerpted from: http://tekmarathon.wordpress.com/2013/04/22/the-internal-architecture-of-jvm/ 

Java Virtual Machine components are depicted in below diagram. Each time we run java_application/java_class, an instance of JVM gets created.

Class Loader Subsystem

Class loader subsystem loads classes and interfaces with fully qualified names into the JVM.

Execution Engine

Execution Engine executes all instructions contained by methods of a loaded class.

While executing a Java program, JVM requires memory for storing objects ,local variables, method arguments, return values, and intermediate computational results and JVM does that memory management on several runtime data areas. The specification of runtime data areas is quite abstract. This abstract nature of JVM specification helps different designers to provide implementation on wide variety of OS and as per choice of the designers. Some implementations may have a lot of memory in which to work, others may have very little. Some implementations may be able to take advantage of virtual memory, others may not.

Method Area and Heap
Each instance of the Java virtual machine has one method area and one heap. These areas are shared by all threads running inside the virtual machine. When the virtual machine loads a class file, it parses information about class type from the binary data contained in the class file. It stored the type information into the method area. As the program runs, the virtual machine places all objects the program instantiates onto the heap.

PC Registers and Stacks
When a new thread is created, it gets its own pc register (program counter) and Java stack. If the thread is executing a Java method (not a native method), the value of the pc register indicates the next instruction to execute. A thread’s Java stack stores the state of Java (not native) method which includes its local variables, the parameters with which it was invoked, its return value (if any), and intermediate calculations.

The Java stack is composed of stack frames (or frames). A stack frame contains the state of one Java method invocation. When a thread invokes a method, the java virtual machine pushes the newly created frame onto that thread’s Java stack. When the method completes, the virtual machine pops and discards the frame for that method.

In JVM ,the instruction set uses the Java stack for storage of intermediate data values. The stack-based architecture of the JVM’s instruction set optimizes code done by just-in-time and dynamic compilers.

Native Method Stacks

The state of native method invocations is stored in an implementation-dependent way in native method stacks, in registers or other implementation-dependent memory areas.

Understanding Priority Inversion

Excerpted from: http://www.embedded.com/electronics-blogs/beginner-s-corner/4023947/Introduction-to-Priority-Inversion

Priority inversions

The real trouble arises at run-time, when a medium-priority task preempts a lower-priority task using a shared resource on which the higher-priority task is pending. If the higher-priority task is otherwise ready to run, but a medium-priority task is currently running instead, a priority inversion is said to occur.

Figure 1 Priority inversion timeline

This dangerous sequence of events is illustrated in Figure 1. Low-priority Task L and high-priority Task H share a resource. Shortly after Task L takes the resource, Task H becomes ready to run. However, Task H must wait for Task L to finish with the resource, so it pends. Before Task L finishes with the resource, Task M becomes ready to run, preempting Task L. While Task M (and perhaps additional intermediate-priority tasks) runs, Task H, the highest-priority task in the system, remains in a pending state.

Many priority inversions are innocuous or, at most, briefly delay a task that should run right away. But from time to time a system-critical priority inversion takes place. Such an event occurred on the Mars Pathfinder mission in July 1997. The Pathfinder mission is best known for the little rover that took high-resolution color pictures of the Martian surface and relayed them back to Earth.

The problem was not in the landing software, but in the mission software run on the Martian surface. In the spacecraft, various devices communicated over a MIL-STD-1553 data bus. Activity on this bus was managed by a pair of high-priority tasks. One of the bus manager tasks communicated through a pipe with a low-priority meteorological science task.

On Earth, the software mostly ran without incident. On Mars, however, a problem developed that was serious enough to trigger a series of software resets during the mission. The sequence of events leading to each reset began when the low-priority science task was preempted by a couple of medium-priority tasks while it held a mutex related to the pipe. While the low-priority task was preempted, the high-priority bus distribution manager tried to send more data to it over the same pipe. Because the mutex was still held by the science task, the bus distribution manager was made to wait. Shortly thereafter, the other bus scheduler became active. It noticed that the distribution manager hadn't completed its work for that bus cycle and forced a system reset.

This problem was not caused by a mistake in the operating system, such as an incorrectly implemented semaphore, or in the application. Instead, the software exhibited behavior that is a known "feature" of semaphores and intertask communication. In fact, the RTOS used on Pathfinder featured an optional priority-inversion workaround; the scientists at JPL simply hadn't been aware of that option. Fortunately, they were able to recreate the problem on Earth, remotely enable the workaround, and complete the mission successfully.

Workarounds

Research on priority inversion has yielded two solutions. The first is called priority inheritance. This technique mandates that a lower-priority task inherit the priority of any higher-priority task pending on a resource they share. This priority change should take place as soon as the high-priority task begins to pend; it should end when the resource is released. This requires help from the operating system.

The second solution, priority ceilings, associates a priority with each resource; the scheduler then transfers that priority to any task that accesses the resource. The priority assigned to the resource is the priority of its highest-priority user, plus one. Once a task finishes with the resource, its priority returns to normal.

A beneficial feature of the priority ceiling solution is that tasks can share resources simply by changing their priorities, thus eliminating the need for semaphores:

void TaskA(void)
{
 ...
 SetTaskPriority(RES_X_PRIO);
 // Access shared resource X.
 SetTaskPriority(TASK_A_PRIO);
 ...
}

While Task A's priority is elevated (and it is accessing shared resource X), it should not pend on any other resource. The higher-priority user will only become the highest-priority ready task when the lower-priority task is finished with their shared resource.

While not all of us are writing software for missions to Mars, we should learn from past mistakes and implement solutions that don't repeat them. Many commercial RTOSes include support for either priority inheritance or priority ceilings. Just make sure you enable one.

Date: Fri, 9 Jan 1998 14:13:58 -0800
From: Mike Jones 
Subject: Re: What really happened on Mars? by Glenn Reeves (RISKS-19.49)

> Date: Monday, December 15, 1997 10:28 AM
> From: Glenn E Reeves 
> Subject:      Re: [Fwd: FW: What really happened on Mars?]
>
> What really happened on Mars ?
>
>By now most of you have read Mike's (mbj@microsoft.com) summary of Dave
>Wilner's comments given at the IEEE Real-Time Systems Symposium.  I don't
>know Mike and I didn't attend the symposium (though I really wish I had now)
>and I have not talked to Dave Wilner since before the talk.  However, I did
>lead the software team for the Mars Pathfinder spacecraft.  So, instead of
>trying to find out what was said I will just tell you what happened.  You
>can make your own judgments.
>
>I sent this message out to everyone who was a recipient of Mike's original
>that I had an e-mail address for.  Please pass it on to anyone you sent the
>first one to.  Mike, I hope you will post this wherever you posted the
>original.
>
>Since I want to make sure the problem is clearly understood I need to step
>through each of the areas which contributed to the problem.
>
>THE HARDWARE
>
>The simplified view of the Mars Pathfinder hardware architecture looks like
>this.  A single CPU controls the spacecraft.  It resides on a VME bus which
>also contains interface cards for the radio, the camera, and an interface to
>a 1553 bus.  The 1553 bus connects to two places : The "cruise stage" part
>of the spacecraft and the "lander" part of the spacecraft.  The hardware on
>the cruise part of the spacecraft controls thrusters, valves, a sun sensor,
>and a star scanner.  The hardware on the lander part provides an interface
>to accelerometers, a radar altimeter, and an instrument for meteorological
>science known as the ASI/MET.  The hardware which we used to interface to
>the 1553 bus (at both ends) was inherited from the Cassini spacecraft.  This
>hardware came with a specific paradigm for its usage : the software will
>schedule activity at an 8 Hz rate.  This **feature** dictated the
>architecture of the software which controls both the 1553 bus and the
>devices attached to it.
>
>THE SOFTWARE ARCHITECTURE
>
>The software to control the 1553 bus and the attached instruments was
>implemented as two tasks.  The first task controlled the setup of
>transactions on the 1553 bus (called the bus scheduler or bc_sched task) and
>the second task handled the collection of the transaction results i.e. the
>data.  The second task is referred to as the bc_dist (for distribution)
>task.  A typical timeline for the bus activity for a single cycle is shown
>below.  It is not to scale.  This cycle was constantly repeated.
>
>     |< ------------- .125 seconds ------------------------>|
>
>     |<***************|                    |********|   |**>|
>
>                      |<- -="" active="" bc_dist="">|    bc_sched active
>     |< -bus active ->|                             |<->|
>
>
> ----|----------------|--------------------|--------|---|---|-------
>     t1               t2                   t3       t4  t5  t1
>
>The *** are periods when tasks other than the ones listed are executing.
>Yes, there is some idle time.
>
>t1 - bus hardware starts via hardware control on the 8 Hz boundary. The
>transactions for the this cycle had been set up by the previous execution of
>the bc_sched task.
>t2 - 1553 traffic is complete and the bc_dist task is awakened.
>t3 - bc_dist task has completed all of the data distribution
>t4 - bc_sched task is awakened to setup transactions for the next cycle
>t5 - bc_sched activity is complete
>
>The bc_sched and bc_dist tasks check each cycle to be sure that the other
>had completed its execution.  The bc_sched task is the highest priority task
>in the system (except for the vxWorks "tExec" task).  The bc_dist is third
>highest (a task controlling the entry and landing is second).  All of the
>tasks which perform other spacecraft functions are lower.  Science

>functions, such as imaging, image compression, and the ASI/MET task are
>still lower.
>
>Data is collected from devices connected to the 1553 bus only when they are
>powered.  Most of the tasks in the system that access the information
>collected over the 1553 do so via a double buffered shared memory mechanism
>into which the bc_dist task places the latest data.  The exception to this
>is the ASI/MET task which is delivered its information via an interprocess
>communication mechanism (IPC).  The IPC mechanism uses the vxWorks pipe()
>facility.  Tasks wait on one or more IPC "queues" for messages to arrive.
>Tasks use the select() mechanism to wait for message arrival.  Multiple
>queues are used when both high and lower priority messages are required.
>Most of the IPC traffic in the system is not for the delivery of real-time
>data.  However, again, the exception to this is the use of the IPC mechanism
>with the ASI/MET task.  The cause of the reset on Mars was in the use and
>configuration of the IPC mechanism.
>
>THE FAILURE
>
>The failure was identified by the spacecraft as a failure of the bc_dist
>task to complete its execution before the bc_sched task started.  The
>reaction to this by the spacecraft was to reset the computer.  This reset
>reinitializes all of the hardware and software. It also terminates the
>execution of the current ground commanded activities.  No science or
>engineering data is lost that has already been collected (the data in RAM is
>recovered so long as power is not lost).  However, the remainder of the
>activities for that day were not accomplished until the next day.
>
>The failure turned out to be a case of priority inversion (how we discovered
>this and how we fixed it are covered later).  The higher priority bc_dist
>task was blocked by the much lower priority ASI/MET task that was holding a
>shared resource.  The ASI/MET task had acquired this resource and then been
>preempted by several of the medium priority tasks.  When the bc_sched task
>was activated, to setup the transactions for the next 1553 bus cycle, it
>detected that the bc_dist task had not completed its execution.  The
>resource that caused this problem was a mutual exclusion semaphore used
>within the select() mechanism to control access to the list of file
>descriptors that the select() mechanism was to wait on.
>
>The select mechanism creates a mutual exclusion semaphore to protect the
>"wait list" of file descriptors for those devices which support select.  The
>vxWorks pipe() mechanism is such a device and the IPC mechanism we used is
>based on using pipes. The ASI/MET task had called select, which had called
>pipeIoctl(), which had called selNodeAdd(), which was in the process of
>giving the mutex semaphore.  The ASI/ MET task was preempted and semGive()
>was not completed.  Several medium priority tasks ran until the bc_dist task
>was activated.  The bc_dist task attempted to send the newest ASI/MET data
>via the IPC mechanism which called pipeWrite().  pipeWrite() blocked, taking
>the mutex semaphore.  More of the medium priority tasks ran, still not
>allowing the ASI/MET task to run, until the bc_sched task was awakened.  At
>that point, the bc_sched task determined that the bc_dist task had not
>completed its cycle (a hard deadline in the system) and declared the error
>that initiated the reset.
>
>HOW WE FOUND IT
>
>The software that flies on Mars Pathfinder has several debug features within
>it that are used in the lab but are not used on the flight spacecraft (not
>used because some of them produce more information than we can send back to
>Earth).  These features were not "fortuitously" left enabled but remain in
>the software by design.  We strongly believe in the "test what you fly and
>fly what you test" philosophy.
>
>One of these tools is a trace/log facility which was originally developed to
>find a bug in an early version of the vxWorks port (Wind River ported
>vxWorks to the RS6000 processor for us for this mission).  This trace/log

>facility was built by David Cummings who was one of the software engineers
>on the task.  Lisa Stanley, of Wind River, took this facility and
>instrumented the pipe services, msgQ services, interrupt handling, select
>services, and the tExec task.  The facility initializes at startup and
>continues to collect data (in ring buffers) until told to stop.  The
>facility produces a voluminous dump of information when asked.
>
>After the problem occurred on Mars we did run the same set of activities
>over and over again in the lab.  The bc_sched was already coded so as to
>stop the trace/log collection and dump the data (even though we knew we
>could not get the dump in flight) for this error.  So, when we went into the
>lab to test it we did not have to change the software.
>
>In less that 18 hours we were able to cause the problem to occur. Once we
>were able to reproduce the failure the priority inversion problem was
>obvious.
>
>HOW WAS THE PROBLEM CORRECTED
>
>Once we understood the problem the fix appeared obvious : change the
>creation flags for the semaphore so as to enable the priority inheritance.
>The Wind River folks, for many of their services, supply global
>configuration variables for parameters such as the "options" parameter for
>the semMCreate used by the select service (although this is not documented
>and those who do not have vxWorks source code or have not studied the source
>code might be unaware of this feature).  However, the fix is not so obvious
>for several reasons :
>
>1) The code for this is in the selectLib() and is common for all device
>creations.  When you change this global variable all of the select
>semaphores created after that point will be created with the new options.
>There was no easy way in our initialization logic to only modify the
>semaphore associated with the pipe used for bc_dist task to ASI/MET task
>communications.
>
>2) If we make this change, and it is applied on a global basis, how will
>this change the behavior of the rest of the system ?
>
>3) The priority inversion option was deliberately left out by Wind River in
>the default selectLib() service for optimum performance.  How will
>performance degrade if we turn the priority inversion on ?
>
>4) Was there some intrinsic behavior of the select mechanism itself that
>would change if the priority inversion was enabled ?
>
>We did end up modifying the global variable to include the priority
>inversion.  This corrected the problem.  We asked Wind River to analyze the
>potential impacts for (3) and (4). They concluded that the performance
>impact would be minimal and that the behavior of select() would not change
>so long as there was always only one task waiting for any particular file
>descriptor.  This is true in our system.  I believe that the debate at Wind
>River still continues on whether the priority inversion option should be on
>as the default.  For (1) and (2) the change did alter the characteristics of
>all of the select semaphores.  We concluded, both by analysis and test, that
>there was no adverse behavior.  We tested the system extensively before we
>changed the software on the spacecraft.
>
>HOW WE CHANGED THE SOFTWARE ON THE SPACECRAFT
>
>No, we did not use the vxWorks shell to change the software (although the
>shell is usable on the spacecraft).  The process of "patching" the software
>on the spacecraft is a specialized process.  It involves sending the
>differences between what you have onboard and what you want (and have on
>Earth) to the spacecraft.  Custom software on the spacecraft (with a whole
>bunch of validation) modifies the onboard copy.  If you want more info you
>can send me e-mail.
>
>WHY DIDN'T WE CATCH IT BEFORE LAUNCH ?
>
>The problem would only manifest itself when ASI/MET data was being collected
>and intermediate tasks were heavily loaded.  Our before launch testing was
>limited to the "best case" high data rates and science activities.  The fact
>that data rates from the surface were higher than anticipated and the amount

>of science activities proportionally greater served to aggravate the
>problem.  We did not expect nor test the "better than we could have ever
>imagined" case.
>
>HUMAN NATURE, DEADLINE PRESSURES
>
>We did see the problem before landing but could not get it to repeat when we
>tried to track it down.  It was not forgotten nor was it deemed unimportant.
>
>Yes, we were concentrating heavily on the entry and landing software.  Yes,
>we considered this problem lower priority.  Yes, we would have liked to have
>everything perfect before landing.  However, I don't see any problem here
>other than we ran out of time to get the lower priority issues completed.
>
>We did have one other thing on our side; we knew how robust our system was
>because that is the way we designed it.
>
>We knew that if this problem occurred we would reset.  We built in
>mechanisms to recover the current activity so that there would be no
>interruptions in the science data (although this wasn't used until later in
>the landed mission).  We built in the ability (and tested it) to go through
>multiple resets while we were going through the Martian atmosphere.  We
>designed the software to recover from radiation induced errors in the memory
>or the processor.  The spacecraft would have even done a 60 day mission on
>its own, including deploying the rover, if the radio receiver had broken
>when we landed.  There are a large number of safeguards in the system to
>ensure robust, continued operation in the event of a failure of this type.
>These safeguards allowed us to designate problems of this nature as lower
>priority.
>
>We had our priorities right.
>
>ANALYSIS AND LESSONS
>
>Did we (the JPL team) make an error in assuming how the select/pipe
>mechanism would work ?  Yes, probably.  But there was no conscious decision
>to not have the priority inversion enabled.  We just missed it.  There are
>several other places in the flight software where similar protection is
>required for critical data structures and the semaphores do have priority
>inversion protection.  A good lesson when you fly COTS stuff - make sure you
>know how it works.
>
>Mike is quite correct in saying that we could not have figured this out
>**ever** if we did not have the tools to give us the insight.  We built many
>of the tools into the software for exactly this type of problem.  We always
>planned to leave them in.  In fact, the shell (and the stdout stream) were
>very useful the entire mission.  If you want more detail send me a note.
>
>SETTING THE RECORD STRAIGHT
>
>First, I want to make sure that everyone understands how I feel in regard to
>Wind River.  These folks did a fantastic job for us.  They were enthusiastic
>and supported us when we came to them and asked them to do an affordable
>port of vxWorks.  They delivered the alpha version in 3 months.  When we had
>a problem they put some of the brightest engineers I have ever worked with
>on the problem.  Our communication with them was fantastic.  If they had not
>done such a professional job the Mars Pathfinder mission would not have been
>the success that it is.
>
>Second, Dave Wilner did talk to me about this problem before he gave his
>talk.  I could not find my notes where I had detailed the description of the
>problem.  So, I winged it and I sure did get it wrong.  Sorry Dave.
>
>ACKNOWLEDGMENTS
>
>First, thanks to Mike for writing a very nice description of the talk.  I
>think I have had probably 400 people send me copies.  You gave me the push
>to write the part of the Mars Pathfinder End-of-Mission report that I had
>been procrastinating doing.
>
>Special thanks to Steve Stolper for helping me do this.  The biggest thanks
>should go to the software team that I had the privilege of leading and whose
>expertise allowed us to succeed: Pam Yoshioka, Dave Cummings, Don Meyer, 
>Karl Schneider, Greg Welz, Rick Achatz, Kim Gostelow, Dave Smyth, 
>Steve Stolper.   Also, Miguel San Martin, Sam Sirlin, Brian Lazara (WRS), 

>Mike Deliman (WRS), Lisa Stanley (WRS)
>
>Glenn Reeves, Mars Pathfinder Flight Software Cognizant Engineer
>glenn.e.reeves@jpl.nasa.gov

Java development IRC channel

I have joined the ##java on irc.freenode.net IRC server recently. I am using XChat IRC client with nick name rwatsh. We first need to register our nick name as follows (words in italics are the ones you need to change):

/msg NickServ REGISTER password youremail@example.com/

Then you get a verification email on the above mentioned email address with the verification command which looks like:

/msg NickServ VERIFY REGISTER username abcdxxxx

After that, you can join the ##java channel on freenode. Its also the only channel for all things Java (so it includes SE, ME, EE).