Thursday, October 11, 2012

JBoss One Day Talk 2012

Yesterday, I had the pleasure of speaking at the JBoss One Day Talk in Munich. Like always the conference was very well organized and it was great to meet up with community, customers and some colleagues from RedHat.

The talk was about OSGi in complex Enterprise Applications on AS7 and for the first time I could speak about how our vision has become reality. It is now possible to leverage middle ware services from AS7 and at the same time using the OSGi standard to address your modularity concerns and dynamic service requirements. I always like to draw the picture of a large and complex web application that is developed by multiple teams in different locations. Each team is primarily concerned with it own functional unit (module) and defines the set of capabilities/requirements of that module.

A fine rule of modularity: 
You can reason about A without having to think about B
At runtime the various modules come together, the caps/reqs get resolved by an external authority and by virtue of using OSGi they have life cycle and may integrate via dynamic services. So at any time you can bring in new modules, start/stop/update existing modules or take functionality out by removing a module. You web app would react accordingly and for example show/hide an option in the UI. Ok, it is rare that you want to modify a complex application in that way that is already in production. Still, 
An application that supports the notion of pluggable functionality is a good candidate for a modular architecture and is likely to use dynamic services to integrate the bits.
Modularity and service dynamicity do not come for free and I surely grant that OSGi can be hard at first. This is not because OSGi is hard it is because a modular system is more complex than a flat classpath and it is also more complex to track dynamic services instead of integrating via interfaces in the traditional way.
It is  sane to stay away from modularity if your system is reasonably simplistic and will stay that way in the foreseeable future. 
 JBoss AS7 supports modularity and dynamic services natively via jboss-modules and jboss-msc. It is good to understand the difference between that or any other home grown solution with respect to the OSGi standard. When in doubt and there is no other compelling reason, I would advise
Use the OSGi standard to secure your investment, stay portable and leverage what is out there in terms of documentation reusable functionality and not to mention last existing know how in the developer community.
I'm also happy to tell you that we plan to bring all that functionality to the JBoss Enterprise Application Platform (EAP 6.1) and it will be fully support.

Here are the slides from the session in Munich.

cheers
--thomas
   

Monday, June 18, 2012

JBossOSGi-1.1.1 Released

I am happy to announce the release of JBossOSGi-1.1.1.

You can download the binary here: jboss-osgi-installer-1.1.1.jar

This is a maintenance release for JBoss-AS-7.1.1 integration.
Please also have a look at the latest version of our User Guide.
Enjoy

Thursday, August 4, 2011

JBoss AS7 OSGi Presentation + Demo

I am happy to announce that the JBoss AS7 OSGi webinar recording is now available on vimeo.
JBoss AS7 is a game-changer for both Java EE developers as well as application server administrators. And while JBoss is best known for being a Java EE container, in this session, we will focus on the OSGi capabilities of the new JBoss AS7.

Starting with background information on OSGi in general, Thomas Diesler introduces the main objectives of this technology and explains the unique JBoss OSGi vision. Combining the best of two worlds we show how modern Java EE applications on AS7 can use the OSGi component model and vice versa.

Presenters: Thomas Diesler and David Bosschaert

Thomas is the OSGi Project Lead at JBoss. As the Red Hat representative in the OSGi Enterprise Expert Group he is involved in the standardization effort of various OSGi technologies that are relevant to the enterprise space.

David, principal software engineer at Red Hat, is a co-chair of the OSGi Enterprise Expert Group and spends the majority of his time on the JBoss OSGi framework and other open source projects. Before joining JBoss/Red Hat in 2010, David worked for IONA Technologies and Progress Software in Dublin, Ireland.

The demo code is available here.

Tuesday, July 26, 2011

JBossOSGi-1.0.0 Final Released

I am happy to announce the release of JBossOSGi-1.0.0 Final.
 
You can download the binary here: jboss-osgi-installer-1.0.0.jar

This is our first OSGi Core 4.2. Framework compliant release. It also comes with improvements in the following areas
  • Integration with JBoss AS7
  • Pass the Core 4.2  Compliance Testsuite from the OSGi Alliance
  • Add Declarative Services (DS) example
  • Documentation now available in Confluence
  • OSGi service invocation from EJB3 and Webap
  • Migration to the Arquillian test framework
  • Better support for execution environments
For details please have a look at the latest version of our User Guide.

Here are the change log details

JBossOSGi 1.0.0

Bug

  • [JBOSGI-438] - Exception when starting Blueprint component
  • [JBOSGI-443] - Intermittent WebAppNegativeTestCase failure
  • [JBOSGI-458] - Warning from webconsole at first access

Feature Request

  • [JBOSGI-192] - Access an OSGi service from EJB3
  • [JBOSGI-313] - OSGi service invocation from EE6 WebApp
  • [JBOSGI-474] - Add Dynamic Services to examples and distro

Task

  • [JBOSGI-434] - Release 1.0.0 Final
  • [JBOSGI-448] - Run OSGi integration tests as part of the AS7 Hudson job
  • [JBOSGI-461] - Update user guide with OSGi in AS7 usage
  • [JBOSGI-463] - Create OSGi in AS7 QuickStart guide
  • [JBOSGI-468] - Update to Arquillian 1.0.0.CR1 or later
  • [JBOSGI-469] - Migrate performance tests to new Arquillian API
  • [JBOSGI-471] - Remove JTA support for standalone Runtime
  • [JBOSGI-472] - Migrate documentation to confluence
  • [JBOSGI-481] - Fix Equinox performance tests

Core Framework 1.0.0

Bug

  • [JBOSGI-433] - [TCK] Intermittent StartLevelControl failure
  • [JBOSGI-439] - Bundle resolution error should provide reason
  • [JBOSGI-442] - Circular class load issue on lazy activation
  • [JBOSGI-444] - Service event failure on Runtime shutdown
  • [JBOSGI-450] - Bundle containing a BundleTracker causes subsequent bundle install/starts to fail
  • [JBOSGI-451] - Timeout starting Framework after shutdown
  • [JBOSGI-452] - ConcurrentModificationException in BundleManager.getBundles()
  • [JBOSGI-456] - Framework events may get delivered out of order
  • [JBOSGI-465] - Framework sets service mode while Container is shutting down
  • [JBOSGI-473] - Unsupported execution environment [OSGi/Minimum-1.2, CDC-1.1/Foundation-1.1, J2SE-1.4] we have [J2SE-1.5, , JavaSE-1.6]
  • [JBOSGI-476] - Cannot acquire start/stop lock for lazy bundles
  • [JBOSGI-477] - Unsupported execution environment OSGi/Minimum-1.1

Feature Request

  • [JBOSGI-441] - Allow for OSGi style Class Filtering

Task

  • [JBOSGI-238] - [TCK] Framework core functionality
  • [JBOSGI-255] - [TCK] Core Framework
  • [JBOSGI-455] - Add services that represent the RESOLVED and ACTIVE states


Enjoy 

Thursday, November 11, 2010

JBoss AS7 OSGi Integration

Now that JBossAS-7.0.0.Alpha1 has come out I'd like to talk about how JBoss OSGi integrates with it and what benefits our community will gain from the OSGi integration. In this post I'll show how non-OSGi components like EJB3, Servlet, CDI Beans, ManagedBeans, MBeans, etc. can access OSGi services and vice versa.

OSGi in the EE Landscape

Generally I believe that OSGi is very much an all-or-nothing technology. If your application is an OSGi bundle (or many of them) you must run within an OSGi Framework and can only have access to services that are provided by other OSGi bundles. In todays EE appplication landscape where folks take (standard) functionality like JPA, Servlet3, CDI, EJB3, JCA, JMS, etc for granted they would need a container that offers these services not only to EE applications but also to OSGi bundles.

In Mar-2010 the OSGi Enterprise Expert Group, has come up with its first EE OSGi specification, which IMHO offers a fraction of what we currently see in modern EE applications. There is no JSP, EJB3, JCA, JMS, CDI not even Annotation processing. The EEG is working on completing the picture, but this is unlikely to become generally available in the near future. David and I are members of this group, so its partially also our call.

Of course, if your application is such that it only requires what is available in OSGi Core, Compendium and Enterprise already (and many of todays larger OSGi applications do) OSGi is a good candidate to solve general modularity and service integration issues.

The JBoss approach to OSGi

At JBoss we take a different approach. With the upcoming AS7 the basic building block is a "module" which can have dependencies on other modules. There are services with basic lifecycle that can also have dependencies on each other. Our OSGi Framework implementation builds on top of this basic infrastructure like any other JBoss subsystem. We integrate at the lowest possible level. Such it becomes possible to access an OSGi service from a non-OSGi component and vice versa.

The questions about the suitability of OSGi is at the very heart of these integration issues IMHO. If EJB3, CDI, etc. offer value-add then it must be possible to leverage that value from an OSGi application. Otherwise you have an either/or situation where you gain benefits from one technology but at the same time loose benefits from another technology that cannot be accessed any more.

You can think of this in terms of two intersecting sets of technology (i.e. EE6, OSGi). I'm interested in the superset of the two and not so much in the smaller individual sets. The aim is that JBoss users who already write great and non trivial EE applications can now "also" use OSGi to address modularity issues in these very applications.

The Theory behind JBoss OSGi

Any deployment, and in-fact any jar you see in AS7, is a Module. A Module may have dependencies and a number of attached ResourceLoaders. A ResourceLoader is typically backed by a VirtualFile. A Module can choose to export a subset of the resource paths from its attached ResourceLoaders. This is equivalent to the Export-Package notion in OSGi. The dependencies on other Modules are equivalent to Require-Bundle. A module can choose which paths to import from a given dependency. A Module can choose to re-export the paths that it imports from a dependency.

An OSGi Bundle is a jar with a set of metadata in it's Manifest. To name a few, we have
Generally speaking however, these are Requirements and Capabilities that a Bundle may have. In JBoss OSGi we translate the OSGi metadata to these general Requirements and Capabilities and feed the Standalone Resolver with it. At bundle resolution time, the Resolver tries to find a consistent set of Wires, such that every non-optional Requirement is wired to a Capability. This is a non-trivial process that leads to consistent class spaces. If successful, the Bundle is said to be RESOLVED.

At the end of the day, every OSGi Framework conceptually creates dependencies between the installed bundles and limits the set of paths at the exporting and importing side of the wires.

So we get the set of Requirements and Capabilities from somewhere, feed them to our resolver abstraction and finally use the resulting Wires to setup the ModuleSpec. The source for our Requirements and Capabilities is not limited to the OSGi manifest. Any AS7 deployment can generate resolver metadata and make itself known to the OSGi layer.

At deploy time every Bundle becomes an AS7 Module and every AS7 Module may become an OSGi Bundle when its provides it's set of resolver metadata and registers with the OSGi layer. OSGi bundles are therefore a subset of the modules that exist in the AS7 module layer.

Lets look at some code ...

Bundle accessing a Non-OSGi Module

In a test case we construct and deploy a module that provides a simple Echo service to the running AS7.

JavaArchive archive = ShrinkWrap.create(JavaArchive.class, "example-xservice-target-module");
archive.addClasses(Echo.class, EchoService.class, TargetModuleActivator.class);
String activatorPath = "META-INF/services/" + ServiceActivator.class.getName();
archive.addResource(getResourceFile("xservice/target-module/" + activatorPath), activatorPath);

The module contains a few classes and a ServiceActivator. The ServiceActivator is the equivalent of a BundleActivator. When the module gets activated we register a simple EchoService.

public static void addService(BatchBuilder batchBuilder)
{
  serviceBuilder = batchBuilder.addService(SERVICE_NAME, new EchoService());
  serviceBuilder.addAliases(ServiceName.of(Constants.JBOSGI_PREFIX, Echo.class.getName()));
  serviceBuilder.setInitialMode(Mode.ACTIVE);
  log.infof("Service added: %s", SERVICE_NAME);
}

Please note, that the service is also registered with an alias. The JBoss OSGi service registry uses the ServiceContainer that contains all services that exist in AS7. For service lookup the OSGi API uses the FQN of an interface that the requested service implements. The alias is needed for this OSGi API call to succeed

ServiceReference sref = context.getServiceReference(Echo.class.getName());
Echo service = (Echo)context.getService(sref);
service.echo("hello world");

When the module is deployed, it is not automatically registered with the OSGi layer. A bundle that imports the package of the Echo interface would not resolve.

Any module can however be registered with the OSGi layer.

ModuleIdentifier moduleId = ModuleIdentifier.create("deployment.example-xservice-target-module");
registerModuleWithBundleManager(moduleId);

Currently this must be done explicitly - there is no automatic registration. In our case the module does not provide resolver metadata explicitly, so it will be generated. We add a PackageCapability for every exported path.

ResolverPlugin resolverPlugin = getPlugin(ResolverPlugin.class);
XModuleBuilder builder = resolverPlugin.getModuleBuilder();
builder.createModule(symbolicName, version, 0);
builder.addBundleCapability(symbolicName, version);
for (String path : module.getExportedPaths())
{
  if (path.startsWith("META-INF"))
    continue;

  String packageName = path.replace('/', '.');
  builder.addPackageCapability(packageName, null, null);
}
XModule resModule = builder.getModule();

Now that the module is known to the OSGi layer. The resolver can use the module's capabilities for bundle resolution. We can now install and start a real OSGi bundle that imports the echo package and calls the Echo service in its BundleActivator.

mvn -Dtarget.container=jboss70x -Dtest=BundleAccessesModuleServiceTestCase test

Running org.jboss.test.osgi.example.xservice.BundleAccessesModuleServiceTestCase
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.009 sec

17:21:16,469 INFO  Activating deployment: example-xservice-target-module
17:21:16,732 INFO  Service added: service jboss.osgi.xservice.target
17:21:16,733 INFO  ModuleIdentifier: module:deployment.example-xservice-target-module:main
17:21:16,854 INFO  Install bundle: deployment.example-xservice-target-module:0.0.0
17:21:18,135 INFO  Install bundle: example-xservice-client-bundle:1.0.0
17:21:18,196 INFO  Echo: hello world
17:21:18,199 INFO  Bundle started: example-xservice-client-bundle:1.0.0
17:21:18,222 INFO  Bundle uninstalled: example-xservice-client-bundle:1.0.0
17:21:18,235 INFO  Bundle uninstalled: deployment.example-xservice-target-module:0.0.0
17:21:18,292 INFO  Undeployed example-xservice-target-module 

Going forward, we can now define a set of rules for every possible AS7 deployment on whether or not it gets registered with the OSGi layer and how it generates it's caps/reqs.

A possible question could be:
"Why isn't the target module a real OSGi bundle in the first place? In other words, why does it not define its capabilities in a standard way in the OSGi manifest"
The target module may have dependencies on other services that are not available in the OSGi layer. Or simply because the folks that provide the target module do not feel sufficiently familiar with OSGi. The important thing here is that with AS7 we have a migration path to and from OSGi.

Module accessing an OSGi Service

This scenario requires a little more thought. Let's first explore the seemingly obvious approach.
  • The target bundle contains the Echo interface
  • The client module has a dependency on the target bundle
Every Module has a ModuleIdentifier, which in case of an OSGi bundle is constructed according to the following pattern
jbosgi.[bundle-symbolic-name]:[bundle-version]
In case of the target bundle that is used in this test case, we would have
jbosgi.example-xservice-target-bundle:1.0.0
This is the Require-Bundle semantic, which is generally considered bad practise in OSGi. In the OSGi core spec we find the following chapter

3.12.3 Issues With Requiring Bundles

The preferred way of wiring bundles is to use the Import-Package and
Export-Package headers because they couple the importer and exporter to a
much lesser extent. Bundles can be refactored to have a different package
composition without causing other bundles to fail.

The Require-Bundle header provides a way for a bundle to bind to all the
exports of another bundle, regardless of what those exports are. Though this
can seem convenient at first, it has a number of drawbacks:

Split Packages – Classes from the same package can come from different bundles with Require bundle, such a package is called a split package. Split packages have the following drawbacks:
  • Completeness – Split packages are open ended, it is difficult to guarantee that all the intended pieces of a split package have actually been included.
  • Ordering – If the same classes are present in more than one required bundle, then the ordering of Require-Bundle is significant. A wrong ordering can cause hard to trace errors, similar to the traditional class path model of Java.
  • Performance – A class must be searched in all providers when packages are split. This potentially increases the number of times that a ClassNotFoundException must be thrown which can potentially introduce a significant overhead.
  • Confusing – It is easy to find a setup where there is lots of potential for confusion. For example, the following setup is non-intuitive.

    A: Export-Package: p;uses:=q
       Import-Package: q
    B: Export-Package: q
    C: Export-Package: q
    D: Require-Bundle: B, C
       Import-Package: p
    


    In this example, bundle D merges the split package q from bundles B and
    bundle C, however, importing package p from bundle A puts a uses constraint on package p for package q. This implies that bundle D can see the valid package q from bundle B but also the invalid package q from bundle C. This wiring is allowed because in almost all cases there will be no problem. However, the consistency can be violated in the rare case when package C.q contains classes that are also in package B.q.

Mutable Exports – The feature of visibility:=reexport that the export signature of the requiring bundle can unexpectedly change depending on the export signature of the required bundle.

Shadowing – The classes in the requiring bundle that are shadowed by those in a required bundle depend on the export signature of the required bundle and the classes the required bundle contains. (By contrast, Import-Package, except with resolution:=optional, shadows whole packages regardless of the exporter.)

Good practise is to use Export-Package to declare a PackageCapability and Import-Package to declare a PackageRequirement. In this way importer does not need to care where a package comes from and the provider of the exported package is free to move the package around from one bundle to another without breaking the importer.

OSGi supports the notion of bundle revisions. This does not show up in the public API. However, every time a bundle is updated (i.e. the bytes change) the framework creates a new revision with potentially the same Bundle-SymbolicName and Bundle-Version. Internally we append the revision suffix to the ModuleIdentifier. As a consequence, binding to a particular ModuleIdentifier identifier ultimately means binding to a particular revision, which negates the purpose update.


Having said all this, we decide against  putting the Echo interface (i.e. the API)  in the target bundle. Instead we put it in a Module which can be registered with the OSGi layer as described above.

JavaArchive getAPIModuleArchive() throws Exception
{
  JavaArchive archive = ShrinkWrap.create(JavaArchive.class, "example-xservice-api-module");
  archive.addClasses(Echo.class);
  archive.addDirectory("META-INF");
  return archive;
}

// Deploy the module that contains the API
String apiDeploymentName = getRemoteRuntime().deploy(getAPIModuleArchive());

// Register the API module with the OSGi layer
ModuleIdentifier apiModuleId = ModuleIdentifier.create("deployment." + apiDeploymentName);
registerModuleWithBundleManager(apiModuleId);

The target bundle is a plain OSGi bundle that imports the package of the Echo service and registers an instance of that service with the OSGi service registry.

@Override
   public void start(final BundleContext context) throws Exception
   {
      context.registerService(Echo.class.getName(), new EchoImpl(), null);
   }

   static class EchoImpl implements Echo
   {
      @Override
      public String echo(String message)
      {
         log.infof("Echo: %s", message);
         return message;
      }
   }

The client module needs to define a dependency on the API module. In JBoss AS module dependencies can be defined in the manifest like this

Manifest-Version: 1.0
Dependencies: org.osgi.core,deployment.example-xservice-api-module

Ouch, this uses an AS7 proprietary manifest header as well as Require-Bundle semantics. For AS7 internal modules (i.e. the ones that we control) this direct and hard coded approach is key to bootstrap performance. It should however be carefully considered before putting in user deployments.

Never mind, for sake of this exercise we continue. When the client module activates it registers a Service.

public class EchoInvokerService implements Service<Void>
{
   InjectedValue<BundleContext> injectedBundleContext = new InjectedValue<BundleContext>();

   static void addService(BatchBuilder batchBuilder)
   {
      EchoInvokerService service = new EchoInvokerService();
      BatchServiceBuilder serviceBuilder = batchBuilder.addService(SERVICE_NAME, service);
      serviceBuilder.addDependency(ServiceName.parse("jboss.osgi.context"), BundleContext.class, service.injectedBundleContext);
      serviceBuilder.setInitialMode(Mode.ACTIVE);
      log.infof("Service added: %s", SERVICE_NAME);
   }

   @Override
   public void start(StartContext context) throws StartException
   {
      BundleContext systemContext = injectedBundleContext.getValue();
      ServiceReference sref = systemContext.getServiceReference(Echo.class.getName());
      Echo service = (Echo)systemContext.getService(sref);
      service.echo("hello world");
   }

   @Override
   public void stop(StopContext context)
   {
   }

   @Override
   public Void getValue() throws IllegalStateException
   {
      return null;
   }
}

This is a native AS7 service that gets the OSGi system BundleContext injected before it is started. Currently, when you download and try jboss-7.0.0.Alpha1 you will see that it bootstraps in very little time (i.e. less than 3sec). This is also because the OSGi subsystem is activated lazily. Installing a service like the one above with initial mode ACTIVE would cause all services that it depends on also to become ACTIVE. The OSGi subsystem would activate and the framework would start.

In the start method of the EchoInvokerService you see the usage of OSGi API that calls the target service.

mvn -Dtarget.container=jboss70x -Dtest=ModuleAccessesBundleServiceTestCase test

Running org.jboss.test.osgi.example.xservice.ModuleAccessesBundleServiceTestCase
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.009 sec

11:10:22,040 INFO  Activating deployment: example-xservice-api-module
11:10:22,228 INFO  Install bundle: deployment.example-xservice-api-module:0.0.0
11:10:23,140 INFO  Install bundle: example-xservice-target-bundle:1.0.0
11:10:23,251 INFO  ModuleIdentifier: module:jbosgi.example-xservice-target-bundle:1.0.0
11:10:23,252 INFO  Bundle started: example-xservice-target-bundle:1.0.0
11:10:23,413 INFO  Activating deployment: example-xservice-client-module
11:10:23,516 INFO  Service added: service jboss.osgi.xservice.invoker
11:10:23,518 INFO  ModuleIdentifier: module:deployment.example-xservice-client-module:main
11:10:23,520 INFO  Echo: hello world
11:10:23,602 INFO  Undeployed example-xservice-client-module
11:10:23,661 INFO  Bundle uninstalled: example-xservice-target-bundle:1.0.0
11:10:23,687 INFO  Bundle uninstalled: deployment.example-xservice-api-module:0.0.0
11:10:23,739 INFO  Undeployed example-xservice-api-module

Conclusion

I have shown how in principle an OSGi bundle can access an arbitrary AS7 service and vice versa. With this distribution we have reached about 80% OSGi Core TCK compliance, which we hope to complete by Q2/2011. The final version of AS7 is currently scheduled for May/2011.

I am hopeful that by the time AS7 goes final we have examples of various EE components interacting with OSGi and vice versa. The aim is that you can use the benefits of OSGi modularity and its service layer in non-trivial applications that make use of AS7 middle ware.

You may also want to check out David's post on "Using OSGi in JBoss AS7".

May this be useful
-thomas

Tuesday, October 26, 2010

JBossOSGi Hudson on Amazon Elastic Cloud

Following Amazons announcement for an AWS Free Usage Tier, I thought I would give it a try. Here my account.

Creating the Instance

If you are new to Amazon EC2 you need to create an account and sign in. Once you are in the AWS Management Console, you can create a key pair like this




I like to have control over the connection ports that are accessible on my instance, so I create a new Security Group, which I can later use with my instance.


Next, you navigate to Instances and launch an new Amazon Linux Micro instance. Basic 32-bit Amazon Linux is a good choice because it can be upgraded to small. A 64-bit instance can only be upgraded to large and above.


Select the key and security group that you created earlier, hit launch and your Linux Micro instance should be up and running.

Accessing you Linux Instance

When you created your keypair you were offered to download and save the private key to a secure location. You can now ssh into you newly created instance like this

chmod 600 .ssh/tdiesler-ec2.pem
ssh -i .ssh/tdiesler-ec2.pem ec2-user@ec2-50-16-7-151.compute-1.amazonaws.com
Last login: Mon Oct 25 13:32:07 2010 from p5b04e10a.dip.t-dialin.net

       __|  __|_  )  Amazon Linux AMI
       _|  (     /     Beta
      ___|\___|___|

See /etc/image-release-notes for latest release notes. :-)
[ec2-user@ip-10-112-99-96 ~]$ 

Installing Hudson on Tomcat

Amazon Liunx supports the Yum Package Manager, which I happen to know quite well because I run Fedora. Lets first check if Tomcat is there

$ yum list installed | grep tomcat
$ 

Ok, that's not the case, so see what's available.

$ yum list available | grep tomcat
...
tomcat6.noarch                               6.0.24-8.12.amzn1 
tomcat6-admin-webapps.noarch                 6.0.24-8.12.amzn1 
tomcat6-docs-webapp.noarch                   6.0.24-8.12.amzn1 
tomcat6-javadoc.noarch                       6.0.24-8.12.amzn1 
tomcat6-log4j.noarch                         6.0.24-8.12.amzn1 
tomcat6-webapps.noarch                       6.0.24-8.12.amzn1 

Let's install tomcat6

[ec2-user@ip-10-112-99-96 ~]$ 
$ sudo yum install tomcat6
...
Installed:
  tomcat6.noarch 0:6.0.24-8.12.amzn1
...
Complete!

Furthermore, we need Git, Ant and Maven. Git and Ant we can install with yum like above. Maven however does not seem to be available in yum. So lets get it from the Apache Maven download page.

$ wget http://apache.linux-mirror.org/maven/binaries/apache-maven-3.0-bin.zip
...
2010-10-25 14:40:23 (44.4 KB/s) - “apache-maven-3.0-bin.zip” saved [2919967/2919967]
$ unzip apache-maven-3.0-bin.zip
$ sudo mkdir /usr/java
$ sudo mv apache-maven-3.0 /usr/java
$ sudo ln -s /usr/java/apache-maven-3.0 /usr/java/apache-maven
$ mkdir bin
$ ln -s /usr/java/apache-maven/bin/mvn bin/mvn
$ mvn
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.108s
[INFO] Finished at: Mon Oct 25 14:44:26 UTC 2010
[INFO] Final Memory: 2M/7M
[INFO] ------------------------------------------------------------------------

Amazon Linux comes with openjdk, which is fine for most purposes.  At the time of this writing however, I saw issues with the maven build and Hudson Captcha. So lets install the Oracle JDK 

$ cd /usr/java
$ sudo wget http://cds.sun.com/is-bi/...jdk-6u22-linux-i586.bin
$ sudo chmod +x jdk-6u22-linux-i586.bin 
$ sudo ./jdk-6u22-linux-i586.bin 
$ sudo ln -s jdk1.6.0_22 jdk1.6
$ sudo rm jdk-6u22-linux-i586.bin

$ vi ~/.bash_profile
...
export JAVA_HOME="/usr/java/jdk1.6"
export PATH=$HOME/bin:$JAVA_HOME/bin:$PATH

$ java -version
java version "1.6.0_22"
Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
Java HotSpot(TM) Client VM (build 17.1-b03, mixed mode, sharing)

 
Building JBoss OSGi

The JBoss OSGi umbrella project that holds the Hudson setup as an integral part of the code base is hosted on GitHub.

$ mkdir git; cd git
$ git clone git://github.com/jbosgi/jbosgi.git; cd jbosgi
$ git submodule init
$ git submodule update

JBoss community projects often do not contain the JBoss Nexus Configuration. Follow the instructions on Maven Getting Started - Users to setup your ~/.m2/settings.xml

Now, lets build the umbrella for the first time

$ mvn install
...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] JBossOSGi JMX ..................................... SUCCESS [1:00.822s]
[INFO] JBossOSGi JMX API ................................. SUCCESS [1:44.154s]
[INFO] JBossOSGi VFS ..................................... SUCCESS [0.008s]
[INFO] JBossOSGi VFS API ................................. SUCCESS [5.442s]
[INFO] JBossOSGi SPI ..................................... SUCCESS [24.504s]
[INFO] JBossOSGi JMX Bundle .............................. SUCCESS [15.142s]
[INFO] JBossOSGi JMX iTests .............................. SUCCESS [2:37.160s]
[INFO] JBossOSGi VFS VFS30 ............................... SUCCESS [9.390s]
[INFO] JBossOSGi Deployment .............................. SUCCESS [12.598s]
[INFO] JBossOSGi Framework ............................... SUCCESS [0.047s]
[INFO] JBossOSGi Framework Core .......................... SUCCESS [32.110s]
[INFO] JBossOSGi Framework iTest ......................... SUCCESS [2:33.963s]
[INFO] JBossOSGi ......................................... SUCCESS [0.026s]
[INFO] JBossOSGi Reactor ................................. SUCCESS [0.006s]
[INFO] JBossOSGi Testsuite ............................... SUCCESS [0.017s]
[INFO] JBossOSGi Testsuite Examples ...................... SUCCESS [3:39.319s]
[INFO] JBossOSGi Testsuite Functional .................... SUCCESS [1:40.045s]
[INFO] JBossOSGi Testsuite Performance ................... SUCCESS [19.179s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16:48.812s
[INFO] Finished at: Tue Oct 26 12:30:18 UTC 2010
[INFO] Final Memory: 22M/68M
[INFO] ------------------------------------------------------------------------

This will take a while, so lets continue with Tomcat and Hudson.

Installing Hudson

In our security group we opened port 8280. Tomcat however runs on 8080 by default. Lets change that.

$ sudo vi /etc/tomcat6/server.xml 
...
    <connector port="8280" connectiontimeout="20000" 
        protocol="HTTP/1.1" redirectport="8443">

Now we configure tomcat to run as the ec2-user, which allows Hudson to reuse the ec2-user's maven repository. HUDSON-HOME can also be set in the same config file.

$ sudo vi /etc/tomcat6/tomcat6.conf
...
# Where your java installation lives
JAVA_HOME="/usr/java/jdk1.6"

# Where HUDSON has its home
HUDSON_HOME=/usr/share/tomcat6/workspace/hudson-home

# What user should run tomcat
TOMCAT_USER=ec2-user

Make sure the ec2-user is in the 'tomcat' group so it can write files to HUDSON_HOME during setup.

$ sudo usermod -g tomcat ec2-user
$ mkdir -p /usr/share/tomcat6/workspace/hudson-home

You might need to re-login for the initial login group change to take effect.

JBossOSGi simplifies Hudson setup a great deal, by including configuration and jobs in it codebase. This also allows us to recreate the QA environment for any given version by running a simple Ant task.

$ cd git/jbosgi/hudson
$ cp ant.properties.example ant.properties
$ ant hudson-setup
...
init-hudson:
     [echo] hudson.home = /usr/share/tomcat6/workspace/hudson-home
...
get-hudson:
      [get] Getting: http://hudson-ci.org/download/war/1.382/hudson.war
      [get] Getting: http://hudson-ci.org/download/plugins/github/0.2/github.hpi
      [get] Getting: http://hudson-ci.org/download/plugins/git/1.1/git.hpi
...
hudson-tomcat-setup:
     [copy] Copying 1 file to /usr/share/tomcat6/workspace/hudson-home/plugins
     [copy] Copying 1 file to /usr/share/tomcat6/workspace/hudson-home/plugins

hudson-setup:
     [copy] Copying 2 files to /usr/share/tomcat6/workspace/hudson-home
     [copy] Copying 9 files to /usr/share/tomcat6/workspace/hudson-home/jobs
     [echo] 
     [echo] *************************************
     [echo] * Hudson setup successfully          
     [echo] * sudo service tomcat6 restart       
     [echo] *************************************

$ sudo service tomcat6 restart
Stopping tomcat6:                                          [  OK  ]
Starting tomcat6:                                          [  OK  ]

Now we should be able to connect to the running Hudson instance.


Hudson Security


By default, Hudson is setup with strict project based security. Only the 'ec2-user' or whoever you configured in ${hudson.admin} can modify the configuration and run the jobs. Lets sign-up as ec2-user


You should now be able to run the jobs.

Upgrading the Instance Type

The Amazon free offer is limited to Micro instances. With it's 613MB of RAM not surprisingly it turns out that this is not sufficient to run the JBossOSGi QA in an acceptable time.

This is however not a problem, because the instance can be stopped and it's type can be changed to m1.small or m1.large depending on what you are willing to pay.

$ ec2-stop-instances i-10429a7d
$ ec2-modify-instance-attribute --instance-type=m1.large i-10429a7d
$ ec2-start-instances i-10429a7d

Note, that this can only be done with instances that are backed by the Elastic Block Store.

Creating an Elastic IP

Every time you stop/start an EC2 instance it's public DNS entry changes. Amazon's solution to this is the Elastic IP. You pay for the IP when your instance is not connected.



This IP address can be associated with your favorite domain name. In our case its a dyndns.org forward. So what was formerly known as


now points to the EC2 instance that we've just setup.


May this be useful
-thomas