Deploying Hawtio on JBoss EAP 6

In this article I describe how to successfully deploy Hawtio on JBoss EAP.

You would need

• JBoss EAP 6 (I have used JBoss EAP 6.2 which can be downloaded from here) 
• Hawtio Web-Application (the version 1.4.37 can be obtained from here)

Before actually deploying Hawtio you would need to do some small changes on hawtio-no-slf4j-1.4.37.war.

• To WEB-INF/lib/guava-15.0.jar add META-INF/beans.xml with the following content

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://java.sun.com/xml/ns/javaee"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:weld="http://jboss.org/schema/weld/beans"
       xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/beans_1_0.xsd
                           http://jboss.org/schema/weld/beans http://jboss.org/schema/weld/beans_1_1.xsd">
 <weld:scan>
   <weld:exclude name="com.google.**"/>
 </weld:scan>
</beans>

        as otherwise the application will not deploy successfully and will report the
        following error

21:15:28,074 INFO  [org.jboss.weld.deployer] (MSC service thread 1-3) JBAS016005: Starting Services for CDI deployment: hawtio-no-slf4j-1.4.37.war
21:15:28,091 INFO  [org.jboss.weld.deployer] (MSC service thread 1-4) JBAS016008: Starting weld service for deployment hawtio-no-slf4j-1.4.37.war
21:15:29,659 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-7) MSC000001: Failed to start service jboss.deployment.unit."hawtio-no-slf4j-1.4.37.war".WeldStartService: org.jboss.msc.service.StartException in service jboss.deployment.unit."hawtio-no-slf4j-1.4.37.war".WeldStartService: Failed to start service
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55]
        at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_55]
Caused by: org.jboss.weld.exceptions.DeploymentException: WELD-001408 Unsatisfied dependencies for type [Set<Service>] with qualifiers [@Default] at injection point [[parameter 1] of [constructor] @Inject com.google.common.util.concurrent.ServiceManager(Set<Service>)]
        at org.jboss.weld.bootstrap.Validator.validateInjectionPoint(Validator.java:312)
        at org.jboss.weld.bootstrap.Validator.validateInjectionPoint(Validator.java:281)
        at org.jboss.weld.bootstrap.Validator.validateBean(Validator.java:144)
        at org.jboss.weld.bootstrap.Validator.validateRIBean(Validator.java:164)
        at org.jboss.weld.bootstrap.Validator.validateBeans(Validator.java:383)
        at org.jboss.weld.bootstrap.Validator.validateDeployment(Validator.java:368)
        at org.jboss.weld.bootstrap.WeldBootstrap.validateBeans(WeldBootstrap.java:379)
        at org.jboss.as.weld.WeldStartService.start(WeldStartService.java:64)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
        ... 3 more

21:15:29,877 ERROR [org.jboss.as.server] (HttpManagementService-threads - 3) JBAS015870: Deploy of deployment "hawtio-no-slf4j-1.4.37.war" was rolled back with the following failure message:
{"JBAS014671: Failed services" => {"jboss.deployment.unit.\"hawtio-no-slf4j-1.4.37.war\".WeldStartService" => "org.jboss.msc.service.StartException in service jboss.deployment.unit.\"hawtio-no-slf4j-1.4.37.war\".WeldStartService: Failed to start service
    Caused by: org.jboss.weld.exceptions.DeploymentException: WELD-001408 Unsatisfied dependencies for type [Set<Service>] with qualifiers [@Default] at injection point [[parameter 1] of [constructor] @Inject com.google.common.util.concurrent.ServiceManager(Set<Service>)]"}}
21:15:29,898 INFO  [org.jboss.weld.deployer] (MSC service thread 1-2) JBAS016009: Stopping weld service for deployment hawtio-no-slf4j-1.4.37.war
21:15:30,003 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-5) JBAS015877: Stopped deployment hawtio-no-slf4j-1.4.37.war (runtime-name: hawtio-no-slf4j-1.4.37.war) in 128ms

• Add WEB-INF/jboss-web.xml as follows

<jboss-web>
  <context-root>/hawtio</context-root>
</jboss-web>

•  Add WEB-INF/jboss-deployment-structure.xml as follows

<?xml version="1.0"?>
<jboss-deployment-structure>
  <deployment>
    <dependencies>
      <module name="org.apache.log4j"/>
      <module name="org.slf4j"/>
      <module name="org.slf4j.impl"/>
    </dependencies>
  </deployment>
</jboss-deployment-structure>

• Remove WEB-INF/classes/log4j.properties

Now you can deploy hawtio-no-slf4j-1.4.37.war to JBoss EAP 6

Using ELK's JMX Plugin to gather JBoss EAP metrics

In this short blog post I outline how to use the ELK (ElasticSearch/LogStash/Kibana) stack to gather data via JMX from JBoss EAP 6.

Ingredients:

- ElasticSearch 1.4.0
- Logstash 1.4.2
- Kibana 3.1.2
- JBoss EAP 6.2

After having gone throught the downloading and installation process first we need to extract some libs out of the modules for JBoss EAP 6.2 that are needed for getting remote JMX calls to work.

In elastic-search-1.4.0/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib create a file jboss_remoting.rb

module JMX
  module JBoss
    java_import java.lang.System

    class JBossRemoting

      sep = System.get_property 'file.separator'
      home = System.get_property 'jboss.home'

      if home != nil
        modulebase = [home, 'modules','system','layers','base','org','jboss'].join(sep)

        libs = [
          ['remoting-jmx','main'],
          ['remoting3','main'],
          ['logging','main'],
          ['xnio','main'],
          ['xnio','nio','main'],
          ['sasl','main'],
          ['marshalling','main'],
          ['marshalling','river','main'],
          ['as','cli','main'],
          ['staxmapper','main'],
          ['as','protocol','main'],
          ['dmr','main'],
          ['as','controller-client','main'],
          ['threads','main']
        ]

        for index in 0 ... libs.size
          Dir[modulebase + sep + libs[index].join(sep) + sep + "*.jar"].each {|file| require file }
        end
      end
    end
  end
end

In elastic-search-1.4.0/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib/jmx4r.rb after

require 'jruby'

        add

require 'jboss_remoting'

In logstash-1.4.2/config/logstash.conf I created the following configuration

input {
jmx{
    path => "logstash-1.4.2/jmx"
    polling_frequency => 30
    type => "jmx"
    nb_thread => 4
  }
}

output {
  elasticsearch { host => localhost }
}

and in logstash-1.4.2/jmx/jmx.conf I configured

{
  // JBoss Remoting JMX URL
  "url" : "service:jmx:remoting-jmx://localhost:9999",
  //username to connect to jmx
  "username" : "<your-user>",
  //password to connect to jmx
  "password": "<your-password>",
  "alias" : "jmx.instance1.elasticsearch",
  //List of JMX metrics to retrieve
  "queries" : [
    {
      "object_name" : "java.lang:type=Memory",
      "attributes" : [ "HeapMemoryUsage", "NonHeapMemoryUsage" ],
      "object_alias" : "Memory"
    }, {
      "object_name" : "java.lang:type=Runtime",
      "attributes" : [ "Uptime", "StartTime" ],
      "object_alias" : "Runtime"
    }, {
      "object_name" : "java.lang:type=Threading",
      "attributes" : [ "ThreadCount", "TotalStartedThreadCount", "DaemonThreadCount", "PeakThreadCount" ],
      "object_alias" : "Threading"
    }, {
      "object_name" : "java.lang:type=OperatingSystem",
      "attributes" : [ "OpenFileDescriptorCount", "FreePhysicalMemorySize", "CommittedVirtualMemorySize", "FreeSwapSpaceSize", "ProcessCpuLoad", "ProcessCpuTime", "SystemCpuLoad", "TotalPhysicalMemorySize", "TotalSwapSpaceSize", "SystemLoadAverage" ],
      "object_alias" : "OperatingSystem"
    } ]
}

As the original JMX input plugin only supports configuration via host and port you need to apply the pull request 141 (https://github.com/elasticsearch/logstash-contrib/pull/141) to your local installation, i.e. by downloading https://raw.githubusercontent.com/jcordes73/logstash-contrib/master/lib/logstash/inputs/jmx.rb and save it to logstash-1.4.2/lib/logstash/inputs

Now you need to setup a JBoss management user via jboss-eap-6.2/bin/add-user.sh to reflect the username and password defined in jmx.conf above.

Because of a change in behaviour in regards to security you need to add the following configuration to elasticsearch-1.4.0/config/elasticsearch.yml

http.cors.enabled: true
http.cors.allow-origin: "*"

and also change http.enabled to true.

We need to set some environment variables so that the JBoss Remoting libraries can be found from an existing JBoss EAP installation:

JBOSS_HOME=/opt/jboss-eap-6.2
export JBOSS_HOME
JAVA_OPTS="-Djboss.home=$JBOSS_HOME"
export JAVA_OPTS

Now you can start ElasticSearch via elasticsearch-1.4.0/bin/elasticsearch and logstash via logstash-1.4.2/bin/logstash -f logstash-1.4.2/config/logstash.conf.

To finally see your JMX data via Kibana, open kibana-3.1.2/index.html with your favourite web-browser.

Hawtio authentication with LDAP on JBoss Fuse

Finally here is the second part on Hawtio authentication with LDAP, this time on JBoss Fuse / A-MQ 6.1

• Download JBoss Fuse 6.1 from https://access.redhat.com/jbossnetwork/restricted/listSoftware.html?downloadType=distributions&product=jboss.fuse&version=6.1.0 and follow the installation instructions

• Create a file named ldap-auth.xml and copy it into the deploy folder (you need to adjust the LDAP settings according to your structure)

<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:jaas="http://karaf.apache.org/xmlns/jaas/v1.0.0" xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0" xmlns:ext="http://aries.apache.org/blueprint/xmlns/blueprint-ext/v1.0.0">
<ext:property-placeholder placeholder-prefix="${" placeholder-suffix="}"/>
<jaas:config name="karaf" rank="2">
<jaas:module className="org.apache.karaf.jaas.modules.ldap.LDAPLoginModule" flags="required">
connection.url=ldap://<LDAP-IP>:389
connection.username=cn=Manager,dc=redhat,dc=com
connection.password=redhat
user.base.dn=ou=User,ou=ActiveMQ,dc=activemq,dc=redhat,dc=com
user.filter=(uid=%u)
user.search.subtree=true
role.base.dn=ou=User,ou=ActiveMQ,dc=activemq,dc=redhat,dc=com
role.filter=(uid=%u)
role.name.attribute=uid
role.search.subtree=true
authentication=simple
</jaas:module>
</jaas:config>
</blueprint> 

 (see also https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_A-MQ/6.1/html-single/Security_Guide/index.html#JAASAuth-LDAPLoginModule)

•  In etc/system.properties add

hawtio.authenticationEnabled=true
hawtio.realm=karaf
hawtio.role=admin
hawtio.rolePrincipalClasses=org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal

(see https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Fuse/6.1/html-single/Security_Guide/#WebConsole))

HornetQ-ActiveMQ Bridge

In case you need a messaging bridge between HornetQ on JBoss EAP and ActiveMQ on JBoss Fuse / A-MQ then the projects on Github at https://github.com/jcordes73/messaging-bridges maybe what you are after.

There I'm featuring

• JBoss Fuse / A-MQ 6.1
• JBoss EAP 6.x (at least 6.1.1 required)
• ActiveMQ on JBoss Fuse / A-MQ
• HornetQ on JBoss EAP 6
• ActiveMQ RAR Adapter on JBoss EAP 6
• SSL for JBoss Fuse / A-MQ <--> JBoss EAP communication
• XA

The bridge comes in two flavours

• Running on JBoss EAP
• Running on JBoss Fuse / A-MQ

How to configure local JMS transactions using Camel with WebSphere MQ on JBoss EAP 6.2

In this article I will show you how to configure local JMS transactions using WebSphere MQ with Camel on JBoss EAP 6.2:

Your Spring context setup should include the following information:

<bean id="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
            <property name="jndiName" value="java:jboss/eis/MQCF" />
            <property name="lookupOnStartup" value="false" />
            <property name="cache" value="true" />
            <property name="proxyInterface" value="javax.jms.ConnectionFactory" />
 </bean>

 <bean id="jmsTransactionManager"           class="org.springframework.transaction.jta.JtaTransactionManager">
            <property name="transactionManagerName" value="java:/TransactionManager" />
  </bean>

  <bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
            <property name="connectionFactory" ref="jmsConnectionFactory" />
            <property name="transacted" value="true" />
            <property name="transactionManager" ref="jmsTransactionManager" />
   </bean>

The important stuff here is to use JtaTransactionManager.

In the JBoss EAP 6.2 configuration the WebSphere resource-adapter should be configured like this:

<subsystem xmlns="urn:jboss:domain:resource-adapters:1.1">
     <resource-adapters>
             <resource-adapter id="wmq.jmsra.rar">
                   <archive>
                        wmq.jmsra.rar
                    </archive>
                    <transaction-support>LocalTransaction</transaction-support>
                    <connection-definitions>
                        <connection-definition class-name="com.ibm.mq.connector.outbound.ManagedConnectionFactoryImpl" jndi-name="java:jboss/eis/MQCF" enabled="true" use-java-context="true" pool-name="WMQConnectionFactoryPool" use-ccm="true">
                            <config-property name="port">
                                MQ.PORT
                            </config-property>
                            <config-property name="channel">
                                MQ.CHANNEL
                            </config-property>
                            <config-property name="hostName">
                                MQ.HOST
                            </config-property>
                            <config-property name="transportType">
                                CLIENT
                            </config-property>
                            <config-property name="queueManager">
                                MQ.QUEUEMANAGER
                            </config-property>
                            <pool>
                                <min-pool-size>1</min-pool-size>
                                <max-pool-size>10</max-pool-size>
                            </pool>
                        </connection-definition>
                  </connection-definitions>
            </resource-adapter>
      </resource-adapters>
</subsystem>

In case you want to use the JBoss CLI to set things up you can use the following script:

if (outcome != success) of /subsystem=resource-adapters/resource-adapter=wmq.jmsra.rar:read-resource() /subsystem=resource-adapters/resource-adapter=wmq.jmsra.rar:add(archive=wmq.jmsra.rar,transaction-support=XATransaction) /subsystem=resource-adapters/resource-adapter=wmq.jmsra.rar/connection-definitions=WMQConnectionFactoryPool:add(class-name=com.ibm.mq.connector.outbound.ManagedConnectionFactoryImpl, jndi-name="java:jboss/eis/MQCF", enabled=true, use-java-context=true, use-ccm=true,min-pool-size=1, max-pool-size=10) /subsystem=resource-adapters/resource-adapter=wmq.jmsra.rar/connection-definitions=WMQConnectionFactoryPool/config-properties=connectionNameList:add(value="${wmq.connectionNameList}") /subsystem=resource-adapters/resource-adapter=wmq.jmsra.rar/connection-definitions=WMQConnectionFactoryPool/config-properties=channel:add(value="${wmq.channel}") /subsystem=resource-adapters/resource-adapter=wmq.jmsra.rar/connection-definitions=WMQConnectionFactoryPool/config-properties=transportType:add(value="CLIENT") /subsystem=resource-adapters/resource-adapter=wmq.jmsra.rar/connection-definitions=WMQConnectionFactoryPool/config-properties=queueManager:add(value="${wmq.queueManager}") end-if

In this script I'm using system-variables that you could define in a property file and pass it on during JBoss EAP via the -P parameter.

Finally to get everything working you should replace the MQ.* values with you actual settings and copy wmq.jmsra.rar to JBOSS_HOME/standalone/deployments.

In case you are using an EAR to package your application and you don't want include WebSphere MQ libraries, you can add a jboss-deployment-structure.xml inside the META-INF directory of the EAR with the following content:

<jboss-deployment-structure>
    <deployment>
        <dependencies>
            <module name="deployment.wmq.jmsra.rar" export="true" />
        </dependencies>
    </deployment>
</jboss-deployment-structure>

To declare a dependency of your EAR to wmq.jmsra.rar also in order of deployment add jboss-all.xml to the same META-INF directory:

<jboss umlns="urn:jboss:1.0">
    <jboss-deployment-dependencies xmlns="urn:jboss:deployment-dependencies:1.0">
        <dependency name="wmq.jmsra.rar" />
    </jboss-deployment-dependencies>
</jboss>

Hawtio authentication with LDAP on Tomcat 7

Motivation

Larger organizations usually do grant access to applications via directory servers, i.e. Active Directory or OpenLDAP. For this purpose I have created this document describing how this can be implemented when using the Hawtio web-application.

This article is part I of three different scenarios:

• Hawtio deployed on Tomcat 7
• Hawtio deployed on JBoss Fuse
• Hawtio deployed on JBoss EAP

Hawtio authentication with LDAP on Tomcat 7

This section will show you how to setup authentication for Hawtio on Tomcat. For this purpose we will

• Install Tomcat 7 (part of EWS 2.0.1)
• Configure LDAP authentication on Tomcat 7
• Install Apache Directory Server
• Deploy Hawtio on Tomcat 7

The steps are described in more details in the following sub-sections.

Install Tomcat 7

• Download Red Hat JBoss Web Server 2.0.1-1 Application Servers for RHEL 6, x86_64
• Extract the archive

      unzip jboss-ews-application-servers-2.0.1-1-RHEL6-x86_64.zip

• Create jboss-ews-2.0/tomcat7/bin/setenv.sh with the following content

    JAVA_OPTS="${JAVA_OPTS} -Djava.security.auth.login.config=${CATALINA_HOME}/conf/jaas.config -Dhawtio.authenticationEnabled=true -Dhawtio.realm=hawtio -Dhawtio.role=admins  
                            -Dhawtio.rolePrincipalClasses=com.sun.security.auth.UserPrincipal"

Configure LDAP authentication on Tomcat 7

• Create jboss-ews-2.0/tomcat7/conf/jaas.config

    hawtio {
       com.sun.security.auth.module.LdapLoginModule REQUIRED
       userProvider="ldap://localhost:10389/ou=people,dc=example,dc=com"
       bindPrincipal="uid=admin,ou=system"
       bindCredentials="secret"
       bindAuthenticationType="simple"
       userFilter="(&(uid={USERNAME})(objectClass=inetOrgPerson))"
       authzIdentity="{OU}"
       useSSL=false
       debug=true;
    };

• In jboss-ews-2.0/tomcat7/conf/server.xml insert into the Host section (after the Valve)

    <Context path="/hawtio-default-1.2.2" crossContext="true">
      <Realm className="org.apache.catalina.realm.JAASRealm" appName="hawtio"/>
    </Context>

Install Apache Directory Server

• Install Apache Directory Studio in Eclipse (more info at http://directory.apache.org/studio/). I have used JBDS 7.1 for this purpose.
  • In Eclipse navigate to "Help / Install New Software" and add http://directory.apache.org/studio/update/2.x/.
  • Choose version 2.0.0.v20130628, but omit "Apache Directory Studio RCP" as there was an issue with missing dependencies
• Create a Apache DS instance
  • Open the LDAP perspective
  • Create a new LDAP server of type "ApacheDS 2.0.0"
  • Create a new Connection
    • Assign port 10389
    • Specify bind user/password as uid=admin,ou=system/secret
  • Import schema
    • Open the connection with the bind user/password uid=admin,ou=system/secret
    • Do a right-click on "Root DSE" and choose Import / LDIF Import
    • Import the example LDIF file you can download here 

Deploy Hawtio on Tomcat 7

• Download Hawtio hawtio-default-1.2.2.war from here
• Copy hawtio-default-1.2.2.war to jboss-ews-2.0/tomcat7/webapps/
• Startup Tomcat by executing

    jboss-ews-2.0/tomcat7/bin/startup.sh

• In a browser enter the URL http://localhost:8080/hawtio-default-1.2.2/
• Observe catalina.out with

    tail -f jboss-ews-2.0/tomcat7/logs/catalina.out

• Login with admin/hawtorwhat. This should succeed and the following output is shown in catalina.out

    [LdapLoginModule] search-first mode; SSL disabled
    [LdapLoginModule] user provider: ldap://localhost:10389/ou=people,dc=example,dc=com
    [LdapLoginModule] searching for entry belonging to user: admin
    [LdapLoginModule] found entry: uid=admin,ou=people,dc=example,dc=com
    [LdapLoginModule] attempting to authenticate user: admin
    [LdapLoginModule] authentication succeeded
    [LdapLoginModule] added LdapPrincipal "uid=admin,ou=people,dc=example,dc=com" to Subject
    [LdapLoginModule] added UserPrincipal "admin" to Subject
    [LdapLoginModule] added UserPrincipal "admins" to Subject

• Logout and try to login with johndoe/notsohawt. This should fail and this output appears in catalina.out

    [LdapLoginModule] search-first mode; SSL disabled
    [LdapLoginModule] user provider: ldap://localhost:10389/ou=people,dc=example,dc=com
    [LdapLoginModule] searching for entry belonging to user: johndoe
    [LdapLoginModule] found entry: uid=johndoe,ou=people,dc=example,dc=com
    [LdapLoginModule] attempting to authenticate user: johndoe
    [LdapLoginModule] authentication succeeded
    [LdapLoginModule] added LdapPrincipal "uid=johndoe,ou=people,dc=example,dc=com" to Subject
    [LdapLoginModule] added UserPrincipal "johndoe" to Subject
    [LdapLoginModule] added UserPrincipal "programmers" to Subject