Wednesday, November 30, 2011

What is Big Data

A new Buzz world which is catching up now days and all top notch companies are craving to showcase their different offerings in the BigData Domain.

Did i say "Domain"? Yes BigData is a Domain and not a technology.

I am fortunate enough to be working in this domain for nearly a year and have been keeping a track of the updates happening on the technology front and market dynamics which is leading most of the big companies to adopt BigData Domain as their next Business Strategy.

Dealing with the different kind of BigData clients have given me a different exposure about the domain, team and off-course technology too.

Going forward in this article I have defined my views about BigData, which would not only help the new bees but also the existing folks to understand this Domain and would also provide a perspective which one should adopt for facing the challenges thrown by BigData.

Brief about the BigData: -

BigData as such is not a technology, it is a domain which requires much deeper understanding of the problems/ solution and technology and that too from a perspective which is far different from the normal web based kind of Architectures.

As the name itself defines - "Large Data" and it is really large, which is beyond the capabilities of an normal web based kind of Architecture to handle.

BigData is spanned over 3 different dimensions, popularly known as 3 V's of BigData: -



1. Volume - BigData is really big, it is really large and size ranging from TB's to PB's to EB's
2. Velocity - It’s really time sensitive, most of the times it is streaming and needs to be analyzed as quickly as possible, so that maximum value can be taken out of it.
3. Variety - It extends beyond structured data and also includes unstructured data of all varieties: text, audio, video, click streams, log files and more.

The 4th "V" -
Recently heard that the 4 "V" have been added, which states "Veracity" - it means "the doubtful data".
It states something which is in doubt about its authencity or correctness or it cannot be validated.

Though somewhere it reflects and extends the concepts of data warehousing which in primitive days (Not more than 2 years back but 2 years is almost stone age for IT) used ETL concepts and high priced data warehousing solutions like informatica, Pentaho, IBM Infoshphere etc.

BigData is much more than the concepts of Data warehousing/ ETL.

It have taken up these concepts to an different heights where analytical capabilities can be applied to continuous and free flowing structured/ un-structured data sizing in PB's or XB.

Not only the technologies have used the capabilities of parallel processing but they have also provided the data processing and analytics in much shorter time and that too on the commodity boxes (leveraging cloud deployments) and removed the need of any dedicated Boxes or the Data Centers.

BigData Technology stack is aim to provide a scalable solutions where computations can be parallelized and distributed over the thousands of nodes, utilizing the processing power of each commodity box and at the same time is capable of detecting and supporting failures.

If this is not enough - all these are available in Open Source world which have tremendously brought down the cost of overall solution.


Technology used in BigData: -

As said earlier - BigData solutions are built on carefully chosen technology Stack and above all it is really important to fit the right technology and that too at the right time, using technology just to make someone happy or just because one knows, ruins the whole solution and adds more complexity which is really hard to get rid off later in the implementations

Still few of the prominent technology players who have made it possible are: -

Hadoop - Based on the M/R concept developed by Google, Hadoop have emerged as a leading solution for large data processing. Though it was developed by Apache and was named as "Apache Hadoop" but at the same time many companies chimed in and did provide their compatibility Stack of technologies over their own hadoop variants (extending Apache Hadoop itself)
e.g. Cloudera, Yahoo (recently yahoo spinned off a new company by name of horton works for their BigData offerings) etc.
There is much more to talk about hadoop, its architecture, tuning, HDFS etc...but i will talk more about his in my next Article

NoSQL - I already explained a bit about NoSQL in my last Article and yes that is already a part of BigData technology stack.
Based on the concept of Google Bigtable/ Amazon Dynamo, It not only have provided a 10x increase in the retrieval and inserting but also have provided something different and unique Approach for varied kind of needs

Some of the popular NoSQL DB's are: -

1. Cassandra - Please refer to my last article

2. Hbase - Built over HDFS (Hadoop File system) and leverages HDFS itself as it underlying Data Store, it provides a tight integration with Hadoop and enjoys a lot of tuning parameters which hadoop uses.

3. GraphDB's - As the name itself defines - A graph database uses graph like structures with nodes, edges, and properties to represent and store data.
A graph database is any storage system that provides index-free adjacency. Nodes are very similar in nature to the objects that we use in object-oriented programming.
Graph databases don’t have rigid schemas and are popularly used in modeling the schemas which are evolved over the period of time.
They do not require expensive join operations, thus can scale more naturally with Large Data sets.
A typical use case would be a Data model of social networking website.

Couple of popular GraphDB's are: -

Neo4j, InfoGrid, hypergraphDB, Bigdata

4. Document DB - Databases which specializes in storing the documents or semi-structured information in form of documents.

Though the implementation of each document database differs but in general, they all assume documents and encode documents in some standard format/ encodings. it could be XML, YAML, JSON, BSON and binary formats like PDF, DOC, Excel etc.
In comparison to relational models they are less rigid and are not required to adhere to standard schema.

Couple of popular Document-DB's are: -

CouchDB, MongoDB

Connectors - There are several connectors which are providing to transfer data from the source and dump it into HDFS (Apache Hadoop File System) for further processing.
for e.g. imagine webserver logs producing millions of logs in each minute and this data needs to be seamlessly transferred to HDFS for further processing and perform analytics.

Couple of popular Connectors are: -

Sqoop - Used to extract data from structured datastores.
Flume - works in a distributed mode and is used for collecting, aggregating, and moving large amounts of log data

Workflows - The next thing in this technology stack was to have a workflow engine which can be used to schedule the jobs (time based or data availability), create a workflow of M/R jobs and also keep a track of all these workflows

Couple of popular Workflow Engines are: -
Oozie - Oozie is a scalable, reliable and extensible system, works very well and haven't been replaced by any other wrokflow engine’s till now

Querying API's: -
Very well integrated with hadoop provides an efficient way to query records lying on HDFS.

Couple of popular Querying API's are: -
Pig, Hive

Analytics - The most important piece of BigData technology Stack!

Analytics basically deals with the predictions by carefully analyzing the past trends e.g.
1. Predicting the behavior of each user
2. Recommending the user with the various products (see recommendations provided by Amazon)
3. Analyzing the Effectiveness of an advertisement.
4. Analyzing the sentiments of public from social networking websites (Facebook, linked-in, G+, Orkut etc.)

And lot more..........

All the above information helps Analyst with the various kinds of trends available, so that they can strategies the future

It is easy to get analytics on structured data but it is difficult to get the Analytics on the semi-structured or un-structured data and here is the real value created by Mahout or NLP like API's.

Couple of popular Analytics API's: -

Mahout, NLP, R

MPP (Massively Parallel Processing Frameworks) - This is something new have emerged in the BigData technology stack, though they used the same concepts but the whole stack is tightly integrated to provide better performance and analytics too.

Couple of popular MPP's are: -

Greenplum,Paracel DB


BigData Clients: -

BigData clients are pretty different from the normal clients which we used to have in a typical web based systems.
Here clients are pretty technical in nature or it would not be wrong to say that here we need to deal with the Enterprise Architects who are: -

1. Highly technical and can talk about Architecture in middle of the night.
2. Know their business very well.
3. Is aware of the next steps for their TB/ PB’s of data.
4. They all have done few POC’s with Hadoop/ MPP or at-least they are well aware what Hadoop/MPP can/ cannot provide.

Now if these customers are already have identified the needs and also aware of technology/ Domain than what they need from us?

Yes, that's really a big question in the world of BigData and here is what i have experienced about their needs: -
“They need people who understand/ implements and are expert in BigData solutions and can provide analytics over structured/ semi-structured/ unstructured TB’s of data for timely decisions, forming new strategizes, acquiring focusing new business/ markets and customers on an ongoing basis”
In next section we will talk more about the team expertise and skill set required to meet the above needs.

Expertise required by BigData Team: -

The team working on bigdata solutions not only requires to be technically competent but should also be well versed with the challenges of large data (in TB or PB etc.).

Any candidate chosen for the BigData domain requires following 3 things: -

Expert in technology: -

1. Technically competent (Software/ Hardware/ Networks) and understand the Large data and is also possess a good experience in dealing with the Architectural issues/ decisions encountered/ taken in processing and parsing of Large Data.
2. Innovation – That’s the key behind any technology creator or implementer - Don’t Limit your thoughts and don’t do things just because someone have told that, put your minds, bring up new ideas and make things work in a better way.

Analytics: -

1. Understands the world of Analytics and also well versed of different solutions available for providing Analytics over the Large Data. This also requires the understanding of Dynamics and constraint imposed by Data.
2. Should not be language specific (can be in python/ shell/ C or C++). Rather it should be focused towards the problem.

Domain/ business Significance: -

1. Should have worked on some domain where he have dealt and understand the dynamics/ behavior and role of the Data in the solutions, at-least he should have dealt with some kind of reporting problems arise due to the size of data and complexity of the reports.
2. implement cases where technology is used to solve the real world problems in the various verticals (Finance/ Insurance/ RiskManagement/ Hospitality etc. ) and not any hypothetical cases.


Competitors in BigData: -

BigData market is still emerging and that’s the reasons nobody have defined list of competitors but every single company dealing or dealt with the Data challenges and having right set of folks to provide solutions over it can be termed as Competitors in BigData domain.

Few of the major players, who have already established themselves as the leaders in this domain are: -

1. Cloudera - One of the early entrants in the BigData market. It also offers its bigdata compatibility stack - CDH
2. Hortonworks - Backed up by Yahoo and Benchmark Capital , it is providing its own BigData compatibility Stack - Hortonworks Data Platform
3. EMC - Quite renowned company in the world of IT infrastructure and have tied up with couple of major players to provide its offerings in BigData.
4. Oracle
5. IBM

Even the majority of the Data warehousing players have also realized the future and have already started their offerings: -

1. Pentaho
2. Informatica
3. JasperSoft

The discussion doesn't ends here, BigData is still a focus area for most of the companies and the future of 2012 looks much brigther and competitive.

Read More!

Tuesday, July 19, 2011

Messing up with NoSQL - Cassandra

Brief About NoSQL: -

No doubt that invention of NoSQL was definitely a breakthrough in the Data storage models.

It not only provided a shift from typical mindset of Relational Model but also provided a way-out to store unstructured data into some Persistence storage area and that too in an efficient query-able format.

If we dig history of Data storage then definitely we will find many such models which were better than the RDBMS and works very well for many solutions
XML Databases, Object Databases were the first which adopted the concept of NoSQL and provided a different structure to store the complex and semi structure data.

It definitely was a breakthrough and involved a typical shift in mindset which was used to imagine everything in the form of rows and columns.

Replacement of RDBMS: -

At the same time people got many misconceptions and tried to implement NoSQL DB's for all kind of solutions, which itself was the biggest mistake made by the developers.

Relational Models have their own world and NoSQL never meant to be a competitor to that.

The idea of NoSQL was plainly to provide a solution which can solve the disadvantages provided by Relational Models and is also helpful in storing unstructured or semi-structured data which is difficult to fit into the traditional relational models.

Going further in this Article we will take up a popular NoSQL DB's - Cassandra and will analyze how it solves the various issues of relational Model.


Introducing Cassandra: -

Cassandra is another such NoSQL concept invented by the facebook folks for their social networking website and than they open sourced it to Apache under LGPL License.

Cassandra is a peer-to-peer, columnar and massively scalable data store.

Data Storage in Cassandra: -

Cassandra introduced the concept of storing the data in an associative Array which is sorted at the time of storage itself.

Associative Array - "An associative array is an abstract data type which consists of unique keys and collection of values. Each key is associated with one or more than one value."

For better and easy understanding here is way the terms used in Cassandra are mapped to the RDBMS: -

Keyspaces - same as Schema in Oracle
ColumnFamilies - Same as different Tables in a Schema
Rows - Same as Rows of a Table
Columns - Same as Columns referenced by different Rows


Here is how the keyspaces and Column Families are organized in Cassandra

Keyspaces have ColumnFamilies - Usually 1 KS per application
ColumnFamilies have Rows - Dozens of CFs per KS
Rows contain Columns - Many per CF
Columns - Contain name:value:timestamp - Many per Row

Now we will look at a typical JSON's style Structure and will see how the data is actually organized in Cassandra: -

Vehicles = {    // this is a ColumnFamily
 Camary:{  // this is the key to this Row inside the CF
      Capacity:"4",     // All these are columns and their values
      Colour: "Black", 
      Extra-Tyres: "1",
      Push-Back-seats: "Y",
      Power-Steering: "Y"           
   },
 Honda: { // this is the key to this Row inside the CF
      Capacity:"4", 
      Colour: "Black", 
      Extra-Tyres: "1",
      Push-Back-seats: "Y",
      Power-Steering: "Y"           
   }

 }     


There is also a term called "super-column" but for the sake of simplicity i will skip it for the time being and will move ahead with Cassandra internals and Architecture.

For more details and explanations please refer this Article which beautifully explains about the way the data is being organized in Cassandra.

Casandra Architecture: -

Over the period of time Cassandra have addressed the various Architecture concerns very well. All these Architecture concerns not only meet the requirements of a distributed environment but it also works very well in a Single site deployment.

Below are few of the Architecture Concerns which are very well handled in Cassandra: -

High Availability: -

All Nodes in Cassandra are independent nodes and *can* serve the user request independently (Going forward we will see how it can be achieved).
We did develop a strategy where each of the Cassandra nodes was capable of responding to user request irrespective of the state of other nodes in the cluster.

Providing low Data consistency levels (we will read about this later) gives us a highly available environment where even a Single node is self sufficient to serve user request.

CAP (Brewer's theorem): -

Consistency, Availability, Partitioning - As per the CAP theorem there is no Application which can provide all the 3 attributes at the same time.
Every application can provide only 2 attributes.

Though there are many RDBMS which provides a highly consistent and partitioned environment at the cost of availability but Cassandra at its best provides a Highly Available environment (i.e. "A") and partitioning of data (i.e. "P") in a cluster but at the same time it also provide the flavor of consistency by implementing the concept of "eventual consistency".

Eventual Consistency - Cassandra provides an excellent mechanism of replication and conflict resolution, which sync's the other nodes of a cluster in an asynchronous mechanism providing an eventually consistent environment.

The biggest benefit of eventually consistent environment is that the user response is not dependent upon the syncing of other nodes in a cluster. User gets an immediate response as soon as data is written to at-least 1 node (this is controllable...we will later in this article) and rest all nodes are being synced in an asynchronous manner.

So in other words Cassandra provides an eventual consistent environment where Data would be replicated and eventually all nodes would be having the same copy of data.

Conflict resolution: -

Cassandra provides a timestamp based conflict resolution Strategy.

Every column is associated with a timestamp and in a situation where we have different data written on different nodes for a same key the data which have higher timestamp wins

Though this is not the optimum algorithm to resolve the conflicts and surely there are better ways like Vector Clocks which provides a n efficient way to manage conflicts. Cassandra code does have flexibility to support any custom built conflict resolution mechanism.
Many of such instances and specially in a distributed environment where we need to control the concurrent read/ writes Cassandra users can use Zookeeper or cages like libraries which provide an out of the box solution and also an elegant way to synchronize the distributed transactions.

Eventual Consistency: -

RDBMS uses strong consistency (as they work on ACID properties) where data is not available to read until and unless its being replicated to all nodes in a cluster and this is done at the cost of performance and availability but as mentioned earlier that Cassandra provides an eventual consistent environment where data across the nodes/ clusters/ Data Centers will be eventually consistent which in turn provides us a Highly available environment and definitely there is big performance boost.

There are 3 different ways by which the data is being made consistent across the clusters/ Data Centers: -

AntiEntropy - Compares all the replicas and updates each replica to the newest version.
Read Repairs – Queries are compared against all the replicas of the key and most recent version is pushed to out-of-date replicas.
HintedHandOff – In case (for writes) replica node for the key is down, Cassandra will write a hint to a live replica node indicating that the write needs to be replayed to the unavailable node.

Cassandra do provides the flexibility to define various consistency levels, so that the users can be sure that read and write is marked as success only after defined consistency levels are met across the clusters/ Data Centers and rest all nodes can be synced in an asynchronous manner.

Write Consistency levels: -

ZERO - Ensure nothing. A write happens asynchronously in background. Until CASSANDRA-685 is fixed. 
If too many of these queue up, buffers will explode and bad things will happen.
ANY - Ensure that the write has been written to at least 1 node, including HintedHandoff recipients.
ONE - Ensure that the write has been written to at least 1 replica's commit log and memory table before responding to the client.
QUORUM - Ensure that the write has been written to N (ReplicationFactor) / 2 + 1 replicas before responding to the client.
LOCAL_QUORUM - Ensure that the write has been written to  / 2 + 1 nodes, within the local datacenter (requires NetworkTopologyStrategy)
EACH_QUORUM - Ensure that the write has been written to  / 2 + 1 nodes in each datacenter (requires NetworkTopologyStrategy)
ALL - Ensure that the write is written to all N replicas before responding to the client. Any unresponsive replicas will fail the operation.

Read Consistency levels: -

ZERO - Not supported, because it doesn't make sense.
ANY - Not supported. You probably want ONE instead.
ONE - Will return the record returned by the first replica to respond. 
A consistency check is always done in a background thread to fix any consistency issues when ConsistencyLevel.ONE is used. 
This means subsequent calls will have correct data even if the initial read gets an older value. (This is called ReadRepair)
QUORUM - Will query all replicas and return the record with the most recent timestamp once it has at least a majority of replicas (N / 2 + 1) reported. Again, the remaining replicas will be checked in the background.
LOCAL_QUORUM - Returns the record with the most recent timestamp once a majority of replicas within the local datacenter have replied.
EACH_QUORUM - Returns the record with the most recent timestamp once a majority of replicas within each datacenter have replied.
ALL - Will query all replicas and return the record with the most recent timestamp once all replicas have replied. Any unresponsive replicas will fail the operation.

Users should keep in mind that higher the consistency levels less will be the availability and performance, so we need to be very careful will defining the consistency levels for any application.

Also it would be nice to see that Cassandra do provide the clients to define the consistency levels, so the server code accepts the consistency levels provided by the client Application.

Failure detection, Replication, Scalability, Partitioning : -

Cassandra works very well in an distributed kind of environment where reads and writes are happening across the locations. It natively enables replication across multiple datacenters.

One of the best Strategies what cassandra layed out is the RackAwareStrategy (NetworkTopologyStrategy), where user defines the layout of a cluster which can consists of various nodes arranged in same Racks and same/ different Data Centers.

It works on Ring topology and each Cassandra node knows about the other nodes in the Ring.

Internally it uses Gossip to track the live nodes across the clusters which may or may not be distributed over the different geographical locations.
It is capable of re-establishing synchronization between two or more datastore and once the failure nodes are live than it automatically replicates the data to it, so that it can also become a part of nodes serving the user request.

The only difference between adding of new nodes bringing back the failure nodes is the initial token assignment for new nodes, which will define its placement in a ring.

The token assignment can be done manually or we can also have system to assign the token automatically to the new Nodes, which itself provides me a highly scalable system in which we can add/ remove nodes without any hiccups.

Linear Scaling - Cassandra runs on the commodity servers, there is no need for specialized servers, so we have advantage to deploy N number of Cassandra nodes in a Single Cluster


The replication mechanism is configurable and can be defined per keyspace and per Data Center. User's have flexibility to define the number of replica's to be present in a particular Data Center.

Now which nodes in a Data Center would be containing the replica depends upon how the data is partitioned...

Partitioning - As of now Cassandra supports 2 different partitioning schemes
1. RandomPartitioner(RP)
2. OrderPreservingPartitioner (OP)

In Both Schemes Cassandra tend to evenly distribute individual keys and their corresponding rows over the nodes in the cluster.

Every time a new node is added to the cluster, it will assign a range of keys to that node such that it takes responsibility for half the keys stored on the node that currently stores the most keys (more on options for overriding the default behavior later).

OP takes up the keys from the different Column Families and distribute over the cluster nodes and in case the distribution of keys used by individual column families is different, their sets of keys will not fall evenly across the ranges assigned to nodes which will lead to a hot spots and un-balanced cluster where one Node might be serving more number of hot keys than the other nodes in the cluster.

RP works on a continuous hashing mechanism and it uses MD5 hash of keys as the real keys for location key and data in a cluster. The distribution of he keys occurring within the individual column family doesn't matter.

It randomly maps any input key to a point between 0...2**12 range. The result is that the keys from each individual column family are spread evenly across the ranges/nodes, meaning that data and access corresponding to those column families is evenly distributed across the cluster.

The only drawback of RP is that the Range Scans over the keys is not possible but thanks to Lucandra/ Solandra, which provides a pretty good solution for the same.

Users do have the flexibility to re-define their replication strategy in future without loosing any of the data but in case of Partitioners it is very important that we decide upon the right data Partition strategy because partition strategy cannot be changed without erasing all the data from nodes.

Performance: -

Cassandra is a highly performant system for reads and writes.

Though you will see that the Cassandra out performs in case of writes compared to reads because of the fact that read_repairs are performed during the reads.

But in case of reads also it provides various levels of optimization which can be fine tuned based on the usage of the various column families

Based on the data usage patterns there are various parameters like KeyCache/ RowCache which needs to be fine tuned for each column family.

Cassandra itself provides different ways to improve the performance like: -
1. Before any disk reads it uses Bloom Filters for all look-ups.
2. All writes are written to commitlogs and from commitlogs it gets flushes periodically (or based on size) it is flushed to SSTables
3. Data Compaction, indexing and deleting tombstones is done for large files at regular intervals

There are many other parameters which are exposed to the users and can be tuned based on the analysis done on the data usage patterns.

Another aspect i.e. Size of data - It doesn't matter much as Cassandra is proven to handle >1TB of data without any issues.
Though identification of data usage patterns, defining Hot Column Families and providing right configuration to various column families (key cache, row cache, memtables throughput, readers/ writers threads etc) should be done in order to efficiently handle large volumes to data.

Avoiding Data Loss - Data Backup's: -

One of the important drawbacks of NoSQL DB's is the data back policies and strategies, which are at present missing and are ignored by most of the NoSQL providers.
Though in Cassandra we have various ways like JSON Backup, Binary backup but it is still in very early stages and does not include the concept of incremental backups.

But we do have a reasonable workaround to minimize the risk of data loss and that is by providing a higher replication factor across the clusters/ Data centers.

Though there is no doubt that in future NoSQL providers (including Cassandra) needs to provide different data back up strategies which are in sync with the relational databases.

Where to Use/ Not to Use: -

In contrast to other NoSQL DB's Cassandra is meant to provide a highly available cluster which if configured carefully can provide the availability of 5-9s.
This is one of the major feature of Cassandra which can be considered whenever we want to implement a high availability systems.

Couple of general questions which need to be answered before we even think of replacing RDBMS with any NoSQL DB: -

1. Do we need a transactional system?
2. Do we need highly consistent data?
3. Do we have a Distributed environment in which writes/ read would be happening from all locations?
4. Do we need Data replication?
5. What is the size of data?
6. Kind of data? Structured, semi-structured?
7. Is the data relational in nature and do we need to maintain the relations along with the data?
8. Does Cost matters?
9. Do we need an unlimited Scalability on the commodity hardware’s?
10. Do we need some sort of Analytic's?

Answer to all the above questions in comparison with the features provided by NoSQL DB will definately help us to identify the need for RDBMS or NoSQL database.

I am sure that folks would now be agree that the NoSQL definitely solves many issues of RDBMS but they are not a replacement of RDBMS and looking at the feature set that is provided by Cassandra, it would be the right choice for a distributed environment where High Availability is one of the major deciding criteria.


Appendix: -

Clients for Cassandra

Cassandra Architecture

Cassandra Wikki

NoSQL Glossary

Download Cassandra

Comparing Cassandra and other NoSQL DB's

Cassandra - A Decentralized Structured Storage System

Read More!

Friday, June 10, 2011

Exposing GIT over HTTP in Ubuntu

Continuing my GIT exploration here I will talk about how to expose GIT over HTTP in UBUNTU

Pre-requisites: -

GIT - GIT Core should be installed (Refer to Working with GIT)
Apache2 - Install it by running the following command: -

sudo apt-get install apache2

The above command will install the apache in the following Directories

1. /etc/apache2 - will contain all the configurations like ports, virtual hosts etc
2. /var/www - Contains the HTML files, CGI scripts etc

GITWEB

GITWEB is a package which can be used to expose GIT repositories over HTTP

It is nothing more than few set of files: -

1. gitweb.css
2. git-logo.png
3. git-favicon.png
4. gitweb.cgi
5. gitweb.js
6. gitweb.conf (Configuration File which usually is found at /etc/gitweb.conf)

sudo apt-get install gitweb will download all of the above files

Preparing Apache for Hosting GIT: -

1. Create a directory by name of "gitweb" under "/var/www".
2. Except gitweb.conf, copy all other files downloaded with GITWEB package to "/var/www/gitweb" directory.
3. Create a another folder by name of "/var/www/gitRepo" and configure your git client and also download the code from git Server (refer to Working with GIT) for all the branches.
4. Open /etc/gitweb.conf and modify the following highlighted Attributes: -

$GIT = "/usr/bin/git";
# path to git projects (.git)
# $projectroot = "/var/cache/git";
$projectroot = "/var/www/gitRepo";

# directory to use for temp files
$git_temp = "/tmp";

# target of the home link on top of all pages
#$home_link = $my_uri || "/";

# Description Text for your Repository
$site_name = "Project Trees";

# html text to include at home page
$home_text = "indextext.html";

# file with project list; by default, simply scan the projectroot dir.
$projects_list = $projectroot;

# stylesheet to use
$stylesheet = "/gitweb.css";

# logo to use
$logo = "/git-logo.png";

# the 'favicon'
$favicon = "/git-favicon.png";


5. Now execute the following Commands in "/etc/apache2/mods-enabled" which will enable the specific modules which are needed for this exercise: -


ln -s ../mods-available/dav.load dav.load
ln -s ../mods-available/dav_lock.load DAVLockDB
ln -s ../mods-available/dav_fs.load dev_fs.load
ln -s ../mods-available/dav_fs.conf dev_fs.conf
ln -s ../mods-available/cgid.load cgid.load
ln -s ../mods-available/cgid.conf cgid.conf
ln -s ../mods-available/rewrite.load rewrite.load

sudo a2enmod rewrite


6. Next create a virtual Hosts, that can expose your git repository

Create a git_vhosts.conf file in "/etc/apache2/sites-available" which will contain the details about the virtual Hosts.
The content of the file will look like this: -

#
# Virtual Hosts
#
# If you want to maintain multiple domains/hostnames on your
# machine you can setup VirtualHost containers for them. Most configurations
# use only name-based virtual hosts so the server doesn't need to worry about
# IP addresses. This is indicated by the asterisks in the directives below.
#
# Please see the documentation at
# <URL:http://httpd.apache.org/docs/2.2/vhosts/>
# for further details before you try to setup virtual hosts.
#
# You may use the command line option '-S' to verify your virtual host
# configuration.

#
# Use name-based virtual hosting.
#
NameVirtualHost *:801

#
# Almost any Apache directive may go into a VirtualHost container.
# The first VirtualHost section is used for all requests that do not
# match a ServerName or ServerAlias in any <VirtualHost> block.
#

<VirtualHost *:801>
    ServerName git.myrep.net
    DocumentRoot "/var/www/gitweb"
    DirectoryIndex gitweb.cgi
    SetEnv  GITWEB_CONFIG   /etc/gitweb.conf

    <Directory "/var/www/gitweb">
        Options FollowSymlinks ExecCGI
        Allow from all
        AllowOverride all
        Order allow,deny

        <Files gitweb.cgi>
            SetHandler cgi-script
        </Files>

        RewriteEngine on
        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteCond %{REQUEST_FILENAME} !-d
        RewriteRule ^.* /gitweb.cgi/$0 [L,PT]
    </Directory>

    <Directory "/opt/git/myrepo.git">
        Allow from all
    </Directory>

    # To debug rewrite rules, which were very painful to figure out. Create directories in case they don't exists
    RewriteLog /var/log/httpd/rewrite_log
    RewriteLogLevel 9

    ErrorLog /var/log/httpd/gitweb
</VirtualHost>

You need to focus on the highlighted text in the above configuration as these attributes are very much specific to the paths exposed on the individual boxes.
In case you have followed all the previous instructions line by line than you just need to change "ServerName" attribute, which should be the IP or Domain Name of your machine.

7. Now to enable the Virtual Host and execute the below command from "/etc/apache2/sites-enabled" directory

ln -s /etc/apache2/sites-available/git_vhosts.conf 002-git_vhost.conf

8. Lastly restart your Apache server by using the following set of Commands

sudo /etc/init.d/apache2 stop
sudo /etc/init.d/apache2 start

NOTE: Make sure it is a clean start and there are no Errors shown on console.

9. Now browse the URL (http://:801/gitweb.cgi?p=.git;a=tree) and you will be able to see the GIT repository.

The window which Appears would be something like the below snapshot



Next I leave to you to play around this UI.

Updating Git repository: -

As we are exposing client repository over HTTP, so naturally the question arises: -

How do you make sure that your repository is continuously updated and doesn't require any manual intervention?

There are 2 ways which can be used to do this: -

1. Create a Cron JOB in Ubuntu which updates the repository at every fixed interval
2. enable the "post-update Hook" of your GIT repository and from there execute a Shell Script which does the job for you.

As both the above the options requires a Shell Script which is capable to logging remotely and updating the repository through various SSH and GIT commands.

Normal Shell scripts are nowhere capable to doing this thing....but thanks to Linux which always have a solution to any problem.

"Expect" is a wonderful utility which helped me to develop a short and simple script which does the needful.

sudo apt-get install expect

Here is actual Script which I created and used with #2 to keep my repository in Sync with each and every commit: -
#!/bin/bash

/usr/bin/expect << EOD

spawn  ssh <username>@localhost

expect "password:"

send -- "<password>\r"

expect "$"

send -- "cd /var/www/gitRepo \r"

expect "$"

send -- "unset GIT_DIR \r"

expect "$"

send -- "git pull ssh://<username>@<IP>/<PATH> <BRANCH> \r"

expect "password:"

send -- "<password>\r"

expect "$"

expect eof

EOD

echo "rep updated"


Read More!

Wednesday, June 1, 2011

Sharing your Git repository

Out of the several configuration parameters provided by Git..."core.sharedrepository" is one of the most important parameter which enables your GIT repository to be shared within/ outside your development team.

By default - GIT repositories are not shareable and can be used only by 1 person.

for e.g. if a developer X add a new file than any other developer apart from X will not be able to update the same file.

The reason behind is that the GIT repositories are created in an non-Shareable mode and the files created in the repository are in read only mode, which can be updated only be the creator and not by anyone else.

Here are the steps which will enable your repository being shared among group of developers (on Unix): -

1. create a group which will be assigned to all developers working on Git

   groupadd {group-name}
  

2. Now assign the primary group of all new or existing users (who needs to work on git) same as what we have created in #1.

   for new users : - useradd -g {group-name} {username}
   for existing users : - usermod -g {group-name} {username}

3. Define GIT_DIR as global variable pointing out to the physical location of the GIT repository

   export GIT_DIR=<Physical Location of your Git Repo>

In order to make this variable available globally you can also define the above parameter in /etc/profile or create a new sh file in /etc/profile.d/*sh

4. Execute the following Command in your Git Repository

   git config --add core.sharedRepository group

The above command will enable your GIT repository to be shared among group of developers.
i.e. Now the repository is in group writable mode, where all users in the same group (as defined
in #1) will be able to read and write, to/ from the repository.

Apart from "group" there are other parameters which are explained very well in the documentation: -

"When group (or true), the repository is made shareable between several users in a group (making sure all the files and objects are group-writable). When all (or world or everybody), the repository will be readable by all users, additionally to being group-shareable. When umask (or false), git will use permissions reported by umask(2). When 0xxx, where 0xxx is an octal number, files in the repository will have this mode value. 0xxx will override user’s umask value (whereas the other options will only override requested parts of the user’s umask value). Examples: 0660 will make the repo read/write-able for the owner and group, but inaccessible to others (equivalent to group unless umask is e.g. 0022). 0640 is a repository that is group-readable but not group-writable. This attribute is false by Default."
Also see git-init to do the same while initializing the repository

5. Lastly fix the permission on the existing directories create in your GIT repository

   sudo chgrp -R coders .
   sudo chmod -R g+ws .

Thats it...from now on any new developer just needs to be added in the same user group (created to git developers) and he/ she will be able to participate in the development.

Read More!

Tuesday, May 24, 2011

Introduction to JMX

JMX is a simple, powerful and effective way to expose the various configurable attributes of a application over HTTP.

It is Widely used for divulging the health of your whole system and is accepted as a universal tool for application monitoring

After providing several implementations of JMX, In this Article I will uncover all the basics of JMX and would also mention few tips & tricks useful for production deployments.

What is JMX and Why do we use it?

JMX is a popular pattern to instrument any application and provide the monitoring capabilities to your system.

Just to make it simple think of scenarios where you need to know the response times of various layers in your systems, so that at any point of time you can decide upon the improvement areas.

There are many other examples and use cases in which JMX fits perfectly, like some of some of the popular ones and which i also like are: -
1. Monitoring DB connections and increasing/ decreasing the connection limits.
2. Activating and de-activating of components. same as OSGi
3. Modifying the attributes of any component and that too at runtime
4. etc...

JMX is a Scalable and true pluggable Architecture and it is the only standard pattern which can help in exposing any application and that too over HTTP.

Though other protocols like SNMP is also supported but exposing applications over HTTP becomes more important in case of production, where you have firewall restricting the access.

JMX Architecture




In the above image you would see that whole JMX Architecture is divided into 4 different levels: -

Instrumentation Level - At this level you will find all your Resources (termed as MBeans) which need to be exposed to the client

Agent level - Basically a virtual server which keep tracks of all resources defined at the instrumentaiton level. It also exposes various Agent Services like MLet, Monitoring, Timer, Realtion (we will discuss about all these services shortly)

Connectors - Defines the protocol which will be used by external applications to retreive the information from the Agents

Distributed level Services - External application or management interfaces (mx4j, JConsole) which is used by the end user to communicate with the resources with the help of Connecters. Clients can also build there proprietry applications but they should confirm to the JMX Specifications.

Components Overview - Instrumentation Level

The instrumentation level is further catagorized into the following sets: -

Mbeans: -

1. Standard Mbean
2. Dynamic Mbean
3. Model Mbean
4. Open Mbean

Notification Model: -

1. Listeners
2. Broadcaster/ Emitter
3. Filter

Lets understand them one by one: -

Components Overview - Instrumentation - MBeans - Standard Mbean

Standard Mbeans are the simplest Mbeans and very easy to design and expose.
There management interface defines the various attributes, operations which need to be exposed to the end users

When to use it: -
1. Management structures are pretty Straightforward.
2. Managed data is well defined in advance and unlikely to change often.

Example: -

1. Define a Management Interface, which states the attributes and operations which will be exposed and managed.

public interface StandardBeanMBean {
 
 public void setName(String name);
 
 public String getName();
}

2. Now define A Simple bean which implements the management interface and defines the action which needs to be taken for the operation and attributes define by management interface

public class StandardBean implements StandardBeanMBean{

 private String name;

 public String getName() {
  System.out.println("Returning name = "+name);
  return name;
 }

 public void setName(String name) {
  this.name=name;
  System.out.println("name have been set to = "+name);
  
 }

}

3. Next is simple Main Class which registers the MBean with the MBean Server.

public class RunSBSample {

 /**
  * @param args
  */
 public static void main(String[] args) {
  new RunSBSample().registerBean();
 }

 public void registerBean() {
  try {
   StandardBeanMBean mBean = new StandardBean();
                        //Sun provides a default implementation of MBean Server.
                        //We get the reference of that MBean Server and registers the MBeans
   MBeanServer server = ManagementFactory.getPlatformMBeanServer();
   server.registerMBean(mBean, new ObjectName("mBean:type="
     + mBean.getClass().getCanonicalName()));

   //Wait forever, so that we connect our management Apps :)
   while(true){
    Thread.sleep(99999999);
   }
  } catch (Exception e) {
   e.printStackTrace();
  }
 }

4. Now compile the above 3 set of classes and run with following VM arguments: -

-Dcom.sun.management.jmxremote.port=6789 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

Something like this: -

Java -Dcom.sun.management.jmxremote.port=6789 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false RunSBSample

5. Next is run Jconsole (open cmd and type JConsole), which is provided as default management apps and connect to the locally running MBean Server (Later in this article I will also tell you the ways to connect remotely) and navigate to the MBeans tab and expand the tree. refer to the below Snapshots: -



6. Now you will see the attribute section in the left hand side of the console, click on that and you will see the attribute exposed by your MBean - "Name".
7. Rest i will leave it to the users to play around and modify the value of this attribute but just wanted to remind that whatever value user is changing, it is actually modified in the actual Bean Object, which is registered within the server.

Components Overview - Instrumentation - MBeans - Dynamic MBean

Dynamic MBean is a bit different and provide more flexibility in comparison to the Standard MBeans

It expose the management interface at runtime for greatest flexibility.

Important Classes and Interface: -
1. DynamicMBean - Interface which should be implemented by all Dynamic Beans
2. MBeanAttributeInfo[] - Defines the attributes which needs to be exposed to the end users
3. MBeanConstructorInfo[]- Defines the details of the constructor which are used while creating the object of Dynamic MBean
4. MBeanOperationInfo[] - Defines the various operations which are exposed by Dynamic MBean
5. MBeanInfo - provides all nessary information about the Dynamic Mbean

When to use it: -
1. Data structures are likely to evolve often over time
2. Instrumentation must provide more flexibility
3. Need to determine the Structures at runtime
4. Need to separate out the Data structures with the management Structures

Example: -

1. Create a Simple Class, which will be exposed and managed by the DynamicMBean

package jmxSamples.mbeans.dynamicBean;

public class DynamicBeanClass {

 private String name="Sumit";
 private String age="10";
 
 public DynamicBeanClass(){
  
 }
 
 
 public void resetData(){
  setName("");
  setAge("");
 }


 /**
  * @return the name
  */
 public String getName() {
  return name;
 }


 /**
  * @param name the name to set
  */
 public void setName(String name) {
  this.name = name;
 }


 /**
  * @return the age
  */
 public String getAge() {
  return age;
 }


 /**
  * @param age the age to set
  */
 public void setAge(String age) {
  this.age = age;
 }
 
 
}


2. Create Dynamic Bean which extends DynamicMBean and define/ expose the attributes which needs to be managed by the user.

package jmxSamples.mbeans.dynamicBean;

import java.lang.reflect.Constructor;
import java.util.Iterator;

import javax.management.Attribute;
import javax.management.AttributeList;
import javax.management.AttributeNotFoundException;
import javax.management.DynamicMBean;
import javax.management.InvalidAttributeValueException;
import javax.management.MBeanAttributeInfo;
import javax.management.MBeanConstructorInfo;
import javax.management.MBeanException;
import javax.management.MBeanInfo;
import javax.management.MBeanNotificationInfo;
import javax.management.MBeanOperationInfo;
import javax.management.MBeanParameterInfo;
import javax.management.ReflectionException;

public class DynamicBean implements DynamicMBean {

 private String dClassName = DynamicBeanClass.class.getName();
 private String dDescription = "Simple implementation of a dynamic MBean.";

 private MBeanAttributeInfo[] dAttributes = new MBeanAttributeInfo[2];
 private MBeanConstructorInfo[] dConstructors = new MBeanConstructorInfo[1];
 private MBeanOperationInfo[] dOperations = new MBeanOperationInfo[1];
 private MBeanInfo dMBeanInfo = null;

 private DynamicBeanClass mbeanObj = null;

 /**
  * Constructor
  */
 public DynamicBean() {
  mbeanObj = new DynamicBeanClass();
  buildDynamicBean();
 }

 /**
  * Gets the value of a given the Attribute
  */
 public Object getAttribute(String attribute)
   throws AttributeNotFoundException, MBeanException,
   ReflectionException {
  if(attribute==null){
   System.out.println("attribute is null");
   return null;
  }
  
  if(attribute.equalsIgnoreCase("name")){
   
    return mbeanObj.getName(); 
   
   
  }
  if(attribute.equalsIgnoreCase("age")){
   
    return mbeanObj.getAge();
   
   
  }
  return null;

 }

 /**
  * Gets the value of a given the Attributes
  */
 public AttributeList getAttributes(String[] attributes) {
  if (attributes == null) {
      System.out.println("attr list is null");
      return null;
  }
  AttributeList resultList = new AttributeList();

  // if attributeNames is empty, nothing more to do
  if (attributes.length==0)
      return null;

  // for each attribute, try to set it and add to the result list if successfull
  for (int i=0;i < attributes.length ; i++) {
      String value = attributes[i];
      try {
       resultList.add(new Attribute(value,getAttribute(value)));
   } catch(Exception e) {
   e.printStackTrace();
      }
  }
  return resultList;
     }

 /**
  * Returns an MBean Info
  */
 public MBeanInfo getMBeanInfo() {
  
  return dMBeanInfo;
 }

 /**
  * Executes a operation and return back the results
  */
 public Object invoke(String actionName, Object[] params, String[] signature)
   throws MBeanException, ReflectionException {
  if(actionName==null){
   System.out.println("Action name is null");
   return new Object();
  }
  if(actionName.equalsIgnoreCase("resetData")){
   mbeanObj.resetData();
  }
  return null;
 }

 /**
  * Set the value of a given Attribute
  */
 public void setAttribute(Attribute attribute)
   throws AttributeNotFoundException, InvalidAttributeValueException,
   MBeanException, ReflectionException {
  if(attribute==null){
   System.out.println("attribute is null");
   return;
  }
  
  String name = attribute.getName();
  Object value = attribute.getValue();
  
  if(name==null){
   System.out.println("name is null");
   return; 
  }
  
  if(name.equalsIgnoreCase("name")){
   if(value==null){
    mbeanObj.setName("");
   }
   else{
    mbeanObj.setName(value.toString()); 
   }
   
  }
  if(name.equalsIgnoreCase("age")){
   if(value==null){
    mbeanObj.setAge("");
   }
   else{
    mbeanObj.setAge(value.toString());
   }
   
  }

 }

 /**
  * Set the value of a given Attribute List
  */
 public AttributeList setAttributes(AttributeList attributes) {
  if (attributes == null) {
      System.out.println("attr list is null");
      return attributes;
  }
  AttributeList resultList = new AttributeList();

  // if attributeNames is empty, nothing more to do
  if (attributes.isEmpty())
      return resultList;

  // for each attribute, try to set it and add to the result list if successfull
  for (Iterator i = attributes.iterator(); i.hasNext();) {
      Attribute attr = (Attribute) i.next();
      try {
   setAttribute(attr);
   String name = attr.getName();
   Object value = getAttribute(name); 
   resultList.add(new Attribute(name,value));
      } catch(Exception e) {
   e.printStackTrace();
      }
  }
  return resultList;
     }
 

 /**
  * Private method, used to build the DynamicBean
  * @param mbeanObj
  */
 private void buildDynamicBean() {
  
  dAttributes[0] = new MBeanAttributeInfo("Name", "java.lang.String",
    "Name: Name of the person.", true, true, false);
  dAttributes[1] = new MBeanAttributeInfo(
    "Age",
    "java.lang.String",
    "Age: Age of the Person.",
    true, false, false);

  Constructor[] constructors = this.getClass().getConstructors();
  dConstructors[0] = new MBeanConstructorInfo(
    "SimpleDynamic(): Constructs a DynamicBean object",
    constructors[0]);

  MBeanParameterInfo[] params = null;
  dOperations[0] = new MBeanOperationInfo(
    "resetData",
    "resetData(): reset Name and Age attributes to their initial values",
    params, "void", MBeanOperationInfo.ACTION);

  dMBeanInfo = new MBeanInfo(dClassName, dDescription, dAttributes,
    dConstructors, dOperations, new MBeanNotificationInfo[0]);

 }

}


3. Now define a Main class which registers this dynamic bean with the MBean Server

package jmxSamples.mbeans.dynamicBean;

import java.lang.management.ManagementFactory;

import javax.management.MBeanServer;
import javax.management.ObjectName;

public class RunDBSample {

 /**
  * @param args
  */
 public static void main(String[] args) {
  try{
  DynamicBean mBean = new DynamicBean();
  
  MBeanServer server = ManagementFactory.getPlatformMBeanServer();
  server.registerMBean(mBean, new ObjectName("mBean:type="
    + mBean.getClass().getCanonicalName()));
  
  while(true){
   Thread.sleep(999999);
  }
  }catch(Exception e){
   e.printStackTrace();
  }

 }

}


4. Next perform the #4 and #5 steps same as you did in the previous example and you will see that you are able to expose and modify a attribute of a bean without actually modifying its actual structure. Basically a wrapper on the original Bean, in form of DynamiMBean does the job for you.

Components Overview - Instrumentation - MBeans - Model MBean

Model Mbeans are Generic, configurable MBeans that anyone can use to instrument almost any resource (any Java resource) rapidly.
It is an extension to the Dynamic MBean but with greater flexibility

Important Classes and Interface: -
1. ModelMBeanAttributeInfo[] - Defines the attributes which needs to be exposed to the end users. Accepets Atrributes and operations wrapped in "Descriptor" object
2. ModelMBeanConstructorInfo[] - Defines the details of the constructor which are used while creating the object of Model MBean
3. ModelMBeanOperationInfo[] - Defines the various operations which are exposed by Model MBean
4. ModelMBeanNotificationInfo[] - Describes the various notifications emitted by the Model Bean
5. Descriptor - Defines and state the description (dataType, name default value etc) of each and every operation and atttributes exposed by the Model Beans
6. ModelMBeanInfo - Define a Model MBean at runtime and encapsulates the various openrations, notifications and attributes which needs to be exposed to the end user.

When to use it: -

1. All reasons which inclined to use Dynamic beans
2. Need a generic template for creating manageable Objects at runtime


Example: -

1. Define or assume that there is an Object which needs to be managed and exposed over the network

package jmxSamples.mbeans.modelBean;

public class ModelMBeanSampleClass {

 
 public ModelMBeanSampleClass(){
  
 }
  
 private String name;
 private Integer age;
 /**
  * @return the name
  */
 public String getName() {
  return name;
 }
 /**
  * @param name the name to set
  */
 public void setName(String name) {
  this.name = name;
 }
 /**
  * @return the age
  */
 public Integer getAge() {
  return age;
 }
 /**
  * @param age the age to set
  */
 public void setAge(Integer age) {
  this.age = age;
 }
 
 public void reset(){
  setName("");
  setAge(0);
 }
 
 
}


2. Now define an another class which dynamically gets the reference of the managed Object and registers its operation and attributes with the MBean Server.

package jmxSamples.mbeans.modelBean;

import java.lang.reflect.Constructor;

import javax.management.*;
import javax.management.modelmbean.*;

public class ModelMBean {

 
 private String dDescription = "Simple implementation of a Model Bean.";

 private ModelMBeanAttributeInfo[] dAttributes = new ModelMBeanAttributeInfo[3];
 private ModelMBeanConstructorInfo[] dConstructors = new ModelMBeanConstructorInfo[1];
 private ModelMBeanOperationInfo[] dOperations = new ModelMBeanOperationInfo[5];
 private ModelMBeanNotificationInfo[] dNotifications = null;
 private Descriptor mmbDesc = null;
 private ModelMBeanInfo dMBeanInfo = null;
 private MBeanServer server = null;

 
 
 public void buildModelBeans(MBeanServer server){
  
  try{
  this.server=server;
  
  String domain = server.getDefaultDomain();
  String mbeanName = "ModelSample";
  ObjectName inMbeanObjectName = new ObjectName(domain + ":type=" + mbeanName);
  
  buildDynamicMBeanInfo(server,mbeanName,inMbeanObjectName);
  RequiredModelMBean modelmbean = new RequiredModelMBean(dMBeanInfo);
  modelmbean.setManagedResource(new ModelMBeanSampleClass(), "objectReference");
     server.registerMBean(modelmbean,inMbeanObjectName);
  }catch(Exception e){
   e.printStackTrace();
  }
 }
 
 
 
 private void buildDynamicMBeanInfo(MBeanServer server, String mbeanName, ObjectName inMbeanObjectName)
 {
  try
  {

  
   
   
   
   Class appBean = Class.forName("jmxSamples.mbeans.modelBean.ModelMBeanSampleClass");

   mmbDesc = new DescriptorSupport(new String[] {("name="+inMbeanObjectName),
           "descriptorType=mbean",
           ("displayName="+mbeanName),
           "log=T",
           "logfile=jmxmain.log",
           "currencyTimeLimit=5"});
   
   
   Descriptor nameDesc = new DescriptorSupport();
   nameDesc.setField("name","name");
   nameDesc.setField("descriptorType","attribute");
   nameDesc.setField("displayName","MyName");
   nameDesc.setField("getMethod","getName");
   nameDesc.setField("setMethod","setName");

   dAttributes[0] = new ModelMBeanAttributeInfo("name",
             "java.lang.String",
             "Name: name string.",
             true,
             true,
             false,
             nameDesc);
   
   Descriptor ageDesc = new DescriptorSupport();
   ageDesc.setField("name","age");
   ageDesc.setField("descriptorType", "attribute");
   ageDesc.setField("default", new Integer("0"));
   ageDesc.setField("displayName","MyAge");
   ageDesc.setField("getMethod","getAge");
   ageDesc.setField("setMethod","setAge");
    
   dAttributes[1] = new ModelMBeanAttributeInfo("age",
             "java.lang.Integer",
             "Age: Age of a Person",
             true,
             true,
             false,
             ageDesc);
   
   Descriptor hardValueDesc = new DescriptorSupport();
   hardValueDesc.setField("name","HardValue");
   hardValueDesc.setField("descriptorType","attribute");
   hardValueDesc.setField("value", new Integer("99"));
   hardValueDesc.setField("displayName","HardCodedValue");
   hardValueDesc.setField("currencyTimeLimit","0");
   /* A currencyTimeLimit of 0 means that the value
      cached in the Descriptor is always valid.  So
      when we call getAttribute on this attribute we
      will read this value of 99 out of the
      Descriptor.  */
   
   dAttributes[2] = new ModelMBeanAttributeInfo("HardValue",
             "java.lang.Integer",
             "HardValue: static value in ModelMBeanInfo and not in my Sample Bean",
             true,
             false,
             false,
             hardValueDesc);
   
  /* Constructor[] constructors = appBean.getConstructors();
   
   Descriptor sampleClassdesc = new DescriptorSupport();
    sampleClassdesc.setField("name","ModelMBeanSampleClass");
    sampleClassdesc.setField("descriptorType", "operation");
    sampleClassdesc.setField("role","constructor");
   
   dConstructors[0] = new ModelMBeanConstructorInfo("ModelMBeanSampleClass(): Constructs a ModelBean App",
          constructors[0],
          sampleClassdesc);*/
   
   MBeanParameterInfo[] params = null;
   
   Descriptor resetDesc = new DescriptorSupport();
   resetDesc.setField("name","reset");
   resetDesc.setField("descriptorType","operation");
   resetDesc.setField("class","jmxSamples.mbeans.modelBean.ModelMBeanSampleClass");
   resetDesc.setField("role","operation");
   
   dOperations[0] = new ModelMBeanOperationInfo("reset",
             "reset(): reset Name and Age",
             params ,
             "void",
             MBeanOperationInfo.ACTION,
             resetDesc);
   
   
   Descriptor getNameDesc = new DescriptorSupport(new String[] {"name=getName",
             "descriptorType=operation",
             "class=jmxSamples.mbeans.modelBean.ModelMBeanSampleClass",
             "role=operation"} );
   
   dOperations[1] = new ModelMBeanOperationInfo("getName",
             "get state attribute",
             params ,
             "java.lang.String",
             MBeanOperationInfo.ACTION,
             getNameDesc);
   
   Descriptor setNameDesc = new DescriptorSupport(new String[] {
       "name=setName",
    "descriptorType=operation",
    "class=jmxSamples.mbeans.modelBean.ModelMBeanSampleClass",
    "role=operation"});
   
   MBeanParameterInfo[] setNameParms = new MBeanParameterInfo[] { (new MBeanParameterInfo("newName",
              "java.lang.String",
              "new Name value") )} ;
   
   dOperations[2] = new ModelMBeanOperationInfo("setName",
             "set State attribute",
             setNameParms,
             "void",
             MBeanOperationInfo.ACTION,
             setNameDesc);
   
   Descriptor getAgeDesc = new DescriptorSupport( new String[] {
       "name=getAge",
    "descriptorType=operation",
    "class=jmxSamples.mbeans.modelBean.ModelMBeanSampleClass",
    "role=operation"});
   
   dOperations[3] = new ModelMBeanOperationInfo("getAge",
             "get Age attribute",
             params,
             "java.lang.Integer",
             MBeanOperationInfo.INFO,
             getAgeDesc);
   
   
   Descriptor setAgeDesc = new DescriptorSupport(new String[] {
       "name=setAge",
    "descriptorType=operation",
    "class=jmxSamples.mbeans.modelBean.ModelMBeanSampleClass",
    "role=operation"});
   
   MBeanParameterInfo[] setAgeParms = new MBeanParameterInfo[] { (new MBeanParameterInfo("newAge",
                  "java.lang.Integer",
                  "new value for Number of Changes") )} ;
   
   
   dOperations[4] = new ModelMBeanOperationInfo("setAge",
             "set NbChanges attribute",
             setAgeParms,
             "void",
             MBeanOperationInfo.ACTION,
             setAgeDesc);
   
   
   
   dMBeanInfo = new ModelMBeanInfoSupport("jmxSamples.mbeans.modelBean.ModelMBeanSampleClass",
              dDescription,
              dAttributes,
              dConstructors,
              dOperations,
              dNotifications);
   
   dMBeanInfo.setMBeanDescriptor(mmbDesc);
   
  } catch (Exception e)
      {
   
   e.printStackTrace();
  }
 }

}


3. Same as we did in previous examples, define a main class which registers this Model Bean with the MBean server.

package jmxSamples.mbeans.modelBean;

import java.lang.management.ManagementFactory;

import javax.management.MBeanServer;

public class RunMBSample {

 /**
  * @param args
  */
 public static void main(String[] args) {
  try{
   
   MBeanServer server = ManagementFactory.getPlatformMBeanServer();
   ModelMBean mbean = new ModelMBean();
   mbean.buildModelBeans(server);
   
   while(true){
    Thread.sleep(999999);
   }
   }catch(Exception e){
    e.printStackTrace();
   }

  }

}


4. Next perform the #4 and #5 steps same as you did in the previous example and you will see that you are able to create template like structure which can be used to expose any java object.

Components Overview - Instrumentation - MBeans - Open MBean

Open Mbeans are used to understand and expose new objects (which needs to be managed/ exposed) whenever they are discovered at runtime. They rely on small, predefined set of universal Java types to advertise their functionality.
Biggest benefit is that they share and use management data and operations at runtime without requiring the recompilation.

Important Classes and Interface: -

1. OpenMBeanInfoSupport - Define a Open MBean at runtime and encapsulates the various operations, notifications and attributes which needs to be exposed to the end user.
2. TabularType - Defines a Tabular structure of the data which needs to be exposed.
3. TabularDataSupport - Defines hashmap like data structure to depict the data defined by TabularType.
4. CompositeType - Defines a Composite type which associate the List of OpenTypes with its name and description.
5. CompositeDataSupport - Defines a set of Complex Open Data which represent the CompositeData structures.
6. SimpleType - Defines the Standard data types for the various attributes
7. OpenMBeanParameterInfo - Defines the attributes, contructor, operations for the OpenBean

When to use it: -

1. Where the management application does not necessarily have access to the Java classes of the agent.
2. Java serialization is not supported by the management application and the agent.

Example: -

1. Unlike all other beans, Open Beans define and maps the structure of heterogeneous objects with the Java types and Structure, so here we will define 1 Class which defines the structure of the object which defines the relationship between the "TShirts", "Colors" and its "Sizes".

package jmxSamples.mbeans.openBeans;

import java.io.ByteArrayOutputStream;
import java.io.PrintWriter;
import java.util.Arrays;
import java.util.Collection;
import java.util.Iterator;

import javax.management.Attribute;
import javax.management.AttributeList;
import javax.management.AttributeNotFoundException;
import javax.management.DynamicMBean;
import javax.management.InvalidAttributeValueException;
import javax.management.MBeanException;
import javax.management.MBeanInfo;
import javax.management.MBeanNotificationInfo;
import javax.management.MBeanOperationInfo;
import javax.management.ReflectionException;
import javax.management.RuntimeOperationsException;
import javax.management.openmbean.CompositeData;
import javax.management.openmbean.CompositeDataSupport;
import javax.management.openmbean.CompositeType;
import javax.management.openmbean.OpenDataException;
import javax.management.openmbean.OpenMBeanAttributeInfoSupport;
import javax.management.openmbean.OpenMBeanConstructorInfoSupport;
import javax.management.openmbean.OpenMBeanInfoSupport;
import javax.management.openmbean.OpenMBeanOperationInfoSupport;
import javax.management.openmbean.OpenMBeanParameterInfo;
import javax.management.openmbean.OpenMBeanParameterInfoSupport;
import javax.management.openmbean.OpenType;
import javax.management.openmbean.SimpleType;
import javax.management.openmbean.TabularData;
import javax.management.openmbean.TabularDataSupport;
import javax.management.openmbean.TabularType;

public class OpenBean implements DynamicMBean {

 // Open MBean Info
 //
 private OpenMBeanInfoSupport OMBInfo;

 // Attributes exposed for management
 //
 private TabularDataSupport tShirts;
 private int nbChanges = 0;

 // Custom open types (and related info) used by this Open MBean class
 //
 private static String[] itemNames = { "model", "color", "size", "price" };
 private static String[] itemDescriptions = { "TShirt's model name",
   "TShirt's color", "TShirt's size", "TShirt's price" };
 private static OpenType[] itemTypes = { SimpleType.STRING,
   SimpleType.STRING, SimpleType.STRING, SimpleType.FLOAT };
 private static CompositeType tShirtType = null;

 // TShirts are indexed according to their model, color and size:
 private static String[] indexNames = { "model", "color", "size" };
 private static TabularType tShirtsType = null;

 // Legal values for TShirt features
 //
 private static String[] legalModels = { "JDMK", "JMX", "JAVA" };
 private static OpenMBeanParameterInfoSupport modelParamInfo;

 private static String[] legalColors = { "black", "white", "red", "green",
   "blue" };
 private static OpenMBeanParameterInfoSupport colorParamInfo;

 private static String[] legalSizes = { "S", "M", "L", "XL", "XXL" };
 private static OpenMBeanParameterInfoSupport sizeParamInfo;

 private static float minPrice = 9.00f;
 private static float maxPrice = 19.99f;
 private static OpenMBeanParameterInfoSupport priceParamInfo;

 static {

  // initializes OpenType instances and ParameterInfo instances
  //
  try {

   // CompositeType instance for a TShirt
   //
   tShirtType = new CompositeType("tShirt", "a TShirt", itemNames,
     itemDescriptions, itemTypes);

   // TabularType instance for the list of TShirts
   //
   tShirtsType = new TabularType("tShirts",
     "List of available TShirts", tShirtType, // row type
     indexNames);

   // Parameter info for the model, color, size and price parameters
   //
   modelParamInfo = new OpenMBeanParameterInfoSupport("model",
     "Valid TShirt model name. Legal models: "
       + Arrays.asList(legalModels).toString(),
     SimpleType.STRING, "JMX", // default model is JMX
     legalModels); // array of legal models
   colorParamInfo = new OpenMBeanParameterInfoSupport("color",
     "Valid product color. Legal colors: "
       + Arrays.asList(legalColors).toString(),
     SimpleType.STRING, "white", // default color is white
     legalColors); // array of legal colors
   sizeParamInfo = new OpenMBeanParameterInfoSupport("size",
     "Valid product size. Legal sizes: "
       + Arrays.asList(legalSizes).toString(),
     SimpleType.STRING, "XL", // default size is XL
     legalSizes); // array of legal sizes
   priceParamInfo = new OpenMBeanParameterInfoSupport("price",
     "Valid product price (ranging from $" + minPrice + " to $"
       + maxPrice + ")", SimpleType.FLOAT, null, // no
                  // default
                  // price
     new Float(minPrice), // Min legal value for price
     new Float(maxPrice)); // Max legal value for price

  } catch (OpenDataException e) {
   // should not happen
   ByteArrayOutputStream bout = new ByteArrayOutputStream();
   PrintWriter pout = new PrintWriter(bout);
   e.printStackTrace(pout);
   pout.flush();
   throw new RuntimeException(bout.toString());
  }

 }

 public OpenBean() {

  try {
   buildMBeanInfo();

   // Create empty TShirts list
   tShirts = new TabularDataSupport(tShirtsType);
  } catch (Exception e) {
   e.printStackTrace();
  }
 }

 /*  Getters */

 /**
  * Returns a clone of the TShirts list
  */
 public TabularData getTShirts() {
  return (TabularData) tShirts.clone();
 }

 /**
  * Returns the number of time the TShirts list has been updated
  */
 public Integer getNbChanges() {
  return new Integer(nbChanges);
 }

 /**
  * Checks param is a valid value for the specified paramInfo and returns
  * param's value (returns the default value if param is null and paramInfo
  * defines one), or throws an OpenDataException otherwise.
  */
 protected Object checkParam(OpenMBeanParameterInfo paramInfo, Object param)
   throws OpenDataException {

  Object result;

  if (!paramInfo.isValue(param)) {
   throw new OpenDataException("parameter " + paramInfo.getName()
     + "'s value [" + param + "] is not valid");
  } else if (param == null && paramInfo.hasDefaultValue()) {
   result = paramInfo.getDefaultValue();
  } else {
   result = param;
  }

  return result;
 }

 /**
  * Builds and returns a new CompositeData TShirt instance from the specified
  * parameters. If parameter values are not legal according to the
  * OpenMBeanParameterInfo instances for this method, it throws an
  * OpenDataException. If model, color or size are null, it uses the default
  * value provided in the OpenMBeanParameterInfo instances for this method.
  */
 public CompositeData addTShirt(String model, String color, String size,
   Float price) throws OpenDataException {

  // Check parameter values are legal, assign default if necessary, or
  // throws OpenDataException
  //
  model = (String) checkParam(modelParamInfo, model);
  color = (String) checkParam(colorParamInfo, color);
  size = (String) checkParam(sizeParamInfo, size);
  price = (Float) checkParam(priceParamInfo, price);

  Object[] itemValues = { model, color, size, price };
  CompositeData result = new CompositeDataSupport(tShirtType, itemNames,
    itemValues);
  tShirts.put(result);
  nbChanges++;
  return result;
 }

 /**
  * Removes the given tshirt from the list if a tshirt with the same index
  * existed in the list, or does nothing otherwise.
  */
 public void removeTShirt(String model, String color, String size) {
  try{  
  
  Collection index = tShirts.values();
  Iterator itr = index.iterator();
  boolean removed = false;
  while(itr.hasNext()){
   CompositeData data = itr.next();
   if(data.get("model").toString().equalsIgnoreCase(model) && data.get("color").toString().equalsIgnoreCase(color) && data.get("size").toString().equalsIgnoreCase(size)){
    Object removedData[] = tShirts.calculateIndex(data);
    tShirts.remove(removedData);
    removed = true;
    break;
   }
  }
  if (removed) {
   nbChanges++;
  }
  }catch(Exception e){
   e.printStackTrace();
  }
 }

 /*  DynamicMBean interface implementation */

 /**
     *
     */
 public Object getAttribute(String attribute_name)
   throws AttributeNotFoundException, MBeanException,
   ReflectionException {

  if (attribute_name == null) {
   throw new RuntimeOperationsException(new IllegalArgumentException(
     "Attribute name cannot be null"),
     "Cannot call getAttribute with null attribute name");
  }
  if (attribute_name.equals("TShirts")) {
   return getTShirts();
  }
  if (attribute_name.equals("NbChanges")) {
   return getNbChanges();
  }
  throw new AttributeNotFoundException("Cannot find " + attribute_name
    + " attribute ");
 }

 /**
     *
     */
 public void setAttribute(Attribute attribute)
   throws AttributeNotFoundException, InvalidAttributeValueException,
   MBeanException, ReflectionException {

  throw new AttributeNotFoundException(
    "No attribute can be set in this MBean");
 }

 /**
     *
     */
 public AttributeList getAttributes(String[] attributeNames) {

  if (attributeNames == null) {
   throw new RuntimeOperationsException(new IllegalArgumentException(
     "attributeNames[] cannot be null"),
     "Cannot call getAttributes with null attribute names");
  }
  AttributeList resultList = new AttributeList();

  if (attributeNames.length == 0)
   return resultList;

  for (int i = 0; i < attributeNames.length; i++) {
   try {
    Object value = getAttribute((String) attributeNames[i]);
    resultList.add(new Attribute(attributeNames[i], value));
   } catch (Exception e) {
    e.printStackTrace();
   }
  }
  return (resultList);
 }

 /**
     *
     */
 public AttributeList setAttributes(AttributeList attributes) {
  return new AttributeList(); // always empty
 }

 /**
     *
     */
 public Object invoke(String operationName, Object[] params,
   String[] signature) throws MBeanException, ReflectionException {

  if (operationName == null) {
   throw new RuntimeOperationsException(new IllegalArgumentException(
     "Operation name cannot be null"),
     "Cannot call invoke with null operation name");
  }


  if (operationName.equals("removeTShirt")) {

   // check params
   if ((params.length != 3) || !(params[0] instanceof String)) {
    throw new RuntimeOperationsException(
      new IllegalArgumentException(
        "cannot invoke removeTShirt: "
          + "expecting params[i] instanceof CompositeData for i = 0"),
      "Wrong content for array Object[] params to invoke removeTShirt method");
   }
   // invoke removeTShirt
   try {
    removeTShirt((String) params[0], (String) params[1],
      (String) params[2]);
    return "Removed";
   } catch (Exception e) {
    throw new MBeanException(e, "invoking removeTShirt: "
      + e.getClass().getName() + "caught [" + e.getMessage()
      + "]");
   }
  }

  // Add TShirt
  else if (operationName.equals("addTShirt")) {

   // check params
   if ((params.length != 4) || !(params[0] instanceof String)
     || !(params[1] instanceof String)
     || !(params[2] instanceof String)
     || !(params[3] instanceof Float)) {
    throw new RuntimeOperationsException(
      new IllegalArgumentException(
        "cannot invoke buildTShirt: "
          + "expecting params[i] instanceof SimpleData for i = 0 to 3"),
      "Wrong content for array Object[] params to invoke buildTShirt method");
   }
   // invoke addTShirt
   try {
    return addTShirt((String) params[0], (String) params[1],
      (String) params[2], (Float) params[3]);
   } catch (Exception e) {
    throw new MBeanException(e, "invoking buildTShirt: "
      + e.getClass().getName() + "caught [" + e.getMessage()
      + "]");
   }
  }

  else {
   throw new ReflectionException(new NoSuchMethodException(
     operationName), "Cannot find the operation "
     + operationName);
  }
  
 } 

 /**
     *
     */
 public MBeanInfo getMBeanInfo() {
  return OMBInfo;
 }

 /*  Open MBean Info */

 /**
     *
     */
 private void buildMBeanInfo() throws OpenDataException {

  OpenMBeanAttributeInfoSupport[] attributes = new OpenMBeanAttributeInfoSupport[2];
  OpenMBeanConstructorInfoSupport[] constructors = new OpenMBeanConstructorInfoSupport[1];
  OpenMBeanOperationInfoSupport[] operations = new OpenMBeanOperationInfoSupport[2];
  MBeanNotificationInfo[] notifications = new MBeanNotificationInfo[0];

  // attribute TShirts (no default or legal values: not supported for
  // tabular types anyway)
  attributes[0] = new OpenMBeanAttributeInfoSupport("TShirts",
    "List of available T-Shirts", tShirtsType, true, true, false);

  // attribute NbChanges (no default or legal values)
  attributes[1] = new OpenMBeanAttributeInfoSupport("NbChanges",
    "Number of times the TShirts list has been updated.",
    SimpleType.INTEGER, true, true, false);

  // constructor
  constructors[0] = new OpenMBeanConstructorInfoSupport(
    "OpenBean",
    "Constructs a OpenBean instance containing an empty TShirts list.",
    new OpenMBeanParameterInfoSupport[0]);

  // operation removeTShirt
  OpenMBeanParameterInfo[] params_remove = new OpenMBeanParameterInfoSupport[3];
  params_remove[0] = modelParamInfo;
  params_remove[1] = colorParamInfo;
  params_remove[2] = sizeParamInfo;

  operations[0] = new OpenMBeanOperationInfoSupport(
    "removeTShirt",
    "Removes the tShirt given in parameter to the list of available tShirts, "
      + "if a tshirt with the same index existed in the list, or does nothing otherwise.",
    params_remove, SimpleType.VOID, MBeanOperationInfo.ACTION);

  // operation buildTShirt
  OpenMBeanParameterInfo[] params_build = new OpenMBeanParameterInfoSupport[4];
  params_build[0] = modelParamInfo;
  params_build[1] = colorParamInfo;
  params_build[2] = sizeParamInfo;
  params_build[3] = priceParamInfo;
  operations[1] = new OpenMBeanOperationInfoSupport(
    "addTShirt",
    "Builds and returns a CompositeData TShirt instance from the specified parameters. "
      + "If parameter values are not legal according to the OpenMBeanParameterInfo instances for this method, "
      + "it throws an OpenDataException. "
      + "If model, color or size are null, it uses the default value provided in the OpenMBeanParameterInfo instances for this method.",
    params_build, SimpleType.VOID, MBeanOperationInfo.ACTION);

  // The OpenMBeanInfo
  OMBInfo = new OpenMBeanInfoSupport(this.getClass().getName(),
    "Open MBean", attributes, constructors, operations,
    notifications);
 }
}

2. Next perform the #4 and #5 steps same as you did in the previous example and you will see that you can expose an Heterogeneous objects by providing their mapping with the various Java Types.


Components Overview - Instrumentation - Notification Model

Notifications as the term itself describes that it is being used to emit the notifications for any change done to any of the resources and at other end there are several other resources who can register there intrerest in the listening of these notifications.
Important Components in Notification Model are: -

Listener - A listener which is shows interest and is registered to listen a certain type of Notifications.

Broadcaster/ Emitter - used to broadcast the notifications emitted by the resources.

Filter - Used for filtering the notifications and invoking listeners only for the relevant notifications.


As shown above the following Steps are followed in case of Notifications: -
1. Mbeans registers themselves with the MBean server as an Notification Broadcaster and emits Notifications which are consumed by the MBean Server.
2. Notification Listeners registers their interest for the notifications with the MBean Server
3. Mbean Server invokes listeners (which have registered their interest in receiving the notification) as soon any Notification is received from broadcasters.
4. Filters are applied before any notifications are processed by the Listeners.

Example: -

1. Define Bean and its management interface which exposes Attributes which needs to be managed and also broadcast the events for every change done with the Attributes.

package jmxSamples.mbeans.notifications.bean;

public interface MyBeanMBean {
 
 public String getName();

 public void setName(String name);

}


package jmxSamples.mbeans.notifications.bean;

import javax.management.AttributeChangeNotification;
import javax.management.Notification;
import javax.management.NotificationBroadcasterSupport;

public class MyBean extends NotificationBroadcasterSupport implements MyBeanMBean{

 private String name;
 
 private static volatile Long seqNum=0L; 

 /**
  * @return the name
  */
 public String getName() {
  return name;
 }

 /**
  * @param name the name to set
  */
 public void setName(String name) {
  
  Notification notification = 
            new AttributeChangeNotification(this,++seqNum,System.currentTimeMillis(),"Attribute of myBean Changed","name","java.lang.String", this.name, name);
   sendNotification(notification);
  this.name = name;
 
 }
 
}


2. Next Define a Listener which will listen to the notifications produced by the above Bean and also a Filter, which filters any unwanted notifications

package jmxSamples.mbeans.notifications;

import javax.management.Notification;
import javax.management.NotificationFilter;

public class MyNotificationFilter implements NotificationFilter{

 /**
  * 
  */
 private static final long serialVersionUID = -4542067417446721223L;

 public boolean isNotificationEnabled(Notification notification) {
  
  return true;
 }

}

package jmxSamples.mbeans.notifications;


import javax.management.AttributeChangeNotification;
import javax.management.Notification;
import javax.management.NotificationListener;

public class MyNotificationListener implements NotificationListener{

 public void handleNotification(Notification notification, Object handback) {
  
  if(notification instanceof AttributeChangeNotification){
   AttributeChangeNotification notificationA = (AttributeChangeNotification)notification;
  System.out.println("Notification receied");
  System.out.println(" Message = "+notificationA.getMessage());
  System.out.println(" SEQ Number = "+notificationA.getSequenceNumber());
  System.out.println(" Old Value = "+notificationA.getOldValue());
  System.out.println(" new Value = "+notificationA.getNewValue());
  System.out.println(" Type = "+notificationA.getType());
  
  }
  else{
   System.out.println("this is not a Attribute change Notification");
  }
 }
 
 

}


3. Lastly define a main class which registers the MBean and also associate the Listeners with this Bean.

package jmxSamples.mbeans.notifications;

import java.lang.management.ManagementFactory;

import javax.management.MBeanServer;
import javax.management.ObjectName;

import jmxSamples.mbeans.notifications.bean.MyBean;

public class RunNotifExample {

 /**
  * @param args
  */
 public static void main(String[] args) {
  new RunNotifExample().registerBean();
 }

 public void registerBean() {
  try {
   MyBean mBean = new MyBean();
   System.out.println("Starting Notificatio examplesss");
   ObjectName beanName = new ObjectName("mBean:type="+ mBean.getClass().getCanonicalName());
   MBeanServer server = ManagementFactory.getPlatformMBeanServer();
   
   server.registerMBean(mBean, beanName);
   server.addNotificationListener(beanName, new MyNotificationListener(), new MyNotificationFilter(), null);
   

   //Wait forever :)
   while(true){
    Thread.sleep(99999999);
   }
  } catch (Exception e) {
   e.printStackTrace();
  }
 }
}


4. Next perform the #4 and #5 steps same as you did in the previous example and you will see that you are able to receive all the notifications broadcasted by the registered Bean.

Some management Apps provide the flexibility to attach Listeners at runtime also


Components Overview - Agent level - Agent Architecture

Below digram clearly states the various resources managed by the Agent and how different ineterfaces to connect to Agents and get the resources.


MBean Server - One of the most important and CORE component of Agent, which not only represents managed resources but also Facilitates searching of Mbeans by its object name or a given pattern. All notification listners registeres their interest for any notification with MBean Server. MBean server can also be proxied by implementing "MBeanServerInvocationHandler".

Components Overview - Agent level - Agent Services - MLets

MLets defines a way to dynamically instantiate the MBeans. Bascially you define your MBeans in a file and these various MBeans are isntantiated and laoded at the runtime and registered within the MBean Server.
Apart from Dynamic Class Laoding it also helps loading and registration of MBeans over the network.

Example: -

1. Define a configuration file which contains the details of the Beans, which needs to be loaded and exposed. For simplicity name it as "mlet.conf" and place it at the root of the Application.
<MLET

CODE=jmxSamples.mbeans.standardBean.StandardBean
ARCHIVE=abc.jar
NAME=example:name=mletStandard >
</MLET>


2. Next define a main class which loads this configuration and registers the MBeans define in the above configuration file with the MBean server.

package 
package jmxSamples.mbeans.mlets;

import java.io.File;
import java.lang.management.ManagementFactory;
import java.util.Set;

import javax.management.MBeanServer;
import javax.management.ObjectName;
import javax.management.loading.MLet;

public class RunMlets {

 /**
  * @param args
  */
 public static void main(String[] args) {
  try{
   MLet mlet = new MLet();

   ObjectName beanName = new ObjectName("mBean:type="+ mlet.getClass().getCanonicalName());
   MBeanServer server = ManagementFactory.getPlatformMBeanServer();
   System.out.println("Starting MLets ");
   //Register the MLet Bean
   server.registerMBean(mlet, beanName);
   
   //Define a configuration file which list down the Beans which needs to be registered
   File conf = new File("mlet.conf");
   //Add the configuration file object to the MLets
   mlet.addURL(conf.getAbsoluteFile().toURL());
   System.out.println(conf.getAbsoluteFile().toURL().toString());
   Set set = mlet.getMBeansFromURL(conf.getAbsoluteFile().toURL().toString());
   System.out.println(" here is the data = "+set);

   
   //Wait forever :)
   while(true){
    Thread.sleep(99999999);
   }
  }catch(Exception e){
   e.printStackTrace();
  }

 }

}


3. Next perform the #4 and #5 steps same as you did in the previous example and you will see that the beans define in the MLet Configuration file is available in JConsole.

Take care that the jar specified in the conf file should be available in the classpath of the application.

Components Overview - Agent level - Agent Services - Monitoring

Monitoring is services which is used to monitor the values defined by a particualr set of variables.

There are 3 kind of Monitors: -

Counter – Monitors the threshold values
Gauge – Monitors a High & Low threshold values
String – Monitors the String Values

All the above beans are based on the Observable Pattern, where the objects which needs to be monitored is defined at runtime.

Example: -

1. Define the All 3 types of beans along with their Management Interface.

COUNTER BEAN
package jmxSamples.mbeans.monitoring.counterBean;

public interface CounterBasicMBean {
 
 public int getCount();
 
 public void setCount(int count);

}


package jmxSamples.mbeans.monitoring.counterBean;

public class CounterBasic implements CounterBasicMBean{

 private int count;
 
 public int getCount() {

  return count;
 }

 public void setCount(int count) {
  this.count=count;
  
 }
 
 

}

GAUGE BEAN

package jmxSamples.mbeans.monitoring.gaugeBean;

public interface GaugeBasicMBean {

 public int getGcount();

 public void setGcount(int gcount);

}

package jmxSamples.mbeans.monitoring.gaugeBean;

public class GaugeBasic implements GaugeBasicMBean{

 private int gCount;

 public int getGcount() {

  return gCount;
 }

 public void setGcount(int gCount) {
  this.gCount = gCount;

 }

}

STRING BEAN


package jmxSamples.mbeans.monitoring.stringBean;

public interface StringBasicMBean {

 public void setName(String name);

 public String getName();

}


package jmxSamples.mbeans.monitoring.stringBean;

public class StringBasic implements StringBasicMBean{

 
 private String name;
 
 public String getName() {
  return name;
  
 }

 public void setName(String name) {
  this.name=name;
 }
 
 

}



2. Define a Listener, which will listen the notifications emitted by these beans

package jmxSamples.mbeans.monitoring;

import javax.management.Notification;
import javax.management.NotificationListener;
import javax.management.monitor.MonitorNotification;

public class MonitoringListener implements NotificationListener {

 public void handleNotification(Notification notification, Object handback) {
  System.out.println("received Notification");
  System.out.println(" Source = "
    + ((MonitorNotification) notification).getSource());

  System.out.println(" Type = "
    + ((MonitorNotification) notification).getType());

  MonitorNotification notif = (MonitorNotification) notification;
  if (notif.getType().equalsIgnoreCase(
    MonitorNotification.THRESHOLD_VALUE_EXCEEDED)) {

   System.out.println("Threshold Value Excedded for Attr "
     + notif.getObservedAttribute());
   System.out.println("Message = " + notif.getMessage());

  }
  if (notif.getType().equalsIgnoreCase(
    MonitorNotification.THRESHOLD_HIGH_VALUE_EXCEEDED)) {
   System.out.println("Threshold High Value Excedded for Attr "
     + notif.getObservedAttribute());
   System.out.println("Message = " + notif.getMessage());

  }
  if (notif.getType().equalsIgnoreCase(
    MonitorNotification.THRESHOLD_LOW_VALUE_EXCEEDED)) {

   System.out.println("Threshold Low Value Excedded for Attr "
     + notif.getObservedAttribute());
   System.out.println("Message = " + notif.getMessage());

  }

  if (notif.getType().equalsIgnoreCase(
    MonitorNotification.THRESHOLD_ERROR)) {

   System.out.println("Threshold Error Occured");
  }

 }

}


3. Now define a Main Class, which registers all types of Beans and also associate the listener with each of them

package jmxSamples.mbeans.monitoring;

import java.lang.management.ManagementFactory;

import javax.management.MBeanServer;
import javax.management.ObjectName;
import javax.management.monitor.CounterMonitor;
import javax.management.monitor.GaugeMonitor;
import javax.management.monitor.StringMonitor;

import jmxSamples.mbeans.monitoring.counterBean.CounterBasic;
import jmxSamples.mbeans.monitoring.gaugeBean.GaugeBasic;
import jmxSamples.mbeans.monitoring.stringBean.StringBasic;

public class RunMonitoringAgent {

 /**
  * Main Method
  * @param args
  */
 public static void main(String[] args) {
  try {
   
   RunMonitoringAgent agent = new RunMonitoringAgent();
   
   MBeanServer server = ManagementFactory.getPlatformMBeanServer();
   MonitoringListener listener = new MonitoringListener();
   
   agent.runCounterMonitor(server, listener);
   agent.runGaugeMonitor(server, listener);
   agent.runStringMonitor(server, listener);   

  } catch (Exception e) {
   e.printStackTrace();

  }

 }
 
 /**
  * Executes and registers the counter Monitor
  * @param server
  * @param listener
  */
 private void runCounterMonitor(MBeanServer server, MonitoringListener listener){
  try{
   
   CounterMonitor counterMonitor = new CounterMonitor();
   ObjectName counterMonitorName = new ObjectName("monitorName:name="
     + "javax.management.monitor.CounterMonitor");
   // Register CounterBean
   server.registerMBean(counterMonitor, counterMonitorName);
   
   // Add Listener to the Counter Bean
   counterMonitor.addNotificationListener(listener, null, null);
   ObjectName standardObsObjName = new ObjectName(
     "CounterBasicBean:name=CounterBasic");
   CounterBasic basicBean = new CounterBasic();
   // Register Basic MBean
   server.registerMBean(basicBean, standardObsObjName);

   // Set CounterBean Attributes
   counterMonitor.addObservedObject(standardObsObjName);
   counterMonitor.setObservedAttribute("Count");
   counterMonitor.setNotify(true);
   counterMonitor.setInitThreshold(6);
   counterMonitor.setOffset(2);
   counterMonitor.setGranularityPeriod(500);
   counterMonitor.start();
   
  }catch(Exception e){
   e.printStackTrace();
  }
 }
 
 /**
  * Sets the executes the Gauge Monitor
  * @param server
  * @param listener
  */
 private void runGaugeMonitor(MBeanServer server, MonitoringListener listener){
  try{
   GaugeMonitor gMon = new GaugeMonitor();
   
   ObjectName gMonitorName = new ObjectName("monitorName:name="
     + "javax.management.monitor.GauugeMonitor");
   
   // Register GaugeBean
   server.registerMBean(gMon, gMonitorName);
   
   // Add Listener to the Gauge Bean
   gMon.addNotificationListener(listener, null, null);
   
   ObjectName standardObsObjName = new ObjectName(
     "GaugeBasicBean:name=GaugeBasic");
   
   GaugeBasic gBasicBean = new GaugeBasic();
   // Register Gauge Basic MBean
   server.registerMBean(gBasicBean, standardObsObjName);
      
   //Set Gauge Parameters
   gMon.addNotificationListener(listener, null, null);
   gMon.setNotifyHigh(true);
   gMon.setNotifyLow(true);
   gMon.setGranularityPeriod(500);
   gMon.setThresholds(10, 6);
   gMon.setObservedAttribute("Gcount");
   gMon.addObservedObject(standardObsObjName);
   gMon.start();
   
   
  }catch(Exception e){
   e.printStackTrace();
  }
 }
 
 /**
  * Sets the String Monitor
  * @param server
  * @param listener
  */
 private void runStringMonitor(MBeanServer server, MonitoringListener listener){
  try{
   StringMonitor sMon = new StringMonitor();
   
   ObjectName sMonitorName = new ObjectName("monitorName:name="
     + "javax.management.monitor.StringMonitor");
   
   // Register GaugeBean
   server.registerMBean(sMon, sMonitorName);
   
   // Add Listener to the Gauge Bean
   sMon.addNotificationListener(listener, null, null);
   
   ObjectName standardObsObjName = new ObjectName(
     "StringBasicBean:name=StringBasic");
   
   StringBasic sBasicBean = new StringBasic();
   // Register Gauge Basic MBean
   server.registerMBean(sBasicBean, standardObsObjName);
      
   //Set Gauge Parameters
   sMon.addNotificationListener(listener, null, null);
   sMon.setNotifyDiffer(true);
   sMon.setNotifyMatch(true);
   sMon.setGranularityPeriod(500);
   sMon.setStringToCompare("Sumit");
   sMon.setObservedAttribute("Name");
   sMon.addObservedObject(standardObsObjName);
   sMon.start();
   
   
  }catch(Exception e){
   e.printStackTrace();
  }
 }

}


4. Next perform the #4 and #5 steps same as you did in the previous example and you will see that all the 3 beans are registered with their appropriate monitors and we can easily monitor them through our management Apps.

Components Overview - Agent level - Agent Services - Timer

Timer services is used to trigger the notifications in 2 different ways: -

1. At Specified Date or Time
2. At Constant Intervals

Example: -

1. Define a Listener which will listen to the notifications emitted by the Time Bean.

package jmxSamples.mbeans.timer;

import javax.management.Notification;
import javax.management.NotificationListener;
import javax.management.timer.TimerNotification;

public class TimerListener implements NotificationListener{

 public void handleNotification(Notification notification, Object handback) {
  TimerNotification notif = (TimerNotification)notification;
  
  System.out.println("Timer Notification received........");
  
  System.out.println("Type of message = " + notif.getType());
  System.out.println("Message = "+ notif.getMessage());
  System.out.println("User Data = "+ notif.getUserData());
  
  System.out.println("Timer Notification END........");

  
  
 }

}


2. Now define a Timer bean, register it with MBean server, associate the above Listener with the Bean and fire some notifications.

package jmxSamples.mbeans.timer;

import java.lang.management.ManagementFactory;
import java.util.Date;

import javax.management.MBeanServer;
import javax.management.ObjectName;
import javax.management.timer.Timer;

public class RunTimerAgent {

 /**
  * @param args
  */
 public static void main(String[] args) {
  try {

   RunTimerAgent agent = new RunTimerAgent();

   MBeanServer server = ManagementFactory.getPlatformMBeanServer();
   TimerListener listener = new TimerListener();

   agent.runTimerAgent(server, listener);
   
  } catch (Exception e) {
   e.printStackTrace();
  }

 }

 private void runTimerAgent(MBeanServer server, TimerListener listener) {
  try {
   
   Timer tim = new Timer();
   
   ObjectName timerMonitorName = new ObjectName("monitorName:name="
     + "javax.management.timer.Timer");
   // Register TimerBean
   server.registerMBean(tim, timerMonitorName);
   
   tim.addNotificationListener(listener, null, null);

  //Will trigger the notification only Once   
  tim.addNotification("String", "Hey I am Timer", new String(), new Date());

  //Will trigger the notification 50 times at the interval of 1 sec 
              tim.addNotification("String", "Hey I am Timer", new String(), new Date(), 1000, 50);
   tim.start();
   

  } catch (Exception e) {
   e.printStackTrace();
  }
 }

}


3. Next perform the #4 and #5 steps same as you did in the previous example and you will see that the events are triggered at regular intervals and Listeners are invoked with the appropriate message.

Components Overview - Agent level - Agent Services - Relation

Relation Sevice is used to define the n-array association between different Mbeans through the named Roles It helps in maintaiting Consistency between the various MBeans.

For example: - In an Library, users shuould not be allowed to register the books which are not available in the Stock .

The only information which is not visible is the apperance of relationship between different MBean's.

Example: -

1. Continuing the same Example lets define the relationship between the Books and their owners and also make sure that we are not having owners for the books which are available in the Library.

Lets define the Owner and Book MBeans: -

package jmxSamples.mbeans.relation.beans;

public interface SimpleOwnerMBean {
 public void setOwnerName(String ownerName);

 public String getOwnerName();
}

package jmxSamples.mbeans.relation.beans;

public class SimpleOwner implements SimpleOwnerMBean {
 private String m_name = null;

 
 public SimpleOwner() {
  
 }
 public SimpleOwner(String name) {
  m_name = name;
 }

 public void setOwnerName(String name) {
  m_name = name;
 }

 public String getOwnerName() {
  return m_name;
 }
}


package jmxSamples.mbeans.relation.beans;

public interface SimpleBooksMBean
{
   public void setBook(String bookName);

   public String getBook();
}


package jmxSamples.mbeans.relation.beans;

public class SimpleBooks implements SimpleBooksMBean
{
    private String m_name = null;

    public SimpleBooks(){
     
    }
    public SimpleBooks(String bookName)
    {
       m_name = bookName;
    }

    public void setBook(String bookName)
    {
       m_name = bookName;
    }

    public String getBook()
    {
       return m_name;
    }
 }



2. Now lets define the relation between the Owners and Books

package jmxSamples.mbeans.relation;

import javax.management.relation.RelationTypeSupport;
import javax.management.relation.RoleInfo;

public class SimplePersonalLibrary extends RelationTypeSupport {


 /**
  * 
  */
 private static final long serialVersionUID = 7029204967709861471L;
 

 public SimplePersonalLibrary(String relationTypeName) {
  super(relationTypeName);

  try {
   RoleInfo ownerRoleInfo = new RoleInfo("owner",
   // the name of the MBean class of which all members must be an
   // instance.
     "jmxSamples.mbeans.relation.beans.SimpleOwner", true, // read
     true, // write
     1, // only one owner
     1, // for 1 book
     "Owner");
   addRoleInfo(ownerRoleInfo);

   RoleInfo booksRoleInfo = new RoleInfo("books",
     "jmxSamples.mbeans.relation.beans.SimpleBooks", true, true,
     1, // and no fewer than 1
     4, // can only own max 4 books
     "Books");
   addRoleInfo(booksRoleInfo);
  } catch (Exception ex) {
   throw new RuntimeException(ex.getMessage());
  }
 }
}

3. Now lets define a Library which provides the access of Books to its various owners and at any point of time if we break any relation than exception is being thrown by the MBean Server.

package jmxSamples.mbeans.relation;

import java.lang.management.ManagementFactory;
import java.util.ArrayList;
import java.util.Iterator;

import javax.management.InstanceAlreadyExistsException;
import javax.management.InstanceNotFoundException;
import javax.management.MBeanRegistrationException;
import javax.management.MBeanServer;
import javax.management.MalformedObjectNameException;
import javax.management.NotCompliantMBeanException;
import javax.management.ObjectName;
import javax.management.ReflectionException;
import javax.management.relation.Role;
import javax.management.relation.RoleList;
import javax.management.relation.RoleResult;

import jmxSamples.mbeans.relation.beans.SimpleBooks;
import jmxSamples.mbeans.relation.beans.SimpleOwner;

public class RunRelationAgent {

 public static void main(String[] args) {

  try {
   RunRelationAgent agent = new RunRelationAgent();
   MBeanServer server = ManagementFactory.getPlatformMBeanServer();

   agent.defineRelation(server);

  } catch (Exception e) {
   e.printStackTrace();
  }
 }

 private void defineRelation(MBeanServer server) {
  try {

   String m_relationServiceClass = "javax.management.relation.RelationService";

   Object[] params = { new Boolean(true) };
   String[] signature = { "boolean" };
   ObjectName m_relationObjectName = new ObjectName("relations:class="
     + m_relationServiceClass);
   server.createMBean(m_relationServiceClass, m_relationObjectName,
     null, params, signature);

   String libraryTypeName = "personal_library";

   SimplePersonalLibrary m_library = new SimplePersonalLibrary(
     libraryTypeName);

   Object[] params1 = { m_library };
   String[] signature1 = { "javax.management.relation.RelationType" };
   server.invoke(m_relationObjectName, "addRelationType", params1,
     signature1);
   printRelationTypeInfo(server, m_relationObjectName);

   createBooksOwners(server, libraryTypeName, m_relationObjectName);

  } catch (Exception e) {
   e.printStackTrace();
  }
 }

 private void createBooksOwners(MBeanServer server, String libraryTypeName,
   ObjectName m_relationObjectName) {

  try {

   String personalLibraryId = libraryTypeName + "_internal";
   
   
   String ownerClassName = "jmxSamples.mbeans.relation.beans.SimpleOwner";
   String bookClassName = "jmxSamples.mbeans.relation.beans.SimpleBooks";

   System.out.println("Creating MBeans to represent our relations");

   ObjectName ownerName1 = new ObjectName("library:name="
     + ownerClassName + "1");
   ObjectName ownerName2 = new ObjectName("library:name="
     + ownerClassName + "2");
   ObjectName bookName1 = new ObjectName("library:name="
     + bookClassName + "1");
   ObjectName bookName2 = new ObjectName("library:name="
     + bookClassName + "2");
   ObjectName bookName3 = new ObjectName("library:name="
     + bookClassName + "3");
   ObjectName bookName4 = new ObjectName("library:name="
     + bookClassName + "4");
   ObjectName bookName5 = new ObjectName("library:name="
     + bookClassName + "5");

   SimpleOwner owner1 = new SimpleOwner("Fred");
   server.registerMBean(owner1, ownerName1);
   
   SimpleOwner owner2 = new SimpleOwner("HP");
   server.registerMBean(owner2, ownerName2);
   
   SimpleBooks book1 = new SimpleBooks("Lord of the rings1");
   server.registerMBean(book1, bookName1);

   SimpleBooks book2 = new SimpleBooks("Lord of the rings2");
   server.registerMBean(book2, bookName2);

   SimpleBooks book3 = new SimpleBooks("Lord of the rings3");
   server.registerMBean(book3, bookName3);

   SimpleBooks book4 = new SimpleBooks("Lord of the rings4");
   server.registerMBean(book4, bookName4);

   SimpleBooks book5 = new SimpleBooks("Lord of the rings5");
   server.registerMBean(book5, bookName5);

   // Create Owner List and Role
   ArrayList ownerList = new ArrayList();
   ownerList.add(ownerName1); // can only add owner to an owner role

   Role ownerRole = new Role("owner", ownerList);

   // Create Book List and Role
   ArrayList bookList = new ArrayList();
   bookList.add(bookName1);
   bookList.add(bookName2);
   bookList.add(bookName3);
   Role bookRole = new Role("books", bookList);

   // Create Library RoleList
   RoleList libraryList = new RoleList();
   libraryList.add(ownerRole);
   libraryList.add(bookRole);

   Object[] params = { personalLibraryId, libraryTypeName, libraryList };
   String[] signature = { "java.lang.String", "java.lang.String",
     "javax.management.relation.RoleList" };
   server.invoke(m_relationObjectName, "createRelation", params,
     signature);

   printAllRelationInfo(server, m_relationObjectName);
   
   //Remove 2 books
   server.unregisterMBean(bookName1);
         server.unregisterMBean(bookName2);
         System.out.println("2 Books are removed succesfully");
         System.out.println("Now removing one more book");
         server.unregisterMBean(bookName3);
         testAllAccessQueries(personalLibraryId, server, m_relationObjectName);
         
   while(true){
    Thread.sleep(999999);
   }

  } catch (Exception e) {
   e.printStackTrace();
  }

 }

 private void printRelationTypeInfo(MBeanServer server,
   ObjectName m_relationObjectName) {
  try {
   ArrayList relTypeNameList = (ArrayList) (server.getAttribute(
     m_relationObjectName, "AllRelationTypeNames"));
   System.out
     .println("The RelationType Names found in the RelationService: "
       + relTypeNameList.toString());
  } catch (Exception ex) {
   ex.printStackTrace();
  }
 }

 private void printAllRelationInfo(MBeanServer server,
   ObjectName m_relationObjectName) {

  try {
   ArrayList allRelationIds = (ArrayList) server.getAttribute(
     m_relationObjectName, "AllRelationIds");
   for (Iterator i = allRelationIds.iterator(); i.hasNext();) {
    String currentRelationId = (String) i.next();
    System.out.println("All RelationIds: " + currentRelationId);
    testAllAccessQueries(currentRelationId, server,
      m_relationObjectName);
   }
  } catch (Exception ex) {

   ex.printStackTrace();
  }
 }

 private void testAllAccessQueries(String relationId, MBeanServer server,
   ObjectName m_relationObjectName) {

  // retrieve all roles
  try {
   Object[] params = { relationId };
   String[] signature = { "java.lang.String" };
   RoleResult roleResult = (RoleResult) (server.invoke(
     m_relationObjectName, "getAllRoles", params, signature));
   RoleList roleList = roleResult.getRoles();
   for (Iterator i = roleList.iterator(); i.hasNext();) {
    Role currentRole = (Role) i.next();
    System.out.println(">>>> role name: "
      + currentRole.getRoleName());
    System.out.println(">>>> role values: "
      + currentRole.getRoleValue().toString());
   }
   System.out.println("No unresolved Roles roleUnresolved size: "
     + roleResult.getRolesUnresolved().size());
  } catch (Exception ex) {
   ex.printStackTrace();
  }

 }

}



JMX Security

JMX Security Model is based on the java Security Model.
So it works in same manner as it works for the other principals.

It defines 3 Basic Permission Classes, which are used to provide the different level of permissions: -

1.MBeanServerPermission - create/ find/ release of MBean Server
2.MBeanTrustPermission - defines only 1 permission – “register” and controls what MBeans can registered by the various servers
3.MBeanPermission - Permissions related to Mbean operations

Lets see a running example which shows how exactly these permissions works: -

1. Define a Policy Files which enables only the "createMBean" permissions and save it to some location and give it the name as "myJMX.Policy"

permission javax.management.MBeanServerPermission "createMBeanServer";
 //permission javax.management.MBeanServerPermission "findMBeanServer";
 //permission javax.management.MBeanServerPermission "newMBeanServer";
 //permission javax.management.MBeanServerPermission "releaseMBeanServer";
 //permission javax.management.MBeanPermission "addNotificationListener", "*";
 
 //permission javax.management.MBeanPermission "getAttribute", "*";
 //permission javax.management.MBeanPermission "getClassLoader", "*";
 //permission javax.management.MBeanPermission "getClassLoaderFor", "*";
 //permission javax.management.MBeanPermission "getClassLoaderRepository", "*";
 //permission javax.management.MBeanPermission "getMBeanInfo", "*";
 //permission javax.management.MBeanPermission "getObjectInstance", "*";
 //permission javax.management.MBeanPermission "instantiate", "*";
 //permission javax.management.MBeanPermission "invoke", "*";
 //permission javax.management.MBeanPermission "isInstanceOf", "*";
 //permission javax.management.MBeanPermission "queryMBeans", "*";
 //permission javax.management.MBeanPermission "queryNames", "*";
 //permission javax.management.MBeanPermission "registerMBean", "*";
 //permission javax.management.MBeanPermission "removeNotificationListener,"
 //permission javax.management.MBeanPermission "setAttribute", "*";
 //permission javax.management.MBeanPermission "unregisterMBean", "*";
 
 //permission javax.management.MBeanTrustPermission "register";

2. Define a java Class which tries to violate or test these permissions

package jmxSamples.mbeans.security;

import java.util.ArrayList;

import javax.management.MBeanServer;
import javax.management.MBeanServerFactory;

public class RunMBeanServerPermissionAgent {

 /**
  * @param args
  */
 public static void main(String[] args) {

  RunMBeanServerPermissionAgent secAgent = new RunMBeanServerPermissionAgent();
  secAgent.displayMbeanServerPermission();

 }

 private void displayMbeanServerPermission() {
  try {

   MBeanServer server = null;
   try {

    // creating severSever
    server = MBeanServerFactory.createMBeanServer();

    System.out
      .println("Congratulations...You have the permissions to \"create MBean Server\"");

    System.out.println("Printing Mbeans in new Server = "
      + server.getMBeanCount());

   } catch (SecurityException se) {
    System.out
      .println("Sorry you dont have permission to create new MBean Sever");

   }

   try {
    ArrayList mServers = MBeanServerFactory
      .findMBeanServer(null);
    System.out
      .println("Congratulations...You have the permissions to \"Find MBean Server\"");
    System.out.println("Got " + mServers.size() + " MBean Servers");

   } catch (SecurityException se) {
    System.out
      .println("Sorry you dont have permission to search for MBean Servers");
   }

  } catch (Exception e) {
   e.printStackTrace();
  }

 }

}


3. Next perform the #4 and #5 steps same as you did in the previous example and also provide the policy file as an added argument: -

-Djava.security.manager -Djava.security.policy=D:/jdk1.6.0_23/Java/jre6/lib/security/myJMX.policy

Now your code will start respecting the policies define in the policy file and will throw the exception for all those permissions which have not been defined in the policy file.

JMX Implementations

The whole JMX implementation is divided into 2 parts: -

JMX Implementations - It is more of the implementation of the JSR-3, where it talks about the implementation of the standard specifications provided by Sun.
Some of popular implementations for the same are: -

- JDK1.5+
- mx4J (http://mx4j.sourceforge.net/)
- Sun reference Implementation (Java DMK5.1)

JMX Management Apps - Which provides the UI and connector to play around with the Data/ Beans exposed by the remote JMX Agent.

Basically these management Apps implement JSR-160 to provide a standard way of connecting to remote JMX Agent.

Few of the popular JMX management Apps are -

- JConsole
- Mx4J (http://mx4j.sourceforge.net/)
- Jolokia (http://www.jolokia.org/)




Customizing JMX Output

This is a very important aspect of deciding upon any of the JMX management Apps.
Till date all management apps do supports the representation of widely accepted data types like String, Integer, Map, List etc and these covers mostly our 75% of the requirements.

But there might be scenarios where we still need to support the display and modifications of custom objects.

Though it mainly depends upon the flexibility provided by the management Apps but most of the popular apps like mx4J do provide the various ways to customize the output produced by the Agents.

For e.g.
mx4j uses XSLT processor to define the views for the various datatypes.
It defines the various XSL files which can be extended (new ones can also be created) for the any Custom data types.

Enhancements in JDK1.6

A new Type of bean is introduced in Mustang (Java6) i.e. MXBeans,

Though MXBeans were initially introduced with Java5 but it didn't provide any API's to define the user defined MXBeans, though some standard MXBeans like MemoryMXBean, ThreadMXBean, RuntimeMXBean, GarbageCollectorMXBean etc were readily available and exposed by JVM

Java6 introduced API's to define the user defined MXBeans.

The idea behind the MXBeans was to define a convenient way to bundle related values
together in an MBean without requiring clients to be configured to handle the bundles.

And to achieve this MXBean were introduced which are based on the concepts of OpenBeans and StandardBeans: -

1. They rely on Open/ Standard Datatypes same as what is defined for the OpenBeans
2. They are much like the StandardBeans but with few notable differences: -
- Your Class and Interface need not follow the naming Conventions, as it was defined by MBeans.
- Management Interfaces can be annotated with "@MXBean" annotation.

Example: -

1. Define a Management interface (same as we did for StandardBeans) and either annotate it with "@MXBean" or use "MXBean" at the end of the Interface name.

package jmxSamples.mbeans.mxBeans;

//@MXBean annotation can also be used 
public interface TestNameIntfMXBean {
 
 public String getName();
 
 public void setName(String name);

}



2. Define an implementation Class.

Note - Implementation class doesn't follow the convention of MBeans

package jmxSamples.mbeans.mxBeans;

public class TestNameClass implements TestNameIntfMXBean {

 String name ="initial value";
 public String getName() {
  
  return name;
 }

 public void setName(String name) {
  this.name=name;
  
 }
 
}


3. Now define a simple Main Class which registers the MBean with the MBean Server.

package jmxSamples.mbeans.mxBeans;

import java.lang.management.ManagementFactory;

import javax.management.MBeanServer;
import javax.management.ObjectName;

public class RunMXBeans {

 /**
  * @param args
  */
 public static void main(String[] args) {
  try {
   TestNameIntfMXBean mBean = new TestNameClass();

   MBeanServer server = ManagementFactory.getPlatformMBeanServer();
   server.registerMBean(mBean, new ObjectName("mBean:type="
     + mBean.getClass().getCanonicalName()));
   
   
   

   while (true) {
    Thread.sleep(999999);
   }
  } catch (Exception e) {
   e.printStackTrace();
  }
 }

}


4. Next perform the #4 and #5 steps same as you did in the previous example and you will see the same behavior as Standard Beans.

Follow this Link to get more understanding and info on MXBeans


Apart from the above there are few Standard MXBeans which are already exposed by JVM and are available to the programmers through the "ManagementFactory" class: -

1. CompilationMXBean - Exposes the attributes and operations related Compilation System managed by the JVM.
2. GarbageCollectorMXBean - Exposes the attributes and operations related garbage collection System managed by JVM
3. MemoryMXBean - Exposes the attributes and operations related Memory like heap, non-heap, gc etc
4. MemoryManagerMXBean- Exposes the attributes and operations related Memory Managers like "CodeCacheManager"
5. ThreadMXBean - Exposes attributes like AllThreads, ThreadCount, PeakThreadCount and many more.
6. OperatingSystemMXBean - Exposes the attributes of OS like physical/virtual Memory, swap space, free memory etc.
7. RuntimeMXBean - Exposes attributes like details about JVM Specs (version, vendor, name etc), classpath, bootclasspath etc
8. ClassLoadingMXBean - Details about the total loaded classes unloaded classes, verbose mode etc.
9. MemoryPoolMXBean - Exposes the attributes and operations related various memory pools like eden, perm gen, Survivor Space, Tenured Gen etc

ManagementFactory.java defines various Static methods to get the references of all the above all the above MXBeans.

For more details Please visit this Link

General Tips & Tricks

Connecting to the JMX Agent Programmatically

JMXServiceURL u = new JMXServiceURL(
"service:jmx:rmi:///jndi/rmi:// “ + hostName + ":" + portNum + "/jmxrmi");
JMXConnector c = JMXConnectorFactory.connect(u);

Remotely monitoring and Connecting the JMX Agents

You can use RMI also to remotely connect to the running JMX Agents.
Provide the below URL in JConsole and select the "Remote Process" Option.

"service:jmx:rmi:///jndi/rmi://:/jmxrmi"

MX4j and other apps also provide the same kind of flexibility

Behind the Firewall

Usually the direct access to IP-Address is not allowed and the Firewall security restrictions forces to use the host name for remote connections.

So while defining the java process, apart from other JMX parameters also define the below param

-Djava.rmi.server.hostname=<public-name of your System>

And than client Apps can connect to the JMX Agents by using the name specified by the above param.

Exposing Agents over HTTP

There are many management apps like mx4j, Java-DMK which defines the HttpAdaptor for exposing all Beans exposed by the Agent over HTTP.
They also provide a decent UI interface for exposing the beans. see here

Securing Your JMX Agents

Certainly in production kind of environments you will definitely need some mechanism to secure your Agents from un-authorized access.

There are 3 different ways by which you can secure your JMX Agents and provide an secure way of access to your Clients: -

1. Using SSL - It implements and uses the concepts of JSSE. Visit this Link for more info.
2. Using Password and Access Files

Also refer to the below Snapshot which defines all the runtime properties which are exposed by JMX (Taken From Here): -


Read More!