An open-source, federated content repository

Custom properties on nt:file and nt:folder nodes

One really nice feature of JCR repositories is that you can use them to store files and folders. Like with all other content, the structure of these nodes is dictated by their node types, and most people use the “nt:file” and “nt:folder” node types defined in the JCR specification. Learning to use these node types can take a little work, because they’re not quite as straightforward as you might expect.

Consider a “MyDocuments” folder that contains a “Personal” folder and a “Status Report.pdf” file. Here’s what those nodes might look like:

Nodes for folders and files

The folders look like what you might expect: they have a name, a primary type of “nt:folder”, and the “jcr:createdBy” and “jcr:created” properties defined by the “nt:folder” node type. (These properties are defined as ‘autocreated’, meaning the repository should set these automatically.)

The file representation, on the other hand, is different. The “Status Report.pdf” node has a primary type of “nt:file” and the “jcr:createdBy” and “jcr:created” properties defined by the “nt:file” node type, but everything about the content (including the binary file content in the “jcr:data” property) is actually stored in the child node named “jcr:content”. This may seem odd at first, but actually this design very nicely separates the file-related information from the content-related information.

Think about how an application might navigate the files and folders in a repository. Using the JCR API, the application asks for the “MyDocuments” node, so the repository materializes it (and probably its list of children) from storage. The application then asks for the children, so the repository loads the “Personal” folder node and the “Status Report.pdf” node, and there’s enough information on those nodes for the application to display relevant information. Note that the “Status Report.pdf” file’s content has not yet been materialized. Only when the application asks for the content of the file (that is, it asks for the “jcr:content” node) will the content-related information be materialized by the repository. (And, some repository implementations might delay loading the “jcr:data” binary property until the application asks for it.) Nice, huh?

Another interesting aspect of the “nt:file” and “nt:folder” node types (and even the “nt:resource” node type) is that they don’t allow adding just any property on the node. The beauty is that they don’t have to, because you can still add extra properties to these nodes using mixins!

Let’s imagine that we want to add tags to our file and folder nodes, and that we want to start capturing the SHA-1 checksum (as a hexadecimal string) of our files. To start, we need to create two mixins (we’ll use the CND format):

[acme:taggable] mixin
- acme:tags (STRING) multiple

[acme:checksum] mixin
- acme:sha1 (STRING) mandatory

(We could have defined a mixin that allows any property, similar to how the standard “nt:unstructured” node type does it. Then, we can add any properties we want. However, I tend to like using more targeted mixins like these, if for no other reason than it makes it very easy to use JCR-SQL2 to query the nodes that use these mixins.)

We then need to register these node types in our repository (perhaps by loading the CND file or programmatically using the NodeTypeManager). Then, we can add the “acme:taggable” mixin to whatever file and folder nodes we want. This is as simple as:

// Find the node ...
Node myDocuments = session.getNode(pathToMyDocuments);
Node personalFolder = myDocuments.getNode("Personal");

// Add the mixin ...

// Set the tags ...
String[] tags = {"non-work"};

// Save the changes ...;

We can do something similar for the “Status Report.pdf” node, as well as use the “acme:checksum” mixin on the “jcr:content” node. The result is something like this:

File and folder nodes with custom properties

As you can see, JCR’s built-in “nt:file” and “nt:folder” nodes really are pretty easy to understand and use, even when you need to place custom properties on these nodes.

Filed under: jcr, techniques

Running tests against different DBMSes

Testing software against multiple database management systems can be tricky. It’s usually a headache to configure and maintain these tests, and then they usually take a long time to run.

Of course there are multiple ways of doing this, and each approach probably has some advantage for your system, the build environment, and whether your developers have access to the test databases.

JBoss DNA is an open source project, and open source projects run into some walls that commercial software developers often don’t have. With open source, each developer likely has a very different environment and likely does not have access to the same databases or same set of DBMSes. It doesn’t work to hard-coding the connection information or even to put it in some property files, because each developer would have to go and change all those settings. Even if we agreed upon a convention, not everyone has access to the same DBMS systems. What we want is to be able to run all of our builds normally without relying upon any external resources, and then at our choosing easily run our tests against the database each developer has access to.

Luckily, we use Maven, and so we can use Maven profiles to define a different environment for each database we want to use. In each database profile, we can add dependencies for the appropriate JDBC driver and specify connection properties. And it becomes very easy to turn profiles on and off, which means developers can choose which databases they want to test against. And, we can use HSQLDB or H2 by default, since these are fast, all developers have them (merely because of Maven dependencies), and there transient.

The only challenge is that we already are using profiles for different kinds of builds we do. One of our profiles (the one used by default) is fast because it simply compiles the source and run only the unit tests. This is the default because we run this build all of the time – it’s actually the easiest way to run all of the unit tests, so I run it constantly throughout the day as I make changes locally.

We have another profile that also compiles and runs the integration tests – these take several minutes to run, so it’s not very nice to have to wait for all these tests while you’re just verifying some changes didn’t cause some unintended behavior. Although I run these tests locally, I suspect most of the developers let our automated continuous integration server do the work.

Other profiles also generate JavaDoc, build our documentation (compiling DocBook into multiple HTML formats and one PDF form), and create the ZIP archives that we publish in our downloads area.

The database profiles are actually orthogonal to these other profiles. Despite this, there is (at least) one way to make them stay independent and play nicely together, and that’s activating them with properties. Our Maven command line becomes:

mvn clean install -Ddatabase=postgresql

and if we want to use any of our other profiles, we can just use the “-P” switch as we’ve always done:

mvn -P integration clean install -Ddatabase=postgresql

That would work great. All we have to do is define each database profile to activate based upon the “database” property.

Oh, you may be wondering why we don’t just explicitly name each profile on the command line. Well, actually we can, and it works as long as the user always remembers to include one database profile along with the other profiles they want to use. With Maven, if the user names a profile, then no other profile is activated by default, which means that our builds may run without a database profile, and this can break the tests. Activating the database profile with a property solves this problem.

Defining the database profiles

As I mentioned earlier, we really want a profile for each database that we want to test against. If we define the profiles in the parent POM, then all subprojects inherit them. So, the first step is to put in our parent POM a separate profile for each database configuration. Here is the HSQLDB configuration:

    <jpaSource.password />

Note how the profile is activated when the “database” property matches a value (“hsqldb” in this case.) We also define the dependencies on the HSQLDB JDBC driver JAR, and we define the properties that we’ll use in our tests. And we have one of these for each database we want to test against.

But before we look at how those properties get injected into our test cases, let’s define the database profile that should be used if the “database” property is not set. We do this with a different activation strategy:

   <jpaSource.password />

I’ve chosen that the default database profile is identical to one of the other profiles, and this means I have some duplication in the POM file. Personally, the cleanliness for the user seems worth it.

Before we leave our parent POM, we also should probably define default values for all of the properties we use (or can use) in the database profiles. This will make it easier to inject those properties into our tests. So in the “<properties>” section of the parent POM, define a few default values:


Injecting the database properties

Each of the database profiles define a bunch of properties, and we want to use those property values in each of our tests. The easiest way to do this is to use Maven filters to substitute the property values in some of our resource files when it copies them into the ‘target’ directory. We could do that in the parent POM, but it’s probably best to do that in each subproject, where we can be specific about the files we want filtered. JBoss DNA has a “dna-integration-test” subproject, and so in its POM file we just need to turn on filtering:

   <!-- Apply the properties set in the POM to the resource files -->

Here the first “<testResource>” fragment copies all of the files in the “src/test/resources” directory without doing any filtering, while the latter fragment copies the “src/test/resources/tck/jpa/configRepository.xml” file and does the substitution of the property values. For example, any “${japSource.url}” strings in the file are replaced with the appropriate value for this property as defined in the database profile (or the default).

The “configRepository.xml” file is a configuration file for the DNA JCR engine. What if we want to inject the database properties into our test cases? Well, one easy way is to have Maven filter a property file and then just have our test cases load that property file and use the data in it. We actually do this in one of our other projects, and it works great.

Running the tests

Now we can see the fruits of our labor. So we can run

mvn clean install


mvn -P integration clean install

to run all unit and integration tests, just as we could before. Since we don’t specify “-Ddatabase=dbprofile” on the command line, these builds will use the default database profile, which for us is HSQLDB. That means any database-related tests we run are fast and require no external resources. Brilliant!

Of course, once all of our tests pass with the default configuration, we can then run all the tests against different databases with a few simple commands:

mvn -P integration install -Ddatabase=mysql5
mvn -P integration install -Ddatabase=postgresql8
mvn -P integration install -Ddatabase=oracle10g

Pretty sweet! Well, as long as we have some patience.

Hat tip to the Hibernate and jBPM team, since our approach was largely influenced by a combination of their setup.

Filed under: techniques, testing

Git and SVN: the beginning

My previous post about Git and SVN hopefully intrigued you enough to want more. Yes, you can use Git locally even though your stuck with using Subversion for the central repository. In this post I’ll show you how to get started, while in the next post we’ll see how the Git Eclipse plug-in.

1. Install Git

You can install it several different ways, depending on your platform and your preference. I have a Mac, so I could download and compile it using MacPorts, or I could grab the latest installer. I like simple, so since I don’t yet have MacPorts I went for the installer. Piece of cake. I also chose to manually set up my path, so in my .bashrc file I added these statements:

export MANPATH=${MANPATH}:/usr/local/git/manexport PATH=${PATH}:/usr/local/git/bin:/usr/local/git/libexec/git-core/

(Okay, I actually used some variables in my real .bashrc to keep things DRY. I thought the above might cut through all that noise.) Note that we added “git/libexex/git-core/” to the PATH. This directory contains all the commands that start with “git-” (e.g., “git-svn …” is really the same as running “git svn …”).

2. Clone a remote SVN repository

The next step for me was creating a local git repository that mirrored the remote SVN repository.

$ git svn clone -s local_dir

The “-s” option tells git that the Subversion repository use the “standard” naming convention of using “trunk”, “branches”, and “tags” directories. When this command runs, it creates a subdirectory called “local_dir”, initializes the git repository in that “local_dir”, connects to the SVN repository at the URL (which does not include “trunk” or any other branch/tag information), proceeds to download all of the history (including branches and tags) in the SVN repository, does some cleanup, and checks out the equivalent of the SVN “trunk” to our working area. And, because of Git’s bridge with SVN, we’ll be able to regularly pull changes from SVN into our git repository, and we can even upload changes we make locally back into SVN.

We now have our new Git repository, so cd into it:

$ cd local_dir

If you look closely, you’ll see a couple of things. First, Git doesn’t proliferate your working area with “.svn” folders. Instead, there’s just one “.git” directory at the top. Git also uses a different technique than SVN for remembering which files and directories should be ignored. But we can bootstrap Git’s ignore information automatically from SVN’s. We’ll use the “.gitignore” file in our working area, since we want this information to be versioned and we want everyone using the repository to be able to use it. If your SVN repository already has a “.gitignore” file, then someone’s done the work for you. Otherwise, you’ll have to run the command to generate the file:

$ git svn show-ignore > .gitignore

This will take a minute or two, depending upon the size of your working area. I would then commit this new file, placing it in the master branch.

3. Committing (locally)

To commit our new file, we first have to tell Git that we want to start tracking this file. Or, in Git parlance, we want to stage the file by adding it to the index:

$ git add .gitignore

We can use the status command to see a summary of what’s already staged, what’s being tracked but hasn’t been staged, and what’s not being tracked at all:

$ git status .

We can also get a more detail report showing all the individual changes that have been staged:

$ git diff

We can then commit our staged changes:

$ git commit -m "Description of commit" .

or, if we want to automatically stage any modified or deleted files, we can add use the “–all” (or “-a”) option:

$ git commit -a -m "Description of commit" .

With the “commit” command, Git records the entire set of staged changes as a single commit to our current branch, and it moves the branch’s HEAD pointer to this last commit. Remember, Git works locally, so this commit is recorded locally. This may sound strange at first, but really it allows you to commit (locally) much more often, which can improve your development process since backing out things that don’t work is really easy.

To see the history of commits, use the “log” command

$ git log

This displays each time, message, and SHA-1 hash (the unique identifier) for the last n commits. You can use these hashes (or the first 4 characters or so) in the diff command.

$ git diff <hash>

displays the difference between a previous commit and changes, while:

$ git diff <hash1> <hash2>

displays the difference between two previous commits.

There are a lot of things Git can do – way too many to cover here. But look at “git bisect”, “git grep”, “git revert” jut to name a few. But let’s get back to our workflow.

4. Updating from Subversion

If you’re familiar with Subversion, you’re hopefully used to doing “svn update” before committing your changes. This pulls any revisions that others have made since you last updated, and its good form to do this and to make sure everything compiles and runs locally before committing.

In Git, you do this with the “git svn rebase” command. This command fetches revisions from the SVN used by the current HEAD (of the current branch), and “rebases” any current commits on the branch to apply to these latest SVN changes. This is analogous to creating a patch file for each local commit (relative to that commit’s parent) since the last rebase, updating the branch with the new SVN revisions, then sequentially applying the patches. In other words, it takes all your local changes (in the form of commits) and reapplies them to the latest SVN revisions.

Here’s the actually command to rebase the current branch:

$ git svn rebase

If you want to fetch all of the revisions on all branches in the SVN repository and rebase any local commits on those branches, you can add the “–fetch-all” option.

5. Committing back to Subversion

Once you have rebased your local commits on a branch, you will still have changes to that branch that aren’t yet in Subversion. We want to do the equivalent of an SVN commit, which in Git is to commit each diff on the branch back to SVN:

$ git svn dcommit

This creates a new revision in SVN for each local commit on the branch. Of course, if you want them to look like a single revision, you’d need to squash the commits before running the dcommit command.


This post focused mostly on how to get started with Git using a remote Subversion repository, so I didn’t talk much about how to go about branching and merging. I think you’ll agree, though, that Git’s Subversion bridge seems very intuitive, and in fact many of the same concepts used by the Subversion bridge are the same concepts used by the rest of Git. And once again I’m going to suggest reading the Git Community Book or watching Bart Trojanowski’s presentation.

Note: Portions of this post were taken from a previous post on my personal blog.

Filed under: techniques, tools

Git it together with Subversion

This is the first in a series of posts about how I use Git to work on JBoss DNA. Like many other projects, all of the code for DNA lives in a central Subversion repository. And for a long time, I naturally used a combination of the Subversion command line client and the Subversive Eclipse plug-in.

Git works a little differently than SVN. Git is a distributed version control system, and that means that you have a local copy of your project’s Git repository. Yup, the whole thing. But that’s a game changer, because you can commit, branch, view history, or whatever while your disconnected from the web. And it’s fast (since everything is done locally) and surprisingly space-efficient. Then, when you’re ready, you can push or pull changes to other Git repositories.

Git also works really well pushing and pulling from a remote SVN repository. In fact it’s built right into Git. So if your remote repository is SVN, you can still use Git locally. The only difference is how you push to and pull from the remote repository.

What really convinced me to try Git, though, was how brilliantly it does branching. Actually, branching and merging. Quite a few source control management systems can branch, but the problem is they don’t merge – or if they do, they don’t do it well. Git, on the other hand, makes merging easy. In fact, it’s really easy to create branches, switch between branches, and merge branches.

So why am I so hung up on branches? Well, being able to branch and merge means that I can use branches for everything – bug fixes, new feature work, prototypes, or maintenance. I can create a branch where I do all my work, and I can commit as many times as I want. I can even use tags to help me remember important commits (e.g., “all tests pass”). And when my work is complete, I can merge my changes onto as many branches as I need to. All of this is done locally, without polluting the main repository’s trunk or branches. Of course, if I’m working with other people on a feature or bug (e.g., “please review these changes and provide feedback”), I could expose my own repository or we can leverage a shared repository. And when the work is all complete, merging onto the official branch(es) is cake, too.

In forthcoming posts, I’ll show how to install and use Git with a central Subversion repository, describe some of the tooling I use with Git, and outline a number of very common use cases: simple branching and merging, stashing your work to fix a bug, and so on.

Until then, if you want to learn more about how to use Git, watch Bart Trojanowski’s presentation. It is 2 hours long, but it really is a brilliant tutorial if you’re interested in learning the fundamentals how Git works and how to use Git.


Filed under: techniques, tools

AtomPub, REST, repositories … and Starbucks?

InfoQ has a great article on how REST and AtomPub work, using a “Starbucks” example. (Via Joel Amoussou.)

One thing this article does well is explain how representations can contain URIs for followup activities (e.g., what can be done next with a resource). In addition to the “next” used in the article, activities might include undoing the last change, or “publishing” some artifact. This works extremely well when coupled with some sort of state transition (process) engine on the back end and allows clients to “discover” the process, even when that process changes! And note that the process might include lifecycle changes and/or workflow.

Now, how does REST and AtomPub relate to JBoss DNA and JCR repositories? Well, it’s always useful to have a REST API into a JCR repository. For the most part, I think a generic repository library like JBoss DNA will provide a generic REST interface. After all, working with a JCR repository doesn’t have a lot of process – it’s mostly just simple get/post/put/delete operations on the nodes (resources). But AtomPub only defines three levels (services, collections, and entries), and this doesn’t really map well to a hierarchical repository. (Yes, I could conceive of several different mappings, but the question is whether any of them really are useful.)

So while a content repository system might not provide a generic AtomPub, a domain that uses a JCR repository may very well want to use AtomPub to allow clients to work with the domain’s content. Consider a SOA governance repository, which may have multiple collections (e.g., “services”, “published services”, “news”, “policies”, etc.), and workflow (e.g., uploading a service, publishing a service, etc.). Here, AtomPub can be mapped to the SOA governance domain pretty easily, but this mapping is probably different than other domains.

It would be nice if JBoss DNA could have a generic “AtomPub” facility that is content-driven: specify the collections (maybe defined as queries), map the entities to workflow processes, etc. But that’s getting ahead of ourselves. First things first.

Filed under: rest, techniques, tools

More on federated repositories




Filed under: features, federation, repository, techniques

To check or not to check (parameters) – Part Deux

In one of my recent posts, I talked about checking method parameters using a utility that throws IllegalArgumentException when a parameter was deemed invalid. The whole point of this check is to prevent a more convoluted exception from certainly happening later on, and using the IllegalArgumentException to give the developer some useful message about what’s allowed or not allowed.

The post generated some good comments. For example, my post didn’t really make too clear that checking method parameters is not a replacement for checking the validity of input data (e.g., data provided by a user, say via a form, or data provided to a database). There are a lot of neat frameworks for doing that, including Commons Validator and Hibernate Validator to name two.

I wanted to follow up on my previous discussion, though, and discuss checking method parameters using Java assertions. In general, this is a bad idea. For one thing, checking arguments with a static utility method (e.g., ArgCheck.isNotNull(...)) should be very fast. Second, assertions can be disabled, and doing so shouldn’t change the behavior of the code. (This is why I’d argue that continuous integration should be run with and without assertions.)

The major requirement for using assert is that removing the assertion should not change the flow of the code – called control flow invariance. This means that using assert for checking parameters

Filed under: techniques

Internationalization in JBoss DNA




I’m not asking for much.

Meet the players


Sweet and to the point


public final class DnaSubprojectI18n {

// These are the internationalized strings ...
public static I18n propertyIsRequired;
public static I18n nodeDoesNotExistAtPath;
public static I18n errorRemovingNodeFromCache;

static {
// Initializes the I18n instances
try {
} catch (final Exception err) {



errorRemovingNodeFromCache = Error while removing {0} from cache

Again, pretty simple and nothing new.

Using in your code



  throw new RepositoryException(DnaSubprojectI18n.propertyIsRequired.text(path));


Testing localizations

public class DnaSubprojectI18nTest extends AbstractI18nTest {
public DnaSubprojectI18nTest() {


One more thing …


Filed under: techniques, testing, tools

Mockito 1.5

Mockito 1.5 has been released with several nice enhancements. Perhaps one of the most useful is the ability to spy on non-mock objects. In other words, you can verify that methods are called on the non-mock object. So, for example (from the release notes):

   List list = new LinkedList();
   List spy = spy(list);

   //wow, I can stub it!

   //wow, I can use it and add real elements to the list!

   //wow, I can verify it!

I haven’t wanted to do this too often, but there was an occasion or two.

Another improvement is supposed to result in more readable code. Instead of


it is now possible to write:


Notice that the code is exactly the same length, so it’s clearly up to you whether you think it’s more or less readable. In addition to doReturn(), there’s also doThrow(), doAnswer(), and doNothing().

Check out the Mockito documentation for examples and details on how to use.

Filed under: techniques, testing, tools

To check or not to check (parameters)?

Some topics in software development are all about preferences and personal choices, and so it is with checking method parameters. When is it worth the extra developer time and execution time to check that the parameters of a method adhere to the expectations of the implementation?

What would you want to happen if you passed in a bad value to a method in some library? Would you want some cryptic exception thrown from the bowels of the library? No, chances are you’d rather get an immediate IllegalArgumentException telling you why your value is bad. Plus, you’d like to read in the JavaDocs what is considered a good (or bad) value for each parameter, and what happens when bad values are used.

Not all code has the luxury of a well-defined public API, where developers can focus on providing meaningful messages and documentation for a limited set of “public” methods. Open source libraries rarely have much control over what classes can be used. And maybe we don’t have to document much at all – can’t the developer using the library just look at the source?

My view is that developers should spend a little extra time to make life easier for their customers. This means good documentation and meaningful parameter checks for (most) methods that will or are likely to be used by clients. If you don’t make your source available to your customers, then your documentation probably needs to be “great”, not just “good”.

So what do we do on JBoss DNA? Well, we don’t really have a well-defined “public API” – we want our customers and users to be able to use JBoss DNA in ways that make sense for them, and that may include extending the behavior by writing a new implementation of an interface, using a small component by itself. In short, they’ll probably use our software in ways we don’t anticipate. Our philosophy is to try to be as helpful as possible, because a little bit of work on our part can make it a lot easier for our customers. So most of the time we’ll check the parameters, but in times where we don’t (because of performance, because a method is protected or private, or because there’s a good reason not to) we’ll still document what the method expects.

To make our lives easier, we’ve created a utility to help us to write simple and readable code that checks parameter values and throws meaningful exceptions when a value isn’t valid. Since JBoss DNA is internationalized, all exception messages should be internationalized, and this utility does this, too. So, we can write code like this:

     * Set the configuration for this component.
     * @param config the new configuration
     * @throws IllegalArgumentException if &lt;code&gt;config&lt;/code&gt; is null
    public void setConfiguration( Configuration config ) {
        ArgCheck.isNotNull(config, "config");

This trivial example shows a couple of things. First, the call to ArgCheck.isNotNull is pretty readable. (In hindsight, CheckArg.isNotNull(config,"config") may have been a better choice. There are other patterns that are even more readable, but then you start trading off performance for readability.) Other examples of ArgCheck methods include isPositive, isNotEmpty, hasSizeOfAtLeast, isGreaterThan, and containsNoNulls. Of course, there are many more.

Second, the ArgCheck methods will internationalize the exception message, so we pass the variable name (which should match the JavaDoc) as the second parameter to isNotNull(...) so that the exception message is meaningful. In this case, the English localization for the message would be “The config parameter may not be null”.

Third, the JavaDoc uses @throws to document the condition when an IllegalArgumentException will be thrown, and we chosen to not duplicate that in the corresponding @param. (Then, in cases where we don’t check the parameter, we don’t put a @throws and we do put the assumption in the @param. For example, “@param config the new configuration; may not be null.“)

When do we not use ArgCheck? We don’t often check parameters in protected methods, since we often have much more control over how the methods are called. And we never need to check parameters of private methods. And some public methods are called so frequently that even the overhead of checking the arguments is not worth it, but in these cases we also try to explicitly say this in the methods’ JavaDoc.

So, in general, we check a lot of our method parameters. What do you do?

Filed under: techniques

ModeShape is







Get every new post delivered to your Inbox.