I’m having at look a GRails 1.0.1. I get a little problem after setup and here is a little trick to help you if you encounter the same kind of problem.

So I installed version 1.0.1 that you can download here. I’ve set up the environment variables and launched “grails” on the command line.

At that point I get some weird message as “Xxxx was unexpected at this time”. This was due to my JAVA_OPTS. To find the problem, just setup the DEBUG environment variable to anything you want.

This way you will see the content of the batch command executed and will be able to find the root cause of your problem.

You can have a look at it …but use firefox as the web site has some display problems on IE (or maybe this is due to my IE configuration).

Finaly the famous Rails framework enter in java world, that should provide excellent reasons to no more see this RubyOnRails video:

[youtube:http://www.youtube.com/watch?v=PQbuyKUaKFo]

Robert Cooper talked about why he hates BPM products but love BPM: XML is not a programming language! I fully agree on this assertion and you can read about a post (in French) I’ve written on this.

This is true for BPM, but also for many enterprise software (see this post from Ayende about what is an enterprise software): they pretend/tend to be highly customizable to be able to match the enterprise processes without requiring high development skills…this usually true, but the drawback is the maintenance, upgrade and evolution capabilities which are usually also very poor.

I think Designer Oriented Architecture for non-techy (non-techy does not mean “no skill” at all…see later in this post) is not compatible with maintenance, upgrade and evolution capabilities. For the rest of this post and as I’m lasy, I will talk only about maintenance but you can replace by “maintenance, upgrade and evolution”. The incompatibility is simply due to the fact that a non-programmers have in fact development skills but are Morts (see Jemremy’s post which I totally agree on), meaning they have to build/customize applications to fulfil some requirements and they learn/use only what they need to do this: “Mort is a developer who only learns just what he needs to complete his project, on the project”.

Microsoft Visual Studio and .Net framework are addressing this very well. This is in itself not a bad thing, but it prevent maintenance. Evan Hoff catch it in his post. Clearly: Designer Oriented Architecture goes against maintainability…and Enterprise Softwares (or BPM products) have not yet understood this. Today, I’ve seen high level requirements for new Enterprise Software platform for an external company with 3 requirements inside the same slide: “Easy to build quality application” (where quality means testability and maintainability), “High level of extensibility without preventing upgrade” and “Ease tayloring, configure and manage applications”.

I can understand people that are looking for a Ferrari able to do cross over and consuming less than 1liter of fuel per 100 kilometre…but I’m doubtful on the success of the search.

Quality, maintenance, ease of upgrade, evolution capabilities are just incompatible with tayloring done by Mort.

Reading an interesting post from Dwhittaker, I think the problematic is broader. It appears through the question on how should I build my entity from the DAO object. To my opinion, the DAO object should be as much simple as possible…not smart at all. The best is to generate it using byte code generation for example using annotation in the real entity object for example, buit this should be addressed into another post

If the DAO object is generated, then the entity (or value object) can be build by generic code using reflection for example. Then the “real” question raise: what about information lost during the entity creation. Let me explain with a very simple example: waht about the Id of the DAO object? It has absolutely no sense in the entity object, but is required when the application push back the entity object to the persistence layer to store changes. This topic is highlighted in this post of Ayende and expose another example: detection of changes.

The problem is about keeping track of information on an object (the entity) without exposing it. more over, this object can be moved from a process to another (or webservice). This sounds very close to the state in ASP net pages, which provide a very good answer. So, I wouold advocate to store a state in the entity object that is crypted and reused by the persistence layer.

Avec l’émergence de l’AOP (Aspect Oriented Programming) et l’IoC (Invertion of Control), on voit fleurir de plus en plus de développement reposant sur l’utilisation d’XML au plus près du code. L’inconvénient majeur (d’où ce billet) que j’y vois est l’utilisation abusive de XML (voire HTML…je vais détaillé un peu plus loin) pour remplacer du code Java ou d’un autre langage. Je ne nie absolument pas la valeur que représente l’IoC, ce qui m’interroge c’est: Pourquoi vouloir utiliser un support tel que le XML pour fraire de la programmation alors qu’il n’est absolument pas fait pour cela. Cela va au delà du couple XML/IoC, on peut parler de HTML/Fit ou encore XML/BPELJ. Lier des composants ou process de haut niveau entre eux de manière découplée avec une configuration en XML à tout a fait du sens. Cependant, plus l’intégration entre ces composant devient intriquée, plus la configuration devient complexe, in-maintenable, sensible aux moindres changement. Pour le BPEL, c’est la même chose, quand l’interaction entre les process devient complexe au point de devoir faire des boucles, il est temps de se demander si on ne doit pas plutôt utiliser un vrai langage de programmation qui se chargera de masquer cette complexité. Dernier exemple: HTML/Fit, qui est un framework de test permettant a des “business analyst” ou “Ingénieurs Qualité” d’écrire des tests en HTML pour valider une fonctionnalité. Cela fonctionne très bien, jusqu’a moment ou une voit apparaitre quasiment des lignes de code dans les ligne de tableau Fit. Il est alors grand temps d’écrire une fiterie de plus haut qui va masquer ce pseudo-code, sinon il faudra être développeur pour pouvoir écrire un test…et c’est exactement l’inverse qui est recherché.

D’une manière générale, laissons aux langages de programmation la responsabilité de cacher la complexité…ou tout simplement les détails d’implémentation, et laissons au XML (ou HTML) la configuration de haut niveau compréhensible à tout le monde.

This post is the third of a serie of posts on persistance stack.

In the previous posts, I describe my humble opinion of the layering for a persistance stak for an entreprise application (for smaller application, there is no problem to mix or get ride of layers). Most of the folks talking about a persistance layer are often forgot to talk about the various services that are providing a strong added value to the applications build on top of it. So, this post is dedicated to some of these services. I’m not saying I will be exhaustive (because I simply do not know all of the potential servics that a persistance stack could provide) but I will try to go beyond what I usually see on this topic.

What is a persistance stack service?

I should;maybe say: why are they so often forgotten? I think this is because we start to deals with business requirements that are not so attractive to technical folks that prefer challenge on the core of the persistance stack. IMHO, a persistance service is an engine that can fulfill extra business requirement from a persistance perspective. That means it’s not mandatory at all from a persistance stack to provide such services as they can be provided through the applciations themselves. The huge benefit to integrate them in the persistance stack is to allow consistent and rapide application development without derundancy in code.

In this post i will details services as query language, scripting, database creation and upgrade, workflow engine, archiving, versioning and partitioning. Some of this service are low level services … I mean they require access to the lower layers of the stack (even if their configuration can be localised in the same area than configuration of higher level services).

Query language

Even if lot’s of persistance stack provide a query language, I do not concider it as a core service of the persistence stack as these persistance framework are usually presenting it as a tuning step or a way to workaround the regular usage of the framework for complexe request. Despite this, this is probably the persistance service having the strongest capabilities and is often a requirement for the other services as partitioning for example. The aim is provide a query pseudo SQL to the framework. I don’t think the key feature of this service is to provide an RDBMS independante query language…I’m sure most of you won’t agree. To me the key feature is to provide a way to provide to the application administrator a way to go beyond the application limitations…if it’s RDBMS independant: great, but we all know that even if SQL is a standard, all implementations are differents and I don’t talked about opitimization! This service is so powerful that it should be

Partitioning

This service is a low level service, that means it’s customization requires the user to use the Query Language as it expose more the database schema than business objectes. Partitioning is a way of doing segregation on (read or write) access to entities (I should say main record of entities). This is tipically a where statement.

Workflow Engine

The persistence layer will trigger this engine to handle integrity check and process steps.

Cette question refait encore surface (cf billet sur slashdot). Il est incroyable de voir tant de guerres de chapelles avec des arguments totalement loin de la vraie problématique. La question classique est “En quoi l’assembleur peut apporter quoi que ce soit alors que tout le monde utilise un langage de haut niveau style Java ou C#?”.

Primo, tout le monde n’utilise pas ce genre de langages et l’assembleur est parfois totalement indispensable dans certains domaines (drivers, pixel shader,…etc). Secundo, même en dehors de ces domaines, l’assembleur ou plus précisément des connaissances en assembleur sont très utiles. On peut sans problème coder sans avoir la moindre connaissance de l’assembleur…c’est vrai, mais il assez impressionnant de voir comment des Bac+5 se plantent sur des problèmes simples genre listes chainées. Ensuite, quand arrivent les problèmes de performances, consommation mémoire, un développeur qui ne sait absolument pas comment cela se passe en interne, sera complètement démuni. On pourrait re-phraser la question initiale en :”Quel est l’intéret de connaitre les pointeurs quand on programme en Java ou C#?”. La réponse est la même: on peut faire sans, mais on se casse les dents sur des bugs tout bêtes et on tombent sur des problèmes de consommation mémoire.

En conclusion, le langage assembleur n’est plus vraiment utilisé (saufs domaines spécifiques a ne pas négliger) mais sa connaissance (ou tout du moins la connaissance des méandres internes des langages de haut niveau) permettent d’éviter beaucoup de problèmes. Les développeurs ayant cette connaissance ont aussi en général un très bon niveau algorithmique et savent optimiser et coder correctement. Ce n’est donc pas indispensable mais reste la preuve d’un niveau élevé.

This post is the second of a set of post on the same topic starting here.

Where do I put business logic?

I’ve seen this question raised a lot of times by developers who are dealing with management applications using a persistence stack. This question is simple and legitimate but the answer is a little bit trickier.

First statement: there are different kinds of business logic which have to be in different layers. Business logic is itself fuzzy and quit large. To some extreme some could argue that the entities definition is part of the business logic, even the primary keys as a way to uniquely retrieve an entity. To the other end, a business process is considered as a kind of business logic. I personally think the business logic is wide but exclude the example I just mentioned.

Even if an entity has obviously some business inside its definition, its only data, so no logic at all. For the primary keys, I consider that a exposing primary keys is a mistake that prevent evolution and prevent implementation of high level technical services as archiving for example. The best is to expose 2 attributes per entity for retrieval: one which is human readable (as the ID of an employee) and another one which will be used for application integration. The idea is that these “IDs” are not varying from a system to another, or during the life cycle of the entities (except for exclusive business reasons), in the contrary of an internal primary key which could vary over time.

Bitwise, you can consider that the mandatoryness of an attribute (or validation) are part of the business logic. I agree some could argue it’s exclusively a data modelling concern. That’s not totally wrong but I think there is a mismatch between engine location and configuration. The engine location is of course in the data modelling layer but as this is part of the business logic (from my point of view) this should be configured in the same area as the other business logic: in the configuration. I will address the problematic of configuration later on. At the same level you have other business logics as: right management, partitioning and entity validity (broader than attributes validity).

Back to the initial question: where do I put my business logic? IMHO, the best is to define it in a set of configuration files and let the entities delegate to the appropriate engine. This provides a flexible approach (separation between the configuration and the engine) which is also generic (usage of an engine). This question is not totally addressed by this approach and hides the real problematic: scope of business logic.

The scope of the business logic is far more meaningful and key for a persistence stack. There are at least 3 different scopes:

• entity attributes
• entity
• application
• inter-applications

The entity attribute scope is for example the scope for validation/mandatoryness of the attribute PONumber in a purchase order for example. The entity scope is for example the global amount of a purchase order which is supposed to be the sum of the amount of the purchase order lines. The application scope is supposed to address the internal workflows to the application as the approval workflow of a user request for example. The inter application scope will address business logic between applications as between the requesting application/service and the catalogue application/service.
You will notice the “entity” word is ambiguous: does a purchase order line is an entity or not? IMHO, from a strict business perspective, I don’t think so, even if it’s convenient. I think a DAO can be considered as entity as soon as it has it’s own validity in the business: does not make sense to talk about a purchase order line outside of the scope of a purchase order. This distinction is sometimes used for value objects versus DAO. I personally prefer use entity and DAO and have both of them in separate layers: Data Access Layer (or data modelling layer) and Domain Layer.
   

This post is my 2 cents contribution to the debate on persistance stack architecture. I try to write short post (the shorter it is…the understanding greater is), but this time —with such a topic— I can only fail on this point. So this post will be followed by other posts .

I will not start from scratch explanation on the different approachs, the wording and so on…you will find some links in this post that I find usefull and I do not want to be redundant with them. I would like to enlarge the scope of the persistance stack to higly customizable environement and also the various features required by an Entreprise Management Application (taht I will name EMA) relying on a persistance layer.

What are the different layers (required or not)

You will find here an excellent article on the different layers of a typical persistence stack and further…to the UI. The UI is key but I will not address this question in this post. You will notice the author talk about a Persistence Layer which is different from an Persistence Stack. I use the word “Persistance Stack” to express all the layers that are requiered to build an application that use a database without the UI. These layers are usually (from the upper level to the RDBMS middleware):

• Service Layer
• Domain Layer
• Data modeling
• Persistance Layer
• RDBMS middleware 

About the interest to respect layering

It’s just about commion sense : by layering correctly you will decouple your code and allow easier replacement of components as there are inricated. Most of the time, developers are doing naturally as it give an impression of over complicated the code. The important point they have to though about that an Entreprise Software relying on database will grow and evolve (faster at the begining) and you will write at the end touthands lines of code. If the rules and delimitations are clear from the begining, you will avoid a real mess of large classes.

Now let see in details the aim of each layers. I will go from down to top to express clearly the boundaries of each layer (starting from top tempt to mix the layer and put the code/requirement at a higher level than it should).

Persistance layer

The persistance layer is juste about persistance…obvious but not always respected. Persisance is only about storing and retreiveing data, nothing else. You can use an ORM (Object Relational Mapping) framework as Hibernate or TopLink for example. It’s only bout storing, that means no validation, no business logic at all, and the objects should be limited to structures, no methods. There is no real interest to build it’s own persistance layer as most of the time your requirement are fulfilled by the most advanced frameworks. That does not means you should not take care in the choice of this framework as this layer is the key for the performances: having a poor framework (even a rich ones depending on how you will use them) will dramaticcaly impact performance by generating useless or not optimized RDBMS queries.

Most of the time these objects are named DAO objects or DAL objects (for Data Access Layer objects).

Tow extensions to EMA that may be include in this layer are rights and partitioning. Rights management is the ability to give read/write access to attributes of the DAL objects. The Partitioning is the ability to segregate the read/write access to DAL objects. These fucntionality have obviously a business aspect,e specially for the partitioning. But for performance reasons (espacially for partitioning which is usually implemented using a where statement), this might be implemented in this layer.

ORM..a myth or not? 

The different post, disucssions and articles I’ve read about persistance stack are highlighting the interest of doing OOP (Object Oriented Programing) especially in the domain layer. Most of the argument yield to the mandatoiness of using an ORM which is the key pivotal framework to step from a relational database to a obejct oriented coding space. This a myth to me and is only relevant for folks that love OOP and particularely love debate about the beauty of it.

ORM use have to be limited to what it is design for: just create DAO objects for upper layer. Some folks abuse of this because they are mixing it with the higher level layers which is strong mistake. I’ve looked at Hibernate and NHibernate (to simplify: NHibernate have the same capabilities than Hibernate v2, but in a .Net environment). As a framework it provides lots of different ways to operate with the RDBMS layer, but you usually use on 10% of them as you are doing some assements. IMHO, this is not a mistake to do so, if you take care that this assement will not have a strong impact on the other layers. You will never be able to take some decisions that have absolutely no impact, but you can mitigate this.

An example of assement is to chose the same type for keys: there is a real advantage to do it, but it will have an impact on the Domain Layer and the evolution of your application (if the type is long…evolve to GUID for example may be tricky if they are exposed and used in the other layers). On this specific point, a good practice is to never expose internal keys outside of the layer but to expose business keys in the Service and Domain layers. For each object, exposing 2 kind of business kay is useful: an international one, and a business one. The international one is a key that won’t varie from a language to another, that is usually a sequence of numbers, which is used by external programs to synchronize data for example. The business one is a key that is usually used by humans over business process as a social key for a employee for example, or a barcode for an asset.

Data Modeling

This layer is most of time mixed with the Domain Layer and Persistence layer, but it can be useful to detail its contract.

In my opinion, part of the contractrthis layer is the dual of the relationship aspect in RDBMS. I deed, you have to define the relationship between various objects in the persistance layer (as a purchase order have order lines for example), but you have to define some low level behaviour. Some of you could say that I start to introduce some business logic here…no, i don’t but you are right: we should not introduce business logic here yet. The behaviour I’m talking about are related to the impact on relationships and complex custom types during CRUD (Create Retrieve Update Delete) operations made by the persistance layer.

A complex custom type is a type that rely on more than one attribute of a DAO object. Here are 2 examples.

Currencies: Handling of currencies is not simple, you have to be able to do arithmetics operations on them, and convert them. Obviously a double type field is not enough, neither a string (when I say string, is a string having only one information, not a string containing an xml description of a currency object with multiple attributes). So in the data modeling you could declare that the currency attribute of the value object will be rely on a set of attributes (amount, currency symbol, reference value in $, date of reference convertion) of a DAL object.

Enumeration: Some enumerations fields may have these requirement. First: the user can add new values. Secondly: some values used by the business layers may not be removed. A typical example is the status of a purchase order. The aim is to present some choices to the end-users (all values already entered by other users for a status plus some predefined fixed values). For technical reasons you may store a predefined fixed value in an attribute and other values into another field, but you want your applciation not to be aware of this.

As you see these are low level behaviour. That the same for relationship: if an object of type A is linked to an (or a collection of) object B, B should be deleted if its parent (from a relationship perspective) A is deleted or B should be attached to another object of type A? Does it makes sense to have an object of type B without an association to an object of type A? All this behaviour have a business rational but this have to be implemented in this layer.

Regarding this last sentence, it really make sense to gather together as much as possible the configuration of almost all layers inside one location. In a metadata driven environement, this is absolutely not an issue to mix configuration of different layer inside a unique location (a file or a a folder)…more over, I would recommend it.

Where do I put business logic?

To continue here

Other interesting links
Segregation/Distinction between Service Layer and Domain Layer

Here is really great post on principals to write maintenable code. More over this can be extended to improve supportability of the products.

Most of the developers are doing development only! It does not sound like a problem for you…I thing we are at the exact barrier between real development and hobby work. I’m not that fan of Agil development as it does not work well if not apply correctly and with a minimum of knowledge, experience and willing to do it. Nevertheless, Agil methodology as the real advantage to melt the concerns and responsability of QA, dev, CM over the entire team. From what I’ve seen, usually, development task are only 20 to 40% of to whole process to implement new features (and I don’t take into account the maintenance, support and bug fixe after release). Do not use the fake argument that it depends on the environment (Language, Framework, QA automation rate…etc), it’s just wrong. I prefer by far having a developper that knows less about a language or a framework but concerned by CM, QA, comprehensiveness and maintenability of features/code than a rely strong developer that will force to use the last framework release that he knows and break build process…even if his coding skills are higher. To make it shorter: don’t care about improving efficiency about 10% on 20-40% of the process, but I care if it’s on 60%-80% of the process plus maintenance, evolution…etc.

A sentence from the article that are fiond terribly true: “there is an inflection point where a coder mindlessly spewing out code transforms into a thoughtful software craftsman capable of creating maintainable code. That inflection point happens the day a coder first stops, lifts his/her nose out of the coding window, and says to him/herself “where should this code go?”.
I’ve seen this many times and is named “Seperation of concerns”. That does not mean using Spring is mandatory do code correctly , but as soon as a developer start to think this way, he will improve maintenability, componentization, and avoid spagetty development with useless interdepedencies. We see new languages as Ruby, Python that are marketed as “less code to implement the same feature” than with other languges as Java and C#. I agree that simplicity is fondamental and less code also, but this should not lead to weird code that can’t be maintained: if seperation of concerns leads to 3 additional lines of code to do a back-and-forth between 2 layers…that’s good.