RPM example - Application packaging

This post is about something quite different than the others.
It concerns packaging your software as an RPM file making it ready for delivery.
If you have never used RPM before and/or you're not an experienced Linux user, this post is intended for you.

I'm far from being an expert in this area, and I would guess there is probably a better way of achieving the goal I want to show here. Please share your thoughts and experience if you have any.

I assume you already heard of RPM, and understand it's benefits and so on.
This post is not to convince you to use RPM, only how to create such RPM file which you could use.

What's in this example?
I created a "Hello World" Java application, and using Ant, I compile it, and Jar it.
This application could be anything you want - this is just a "place-holder" for you, to replace with
your real stuff.

So, after having an RPM file, we'll be able to install it to the system and remove it afterwards.
The path which the program will be installed into will contain the "Hello World" application Jar.

RPM Structure quick overview

As you probably know by now, RPM files are created using a ".spec" file.
This file tells the rpmbuild tool which we will use how to create the RPM package.
The .spec file contains several sections:

1. Headers - Headers of the spec file contain stuff like a Summary of what the package is,
   Name of the package, Version & Release information, License & Package and an
   attribute named BuildRoot. BuildRoot is important - it specifies the location
   of where your package will be temporarily installed.
   What I mean by temporarily, we'll get to later.
2. %description step - This is where you can type just about anything you want describing
   your awesome application.
3. %prep step - In the step, preparing your Source files to be built (using Ant in our example).
   "What's to prepare?" - you ask? - Well, in order to build an RPM package, you must provide
   RPM a .tar file or a zipped file of some kind to it's SOURCES directory, (I'll explain the
   structure of RPM further down), so what you need to do in this step is to extract your
   archived sources.
4. %build step - This is where the actual build of your application happens. In our case, we'll
   execute ant on our build.xml file, that compiles and Jars the application - real simple.
5. %install step - This is the step that happens after the %build step, and will determine how
   your installed package will look like. We will copy the necessary binaries (Jar in our case) to a relevant directory.
6. %files section - This section determines which files from the %install step to pack, and with what access rights.
7. %clean section - This section runs after packaging was done, and cleans up all the leftovers from your build and your install steps. It's important to delete all the old files, especially if you build several RPM packages, you don't want to have files of another RPM build process in your way.

Okay. I know this still doesn't tell you how to do stuff, but this introduction is quite important in my opinion.

So take a deep breathe one last time, and we'll go over the rpmbuild directory structure and understand the steps we need to take in order to accomplish our mission.

"rpmbuild" Directory structure

An important part of the way RPM works, is its directory structure where work is being done.

You should be able to locate your rpmbuild directory on your machine, which usually should

show under your home directory.

The rpmbuild directory contains the following sub-directories:

1. BUILD - This is the directory where your sources will be built, and where the artifacts of your build will be located. The %build step, reads from the directory and writes to it. 
2. BUILDROOT - This is the directory where you will "temporarily" install your binaries to.
   You can think of it as the "INSTALL" directory. After the build step, the BUILDROOT
   serves as a directory to contain the files in a given directory structure. Those files in this
   specified directory structure will be installed afterwards using RPM.
   To clarify: If you place under BUILDROOT a directory called my-app-123 then on the
   real install of the RPM package, your package will be installed under /my-app-123.
   This step reads from the BUILD directory and writes to the BUILDROOT directory.
3. RPMS - This is the directory that will contain your RPM package in the end.
4. SOURCES - This is the directory where you should place your archived sources.
5. SPECS - This is the directory where you should place your .spec file which will create
   the RPM package.
6. SRPMS - This is a directory where source RPMs are created and stored. 

Each of those directories, can be referenced in the .spec file using RPM variables.

Variables

%_specdir	~/rpmbuild/SPECS
%_sourcedir	~/rpmbuild/SOURCES
%_builddir	~/rpmbuild/BUILD
%_buildrootdir	~/rpmbuild/BUILDROOT
%_rpmdir	~/rpmbuild/RPMS
%_srcrpmdir	~/rpmbuild/SRPMS

Now that we finished our overview. Let's go over our 8 steps to create a packaged application with RPM.

1. Create an application (write source code) - in our case, a simple "Hello World" application.
2. Create the build file for the application - Ant's build.xml file in our case.
3. Create an archive from our application's sources.
4. Place the archive (tar) at ~/rpmbuild/SOURCES directory.
5. Copy the .spec file into the ~/rpmbuild/SPECS directory.
6. Create the RPM package using: rpmbuild -ba <spec_file>
7. Install the package.
8. Uninstall the package.

Now, lets go a bit into details:

1. You can download the pre-made application entirely from GitHub: rpm-example-project.zip
   in order to skip steps 1,2 and 3.
   (You can also access the repository - http://github.com/nirgit/RPM-Project-Example).
2. After downloading the zipped application, and extracting it somewhere on your machine,
   you can take a closer look into its hierarchy. You will find under the Application directory,
   a directory called src which contains the sources of our application, and you will find
   the build.xml file which builds the application.
   You will also find 2 other files - the application.tar containing the application sources,
   and the project.spec file.
3. You need now to copy the .spec file into the ~/rpmbuild/SPECS directory, and the
   application.tar file into the ~/rpmbuild/SOURCES directory.
4. In order to create the package now, you're required to run the rpmbuild tool, by
   executing: > rpmbuild -ba ~/rpmbuild/SPECS/project.spec
5. Since I use Ubuntu, I also use alien to install the package. In case you're running on
   a Linux flavor such as CentOS, I think you can simply run the regular RPM install.
   With alien: > sudo alien -i ~/rpmbuild/RPMS/i386/Example-RPM-Project-1.0-1.i386.rpm
   Without: > rpm -i ~/rpmbuild/RPMS/i386/Example-RPM-Project-1.0-1.i386.rpm
6. After the install you should be able to find the package installed under the root:
   /Example-RPM-Project-1.0-1
   Under it, you can find the app.jar.
7. You can uninstall the package using RPM's command: rpm -e <package_name> 
   or (in my case using Ubuntu): sudo dpkg --remove example-rpm-project

The SPEC file

This is the SPEC file associated with the RPM package.
I hope it will serve you as a good starting point for your own SPEC file.

################################################################

#

# This is an example of a simple RPM spec file.

#

################################################################

Summary: An RPM Spec example

Name: Example-RPM-Project

Version: 1.0

Release: 1

License: Apache 2.0

Group: Applications/Sample

URL: http://www.mycompany.com

Packager: Nir Moav <getnirm@gmail.com>

BuildRoot: %{_buildrootdir}/%{name}-%{version}-%{release}

%description

This is a sample SPEC file for the RPM project

demonstrating how to build, package, install(deploy)

%prep

# extract the tar file containing all the sources, to the build directory.

tar -xvf %{_sourcedir}/*.tar -C %{_builddir}

%build

echo "Building the project..."

cd Application

# running ant to build the java project (could be make/maven/gradle or anything else).

ant

%install

# This is the hierarchy which is going to be inside the package (RPM/Deb) eventually.

echo "Install phase..."

mkdir -p %{buildroot}/%{name}-%{version}-%{release}

cp -R %{_builddir}/Application/output/jars/* %{buildroot}/%{name}-%{version}-%{release}

%post

#This runs post the install - maybe you want to execute the application already then

echo "Post install.."

%postun

#this runs after the uninstall

echo "Post Uninstall..."

%files

# tells which files to contain in the package and with what access rights

# the triplet contains of (<file mode>, <user>, <group>). Make the necessary changes.

%defattr(-,nir,nir)

/*

%clean

# Clean up! Must run this! after build and install steps execute, this will make

# sure that the directories are remained clean, so in case you're building another

# package, you don't want to pack the previous build's artifacts.

rm -rf %{_builddir}/*

rm -rf %{_buildrootdir}/*

That's all folks!

I hope you managed to read through this long post, and found it useful.

Enjoy,
Nir

SpacePong - version 2 - HTML5 Game with GWT

Hey,
This is just a small update of the game, to make it a bit more fun to play actually.
So now you can pick different levels to play and have something a bit more challenging.

The source code has changed of course, and you can find it like before on GitHub:
https://github.com/nirgit/GwtPong/tree/space_pong_ver_2_0_0

Space Pong

Enjoy!

GWT HTML5 Game Example - Pong

Hi everyone,

This post is intended to present a very simple example of a game, made with the HTML5 support of GWT.
The game uses only GWT, without any other 3rd party libraries such as PlayN which works with GWT and is rather famous and meant for creating games based on HTML5.
Take a look at PlayN's website: http://code.google.com/p/playn/

GWT provides an API to the underlying HTML5 Canvas, and lets you use it to create
custom graphics.

This example shows the game of Pong. Though the game might be a little buggy, it nevertheless makes use of the fundamental HTML5 Canvas features, such as drawing simple geometric shapes, such as rectangles, circles, gradients and with those elements creating a very simple and compact code for a game.

You can find all the code on GitHub for you to clone: https://github.com/nirgit/GwtPong

Click the screenshot to play:

Have fun playing!

Gradle - Opinion, experience and examples

Opinion and Experience

I had quite a short experience with Gradle, but the impression it left is of an incredible tool.

I don't want to diss any other build tools out there like Ant or Maven, they're great,
and if you're a real savvy, you'll get the job done probably pretty fast.
I generally believe that one needs to use the right tool for the right task, but I'd dare say
that whatever you're doing with Maven or Ant you could probably do better with Gradle -
it is simpler, faster, shorter to write any build file, much more concise yet very powerful.

I have also my experience with Ant for years and with Maven, and I can tell you that I appreciate
Ant very much as a build tool. It does exactly what you expect it to do.
When you read a build.xml file, you'd hardly find something you don't understand.
It's just clear how things work, and what is expected at the end.
The problem is, you need to write so much "build code" in order to get your job done, especially
in a complex system, when you have includes of other extension.xml files you rely on and so on.
What about dependency resolution? - OK, so you could use Ivy, but that's just for dependency resolution.
It's not a "full" build tool in the sense of Maven that comes with creating your packages out of the box, deploying it to your tomcat and distributing it to your Repository.

Yes, Maven can be great but I must say that my 'like' metric towards it is not so stable.
It is too complicated to get simple stuff done... compiling GWT? - Get a plugin!
In Ant, it's really simple to compile a GWT project + you get the example from the GWT website.
In Gradle you can simply execute Ant Tasks, so no plugin extra work needed.
Want to include / exclude resources from your WAR file... why does it have to be complicated?
Dependency management and resolution is awesome, distributing your artifacts into a repository
is great, and fairly easy to do.

However, despite the good stuff of these build tools, I must say that from my personal experience  Gradle tops them.
Gradle like many other posts on the internet would say - brings all the good stuff from the worlds
of Ant and Maven while keeping the ugly stuff out.

What do I mean by that?

1. It is the easiest build tool I've seen to get started with - simply download, extract and set the GRADLE_HOME environment variable on your path.
   Like Ant - no other configuration is needed! Sweet.
2. Very well documented!
3. You can execute any Ant task from the Gradle build files. Any.
4. Dependency resolution - from Maven repositories, it leverages the existing repositories you may already have in your corporation without needing to change anything, while decreasing the verbosity of dependency declarations in the build file itself.
5. Gradle is based on Groovy, so you can just program your build.
   And you can use all the libraries of Groovy. And it runs on the JVM, so you could theoretically use any Java library you like when running the build.
   Did I already say powerful?
6. It is dynamic - You can tell Gradle in run-time for example, which sub-projects you'd like to include in your Project's build. I did that, using naming conventions for instance when I had a process that was auto-generating entire projects ready to be built in a pre-compilation phase. Incredible!
7. No need to sit and rewrite ant targets to do jobs like clean, compile, jar, war etc... it is all there already!
8. The integration with Eclipse is excellent. All you need is there.
9. It just works!

Let me give you an example of how a simple Gradle build (build.gradle) file looks like:

apply plugin: 'java'
apply plugin: 'eclipse'
sourceCompatibility = 1.6
version = '1.0'
jar {
    manifest {
        attributes 'Implementation-Title': 'My project', 'Implementation-Version': version
    }
}
repositories {
    mavenCentral()
}
dependencies {
    compile group: 'commons-collections', name: 'commons-collections', version: '3.2'
    testCompile group: 'junit', name: 'junit', version: '4.+'
}

I think that what is so nice about this build file, is that you can understand it right away.
Lets go over the lines to try and understand this file:

1. The first two lines declaring the "apply plugin...." means that you are telling Gradle to use plugins such as 'java' for example to it would know how to compile Java source to classes.
   Gradle is based on plugins, which facilitate all the needed 'know-how' and logic of performing
   various tasks. - You could write your own custom plugins as well.
2. The 'eclipse' plugin for example generates the needed files for importing your Gradle project into Eclipse.
   You can read more on that on - The Eclipse Plugin.
3. sourceCompatibility - tells Gradle what version of Java your source files are written in.
   You can also specify the 'targetCompatibility' attribute in order to compile your sources into a lower version of Java.
4. version - Declares a variable called 'version' with the version number of your package
   (Be it a JAR or WAR file). The Jar files that are generated with the Gradle 'build' task, have a Maven convention. So the version number is appended to the name of the package at the end. (for example: "MyPackage-1.2.jar", in case version was given the value "1.2").
5. jar { ... } - This is a Gradle Task is a "task" (can be thought of an Ant task) which is part of the 'java' plugin that assembles a Jar file from your compiled code.
   You can add to the manifest file your own attributes, such as a version number.
   Note that  'Implementation-Version': version is using the version number declared earlier in paragraph 3, but you could have used any other variable to place your value in there.
6. repositories { mavenCentral() } - determines the repositories from which you want to perform dependency resolution.
   In this case - using the Maven Central repository.
7. dependencies { ... } - determines the needed dependencies for your build.
   Like Maven's dependency scope, in Gradle there are also so called 'scopes' which you declare for your dependencies: compile, runtime, testCompile & testRuntime.In this case we declare for example the following:
   testCompile group: 'junit', name: 'junit', version: '4.+'Which makes sure to use the JUnit version 4 and above during compilation of test files.

That's it.
It's really short and simple!
Now you can just run from your command line : > gradle clean build
And you will get your project cleaned, compiled, packaged ("Jarred"), tested and with a JUnit test report, all under the 'build' directory of your project.
Of course, you can also distribute the artifacts (your Jars) to your repository, or just copy them
flat to an "Artifacts" directory of your project and so on.

But this was only to demonstrate how easy it is to create a build file, that does so much.

Examples

Running an Ant Task from Gradle build file:

You can run an Ant tasks from a Gradle build file simply by writing something like this:

ant.echo(message: "Building a project....")

This task in particular is not that useful... it's just a print, but it demonstrates how easy it is to execute Ant tasks!
Especially if you consider migrating from Ant, and you have custom Ant Tasks written
which you want to re-use - then you could easily do this.



Running an external Java program from an external Jar:

This could be handy if you want to run anything which is for example at a pre/post phase
of the build.
The snippet below shows how you would apply an XSLT on a XML file, using
the xalan library:

ant.java(jar: "lib/xalan.jar", fork: "true") {
arg(line: "-xml" + " abc.xml")
arg(line: "-xsl" + " myXslt.xsl")
}

(the "lib" directory is at the project's root directory as were the "abc.xml" and "myXslt.xsl" files).


Including Sub-Projects in your Project's build dynamically:

When your project is comprised from sub-projects, it is necessary to tell Gradle about
those sub-projects in a dedicated file called settings.gradle.
In that file, one needs to "include" the sub-projects that need to be built by Gradle.
So typically you could see something like this:

include 'server', 'client'

The great thing about Gradle is that in the settings file you can pretty much put just about anything
you put in a build file. The settings file is executed in the initialization phase of the build.
So if you want to dynamically include sub-projects into your build because you're generating them
from another process, or because you have many of them, or for any other reason, you can do
something like this:

new File(".").eachFile(groovy.io.FileType.DIRECTORIES) { f ->    if(f.name.startsWith("subProjectPrefix") {                

                            include f         
                 }          }

This will just iterate over the root directory of your project, check if the current directory
it iterates on starts with the "subProjectPrefix" prefix, and includes it if it does.


This was really just the tip of what Gradle can offer you.
You can find many examples on the internet as well as on Gradle's installation directory.
(Look under %GRADLE_HOME%/samples - there are many build files examples).
So if you're using Ant, Ivy or Maven I recommend you to look into it, and give it a try,
I don't think you will regret it.


Gradle - http://www.gradle.org/

Merge on check-in, is there more to this?

In case you find yourself merging code very frequently when trying to commit/check in/push your changes to the version control system (VCS like GIT or SVN), you should start asking yourself - 

"Why am I merging so many times? What is the reason for this?".

If you are making lots of merges to your code - it could indicate a design problem in your project.

(I am trying to be very careful here, as each project is unique in its characteristics - the amount of code and modules, the amount of people working on it, and so on, so it does not necessarily point that out).

"How come?" you might ask...
Basically merging happens when more than one party checks out a piece of code makes changes to it and then checks it in. Only the first party that checks in is avoiding a merge, but all others that made changes (to a now stale version of a file) will have to merge their changes with the latest commits.

I heard many times the good argument that claims that to check-in small pieces of code at a time reduces the amount of merges and helps avoiding big sudden changes to the code which could render it buggy. 

While a "small" size of code is something subjective the idea is clear and makes sense.

It is easy to keep track of small changes. A single bug fix or small enhancement and so on.

Although this is a good practice and will help you reduce the amount of merges, there are sometimes still situations where merges happen more than they should.

There could be another reason why you're merging more than you should, and this reason could be - a design "problem" in your code.

Such "problem" could be for example :

1. Classes that represent two (or more) logical units or well defined purpose.

Then it's clear that if 2 people (or more) would need to make some changes regarding different functionalities of this class, because this is the way work was distributed in their team, those people are very likely to face merging issues when they will try to check-in their code.

While if the class would have been splitted into 2 (or more) distinguished classes with clear purpose, then the 2 parties wouldn't need to face a merging issue.

Unless... (here comes #2)

2. Dependencies - sure almost everyone has dependencies and that is how things work. 

But what happens if you are starting to have a lot of dependencies? Then you're also likely to face a merge. 

Only this time you'll have to change your dependants' code (code who uses your class/interface as API) in order not to break compilation, and in those dependants is exactly where the merge lies. 

(Someone who just worked on that class earlier using your now 'old' API... and now you have to adapt their new code! - I'm not talking about changing your interface or API as part of a general change in the infrastructure of the system, that is obvious, and intended to affect client code. I'm talking about the times where it was not meant to be or shouldn't be due to bad design, wrong API usage and so on.)

Although adapting client code might sound less problematic, reducing those incidents as much as possible by reducing the amount of your dependencies, will help you avoid more merges. (Changer point of view).

Also as a client, It would be less likely that someone would come and change your client code, and adapt it in a way that you did not mean.

So if you're having lots of merges, try to understand why you have them before trying to resolve them. Sometimes it is very tempting to go on with the merge, as today there are good automatic tools to perform a merge. But, maybe you'll find something interesting, or come up with a way to refactor code that was taken for granted as a "law of nature". 

You'll eventually earn less merges, a more modular well defined code, and become less error prone, 

After all, who hasn't made a mistake once of accidentally dropping some code on a merge?

So not every merge is a reason to start wondering why this is happening, but it should make you at least aware of the situation and think whether it could be different.

Hope  this post wasn't stating the obvious.