Pages

Showing posts with label Build. Show all posts
Showing posts with label Build. Show all posts

Saturday, January 25, 2014

Grunt - Create your custom local Grunt plugin

This post describes how you can write your own Grunt Plugin in Javascript and install it locally,
in order to use it on your own (Grunt) project build file.

If you don't know what Grunt is then to explain it on one leg -
"Grunt is a pluggable (JavaScript) task runner", which can be used to build JavaScript projects.

Such typical build of a Javascript project may contain a Linting task, minification task, and so on...

So, you can read more about Grunt if you want to get a better understanding.

And now, to the recipe!
Ingredients:
1 NPM installed on your machine
1 GIT installed on your machine
1 Skill writing a tiny bit of Javascript
A zest of patience

Instructions:
1. First thing would be to install a Grunt plugin that is used for templates scaffolding.
In order to do that, you need to run the following in your shell:
$ npm install -g grunt-init
2. Clone a GIT repository which is going to be used as your template for creating your glorious
Grunt Plugin. (https://github.com/gruntjs/grunt-init-gruntplugin)
So, run the following in your shell:
$ git clone https://github.com/gruntjs/grunt-init-gruntplugin.git ~/.grunt-init/gruntplugin
3. Create an empty directory where you want to place your Plugin like "my-custom-plugin", and cd into it.
$ mkdir my-custom-plugin
$ cd my-custom-plugin
Then run the following in your shell:
$ grunt-init gruntplugin
4. You should now be prompted with a serie of questions in the shell, in order to create your plugin.
You should be seeing something like this:

Please answer the following:

[?] Project name (grunt-my-custom-plugin)

[?] Description (The best Grunt plugin ever.) the best custom plugin ever!

[?] Version (0.1.0)

[?] Project git repository (ssh://<Your GIT remote Repository URL>)

[?] Project homepage (none)

[?] Project issues tracker (none)

[?] Licenses (MIT)

[?] Author name (nirgit) Nir M

[?] Author email (getnirm@gmail.com)

[?] Author url (none) tech-drum.blogspot.com

[?] What versions of grunt does it require? (~0.4.2)

[?] What versions of node does it run on? (>= 0.8.0)

[?] Do you need to make any changes to the above before continuing? (y/N) N



Writing .gitignore...OK

Writing .jshintrc...OK

Writing Gruntfile.js...OK

Writing README.md...OK

Writing tasks/my_custom_plugin.js...OK

Writing test/expected/custom_options...OK

Writing test/expected/default_options...OK

Writing test/fixtures/123...OK

Writing test/fixtures/testing...OK

Writing test/my_custom_plugin_test.js...OK

Writing LICENSE-MIT...OK

Writing package.json...OK



Initialized from template "gruntplugin".

You should now install project dependencies with npm install. After that, you

may execute project tasks with grunt. For more information about installing

and configuring Grunt, please see the Getting Started guide:



http://gruntjs.com/getting-started



Done, without errors.

5. Now you should have your directory containing a Grunt Plugin template.
Open the file you have inside the "tasks" directory in order to edit it.

6. The file should look as follows:


7. Just in order to get started, lets strip the file and change it to contain the following:
/*
 * grunt-my-custom-plugin
 * 
 *
 * Copyright (c) 2014 Nir M
 * Licensed under the MIT license.
 */

'use strict';

module.exports = function(grunt) {

 grunt.file.defaultEncoding = 'utf8';

 // Please see the Grunt documentation for more information regarding task
 // creation: http://gruntjs.com/creating-tasks

 grunt.registerTask('my_custom_plugin', function() {
  grunt.log.writeln('Hello my great Grunt Plugin!') ;
 });
};

The my_custom_plugin shown in bold above, is the name of your task in the Grunt build, that
you'll execute at the end.

Lets change the Gruntfile.js in the root directory of the Plugin so tests won't execute when
we will run "grunt" on the plugin's source. Otherwise the build would fail.
So now the Gruntfile.js last line should look like:
grunt.registerTask('default', ['jshint']);

8. Go back to the root directory of your Grunt Plugin and run the following:
$ npm install
$ grunt
Running "jshint:all" (jshint) task >> 3 files lint free. Done, without errors.
9. Almost done.
Now go to your own project where you want to use your plugin and add the following line to the
Gruntfile.js of your project:
grunt.loadNpmTasks('grunt-my-custom-plugin');
This will load your Grunt plugin task to your Grunt's project.

10. Next step would be to install the plugin locally in the node modules of your Proect.
You should now run in the root directory of your Project (where you want to use your plugin) the following:
$ npm install /<PATH_TO_YOUR_PLUGIN>/my-custom-plugin
11. You should now be able to execute:
$ grunt my_custom_plugin
And see:
Running "my_custom_plugin" task

Hello my great Grunt Plugin!



Done, without errors.



That's all folks.
Hope this was helpful to get you started with Grunt Plugins.
Don't forget to publish it (npm publish) afterwards to the rest of the Node community if you think it may be helpful to others as well.

Check out the official guide for creating Grunt plugins at:
http://gruntjs.com/creating-plugins

Good luck!





Wednesday, January 30, 2013

RPM example - Application packaging

This post is about something quite different than the others.
It concerns packaging your software as an RPM file making it ready for delivery.
If you have never used RPM before and/or you're not an experienced Linux user, this post is intended for you.

I'm far from being an expert in this area, and I would guess there is probably a better way of achieving the goal I want to show here. Please share your thoughts and experience if you have any.

I assume you already heard of RPM, and understand it's benefits and so on.
This post is not to convince you to use RPM, only how to create such RPM file which you could use.

What's in this example?
I created a "Hello World" Java application, and using Ant, I compile it, and Jar it.
This application could be anything you want - this is just a "place-holder" for you, to replace with
your real stuff.

So, after having an RPM file, we'll be able to install it to the system and remove it afterwards.
The path which the program will be installed into will contain the "Hello World" application Jar.

RPM Structure quick overview

As you probably know by now, RPM files are created using a ".spec" file.
This file tells the rpmbuild tool which we will use how to create the RPM package.
The .spec file contains several sections:

  1. Headers - Headers of the spec file contain stuff like a Summary of what the package is,
    Name of the package, Version & Release information, License & Package and an
    attribute named BuildRoot. BuildRoot is important - it specifies the location
    of where your package will be temporarily installed.
    What I mean by temporarily, we'll get to later.
  2. %description step - This is where you can type just about anything you want describing
    your awesome application.
  3. %prep step - In the step, preparing your Source files to be built (using Ant in our example).
    "What's to prepare?" - you ask? - Well, in order to build an RPM package, you must provide
    RPM a .tar file or a zipped file of some kind to it's SOURCES directory, (I'll explain the
    structure of RPM further down), so what you need to do in this step is to extract your
    archived sources.
  4. %build step - This is where the actual build of your application happens. In our case, we'll
    execute ant on our build.xml file, that compiles and Jars the application - real simple.
  5. %install step - This is the step that happens after the %build step, and will determine how
    your installed package will look like. We will copy the necessary binaries (Jar in our case) to a relevant directory.
  6. %files section - This section determines which files from the %install step to pack, and with what access rights.
  7. %clean section - This section runs after packaging was done, and cleans up all the leftovers from your build and your install steps. It's important to delete all the old files, especially if you build several RPM packages, you don't want to have files of another RPM build process in your way.

Okay. I know this still doesn't tell you how to do stuff, but this introduction is quite important in my opinion.
So take a deep breathe one last time, and we'll go over the rpmbuild directory structure and understand the steps we need to take in order to accomplish our mission.

"rpmbuild" Directory structure

An important part of the way RPM works, is its directory structure where work is being done.
You should be able to locate your rpmbuild directory on your machine, which usually should
show under your home directory.

The rpmbuild directory contains the following sub-directories:
  1. BUILD - This is the directory where your sources will be built, and where the artifacts of your build will be located. The %build step, reads from the directory and writes to it. 
  2. BUILDROOT - This is the directory where you will "temporarily" install your binaries to.
    You can think of it as the "INSTALL" directory. After the build step, the BUILDROOT
    serves as a directory to contain the files in a given directory structure. Those files in this
    specified directory structure will be installed afterwards using RPM.
    To clarify: If you place under BUILDROOT a directory called my-app-123 then on the
    real install of the RPM package, your package will be installed under /my-app-123.
    This step reads from the BUILD directory and writes to the BUILDROOT directory.
  3. RPMS - This is the directory that will contain your RPM package in the end.
  4. SOURCES - This is the directory where you should place your archived sources.
  5. SPECS - This is the directory where you should place your .spec file which will create
    the RPM package.
  6. SRPMS - This is a directory where source RPMs are created and stored. 

Each of those directories, can be referenced in the .spec file using RPM variables.
Variables
%_specdir~/rpmbuild/SPECS
%_sourcedir~/rpmbuild/SOURCES
%_builddir~/rpmbuild/BUILD
%_buildrootdir~/rpmbuild/BUILDROOT
%_rpmdir~/rpmbuild/RPMS
%_srcrpmdir~/rpmbuild/SRPMS

Now that we finished our overview. Let's go over our 8 steps to create a packaged application with RPM.
  1. Create an application (write source code) - in our case, a simple "Hello World" application.
  2. Create the build file for the application - Ant's build.xml file in our case.
  3. Create an archive from our application's sources.
  4. Place the archive (tar) at ~/rpmbuild/SOURCES directory.
  5. Copy the .spec file into the ~/rpmbuild/SPECS directory.
  6. Create the RPM package using: rpmbuild -ba <spec_file>
  7. Install the package.
  8. Uninstall the package.
Now, lets go a bit into details:
  1. You can download the pre-made application entirely from GitHub: rpm-example-project.zip
    in order to skip steps 1,2 and 3.
    (You can also access the repository - http://github.com/nirgit/RPM-Project-Example).
  2. After downloading the zipped application, and extracting it somewhere on your machine,
    you can take a closer look into its hierarchy. You will find under the Application directory,
    a directory called src which contains the sources of our application, and you will find
    the build.xml file which builds the application.
    You will also find 2 other files - the application.tar containing the application sources,
    and the project.spec file.
  3. You need now to copy the .spec file into the ~/rpmbuild/SPECS directory, and the
    application.tar file into the ~/rpmbuild/SOURCES directory.
  4. In order to create the package now, you're required to run the rpmbuild tool, by
    executing: > rpmbuild -ba ~/rpmbuild/SPECS/project.spec
  5. Since I use Ubuntu, I also use alien to install the package. In case you're running on
    a Linux flavor such as CentOS, I think you can simply run the regular RPM install.
    With alien: > sudo alien -i ~/rpmbuild/RPMS/i386/Example-RPM-Project-1.0-1.i386.rpm
    Without: > rpm -i ~/rpmbuild/RPMS/i386/Example-RPM-Project-1.0-1.i386.rpm
  6. After the install you should be able to find the package installed under the root:
    /Example-RPM-Project-1.0-1
    Under it, you can find the app.jar.
  7. You can uninstall the package using RPM's command: rpm -e <package_name> 
    or (in my case using Ubuntu): sudo dpkg --remove example-rpm-project

The SPEC file
This is the SPEC file associated with the RPM package.
I hope it will serve you as a good starting point for your own SPEC file.


################################################################
#
# This is an example of a simple RPM spec file.
#
################################################################

Summary: An RPM Spec example
Name: Example-RPM-Project
Version: 1.0
Release: 1
License: Apache 2.0
Group: Applications/Sample
URL: http://www.mycompany.com
Packager: Nir Moav <getnirm@gmail.com>
BuildRoot: %{_buildrootdir}/%{name}-%{version}-%{release}

%description
This is a sample SPEC file for the RPM project
demonstrating how to build, package, install(deploy)


%prep
# extract the tar file containing all the sources, to the build directory.
tar -xvf %{_sourcedir}/*.tar -C %{_builddir}


%build
echo "Building the project..."
cd Application
# running ant to build the java project (could be make/maven/gradle or anything else).
ant


%install
# This is the hierarchy which is going to be inside the package (RPM/Deb) eventually.
echo "Install phase..."
mkdir -p %{buildroot}/%{name}-%{version}-%{release}
cp -R %{_builddir}/Application/output/jars/* %{buildroot}/%{name}-%{version}-%{release}


%post
#This runs post the install - maybe you want to execute the application already then
echo "Post install.."


%postun
#this runs after the uninstall
echo "Post Uninstall..."


%files
# tells which files to contain in the package and with what access rights
# the triplet contains of (<file mode>, <user>, <group>). Make the necessary changes.
%defattr(-,nir,nir)
/*


%clean
# Clean up! Must run this! after build and install steps execute, this will make
# sure that the directories are remained clean, so in case you're building another
# package, you don't want to pack the previous build's artifacts.
rm -rf %{_builddir}/*
rm -rf %{_buildrootdir}/*



That's all folks!
I hope you managed to read through this long post, and found it useful.

Enjoy,
Nir

Sunday, November 11, 2012

Gradle - Opinion, experience and examples

Opinion and Experience

I had quite a short experience with Gradle, but the impression it left is of an incredible tool.

I don't want to diss any other build tools out there like Ant or Maven, they're great,
and if you're a real savvy, you'll get the job done probably pretty fast.
I generally believe that one needs to use the right tool for the right task, but I'd dare say
that whatever you're doing with Maven or Ant you could probably do better with Gradle -
it is simpler, faster, shorter to write any build file, much more concise yet very powerful.

I have also my experience with Ant for years and with Maven, and I can tell you that I appreciate
Ant very much as a build tool. It does exactly what you expect it to do.
When you read a build.xml file, you'd hardly find something you don't understand.
It's just clear how things work, and what is expected at the end.
The problem is, you need to write so much "build code" in order to get your job done, especially
in a complex system, when you have includes of other extension.xml files you rely on and so on.
What about dependency resolution? - OK, so you could use Ivy, but that's just for dependency resolution.
It's not a "full" build tool in the sense of Maven that comes with creating your packages out of the box, deploying it to your tomcat and distributing it to your Repository.

Yes, Maven can be great but I must say that my 'like' metric towards it is not so stable.
It is too complicated to get simple stuff done... compiling GWT? - Get a plugin!
In Ant, it's really simple to compile a GWT project + you get the example from the GWT website.
In Gradle you can simply execute Ant Tasks, so no plugin extra work needed.
Want to include / exclude resources from your WAR file... why does it have to be complicated?
Dependency management and resolution is awesome, distributing your artifacts into a repository
is great, and fairly easy to do.

However, despite the good stuff of these build tools, I must say that from my personal experience  Gradle tops them.
Gradle like many other posts on the internet would say - brings all the good stuff from the worlds
of Ant and Maven while keeping the ugly stuff out.

What do I mean by that?

  1. It is the easiest build tool I've seen to get started with - simply download, extract and set the GRADLE_HOME environment variable on your path.
    Like Ant - no other configuration is needed! Sweet.
  2. Very well documented!
  3. You can execute any Ant task from the Gradle build files. Any.
  4. Dependency resolution - from Maven repositories, it leverages the existing repositories you may already have in your corporation without needing to change anything, while decreasing the verbosity of dependency declarations in the build file itself.
  5. Gradle is based on Groovy, so you can just program your build.
    And you can use all the libraries of Groovy. And it runs on the JVM, so you could theoretically use any Java library you like when running the build.
    Did I already say powerful?
  6. It is dynamic - You can tell Gradle in run-time for example, which sub-projects you'd like to include in your Project's build. I did that, using naming conventions for instance when I had a process that was auto-generating entire projects ready to be built in a pre-compilation phase. Incredible!
  7. No need to sit and rewrite ant targets to do jobs like clean, compile, jar, war etc... it is all there already!
  8. The integration with Eclipse is excellent. All you need is there.
  9. It just works!



Let me give you an example of how a simple Gradle build (build.gradle) file looks like:

apply plugin: 'java'
apply plugin: 'eclipse'
sourceCompatibility = 1.6
version = '1.0'
jar {
    manifest {
        attributes 'Implementation-Title': 'My project', 'Implementation-Version': version
    }
}
repositories {
    mavenCentral()
}
dependencies {
    compile group: 'commons-collections', name: 'commons-collections', version: '3.2'
    testCompile group: 'junit', name: 'junit', version: '4.+'
}



I think that what is so nice about this build file, is that you can understand it right away.
Lets go over the lines to try and understand this file:


  1. The first two lines declaring the "apply plugin...." means that you are telling Gradle to use plugins such as 'java' for example to it would know how to compile Java source to classes.
    Gradle is based on plugins, which facilitate all the needed 'know-how' and logic of performing
    various tasks. - You could write your own custom plugins as well.
  2. The 'eclipse' plugin for example generates the needed files for importing your Gradle project into Eclipse.
    You can read more on that on - The Eclipse Plugin.
  3. sourceCompatibility - tells Gradle what version of Java your source files are written in.
    You can also specify the 'targetCompatibility' attribute in order to compile your sources into a lower version of Java.
  4. version - Declares a variable called 'version' with the version number of your package
    (Be it a JAR or WAR file). The Jar files that are generated with the Gradle 'build' task, have a Maven convention. So the version number is appended to the name of the package at the end. (for example: "MyPackage-1.2.jar", in case version was given the value "1.2").
  5. jar { ... } - This is a Gradle Task is a "task" (can be thought of an Ant task) which is part of the 'java' plugin that assembles a Jar file from your compiled code.
    You can add to the manifest file your own attributes, such as a version number.
    Note that  'Implementation-Version': version is using the version number declared earlier in paragraph 3, but you could have used any other variable to place your value in there.
  6. repositories { mavenCentral() } - determines the repositories from which you want to perform dependency resolution.
    In this case - using the Maven Central repository.
  7. dependencies { ... } - determines the needed dependencies for your build.
    Like Maven's dependency scope, in Gradle there are also so called 'scopes' which you declare for your dependencies: compile, runtime, testCompile & testRuntime.In this case we declare for example the following:
    testCompile group: 'junit', name: 'junit', version: '4.+'Which makes sure to use the JUnit version 4 and above during compilation of test files.


That's it.
It's really short and simple!
Now you can just run from your command line : > gradle clean build
And you will get your project cleaned, compiled, packaged ("Jarred"), tested and with a JUnit test report, all under the 'build' directory of your project.
Of course, you can also distribute the artifacts (your Jars) to your repository, or just copy them
flat to an "Artifacts" directory of your project and so on.

But this was only to demonstrate how easy it is to create a build file, that does so much.

Examples

Running an Ant Task from Gradle build file:

You can run an Ant tasks from a Gradle build file simply by writing something like this:
ant.echo(message: "Building a project....")

This task in particular is not that useful... it's just a print, but it demonstrates how easy it is to execute Ant tasks!
Especially if you consider migrating from Ant, and you have custom Ant Tasks written
which you want to re-use - then you could easily do this.



Running an external Java program from an external Jar:

This could be handy if you want to run anything which is for example at a pre/post phase
of the build.
The snippet below shows how you would apply an XSLT on a XML file, using
the xalan library:
ant.java(jar: "lib/xalan.jar", fork: "true") {
arg(line: "-xml" + " abc.xml")
arg(line: "-xsl" + " myXslt.xsl")
}

(the "lib" directory is at the project's root directory as were the "abc.xml" and "myXslt.xsl" files).


Including Sub-Projects in your Project's build dynamically:

When your project is comprised from sub-projects, it is necessary to tell Gradle about
those sub-projects in a dedicated file called settings.gradle.
In that file, one needs to "include" the sub-projects that need to be built by Gradle.
So typically you could see something like this:
include 'server', 'client'
The great thing about Gradle is that in the settings file you can pretty much put just about anything
you put in a build file. The settings file is executed in the initialization phase of the build.
So if you want to dynamically include sub-projects into your build because you're generating them
from another process, or because you have many of them, or for any other reason, you can do
something like this:


new File(".").eachFile(groovy.io.FileType.DIRECTORIES) { f ->    if(f.name.startsWith("subProjectPrefix") {                
                            include f         
                 }
          }


This will just iterate over the root directory of your project, check if the current directory
it iterates on starts with the "subProjectPrefix" prefix, and includes it if it does.


This was really just the tip of what Gradle can offer you.
You can find many examples on the internet as well as on Gradle's installation directory.
(Look under %GRADLE_HOME%/samples - there are many build files examples).
So if you're using Ant, Ivy or Maven I recommend you to look into it, and give it a try,
I don't think you will regret it.


Gradle - http://www.gradle.org/