Announcing Pentaho Data Integration version 3.0.0 GA

Dear Kettle friends,

After months upon months of hard work the Kettle community is proud to announce that version 3.0.0 of Pentaho Data Integration is generally available.
Here are the key changes in version 3.0 compared to version 2.5:

  • Separation of Data and metadata to reduce object creation and improve performance
  • Faster flat file support including parallel reading and lazy conversion
  • New look and feel and improved usability
  • 7 new Transformation Step types
  • 7 new Job Entry types
  • An integrated debugger
  • ….

For a more extensive list of all the changes, have a look at the change log.
After you upgraded to Java version 1.5 or later, you can grab the goodies for version 3 over here:

Also take a look at our updated homepage and the PDI launch site.

I would like to take the opportunity to pass a bigthank you to everyone that helped to make this release happen!

Go, go three dot oh!

😉

Matt

Spoon screen shot

Our little v3 launch party

Someone from the community pointed me to the new Llays Kettle chips and I actually did find them in the store, Jens Bleuel made a nice chocolate cake for the occasion and as you can see from the picture below, Rob Van Winden from company Red Dolphin brought his Ketel1 drink. (Alcoholic: the flammable kind)

Kettle 3 launch event goodies

The feedback was great for me, we should do it again.

Until next time,

Matt

The beauty of OSX

DISCLAIMER: if you’re not up for yet another piece of OSX fan-mail, you can safely skip this blog post!

OK, so with the help of a few Apple fans I finally managed to figure out how to type pipe (|) and tilde (~) symbols on my Mini-Mac.  After a while I even figured out that all the CTRL key-bindings in Eclipse where magically replaced by [Apple-Key] key-bindings and that word navigation using CTRL-LEFT/RIGHT is done using ALT-LEFT/RIGHT.

It was a start, but I didn’t stop there…  With bugs like PDI-406 open, there was a lot of work left to do.

Using an excellent packaging tool JarBundler we created a DMG image for our OSX users…   Launching Kettle and Spoon has never been easier on any platform.

The package management is certainly one of the better parts of OSX(**).  That being said, we’re just beginners here, so why don’t the OSX users amongst you try out this image over here:  Kettle-3.0.0.dmg (Java 1.5 required!) to see if it works to your satisfaction.  For the non-OSX users out there, the installation process is a double click on the “dmg” package filename and you’ll see a Kettle icon on which you can double click again to launch Spoon:

OSX dmg package

Interestingly, if you click right on the Kettle icon, there is an option called “Show package content”.  You can then navigate to Contents/Resources/Java.  You can copy this folder to any place you like and it will behave like the regular .zip package.  It will contain the documentation, samples, README files and the shell scripts.  That way you don’t have to download the same thing twice. (to run Pan, Kitchen and Carte for example)

So, let me know how you like the new and improved Pentaho Data Integration for OSX!  I’ll be in Madrid the next couple of days, but I’ll keep an eye out for the reactions.

Until next time,

Matt

(**) I still think that global software repositories like the ones used in Debian/Ubuntu are the way to go, but there is a certain elegance in the way OSX does it too.

The distaster called OSX

DISCLAIMER: If you’re not ready for yet another OSX rant, you can safely skip this blog post 🙂

OK so yesterday I did The Right Thing(tm) and I went out to buy an OSX box. I’m actually typing this blog post on it right now.

The goal of the purchase was to fix a number of long-standing GUI issues in Pentaho Data Integration. As I’m familiar with nearly all versions of Windows (except Vista) and Linux I was expecting some un-ease with the platform.

Boy, was I ever right about that. I guess if I would use it exclusively for the next couple of months, I would eventually get used to the quirks. However, as a Java development platform, it’s less than useless to me.

I did like the funky hardware. The (1.83Ghz/1GB RAM/80GB HD) mini fits nicely on any desktop as it is so small. It’s also virtually silent in operation. The keyboard is a bit heavy and a bit too rubbery for my taste, but I could get used to it.

The #1 problem on OSX is the horrible keyboard layout. When you develope code in Eclipse or when you write shell scripts, you want access to keys like these:

  • Home
  • End
  • Pipe (I would type a vertical line, but I can’t)
  • Tilde (I would type it, but I can’t)

Unfortunately, it is not possible to use these keys. They are simply not on my Belgian keyboard. (Note that it’s virtually impossible to find a store that sells a US keyboard over here). The pain doesn’t end there. The Terminal supplied with OSX doesn’t support anything but US7-ASCII. Any character out of the ordinary is converted to escape codes.

Can you imagine writing code without a pipe symbol? You need it in every other “if” statement and it’s used in SWT development a lot to combine options for widgets.

It gets worse: I found no solution. I found a few rants in the blogosphrere about people that had the same problems. I found vendors selling “improved” keyboards for OSX. I found plenty of reference pages at apple.com for all kinds of funky keyboard shortcuts. Tilde and Pipe are still eluding me though. I saw some proposals on various blogs: none of them worked. I even looked for some sort of Character Map tool like you have in Windows and Linux: nothing there. Then I tried to type all Alt-Gr+Key combinations on the keyboard. Nothing. Then all CTRL-AltGr+Key combinations, SHIFT-AltGR+Key,… you get the point. Still … absolutely… nothing. One has to wonder what kind of idiot would hide the pipe symbol under such a key combination, but hey, I could probably live with it over time.

As such fixing the issues by copy-pasting Pipe symbols from other places in the code, laborously manouvering through the code without home and end, took me hours instead of minutes.

Not all is bad though. Installing software on OSX was not as clean as it was on my Kubuntu machine, but I managed to install these in less than an hour:

  • Subversion
  • Eclipse
  • Ant
  • Firefox

However, let’s get one other myth out of the way here: in order to install those software packages, I **DID** most certainly use a terminal to copy ant and subversion stuff to the correct location. At least on Linux they moved beyond the copy-zip-file-contents-to-/usr/local/bin nonsense. I did have to change /etc/profile to include /usr/local/bin in the path too.

On the positive side, the Java JDK version 1.5 was installed by default on the box. That’s a good thing because I wouldn’t have known where to turn if it was missing or if it had 1.4 instead of the 1.5 version I need.

So then I tried to get my webcam to work on the Mini-Mac. Tried to see if the older Creative Labs NX Pro worked on Tiger. Well, fanboys, you can shove the “It just works” attitude as well since it doesn’t. No drivers available. Tried the open source “macam” project that supposidly has it working. Nothing. Zip. This was once the weak point of Linux too, however the NX Pro works without a problem on my Kubuntu box.

Chat client “iChat”: needs a €99/year .mac account it seems. The alternative, my AIM account, didn’t take. Can’t log in for some reason. Not even Microsoft is that stupid. WTF??

I actually had been looking forward to working with the Mac, but unil now it turned out to be one big disapointment. Pretty soon I’ll be looking for instructions to put Linux on the box. The problem is then that it would no longer serve its purpose as a testing box. Over time it will propbably suffer the same fate as my Windows machine by gathering dust 99% of the time.

Until next time,

Matt

PDI 3 RC2 on OSX

Ohloh top 10

People sometimes ask me if I still do a lot of development.

Well, Ohloh keeps track of that these days and it seems that between September and November 2007 I was the 7th most active contributor:

Ohloh top 10

Ohloh tracks 90655 developers in 8985 projects including Firefox, Apache HTTP server, Subversion, MySQL, PHP, Open Office, the Linux kernel, Ubuntu and many more.  As such, I’m kinda proud of that 7th spot.

If version 3 of Pentaho Data Integration has any bugs left when it launches, it won’t be because I was having a vacation 😉

Until next time,

Matt

PDI 3.0 : Second Release Candidate available

UPDATE: version 3.0 has been released

Dear friends,

Today it is with great pleasure that on behalf of the entire PDI team, I’m able to offer you RC2 of the long-awaited version 3 of Pentaho Data Integration.
Thanks to our enormous efforts we have been able to bring the number of bugs down to a minimum (See: http://jira.pentaho.org/browse/PDI).

Even though you should not use any development versions such as RC2 in production, we do hope you will continue to help us test PDI and that you file bug reports for any issue you might find.
Over 250 issues have already been resolved in the last couple of months and we went through and fixed all open issues with priority Blocker, Critical, Severe and most of the Medium priorities too. A number of those still have to be re-tested and closed, but it’s a good sign nevertheless.

The Spoon user guide documentation has also received an update and we are also continuing work on our version 3 migration guide.

Here are the zip files for Kettle 3.0.0 RC2 :

Please note that for version 3 you need version 1.5 (Java 5) or higher of a Java Runtime Environment in order to use our software.

Until next time,

Matt