Executing Talend Jobs in Kettle

Dear Kettle friends,

I know a number of you have been asking for this feature to facilitate your migration projects for a while. However, today the “Talend Job Execution” job entry has finally arrived in Kettle!

Take a look at this screen shot demonstrating how easy it is to integrate:

Executing a Talend Job in Kettle: screenshot

The way that this works is by first doing an export of your job in Talend Open Studio:

Then you can configure your new job entry:

The name of the class can be derived from the generated code in TOS “Code” tab (behind your Designer view).

I’m looking forward to your (non-hostile, non-flamebait, non-trolling as always) comments!

Until next time,

Matt

P.S. You can see this new job entry yourself in the upcoming PDI 4.2 milestone release or download it now from CI.

JSON / Portugal

Hello Kettle people!

It’s been a while since my last blog but I since there is always so much to tell you about Kettle, it’s high time to pick up the pace again.

This time I want to inform you that Kettle developer superstar Samatar informed me that he’s working on JSON steps as par of JIRA case PDI-3323 (input/output).

(click to see full size image)

Let’s hope the input step arrives in time to test Aaron’s (phytodata) REST/JSON web services on the BI server 3.7.

I’m also happy to inform you that Samatar will be joining us in Portugal for Pentaho’s yearly Community Meeting, September 25-26.  And, just to make sure you’ll recognize us there I’m posting a goofy picture of the both of us.

The picture was taken a few years ago when I visited his office in Paris.  I told him back then the picture would surface sooner of later 🙂

Until next time,

Matt

Book Review : Pentaho Reporting 3.5 for Java Developers

Hi Pentaho fans,

These are exciting times for Pentaho for sure.  These are also extremely busy times.  However, that doesn’t mean we can’t look around once in a while.  Today we’ll take a quick look at a new book that arrived on my doorstep a few weeks ago.  It’s titled

Pentaho Reporting 3.5 for Java Developers

I’m very pleased to be able to review this book as it is written by one of the smartest but more importantly also one of the nicest people at Pentaho: Will Gorman.  Not only that, he apparently had help from KC (Kurtis Cruzada) and Jem (Matzan) completing the dream team for this book.

And what a great book it turned out to be.  It covers pretty much everything from basic reporting, over mobile reporting, calculations and formula, sub-reporting, cross-tabs, charting down to the Java API.

Obviously, this book as been reviewed many times before by various people and websites. (Yes, it’s that popular)   To me that means that I can’t just do a quick review, I’m going to have to actually use and read the book.  And that’s what we’ll do today for this review.

We’re going to create a report in the form of a PDF.  The data for the report comes from a Kettle transformation.  We’re going to do it with my favorite programming language (Java) and a complete stack of Open Source Software…

I began by creating a new Eclipse project called KettleBook, download the source over here.
To make sure I didn’t miss any library dependencies, I used the complete “lib” folder of Pentaho Report Designer 3.5 as my class path. (not included in the download)

First, I went to Chapter 10 in the book and started reading the paragraph titled “Building a report using Pentaho Reporting’s API” as that seems to fit the bill. (page 266)

That part explains plain and simple how to create a new Master Report, how data sources work.  But wait, I don’t want a DefaultTableModel, I want to read from Kettle!  Well, a few page flips later we find ourselves on page 143 reading about the KettleDataFactory.  That got me quite far actually as the sample is quite descriptive.

So then I created a small transformation to read from a sample customer file using Pentaho Data Integration 3.2.  This is it:

It reads 100 rows of sample customer data, filters out the people from California, Florida and New York state.  That gives us 91 records.  We’re going to read from the RESULT step placeholder.

The part on page 147 I needed was this block:

KettleTransFromFileProducer producer = new KettleTransFromFileProducer("Customer data", transFile, stepName, "", "", new String[0], new ParameterMapping[0]);
KettleDataFactory factory = new KettleDataFactory();
factory.setQuery("default", producer);

This part describes a producer to the engine.

I then proceeded on page 269 and put a document header and footer on the report and an item band.  Then I put 4 columns on the page and the report was written.  This took me all of about 30 minutes. The nice folks at Pentaho Orlando will have to forgive me, reporting is not my specialty. Personally I was quite pleased that it was that easy to do.

So, with the report definition ready, I now wanted to create an actual PDF out of that.  More reading revealed that we needed a PDF Output processor (to generate the actual file) and a page-able report processor to paginate and process the report definition.  This is how it looks in my case:

  FileOutputStream fos = new FileOutputStream("files/output.pdf");
DefaultConfiguration configuration = new DefaultConfiguration();
PdfOutputProcessor processor = new PdfOutputProcessor(configuration, fos);
PageableReportProcessor reportProcessor = new PageableReportProcessor(report, processor);
reportProcessor.processReport();

5 lines of code to generate a PDF! Suffice it to say I was very happy.

In total I spent a little over an hour to produce this document:

It’s quite simple: if it weren’t for the book I would have a really hard time figuring out where to begin.  I probably would have had to talk to Thomas Morgner, the brain child of Pentaho Reporting.  A nice fellow as he is, communicating to him is not for the faint hearted. (Fortunately he recently moved to Ireland so things will get better soon)

All joking aside, if you are planning to create reports using the Java API, do yourself a favor and buy this book right away.  Even if you’re not going to use the API, Pentaho Reporting principles and concepts are explained in great detail.

Many thanks to Packt publishing for sending me the book to review and congratulations to Will Gorman and the reviewers for an excellent job.  Congratulations to Thomas and his community too for making Pentaho Reporting 3.5 a smash hit.

Until next time,
Matt

P.S. I’ll be obviously covering more of this Java API sample at the upcoming Devoxx conference in Antwerp.

Blogging on the HST

There has to be a first time for everything and for me this is the first blog post on the high speed train back home from Paris to Brussels.  OK, so the ping times to ibridge.be range from 180-800ms, but so far everything seems to work absolutely great.

All in all, I’m quite happy with the setup. Took a picture with my phone, sent it to the laptop using Bluetooth, then over WIFI to my website.  All traveling at +300kph (200mph) with the laptop plugged in.

The downside obviously is that you can work on the train this way 🙂

Until next time,
Matt

Meme(me)

I guess it’s a new type of chain-blogging like we used to have chain-mails.
Just passing along the instructions for other bloggers….

1. Take a picture of yourself right now.
2. Don’t change your clothes, don’t fix your hair…just take a picture.
3. Post that picture with NO editing.
4. Post these instructions with your picture.

It’s kinda fun.

Until the next time,
Matt

Black holes : ban all matches!

Because of the construction work on our house we had to move to a different place.  We have a gas driven stove to cook our daily food on.

Today, when I wanted to light a match to start the fire, I quickly stopped when the following thought occurred: what if lighting that match would create a black hole and suck up the complete earth?

OK, I hear you say, it happens many times all over the world, people light up matches all the time!  Yeah, but it would just be MY luck if a black hole happens to get created in MY match, right?

I mean, those things are highly explosive things, you never know what goes on in there?  Has this process been investigated long enough?  I certainly have no clue how a match ignites!

Personally, I think they should just switch to electronic lighters and ban all matches on the planet.  I’m sure most of you agree with me!

Until next time,

Matt