Kettle data in a browser

Dear Kettle fans,

As you can tell from the Kettle JDBC driver project and also from the Talend job execution job entry (if you’re still wondering, that was NOT a joke) we announced a few weeks ago, we’re constantly looking for new and better ways to integrate Kettle into the wide world.

Today I’m blogging to spread the word about a new exciting possibility: expose Kettle data over a web service.

Here’s the situation: you have a 3rd party vendor that wants to read data from you.  However, you don’t want to spend a lot of time coding web services and what not to deliver the data in a language neutral format.  Anybody can read data from a simple web service.  You can use any programming language and it’s easy to test in a browser.

The way that it works arguably couldn’t be easier…  Let’s take a look.

We start with a simple transformation that reads some customer data.  We take the “Read customer data” example from the samples:

samples/transformations/CSV Input – Reading customer data.ktr

Next, we’re going to replace the dummy step with a “Text File Output” step (or “XML Output” if you prefer that):

Save the transformation in the same samples folder under the “Servlet Data Example.ktr” filename.  In my case the full filename is: (from the transformation settings dialog)

/home/matt/svn/kettle/trunk/samples/transformations/Servlet Data Example.ktr

Now you can use an instance of Carte that runs on your local host on port 8282 (execute carte.sh 127.0.0.1 8282) to get the data in your browser.  The URL is easy to construct.

For file (XML) based transformations:

http://username:password@hostname:port/kettle/executeTrans/?trans=PathToFile

For transformations stored in a repository:

http://username:password@hostname:port/kettle/executeTrans/?rep=RepositoryName&user=RepositoryUser&pass=RepositoryPassword&trans=PathToTransformationInRepository

Please note that you need to replace forward slashes with %2F in your browser.  That leads to this URL in our example:

http://cluster:cluster@127.0.0.1:8282/kettle/executeTrans/?trans=%2Fhome%2Fmatt%2Fsvn%2Fkettle%2Ftrunk%2Fsamples%2Ftransformations%2FServlet Data Example.ktr

The result is can be tested with your browser:

For the script kiddies among you it is possible to get a hold of the servlet print writer in JavaScript too:

var out = _step_.getTrans().getServletPrintWriter();
out.println(“<H1>Hello, world!\n</H1>”);

Well, there you have it. I hope you like this idea.  You can try it out yourself if you download a new build of Pentaho Data Integration from our Jenkins CI build server.

Now excuse me while I put the same button in the “JSON Output” step…

Until next time,

Matt

UPDATE Apr 27: “JSON Output” now also supports the new “Servlet output” option.  Furthermore it is now also possible to pass parameters and variables through the URL.  Simply add &PARAM=value and the appropriate parameter or variable in the transformation will be set prior to execution.

Executing Talend Jobs in Kettle

Dear Kettle friends,

I know a number of you have been asking for this feature to facilitate your migration projects for a while. However, today the “Talend Job Execution” job entry has finally arrived in Kettle!

Take a look at this screen shot demonstrating how easy it is to integrate:

Executing a Talend Job in Kettle: screenshot

The way that this works is by first doing an export of your job in Talend Open Studio:

Then you can configure your new job entry:

The name of the class can be derived from the generated code in TOS “Code” tab (behind your Designer view).

I’m looking forward to your (non-hostile, non-flamebait, non-trolling as always) comments!

Until next time,

Matt

P.S. You can see this new job entry yourself in the upcoming PDI 4.2 milestone release or download it now from CI.