IBM developerWorks Interview on Open Source Business Intelligence
A few weeks ago, I had the very enjoyable experience of chatting with my friend, Andy Glover, on the phone for a little while. We talked about open source business intelligence, including a few popular tools and a few key concepts that are well known to database professionals but not so well known to developers. Or as Andy put it, “Tools techniques, and sardonic wit.” That’s me! Give it a listen today:
http://www.ibm.com/developerworks/java/library/j-gloverpodcast/
Matt Casters Said,
September 6, 2010 @ 1:24 pm
Tim, nice job on the developerWorks Interview.
It gave me a nice insight into the viewpoint of the developer on business intelligence. It nicely contrasts with the viewpoint of the typical data warehousing consultant that works with the business users to deliver a BI solution. In that situation you want your solution to live as close to the business requirements as possible.
From the technical Java/Eclipse developer viewpoint you probably have little need to use metadata layers and so on, or so it seems at least. Perhaps that explains the absence of those in the developer oriented software stacks you mentioned.
Just thinking out loud, keep up the good work!
Regards,
Matt — Pentaho/Kettle
tlberglund Said,
September 6, 2010 @ 3:12 pm
Thanks for you comment, Matt. Whether the stacks are really developer oriented or merely my perspective is, it’s definitely the case that you could employ a proper metadata layer in Talend, as I’m sure you could in Kettle. The fact that I de-emphasize that architecture is probably not to my credit.
I’ve been thinking about reworking my No Fluff Just Stuff BI workshop with some Kettle examples. From what I’ve seen, it would probably be easier to transfer Kettle jobs between users than it is with Talend jobs, the latter being Eclipse-based.
Matt Casters Said,
September 7, 2010 @ 2:43 pm
I’m sorry I wasn’t clear Tim. I wasn’t talking about ETL metadata but about the lack of real, complexity hiding, metadata layers in the reporting tool stack. As far as I can tell, the only one available in open source comes from Pentaho. This has lead to interesting and strange developments like this plugin for BIRT : http://www.osbi.fr/?p=1697
As far as the popularity of the open source stacks is concerned (something you tried to get a feel for) there has been a study that was done by Mark Madsen and the BeyeNetwork at the end of last year: http://www.beyeresearch.com/study/12261 This study ranks Pentaho as the dominant player, even in ETL.
Now, what I found interesting with your remarks was that for *developers*, this picture might be completely different. My stance on this has always been that the domain is called “Business Intelligence”, not “Developer Intelligence” hence my efforts in the past to try and remove the need for development and code from Kettle as much as possible. From my background as a BI consultant I guess that made sense. However, in the recent past we at Pentaho have come to realize that this developer community is just as important to us. As such, for the past year we have been delivering steps like the “User Defined Java Class” step, a new plugin architecture and even a metadata injection system ( ) to make it easier for folks to embed and extend Kettle in all sorts of complex situations. Our LGPL license makes this extra interesting. We even devoted a complete chapter on the subject in our upcoming Pentaho Kettle Solutions book (Wiley at the end of the month, the 2nd book released on the subject of Kettle this year)
Anyway, I hope that my long soapbox rant explains my previous post a bit. Please see my ramblings as an invitation to you or any other developer to let us know what we can do for them to make things easier and better.
Regards,
Matt