Data Management & Decision Support

War

Check this out - http://mshiltonj.com/software_wars/current/

Tip by Marc van Waardenburg...

High performance database processing

Just a heads up here.

At the moment I have drowned myself into the area of high performance database processing. So many developments have urged me to get even more in depth knowledge of this field of expertise. So what am I talking about:

- column oriented databases vs. row oriented databases - new players on the market (e.g. ParAccel - look at this one guys - they seem to do really well in high speed data warehousing)

- Data partition in all it's forms...

- shared nothing, shared memory, shared disk architecture, scale up, scale out

- TPC-H results

- the rise of different types of appliances

It's like I always said....the average Data warehouse architect (and I off course I don't mean you!) does not seem to care for DBMS infrastructure and it's underlying hardware architecture. They all shout that performance is vital to a succesfull DWH but yet they do not seem to incorporate the DBMS infrastructure and the hardware architecture into their overall DWH architecture.

When data is delivered in actual projects I already hear the bells; "why is it not performing?" I got great ETL, I got a hugely fine data model, what's wrong? Duh..u forgot to carefully decide on the DBMS infrastructure and hardware architecture. We (DWH arhitects) can not afford to not know your potatoes in this field of expertise. Data warehousing results in fundamentally other requirements for your DB infra and hardware architecture. You can not leave this to Database and hardware folks that only know the OLTP world.

So fellow architects - if you do not have a least a basic understanding of DBMS infrastructure and hardware architecture specificcaly for data warehousing - well...simple put and quoting a fellow expert; YOUR CLIENT IS LOOSING MONEY!

I will be posting several subjects on this matter in the coming months.

Application integration or data integration....

There is no such thing as coincidence.....today I worked on new material for the masterclass Data Warehousing in Depth for DNV/CIBIT and I worked out this data-centric case (I was triggered by Donald Feinberg of Gartner by the way; 'operational databases will eventually die, the data warehouse will take over') with reference to SOA and it's impact on data warehousing.

Dan Linstedt wrote today a great blog entry about such a future. In the root of this blog entry lies a very fierce discussion between federated and data centric approaches.

http://www.b-eye-network.com/blogs/linstedt/archives/2008/02/the_new_evoluti.php

I responded to this entry allready. It's really mindboggling if u think about the consequences....

Really guys...this is great stuff.

Mega Vendor: BO unveils XI 3.0

Business Objects just unveiled their 3.0 release. I have read throught the changes and it does seem like good new release. A redesigned CMC (oh my god - I hope it's workable now. Long overdue this one), the web intelligence client approaches it's full client even closer now, the personal data provider (I like that one very much), hyperlink capabilities made easier (any BO expert know that this was a major pain!! - hope they fixed this), smart cubes (nice functionality! - again...overdue..but hey...who complaining), Native printing (omg - YESSSS, not via pdf...hooray), xcelcius stuff (off course) and widgets (which seem very cool!!), a new and improved scheduling and publishing funcitonality (again - long overdue - hope it works now)....and much more. So, quite a few improvements which seem to be worth it a lot!

But...

What about SAP?

Well; they are writing some parapraphs about this XI3.0 with 'great' BW intergration:

- Managing SAP BW universes (pff...nothing new - they had this functionality for years - think they bought it from a french company - not sure though).

- Extend support for BW metadata (I read this one..over and over again..don't get it....)

- mass publication of personalized SAP reports (again - don't understand this one - i must be stupid)

- improved SAP portal integration (not sure what's new about this - didn't XI2.0 already offer this?)

So...XI3.0 sounds like a major new release for BO customers. That's great and a bit overdue - but hey, who's complaining. The first signs of SAP BW integraion are emerging.

SAP customers will benefit from having a decent BI tool on BW....but how are BO customers benefit? I don't see it. I am even a bit worried for these customer....

Mega vendors: IBM and Cognos reveil....

Johan pointed me to the strategy of yet another mega vendor; IBM and Cognos

http://www.ciol.com/EC/News-Reports/IBM-unveils-Cognos-Strategy/8208103531/0/

Let's analyze this one a bit:

1. 'The new IBM offerings include 10 industry solutions that help customers turn information into a strategic asset.  These solutions include an expansion of the IBM Retail Integration Framework to incorporate Store Operations and Planning blueprints from Cognos that have been pre-integrated with IBM retail data models.'

Ronald: To be honost and fair - it seems that some actual integration work has been done. But I am very carefull here. Why am I carefull? Well upon reading this over and over again I can't understand what the actual integration would be? I mean planning blueprints from Cognos? The planning module was already an acquired application from Cognos and very poorly integrated into the Cognos suite.  So IBM bought a planning app that Cognos also bought (Adaytum)....hmm...sounds like very nice integration to me. The press release mentioned10 industry solutions, but only the retail framework is highlighted, that's rather strange? Is the banking model also included in here? But the banking model is just a model...nothing more. How do u integrate that with Cognos? Maybe deliver pre-made packages? That would be a hell of a job - and sorry...I don't believe it.

In summary; a lot of questions...time will tell. But it would be an interesting strategy to sell vertical-branche-specific stacks!

2. Cognos 8 BI "starter pack" for IBM InfoSphere Warehouse and distribution of InfoSphere Warehouse with Cognos 8 BI solutions, making it easier for clients to adopt complete business intelligence and data warehousing solutions that quickly generate a positive return on investment.

Ronald: I am sorry...i am still completely in the dark of these new marketing names IBM found for their software and hardware pertaining to data warehousing and Business Intelligence. 'IBM infosphe Warehouse'...what the hell is that? Tried to look it up on their site....could not found much. I need to invest more time in this (...)

3.Pre-integration of Cognos 8 with the industry-first IBM Information Server software platform to deliver seamless data integration and quality, and metadata integration to provide data lineage to track the basis for business decisions.

Ronald: Ok....another product; IBM Information Server. I am going to guess this is the old Ascential stuff. In my view, this is like I already said in my previous blogs on mega vendors....just a sales package where IBM and Cognos package their software. And right they are!!! There is not much overlap, so I can dig this a lot. I got offcourse a few questions. Like, how is the metadata from Ascential stuff and Cognos stuff integrated (imported, bridged, etc..)? I assume we still need Transformer to make cubes (would like to see a strategy where they adapt Ascential ETL to build PowerPlay cubes)?

4.Pre-configured templates for integrating Cognos 8 BI with IBM FileNet Business Process Management (BPM) software, enabling customers to use Cognos 8 BI reports and analytics to support critical business decisions.

Ronald: Could be some integration work or just some utalisation of previous work Cognos and IBM did with other customer (they got some history together). Still in the dark what the integration part is though. Packages? pre-build powerplay cubes..? Need more information on this one...

5.Pre-integration and bundling of an IBM Dashboard Accelerator "starter kit" with Cognos 8 BI for rapidly building portal-based, actionable dashboards that enable users to take action and initiate operational transactions based on reviews of key business metrics.  The Dashboard Accelerator will also now include the Cognos 8 BI "starter pack" along with a new Cognos builder, which will make it easy for users to create portal and dashboard components that include Cognos reports or analytics.

Ronald: Sheer brillance...a package where webspher products and Cognos products are sold. And again - right they are!! Not very much overlap I think. Very good marketing strategy.

6.In addition, IBM Cognos 8 BI software will be included in C-Class Balanced Warehouse offerings, which provide pre-tested, scalable and fully-integrated system components of IBM data warehouse software, server and storage technologies.  The C-Class Balanced Warehouses offer small and medium-sized businesses a simple, complete solution for integrating and maintaining multiple data and information resources in a scalable and versatile data warehouse

Ronald: and...as expected. They combined Cognos in the above with the IBM websphere stuff, the Ascential stuff, the industry solutions...1 was missing (or is it part of bullet 2??? - help pls)...IBM's data warehouse appliance; the balanced warehouse. They now sell cognos in the same package. Just wondering whether u get a pre-configured, pre-installed, pre-balanced Cognos environment. Again, lacking information here quite a bit.

Overall; IBM and Cognos do a fairly neat job with this strategy. Although I got quite a few questions, the strategy is really to combine IBM hardware/software/solutions with Cognos in various packages. The direction of IBM/Cognos is quite clear if u compare it to the other mega's. IBM did a perfect job in acquiring Cognos, not much overlap with existing IBM products. I especially like the verticals combined with Cognos software. It's a potential sales killer! I once said that unlike Oracle, SAP and Microsoft, IBM has got no business content (SAP R/3, Microsoft Dynamics, Oracle EBS/Siebel/JDedwards). That offcourse is true, but with combining IBM's vertical Industry solutions with Cognos, they sure compensate for it.

Thx Johan for pointing me to this press-release. I need to get deeper into the IBM - freshly renamed - product suite for data warehousing and Business Intelligence. I am very much confused with these new names. And since IBM got 350.000 employees...it will take some time for IBM peeps to get to know these new terms.

Now..did u guys see anything about 'dynamic data warehousing' in this press release? Or did they rename it?

Mega Vendors: SAP and Business Objects Unveil First Joint Offerings

In one of my previous blog on mega-vendors I highlighted the huge integration-work these mega-vendors need to do. SAP/BO unveiled their first joint offering: http://www.businessobjects.com/news/press_release.asp?id=20080116_005992

In my blog about the mega-vendors I warned for:

'The first thing these vendors do is trying  to market packages (it is the fastest way of creating cashflow)...lock u in....do not fall into tis trap. U will do the integration work for them!'

I am so sorry to be right again.....It seems that this first offering of SAP/BO is nothing more than a bunch of sales packages. I have yet to see any technical integration or any real vision of SAP and BO on their overlapping functionality or any clear direction of this new company to the BO-non SAP-customers.

I am still very curious......

"BI 2.0......marketing bs"

On the Gartner Summit I finally heard something from Gartner I agree with........

Gartner BI Summit: mega vendors - selling BI stacks

Gartner is pushing hard to let us convince that 4 mega vendors will (and maybe are) rule the BI-space:

More application oriented vendors:

- SAP (with Business Objects)

- Oracle (ofc. with Siebel, EBS, JDedwards - Hyperion, Siebel suite, Warehouse builder, database)

More infrastructure oriented:

- IBM (especially with Cognos)

- Microsoft

I am amazed by the conclusion that these vendors will rule the game. Almost all vendors (maybe microsoft is the exception) have huge integration challenges. If u do business with one vendor....u will not buy an integrated stack. Even worse, u will buy a huge load of uncertainty as in the future of your freshly bought software.

The first thing these vendors do is trying  to market packages (it is the fastest way of creating cashflow)...lock u in....do not fall into tis trap. U will do the integration work for them! And what about all those overlapping software tools....IBM....what are u going to do with Applix - let it die? Or how are u going to load powerplay cubes - with Ascential would be a nice integrated step...? Oracle....u got 2 MOLAP engines (Hyperion Essbase and the Express stuff)..how?? and what?? Is your ETL going to load this MOLAP cubes? Both? What about those analytical application on Siebel? They are loaded with Informatica....Is Oracle selling Informatica licenses now? How about the integrated metadata (question to all mega vendors)........And SAP, are they offering data warehousing and BI to non-SAP-companies now? I don't think so.....How about Data Integrator (ETL of BO) and SAP BW??? two tools...same functionality? Or do I now need to buy two tools; BW for SAP, BODI for non-SAP?? How about the metadata (again)? Planning software? BO acquired it (i have not seen much integrating efforts there)....but SAP has got planning modules....and BW content on top of it.....what about these 2? I can continue the list of integration-questions for a very long time....but this is a blog....i need to be short....

Loads of questions and hazards.....If I was a customer (and I am not) I would seriously look for the more or less independent vendors (SAS, Microstrategy, Informatica, Information Builders..).....They seem (unless they get 'eaten' as well) to offer a more secure and solid future....where the 'mega' vendor's offer a lot of uncertainty.

But what if these more independent vendor's get eaten by the mega-four (or 5 - HP???)? Well, lets just all go open source then....and wait for the mega's to integrate their software and deliver on the promise.

Again; i wanna exclude Microsoft in this conclusion. They tend to buy a lot smarter.They tend to buy innovative new technology and integrate it steadily into their suite. Most of the stuff however is integrated by design (where the mega's need to integrate by aquisition). I am very much impressed by their BI strategy and their stack of software that includes SQL server (IS, AS, RS), Sharepoint, Office, Performance Point Server. Very impressive stack!!

Gartner BI summit: IBM's dynamic datawarehouse

Ok...I am attending the Gartner BI summit in Amsterdam from 5-7 february. Till now - and I am in the second day - I am quite dissapointed in the quality of content that is delivered. It's hugely aimed at selling stuff.....The 'how' is not addressed or very limited. But...one day to go and on paper, some sessions look interesting....who knows.

One of the sessions I was particulary interested in was the session of IBM, explaining their 'Dynamic Data Warehouse' offering. There is very little known on this concept, so I was interested. I am going to be very direct here.....based on the presentation that was held I can conclude:

1- Dynamic data warehousing is a bunch of tools (and IBM got plenty)

2- Dynamic data warehousing is very much marketing (faster information, more reliable blabla)

3- Dynamic data warehousing is not compliant to what I think dynamic data warehousing should be about

4- oh....IBM sells their 'balanced warehouse' as an appliance.......HUH??its hardwarde with DB2...and some other stuff and 1 phone number a customer can call....that aint an appliance, or is it. Maybe it's a sales-appliance..."oh we got so many software packages, lets bundle them and sell them with our hardware, and we also have an appliance"....duh

About number three....I very much like the definition made by Dan linstedt:

The ability of a system to 1) interrogate arriving information at run-time 2) discern new "structure" from old "structure" 3) separate the new structure, and build or attach new structural elements to the existing structure, 4) mine existing structural elements for unseen relationships and finally 5) Follow a series of "alert" patterns to notify operators that new nodes or elements have been added, and need to be checked.

In a business sense, or the simplistic definition is: to add and/or change the structure of information on the fly based on "content analysis". The adaptation of the structure is in near-real time, and will result in learning things we didn't know before. It basically changes the data model underneath the covers by using neural net techniques and structural analysis ideas.

In later writings Linstedt is referring to a major misuse of the term dnamic data warehousing (hmm...IBM maybe?) and he uses another term for the above:Automorphic data models. Let's be very clear; IBM's definition of Dynamic Data Warehousing is not at all close to what Linstedt means if he talks about Dynamic Data Warehousing.

Based on the presentation of IBM I was dissapointed......Now, informally I met a representative from BIReady.nl. And they told me that dynamic datawarehousing of IBM should be translated as a partnership of BIReady (uses Data Vault principles) with IBM hardware (p-series) and IBM referential models (e.g. banking model) and sogeti for the services (why not IBM global services is off course an interesting question). Now, this is interesting because I am a firm believer in generating ETL instead of programming it. This 'dynamic' nature of data processing -not data warehousing - (completely other definition as compared to Linstedt's) could hold some truth and is certainly interesting enough to explore even more, BUT WHY ISN'T IBM TELLING ME THIS?

To conclude this story....I am puzzeled as to the direction IBM is heading into with regard to data warehousing. If I don't get it....why should any1 buy it?? And what the hell are you buying then? I am going to try to investigate a bit more. It's simply beyond my comprehesion that IBM does not have a valid and strong case in their 'dynamic data warehouse' offering...I am just not able to grasp it...see it...or understand it.

More will follow..or not...