Xcelsius scalability

[Neil Phillips]

Can I just ask you all for a bit of feedback on the sizes of the Xcelsius implementations you've developed.

I am having a real problem with what appears to be the scalability of the various components which make an Xcelsius dashboard.

Currently we have an .XLS model which has a raw pre-aggregated dataset of about 5000 rows by 25 columns (Current YTD monthly trend, Previous Year monthly trend, etc). All this data is normal business data for Performance Mgmt.
This does not seem an unreasonable amount of data for Excel to deal with.
This will be updated on load of the SWF file by an XML feed.
There are then various other Tabs in the .xls for lookup, sumif, and other formulas.
So I have an .XLS of about 650kb.

The XLF (even with minimal charts and no dynamic visibility) generates at 12MB and the SWF about 1MB.

The time to save the XLF is massive (up to 5 minutes) and then to generate the SWF also a few minutes.
Updating the SWF via XML is OK when only updating 2 columns, but extend the arrray and this also grows (exponentially it seems).
Also performance of the SWf itself is fairly clunky when used as it would be by the business.

I am stumped as to why these take so long. I can obviously understand there is complexity in creating the SWF because of embedding Excel formulae, etc but this seems over the top.

I know that Xcelsius is not meant to be great at dealing with large datasets, but I cannot imagine many Organisations with a large BO deployment having much less than we are trying to implement here.

Is this something anyone can offer advice on?
I don't know whether I am flogging a dead horse by proceeding or whether I'd be better using Xcelsius simply for current PI positions and then embedding WEBI reports for the trending.

Thanks in advance
Neil

[rgoodman]

The use of Xcelsius with enterprise data should be reserved for presenting summarized information. The goal is to use tools like QaaWS or Crystal/WebI to do all of the heavy aggregation and calculations for your data sets. Because this SWF serves as a visualization layer, you want the data volume and calculations to be minimal during runtime.

Many customers use multiple parameterized queries and/or role level security (when used with QaaWS) to dictate the data that is fed into the dashboard. Depending on the analysis and number of analytics, there are ways to scale the dashboard to multiple SWFs if necessary. For extremely large data sources, we have seen people use aggregate tables which allows for great query response time.

Intuitively we want to push all of the data to the dashboard with one pass to the data base. How many passes and when the queries are run depends on the natural workflow that the end user will step through when accessing information.

I agree that there needs to be some more messaging around this.

[Neil Phillips]

Thanks for the feedback Ryan. I think we've come to the same conclusion, that to maintain performance within Xcelsius SWF files it is necessary to minimise the amount of data within the embedded Excel model at any one time.
Our solution was to break our XML files down into smaller and smaller sizes until we achieved a suitable runtime performance. An XML file of about 500 rows by 4-5 columns seems about the max we can deal with. Saying that, the Excel model is heavy on calcs, especially VLookups and some Match stmts which are not very efficient.

This worked OK because our model will support it, though it has been a bit of a headache to get to the point where we knew why Xcelsius was crashing, and then working backwards to amed the Excel model and Data Integrator XML outputs.

The " A movie in this script is causing Flash to run slowly" (or similar wording) baffled me for some time as I assumed there was an error or problem. In fact it was simply too big. The decompiled SWF came out at 15MB, so no wonder Flash wasn't dealing with it very well.

My hope would be that BO will rewrite the Flash engine so that a whole dashboard is not defined as one 'movie', though I don't know if this is possible. Or perhaps I need to look at embedding SWFs within SWFs.

I am sure more info will come to light around performance of SWF files as more and more people start using it.

Neil

[MHerlund]

I am having the same issue with the flash error where the script is taking a long time to run...or whatever it was. My swf file is only ~.5 MB so it is not that big at all. My data is <300 rows with ~27 columns (not all are used). All that is being displayed is 3 graphs. I don't see why it would be that hard to run, does anyone have any ideas?

Thanks.

[Neil Phillips]

[MHerlund]

Thanks for the reply.

I am using XML as data but I even placed static data in Excel and only put little data in there and I am still getting the error.

I am using a few simple calculations like divide one cell by another and I even removed these formulas and still get it.

[Neil Phillips]

Only thing I can think is that the XML ranges you've defined are huge.

Even if most of the cells are empty Xcelsius doesn't know that they always will be, so will account for the full range when it converts to SWF (I think).

Email me the XLF and XLS and I'll try to have a look. Not sure when, but mail it over anyway if you want. Put some nonsense data in if you don't want me to see it.

Neil

[turingcomplete]

I've had luck by using multiple tabs, and limiting the data on each tab to 500 rows. In this way I could simulate many more rows. I would move the data onto the first tab using IF() formula's.

Otherwise Xcelsius would crap out with a "C++ library error" after about 10 minutes of trying to export to a swf.

The fact that it can't handle a few thousand rows is ridiculous, I don't know how it ever made it into production, this is a huge and obvious bug.

[anthonyjones]

I have also found that using sumif's on anything over about 300 rows causes the swf file to perform badly.

Again, i used multiple tabs and addtional data sources to aggregate the data. I used Liveoffice with 16 connections to Webi reports that return data at all levels.

Then removing sumif's and replacing with either vlookup or hlookup, the swf performs so much better. Although this improves the performance of the model, it also has its downfalls. For example you can not use iserror() in conjucttion with the lookups so additional excel functions will have to be used.

[MargaretG]

iserror() and several other information functions are going to be supported in the Service Pack 1.

[Andreas]

I can only hope that BusinessObjects SAP will integrate the Xcelsius graphical capabilities/visualizations with the existing reporting tools (Crystal Reports, WebIntelligence). Most dashboards can be built with Webi or Crystal Reports (no need to use Xcelsius). And Webi and Crystal Reports can easily handle 100k+ data rows, most charts from Xcelsius are also supported in Webi/Crystal Reports (think bar charts, pie charts, radar charts, area charts, etc.).
If you just count how many data points one can visualize in an Xcelsius model compared to a WebIntelligence report or Crystal Reports and have decent performance...

Xcelsius is essentially MS Excel on steroids, and the architecture beneath does work, but there is room for improvement.

Where Xcelsius shines IMHO is:
- Ease of use
- Time to deliver (as a MS Excel spreadsheet is all you need to produce your first dashboards)
- Interactivity/capability to do simple simulations
- Capability of almost live feed of data/operational dashboarding
- Inclusion of geographical maps