Problem with the time performance with large data sets

rbuser

I have a TClientDataset connected also with a pipeline. The ClientDataSet
have over 8000 data records. I start the print appear for a very long time
the message "seize on data too". Afterwards approximately 12000 sides are
developed. This entire procedure lasts over 3 hours. The technology of the
soracle-servers and the PC-Client is in the best way equipped.
Which possibilities do I have to generally accelerate the procedure?

Thanks for the help

best regards

Dirk

nicocizik

Dirk,

Be sure that you have the Report Outline disabled before you run your
report. In testing we have found that the Report Outline tends to slow the
generation of larger reports. Another option would be to use ReportBuilder
Server Edition so you would not have to load the entire report to view and
print. The article below explains what the Server Edition offers.

-----------------------------------------------------------
R B S E R V E R E D I T I O N
-----------------------------------------------------------


Live Demo!
----------

http://www.digital-metaphors.com/RBServerLive


Trial Edition
--------------

http://www.digital-metaphors.com/server/TrialEdition.html


Server Edition Overview
------------------------
The Server Edition makes it easy to create and deploy report applications to
the web. Using basic component configuration you can get a Windows
service-based report server application up and running, a web application up
and running, and have users previewing reports in a web browser in no time.
And you can do it all without having to master multi-threaded programming,
Windows Services, Windows Sockets, Windows System Tray applications, COM,
I/O Completion Ports/Worker Thread Pools, XML, SOAP Services, ISAPI Thread
Pools, XHTML, or JavaScript. Sound interesting? Then read on...


1. ReportBuilder Services

ReportBuilder Services is a Windows service designed to act as a host to a
report server application. This two-piece architecture isolates a report
server application in its own process, allowing it to be started and stopped
from the service. By default, the service is configured to start
automatically each time the server machine boots. Once the service is
started, it launches the report server application. The service is also
configured to automatically restart the report server application in the
event of a crash.

To install ReportBuilder Services, you simply run the included
RBWinService.exe installation program. The program installs and starts the
Windows service and deploys a server management application, accessible from
both a system tray icon and a Start menu item. You can then access the
management application and point the Windows service to the location of your
report server application.


2. Report Server Component

The Report Server component implements a multi-threaded report server that
can execute reports and deliver report pages to clients. You build a report
server application by dropping a ReportServer component on a Delphi form,
creating data modules which contain ReportVolume components (explained next)
and compiling the application. And that's it; you can build an entire server
application without writing any code. Once the application is built you can
then use ReportBuilder Services to quickly and easily designate it as the
official report server.

The Report Server is implemented using an I/O completion port/worker thread
architecture that optimally manages persistent report sessions to maximize
performance and throughput. The server can generate reports incrementally
(that is, page-by-page) in order to optimize response time. Pages are cached
on the server and on the client to minimize network traffic. The amount of
time that passes before a session expires (and the cache is cleared) can be
controlled via the TimeOut property.


3. Report Volume Components

Report Volume components are used to register a collection of reports with
the server. The server uses a report catalog to build and store the tree
structure which forms as the various reports are registered.

There are several types of report volume components, one for each of the
ways that reports are most commonly deployed. For report templates stored in
file directories or database tables, there is the ReportTemplateVolume. For
reports stored in the Explorer database structure (i.e. RB end-user
reports), there is the ReportExplorerVolume. For report archives stored in
file directories or database tables, there is the ReportArchiveVolume. For
reports stored on a Delphi form, there is the "behind-the-scenes" report
volume object. The bottom-line: wherever or however your reports are
deployed, there is a simple way to register them with the server, usually
without writing a single-line of code.

Report volumes are also dynamic; as you add reports to a designated file
directory or database storage medium, the report server will automatically
add or remove those reports from the catalog. Yes, this does mean that you
can deploy reports to the server without bringing the server down.


4. Windows Rich Client Components

With one ClientReportExplorer, one ClientReport, and one line of code, you
can build a full-featured Delphi reporting application which provides access
to reports on a remote server. The ClientReportExplorer uses a Windows
Explorer style interface to display a folder tree structure of available
reports. The ClientReport makes it easy to preview and print. When
previewing a report that requires search criteria, the AutoSearch dialog is
automatically displayed and any search values entered are returned to the
server. The report is then generated and pages are returned.


5. WebTier Component

When compiled as part of a web application, the web tier acts as a middle
tier, providing access to a report server application from a web browser. A
user may select reports from a folder tree structure, enter search criteria
when applicable and preview report pages from the web browser. All of these
capabilities are made possible via a zero-client application (100%
JavaScript/XHTML) generated by the web tier. The zero-client is compatible
with Netscape 4, Netscape 6, and Internet Explorer (5 and 6.)

Designed to integrate seamlessly with Delphi's various web technologies, the
web tier requires only a single line of code to compile successfully as part
of an ISAPI DLL or Apache Shared Module. ASP is also a snap, as a single COM
interface (easily generated with the help of Delphi's ActiveX support) is
all that is needed to deploy the web tier as part of an Active Server Page.

The web tier implements a sophisticated concept of "session", grouping all
of the communications received from a single web browser under a single
session object. In order to optimize performance, both the state of the
session, and any report output generated for the session are cached. For
example, assume a user previews report A, then report B, then returns again
to report A. The web tier would manage all of this activity as a single
session, and whenever possible, would return pages from the session cache,
instead of re-requesting them from the report server application. The amount
of caching which occurs, the TimeOut for a session and the frequency with
which garbage collection is performed on expired sessions are just a few of
the configuration options available on the web tier.

Implemented as a set of Delphi classes, the WebTier consists primarily of
objects which descend from TrsWebContentProvider. Web content providers
communicate with the report server application, converting the response from
SOAP/XML to native Delphi objects to XHTML/JavaScript, suitable for
consumption by a web browser. The content providers included in the Server
Edition make it easy to provide a complete, robust, and professional web
reporting solution right out of the box. You can customize the HTML created
by the web tier by creating your own content provider descendants.


6. RB ISAPI Thread Pool

The Server Edition includes a custom ISAPI thread pool, implemented using an
I/O completion port/worker thread architecture designed to optimally manage
incoming requests. The thread pool queues requests based on session, and can
eliminate redundant requests. This greatly improves both throughput and
performance, as the web tier passes only relevant requests through to the
report server application.


7. Server Farm

The web tier can be configured to utilize any number of report servers to
maximize scalability. This allows the web tier to scale gracefully as a
greater number of users access reports. There are two load balancing options
provided: Round Robin and Minimum Load. Round Robin simply steps through a
list of servers, each time returning the next server in the list. Minimum
Load checks the CPU usage of each server (based on a sixty second moving
average), and returns the server with the minimum usage.


8. Documentation, Help & Examples

The Server Edition Developer's Guide is provided in PDF format. The concepts
behind the web server are covered, along with numerous step-by-step
tutorials that guide you through everything from creating a report server
application to deploying the web tier with a server farm. The online help is
a reference for all of the components and major classes which make up the
product. The help is integrated with the Delphi help and is accessible via
the help menu or the F1 key. Numerous examples of report server
applications, web applications and client applications are also provided.

--

rbuser

Nico,

the Report Outline is disabled. I bought the Server Edition.

Why is the printing of the report with the servers edition to run faster?

The problem is the also following:

I must let the print run three times.

1. As the first the originals must be printed. Those are approximately
12000 sides. That lasts 2 hours.

2. As the second the document copies are printed. These are however
differently sorted equal the originals and have additionally the word copy.
That lasts 2 hours.

3. Each individual document (approximate 8000) is printed last into
archives. That lasts over 5 hours.

Is there a possibility time to save here?

Thanks for the help

Dirk

rbuser

Hallo Nico,

I have still some questions.

How get do I mean data from the ClientDataSet from the client application
into the server application? Do I have to store the data set physically
firmly?

How can I new of report put on or on existing report changes about the
server make?

Best regards

Dirk

nicocizik

Dirk,

Do the reports take this long to generate to to the print queue or to print
completely? Be sure you have sufficient hard disk space and memory to
support such a large report. Try just generating to an Archive file. Does
this still take 5 hours? If you create a minimal report and run it with the
same database, do you see the same time delay?

--

rbuser

The reorts take thislong to generate to the print queue. The hard disk space
and the memory is excellent. I have 1 GB memory.

Even if I 100 or 1000 data records select to the print the time performance
is okay.

rbuser

Hallo Nico,

I print the 8000 data records into an archive file last it somewhat longer
as into the queue of the printer.

Best regards

Dirk

nicocizik

Hi Dirk,

Try taking the TClientDataset out of the equation and pulling the data you
need directly off of the database you are using and see if that helps. In
testing ReportBuilder, we have successfully processed 10 to 20 thousand page
reports very quickly. Usually a performance constraint such as the one you
are experiencing is due to the data access, not the report engine.

--

rbuser

Hi Nico,

Now the print data lie in the Oracle data base. But the time performance
became still worse as with the ClientDataSet.
Perhaps it is there because of:

I have two data pipelines, which are over a master/detail relationship from
each other dependent.
The dependent field is indicated in both datasets.

I leave the detail dataset empty am very good the time performance.
In the main quantity about 8000 data records are, in the subset are it 12000
data records.
The execution print lasts over 3 hours. What can I make?

Thanks for help

best regard

Dirk

nicocizik

Dirk,

How are you linking your dataset? Do not link your master-detail datasets
using Delphi parameters as this will slow your performance substantially.
Try ordering your datasets and using DataPipeline links. To see an example
of using pipeline linking, please refer to the ReportBuilder Developer's
Guide tutorials or the RBuilder\Tutorials\... directory.

--

rbuser

Hi Nico,

Thank you for the past assistance. The times become better. And with also
without server edition from reportbuilder. We will train those now into our
software.

I have still another problem with the letter of the 8000 data records into
archives. In my data base there is a archives table, into which in a BLOB
field the report is written. I implement thus 8000 times the print, produce
in each case a raf file and copy these then into the Blob field of the
table. The user of the software can regard thus to each procedure the
produced pressure. Printing and writing the 8000 data records last however
for a very long time. Is there here perhaps still another another variant?

Best regards

Dirk

nicocizik

Hi Dirk,

If you've taken out the Delphi Parameters and disabled the Report Outline,
there is not much else you can do with ReportBuilder to speed up the
process. The rest of the speed restrictions would have to lie within your
data access.

--