Upgrade to XE and RB12.03 - problem with string fields

rbuser

I have upgraded from Delphi 7 to Delphi XE, and RB7.03 to 12.03.
I'm upgrading a large app. Just reviewing reports, which mostly run.
Some character fields do not display. For example, the Account Name in
the title of some reports, and the product name on the lines in reports.
Numeric fields don't seem to be having a problem.

The data comes from Interbase Views, which do contain the data.
The TppReport is connected to a TppDBPipeline to a TDataSource to a
TClientDataset to a TDataSetProvider to a TIBQuery.

Looking at 2 very similar reports, one has the Account Name in the Title
and one has a blank in that spot. I've moved to UTF8 in Interbase XE,
but this works in one report and not the other with the same Account
Name. Both reports are getting good numeric data. These reports are very
similar, but I can't see what is different.

nicocizik

Hi Patrick,

We have not seen this behavior before but it sounds like it may have
something to do with the Unicode capabilities of Delphi XE/Interbase.
Are you able to recreate this with a new report or just with old saved
report templates? If you create a new report that uses the same fields
that do not display, are they visible?


Regards,

Nico Cizik
Digital Metaphors
http://www.digital-metaphors.com

rbuser

On 3/29/2011 11:59 AM, Nico Cizik (Digital Metaphors) wrote:

Nico,
Thanks that gives me something to try.
Converting from D7 to D-XE with Unicode has kept me busy for months. I'm
only now addressing the reports.
I did have a lot of trouble with ClientDataSets, which are used here,
but I always got errors for Widestrings on forms, although I don't get
any erros on the reports. I looked here but I didn't see a problem.
I'll keep looking and reply here if I find an answer.

rbuser

On 3/29/2011 11:59 AM, Nico Cizik (Digital Metaphors) wrote:


Nico,
I gave it a try, creating a new report. I had the same problem as soon
as I started building it. I put a simple "Select * from View" in the
CommandText of the ClientDataSet, and checked Active.
Then I placed a couple of fields on the report. Even in Design view the
numeric fields were populated and the string fields were blank.

As I've been upgrading my app I had the same problem with data displayed
in DBGrids on forms. Those were getting their data from stored
procedures. Any persistent fields stored in the ClientDataSet gave
errors of WideString data and String definitions for data coming from
Varchar columns in the database. For Char columns, I have to Trim() the
data to get it to work. I think that's a Delphi problem. I don't think
either problem occurs coming from Query components, rather than stored
procedure components.

Now that I've gotten to work on the reports, I'm first addressing data
from Views. There seems to be a problem here too. I'm not as comfortable
with how to get at the data as it goes through the CDS to the Pipeline
to the report.

As part of my upgrade from D7 to Xe, I defined my database as UTF8 Data,
which is a change from D7. I didn't transfer data. I just have a few
records here that I manually entered into my app. I can see that the
View is fully populated in Interbase.

So far, none of my reports are working. Obviously some assistance would
be appreciated.

Thanks,
Patrick

nardmoseley

So in D7 you had this...

Interbase (ansi string) ---> TClientDataSet (TField.DataType of ftString)

where for D7 ftString is AnsiString

In DXE you have this....

Interbase (utf8 string) --> TClientDataSet (TField.DataType of ftString)

Only now for DXE ftString is UnicodeString.

Somewhere in reading in the data, the UTF8 strings have to converted to
UnicodeString. If that does not happen, you will have issues all over the
app.

For a Unicode VCL application, you want all data inside the app to be
UnicodeString. Any issues you are encountering are due to reading in string
data that is not converted properly.

Much of the Unicode VCL, including the database access components can do
conversions automatically, so you don't have to write the code to do them.
Try Googling about your specific configuration and issues, it might be a
simple configuration issue with your data access. For example, here is a
post I found that discusses configuring IBExpress with the correct encoding
type (ic_ctype). If you are using IBExpress, that might be your issue.

https://forums.embarcadero.com/message.jspa?messageID=20886



-
Nard Moseley
Digital Metaphors
www.digital-metaphors.com

rbuser

On 3/31/2011 12:11 PM, Nard Moseley (Digital Metaphors) wrote:

There are two issues. On my forms that display with a dbgrid, I
immediately got a compile error saying that my fields were tstring and
the data was widestring. So I changed those and it worked ok. Those were
only where I had persistent fields defined. Where it was dynamically
created, it is creating it properly. So all my forms are working for
that issue. I'm sure my IBX configuration is good since I have over 100
forms working properly.

On the reports, I think a similar thing is happening, but I'm not
getting the error message when I compile. And, I don't know where to fix
it either. I think it's probably happening in the datapipeline. I tried
creating a new report but it didn't fix the problem.

I think there is a separate issue with Char() type data coming out of
stored procedures from interbase. I believe there is an actual error in
the IBX for that. I'm not sure if that is nested in here too, but most
of my data is from varchar fields.

The reports are the only thing now working now. I hope you can find
something there.

rbuser

I have a similar report that is working. They are both inherited from a
common template that I had created. I compared the reports to try and
find where the problem is.

I don't have a firm answer to the problem but there are two things of
interest. In the Report Designer, the right panel shows the "Fields for"
the dataPipeline. These come from separate Interbase Views, but they
share many fields. The working report shows traditional field sizes. The
failing report shows Unicode sizes which, for strings, are 4 times as
large. The failing report, which I have been working on, probably has
had the field definitions refreshed during my work.

Next, if I click on a field on the report designer, one of the
properties is "Font". Here the Charset is ANSI_CHARSET. None of the
choices include Unicode. So, where is ReportBuilder getting the list of
character sets? On my working report, this selection probably agrees
with the data as it is seeing it, which is the traditional field size.
But I may have gotten all the data corrected to Unicode on the failing
report, but the Font setting is still ANSI. The ppReport component,
likewise has Font settings, which are set to "Default". These also
probably equate to ANSI as there is no Unicode manual choice there.

I hope this helps.

Patrick

nardmoseley

Delphi 2009 introduced the Unicode VCL. Since that time, customers have not
reported any issues like this. Reports created in prior Delphi versions just
work, as do reports created in prior RB versions. The only thing I can think
of that is different is that you changed your database to be UTF-8.

The DBPipeline relies upon the TDataSet (or ClientDataSet) to supply the
field information and to do the conversion to TUnicodeString.

Fonts are handled by Windows. If you use a Font that includes support for
the characters you need to render, then it should just work. If the Font
does not support the characters, then you will typically see ??? characters.
The Arial font is a good one to use for a test, it includes good support for
Unicode.

Here are some simple tests to try. The goal is to try some of the most
basic tests possible, to try to isolate where the issue is located.

1. Basic Font Test. Create a new Delphi project. Put a Label on a form and
type the characters into the caption. And then likewise, put a TppLabel on a
report and type the characters into the caption. Configure the font for the
Label and TppLabel and test whether it renders correctly.

2. Basic ClientDataset/View test. Create a new Delphi project. Place the
ClientDataSet (and provider) on the form and connect a DBGrid. And then
connect a DBPipeline and Report. Create a new report layout. Does they both
work?

Post a follow up here as to the results.

If you want to see how DBPipeline works, add RBuilder\Source to your Delphi
library path and put some debugger stops in ppDBPipe.pas. Methods of
interest would be GetFieldValue, GetFieldAsString, GetFieldSize.




-
Nard Moseley
Digital Metaphors
www.digital-metaphors.com

rbuser

On 4/4/2011 2:31 PM, Nard Moseley (Digital Metaphors) wrote:

Nard,
Thanks for your reply.
I don't have any special characters, just good ole American English.
Same database that was previously ISO8859-1. And, I haven't had any font
problems on my forms, which are far more numerous than my reports.

While many clients have used recent Delphi versions with RB, if their
data has not been changed to Unicode, I'm not sure if the lack of any
reported problems means anything.

I did have problems on my forms with ClientDataSets, but Delphi gave an
error message. I didn't get any message with the reports - just missing
string data. I used what I learned in making my forms work, but couldn't
see where to change anythings with the reports. I did create a small
report from scratch, using a view from my database, and it too had the
problem - which surprised me.

I will try to locate the problem and post anything I find.

Thanks,
Patrick

rbuser

On 4/4/2011 2:31 PM, Nard Moseley (Digital Metaphors) wrote:

Following your suggestion #2 I built a simple form. I experienced the
problem with the DBGrid and never got as far as the report. It appears
this has something to do with the ClientDataSet component.

I did use two of my actual views from my reports - one that works and
one that doesn't. Same result in DBGrid from both when using the
ClientDataSet - one worked and one didn't. In all cases I got the right
number of records and the numeric data - only the string data varied.
The only thing I can easily see is that the failing displays come from a
View that includes a Union. And of course, this has now been converted
to UTF8. Otherwise, in Delphi 7 this code works.

I next used a TIBQuery as the Dataset, removing the ClientDataSet and
Provider. This always worked. Previously the IBQuery was feeding the
ClientDataSet. The problem seems to be in the DataSetProvider or the
ClientDataSet.

Patrick

rbuser

Nard,

FYI, I have resolved this issue and pass it along.
The problem was reports coming from InterBase Views that were missing
some string data but had all the records and numeric data.
It seems the reports with problems got data from Views which included
Unions. Something there was failing under Unicode. I changed the
ClientDataSet on my report to request data from a new IB procedure,
which simply said Select * from View. Running the request through a
procedure cleaned up whatever is wrong with the view. No changes were
made to the View. I suspect it is primarily the passing of metadata.

I also created the same database as ANSI and accessed it with the same
Delphi application (prior to making the corrections). It ran correctly
as ANSI.

Patrick Moloney