pointers, sizes

Just a small remember for myself:

If

TMyRecord = record
  pointer1: pointer;
  pointer2: pointer;
  pointer3: pointer;
  pointer4: pointer
end;
PMyRecord = ^TMyRecord;

then

  sizeof(TMyRecord) <> sizeof(PMyRecord)

So

  var rec: PMyRecord;

  rec = AllocMem(sizeof(rec));

is probably not a sensible thing to do (at least not if you intend to actually put something into that space the pointer points to).

At least it began breaking very, very soonish and consistently once TMyRecord got enough members – too bad though that I first looked at the completely wrong space.

Nothing beats the joy of seeing a very non-localized access violation go away after two hours of debugging though.

Reporting Engines

There are quite many solutions available to you if you want to create printed (or PDFed) reports from your delphi application, but surprisingly, one one is really convincing.

The so called professional solutions like Crystal Reports or List&Label are very expensive, require a large runtime to be deployed and may even require you as the developer to pay royalties per delivered client

So for my particular problem, the only solution was to use a VCL based engine that can be compiled into the exe and does not need any additional components to be deployed.

Years ago, I was told to use ReportBuilder and maybe people were even right there: Quick Reports (as in included into delphi back then) had and still has a very bad reputation, and RB was the only other VCL based product available

RB has some problems though. Lacking Delphi 2006 support, limited report templates, field formating on the database access layer and last but not least: A nasty bug preventing barcodes from being correctly rendered to PDF-Files.

Then, I had a look at Fast Report and I tell you: That piece of software is perfect!

Granted, switching will come with a bit of work, though the paradigms of both engines kinda match. But once you’ve done the stupid rebuilding of the old templates, you’ll notice how wonderful Fast Report actually is. And you will notice immediately as it’s very, very intuitive to use – compared to RB. Things that required custom programming or even a little hacking here and ther in RB just work in FR. And they even work without forcing you to read through lots of documentation in advance

And everything – just everything is in the report template. Number formats, even small scripts for changing the report in subtle ways while it’s being printed. Just perfect for what I was doing.

So, if you are looking for a nice, powerful really easy to use reporting enginet that can be fully compiled into your EXE, you should really go with FR. It even costs less than RB.

The myth of XCOPY deployment

Since the advent of .NET, everyone is talking about XCOPY deployment.

XCOPY deployment means that the applications are distributabe without a setup routine. Just copy the file(s) where you want them and that’s it.

We are being told that this is much easier and safer than the previous non-.NET approaches which – as they continue – always required a setup program.

The problem with those statements is that they are all false.

First the ease of use: Think of it: Say you want to install Cropper (which made me write this entry. I found that screenshot utility via flow|state). What you are getting is a ZIP-File, containing 5 files and a folder (containing another 6 files). Nearly all the files are needed for the application to run.

XCOPY deployment in this case means: Create a folder somewhere (Windows guidelines advocate you create that in c:Program Files which is a folder windows does not want you to mess with and per default does not display its contents) and copy over all those files, being aware not to forget a file or the folder in the archive.

But it does not end there: As you have to launch the application and going all the way through those folders, you will want to have a shortcut in the start menu or on the desktop. With this new and “better” method of deployment, you’ll have to do that yourself.

This is a tedious task involving lots of clicks and browsing. An unexperienced user may not be able to do this at all.

What an unexperienced user will want to do is to copy that application right to the desktop. But in this case this does not work well as the whole application consits of multiple interdependant parts. Copying only the .EXE will break the thing.

Compare this with Mac OS X

In Mac OS X, application also consist of multiple parts. But the shell is built with XCOPY deployment (not called like this, of course. As a matter of fact, it does not have a name at all) in mind: In OS X, you can create a special kind of folder which is a folder only on the file system. The shell displays it to the user as a single file – the application.

Whenever you move that “file” around, OS X will move the whole folder. When you double click the “file”, the application will launch (the binary is a file somewhere in this special folder. The shell is intelligent enough to find and launch that). When you delete it, the shell will delete the folder including it’s contents (of course).

This makes XCOPY deployment possible as the applications become one piece. You want it on the desktop? Drag it there. In the Application folder (without warnings about not being allowed to mess with its contents, btw)? Drag it there? On an USB-Stick? Drag it there.

Well. There’s one other thing: It’s the users data and the applications data. Most of the applications will be used to create data with them. And all application somehow create their own data (for saving things like the window state or position for example). As all modern OSes are multiuser ones where a user does not necessarily have to have write access everywhere, there’s the concept of the home directory. That one is yours. You may store whatever you want in there.

So naturally, this is the place where the applications should store data to0.

User data goes to a specific folder of the users choice. Per default, applications should suggest some Documents-Folder. Like “My Documents” in Windows or “Documents” in Mac OS. In most of the cases you don’t want to delete that on uninstall.

Application settings are in Windows stored in the Registry (under HKEY_CURRENT_USER – a hive that belongs to the current user like his home folder does. And actually, the file behind that is stored in the home folder aswell (USER.DAT)) or in the Application Data folder below the users home folder.

Mac OS X Applications are advised to use the Preferences-Folder inside the Library Folder inside the users home directory<./p>

Now. Application data is something you want to remove when you uninstall the application (which means deleting a bunch of files in Windows or one “File” in Mac OS). Application data is created by the application, for the application. No need to keep that.

In Mac OS, you can do that by going into the folder I’ve described above and delete the files – mostly named after your application. There are no warnings, no questions, no nothing. Just delete.

In Windows, editing the registry is off-limits for end-users and very, very tedious to do for experienced users (due to the suboptimal interface of regedit and because the whole thing is just too large to navigate it easily), so you generally let the stuff stick there. Deleting the Application Data in the same-named folder is also impossible for the end user: That folder is hidden by default. Explorer does not display it. And it’s hard as hell to find, as you have to manually navigate into your home directory – there’s not easy GUI-access to that. So that sticks too.

All in all, this means that windows is – at least in its current state – very unsuited for XCOPY deployment:

  • It does not help at keeping together things that must be together
  • Its complex file system structure makes it hard to copy the application where windows wants it to be
  • Manually creating shortcuts is not feasible for an unexperienced user
  • Uninstallation of Application Data is impossible

So, we found out that XCOPY deployment is not easy at all. Now let’s find out how it’s not true that only .NET enabled you to do this.

Ever since there is Delphi, there theoretically is XCOPY deployment.

Delphi is very good at creating self-contained executables.

With delphi it’s a breeze to create one single .EXE containing all the functionality you need. That one single .EXE can be moved around as a whole (obviously), can be deleted, can even be put right into the start menu (if you want that). It can even create the start menu shourcuts, delete application data – basically configure and clean itself

It can even uninstall itself (embed an uninstaller, launch that with CreateProcess and set the flag to delete the .exe after it ran). And it can contain all it’s image and video and sound data it needs.

Just because nobody did it does not mean it was not possible.

Face it: Windows users are used to fancy installers. Windows users are not at all used to dragging and dropping an application somewhere. And currently Windows users are not even able to do so as dragging and dropping will break the application.

OS X and now Linux allow for true XCOPY deployment of desktop applications.

Well, you say… then maybe XCOPY deployment is just for those fancy ASP.NET web applications?

Maybe. But after XCOPY you need to configure your webserver – at least create a virtual directory or host. A good installer could do that for you – if you want it to.

Microsoft too has seen that this XCOPY thingie is not as great as everyone expected, so they added the new “One-Click Install” technology, which is not much more than a brushed-up MSI file which does a old-fashioned install.

To really make XCOPY deployment a reality (btw, I’m a big fan of depolying software like this), there must be some changes within Windows itself. Microsoft, copy that application bundle feature from OSX. That one works really, really good.

Btw: Am I the only one that thinks “XCOPY deployment” is a very bad term? What is XCOPY? Who the hell still uses XCOPY these days? And when we are using the command line: COPY would be enough.

No topic-based help system installed

Recently I had to do some Delphi-work again. To my surprise, the online help seemed to have stopped working. I always got this error message:

No topic-based help system installed

Programming without an online help is very tedious and sometimes nearly impossible.

When I had to look up in which Unit TWinControl is declared, I had two possibilities: Either look it up in the source code (Borland ships the full source code to their class library) or fix the help system once and for all.

I deceided to do the latter (searching after TWinControl is no fun).

Googling in the web turned out nothing. In Groups most of the time, the suggestion was to reinstall the whole thing

I absolutely did not have time for this, so I dug deeper.

The problem is caused by the installation of the VS2005 Beta which resets some AppID-GUUID. Afterwards delphi crashes while loading the IDE-package htmlhelp290 which in the end causes delphi to think that there’s no help installed

I fixed it doing the following:

  1. Reset the help-viewer-appid. In the registry under HKEY_CLASSES_ROOTAppIDDExplore.exe, set AppId to {4A79114D-19E4-11d3-B86B-00C04F79F802}
  2. In HKEY_CURRENT_USERSoftwareBorlandBDS3.0Disabled IDE Packages remove the entry for htmlhelp290 that has been created.
  3. Start Delphi and use the help again

What I don’t know is if this has a negative effect to VS, but this does not matter for me: I need Delphi to work.

The whole thing is a consequence of the .NET orientation of delphi: Earlier, Delphi was as self-contained as the executables it can build: Drop it into a directory and run it. No problems, no questions asked.

With Delphi integrated into .NET and using .NET-Components, problems begin to rise: First there was a bug in D8 causing it to stop working after .NET 1.1 SP1, now this.

Hopefully, they find a way back to both .NET (for the acceptance in the buzzword-centered world where you can’t have a dev-tool not .NET capable) and self-containment.

Productive with Delphi 2005

Yesterday and today, I finally had the opportunity to do some real work on PopScan with Delphi 2005. Here’s what I really like besides the obvious:

  • Those .bdsproj-Files are incredibly useful. They replace the old DSK and DOF-Files, have a convenient XML-format and are the new project file, you open with in Delphi. This is very nice as the old project file (.dpr) is actually program code and does not contain any project metadata. This is what those .dof and .dsk-Files where used for, but I never understood which setting is in which and the format has not been XML either. So this consolidation really is convinient.
  • The history-feature really saved my day. With me hitting Ctrl-S on nearly every line I write, the older .~pas-approach wasn’t very useful and CVS was no help either because I don’t commit as often as something could go wrong in the code.
  • The new exception catching-dialog of the debugger is really nice. I like this “Dont halt on this exception again”-Checkbox.
  • While it makes the application significantly slower under the debugger, the new “event log” is great.
  • Speaking of debugging: The “Local Variables”-Window is great too.
  • Delphi now distinguishes between a “Default Layout” and a “Debug Layout”. You can configure both of them as you like and Delphi automatically switches between them. This is much more intuitive than before.
  • Maybe I’m the only person on earth, but I like the single-window-approch: It’s much cleaner than before. No more tons of clutter on the screen and the important screens are always wisible. No more Ctrl-Alt-F11 either.

Additionally, I don’t have as much speed problems as others seem to have: While starting the IDE takes it’s time, working with Delphi when it’s open goes quite smoothly.

My only problem is opening the form designer. This definitely takes too long, but not long enough for me to switch back to the undocked layout.

Memory usage is of no concern to me. I have 1 GB of RAM and even after a day of using delphi, my Thinkpad remains responsive even though not only delphi, but also eclipse, Zend Studio, Firefox and many other programs are running. For me the figure the task manager tells me is not nearly as important as the responsiveness. If delphi uses 500 MB of RAM, fine – as long as my PC stays responsive.

All in all, I really like this new Delphi and I already have uninstalled D7 (thus breaking FeedDaemon).

Delphi 2005

I got my hands on the demo-version of Delphi 2005 (download it here), and I actually have configured the beast already, so I have my usual environement to work on PopScan with it. These are my first impressions (I won’t talk about this File-Download-Window-Popping-Up Problem as all know it’s a nasty problem with a security-patch from Microsoft which will soon be fixed. Read about it here on Steves blog

  • It takes quite some time to start up. After removing the Delphi.NET and c# personalities (don’t need them), it starts about as fast, als my Delphi 7 did. Just a little bit slower
  • The compiler got faster, if you ask me.
  • Besides the great new features Borland is talking about, there are very nice usability-tunings everywhere which make working quite a bit easier.
  • The VCL form designer is extremely slow on my machine. Just displaying the PopScan Main Form within the designer takes nearly 10 Seconds. Delphi 7 does that instantly.
  • The debugger is slower too, which certainly has to do with the many great feature additions. I can live with that.
  • It’s extremely compatible to Delphi 7: I could install every single third party component without any problems. This is quite impressive considering Delphi 2005 is quite a rewrite.
  • While I like the new docked form designer, there’s one usability-problem with it: When you have components that use their own property editors (like Toolbar 2000), those editors are opened in their own window (understandable). Now, if you select a button in the component editor and then click the Project inspector to change a property, the Delphi Main Window will cover the property editor rendring it invisible. An easy fix would be to define the property editor always-on-top – a better fix would be integrating it somewhere in the IDE
  • Even JCLDebug could be compiled and installed (even the IDE Expert did work, though you have to manually install it)

All in all, this release of Delphi is a very great release providing the user with a ton of new features and fixes to long-standing usability problems (so long that you got used to them and now are missing them…). I have not expirienced any crashes so far (besides the one where the expat-parser of a debugged application took all the ram on my system, but I don’t blame Delphi for that), which is very nice.

Now, if only the beast could be made to run a bit faster (which will be done, I’d say, it’s the best Delphi since Delphi 2 which means quite a lot…

Thanks Borland.

PS: I know that it’s currently more in fashion to bash Borland and to whine about everything they do. And for the fourth consecutive year now I read posting about Delphi’s impending doom everywhere on the net. But consider this: Delphi still is the only RAD tool out there producing 100% native windows executables. And it still has one of the most lively communities I know of in the Windows-world. Even if Borland would kill off delphi, I’m quite certain, it will not go so easily. Not with this community.

On and speaking of killing off delphi: Seeing this great release of Delphi 2005, I am quite assured that Borland will continue supporting us.

So: Quit whining around!

Copying with MOVE? Moving with copy?

Today I came across the situation where I had to copy – using delphi – some chunck of memory from one place to another. I nevery did that before (using OOP techniques gets you around that most of the time – at least in Delphi), so I had no idea how to do it. What I knew is that in C, I’d do that with memcpy. As a convinced fan of Pascals intuitive API notation, I looked in the help for MemCopy or CopyMem. Nothing (which is strange, considering things like AllocMem actually exist).

Some googling around turned out the name of the function: it’s

procedure Move(const Source; var Dest; Count: Integer);

Move? That can’t be. Can it? I want to copy, not to move. A quick glance at the help file revealed that it’s the truth: Move actually copies…

Move copies Count bytes from Source to Dest. No range checking is performed. Move compensates for overlaps between the source and destination blocks.

Descriptive procedure names? Usually, yes. But this can only be described as way beyond the optimum ;-)

Oh… on another note: What do you think, Copy does? Copying memory? No way:

function Copy(S; Index, Count: Integer): string;

function Copy(S; Index, Count: Integer): array;

S is an expression of a string or dynamic-array type. Index and Count are integer-type expressions. Copy returns a substring or subarray containing Count characters or elements starting at S[Index]. The substring or subarray is a unique copy (that is, it does not share memory with S, although if the elements of the array are pointers or objects, these are not copied as well.)

If Index is larger than the length of S, Copy returns an empty string or array.

If Count specifies more characters or array elements than are available, only the characters or elements from S[Index] to the end of S are returned.

Yeah!. Right.

Oh and on second thought: The move-thing may have its roots in the assembler language, where MOV actually copies the data too – at least I think so. But anyway: If even C got it right, why has my beloved Pascal to fail in such an easy case?

Responding to search-strings (II)

While looking through the logfile analysis of gnegg.ch I saw that someone came to this site searching with

set ie proxy delphi

so I deceided that it’s time for anther episode of “Responding to search-strings” (the other being here). This time it’s about setting the IE’s proxy server from a delphi application.

When you do it manually, you access the proxy server settings from Tools / Internet Options / Connections in Internet Explorer. Whatever you change there, is used not only by IE, but by every application on your system using the WinInet API function InternetOpen with the flag INTERNET_OPEN_TYPE_PRECONFIG set. Additionally, many applications use the WinInet-API to get the Proxy Server settings and then use their own routines to actually connect to the server via the proxy they got before

So, if you want to change the Internet Explorer Proxy settings, you actually change it for the bigger part of the wohle system

When you go to Tools / Internet Options / Connections, you will immediately see that setting the proxy is going to be quite a task: You don’t just set one proxy server, you actually set one for LAN-Connections and one for each dialup connection that is installed on your system. Finally the proxy being used depends on the state of the radio buttons you see in the middle of the dialog because they define whether IE should even bother connecting to the LAN or just call one of the connections defined.

But it gets even more complicated: The proxy settings provided changed for each version of Internet Explorer. As always, it was an evolutionary process getting more complex in every iteration, so you will have to cope with that too.

But now to the details: While the settings are stored in the Registry, this is not the recommended way for changing them. Microsoft has created some API functions specifically for that, so you should use them as this is the only way guaranteed to be portable even for future versions of Windows.

The problem: The API is very painful to use – even more so because it is somewhat different for each version of IE (getting more complex along the proxy feature itself). Oh, and please don’t ask me how to get the version of the installed IE – that I do not know.

All is about InternetSetOption and InternetQueryOption respectivly. Both require a parameter to tell them which option you are interested in. Have a look at INTERNET_OPTION_PER_CONNECTION_OPTION (for IE5 and later), INTERNET_OPTION_PROXY and INTERNET_PER_CONN_OPTION

In the end, you will be calling InternetQueryOption quite a lot of times and change some settings with InternetSetOption, but you will soon see that it’s not actually worth it: There is always the possiblity that you have not anticipated some obscure setting a user may have which will distrub your application greatly

And additionally, changing the proxy server settings is a task for an adminitrator, not for a simple application. Before asking the question “How can I change the proxy server?”, the question should be “Do I really have to change the proxy server? Isn’t there a better way?”

Refactoring – It’s worth it

Just shortly after complaining about not having time to do some refactoring, I reached a place in my code where it was absolutely impossible to add feature x without cleaning up the mess I created three years ago. And – what’s even better: I had the time do really fix it. Cleanly

What I did was to sit down and recreate the whole module in a new Delphi project. I knew what features I wanted to have when finished and I somewhat knew the interface I had to comply to. The latter proofed inpractical, so I did some modifications to the interface itself (the thing was hacky too). Redoing the whole module took about a week (it’s about downloading stuff, exctracting and then XML-parsing it – everything in a thread while still providing feedback to the main thread), but it was absolutely worth it:

  • The code is clean. And with clean I mean so clean that adding further features will still be clean, depite not being needed as the new framework I’ve created is extremely powerful.
  • The thing is fast. Twelve times faster than the old version. I’m processing 7000 datasets in just 20 seconds now (including time needed for downloading and decompressing) which took me four minutes before.
  • The thing is more usable. Status reporting to the end user went from nearly nothing to everything the user may need. And she can now cancel the process – of course.

A task fully worth of undertaking. I’ve not been that pleased with my code for quite some time now

Refactoring – If only I’d had time

Refactoring is a cool thing to do: You go back to the drawing-board and redesign some parts of your application so that they fit better to the new requirements building up over time. Sometime you take old code and restructure it, sometime you just rewrite the functionality in question (or even the whole application, but I don’t count this as refactoring any more)

Code always has the tendency to get messy over time as new requirements arise and must be implemented on the basis of existing code. Not even the most brilliant design can save your code. It’s impossible to know what you are going to do in the future with your code.

Let’s say you have an application that is about orders. Orders with ordersets that somehow get greated and then processed. Now let’s say you create quite an usable model of your order and ordersets. Very well. It’s nice, it’s clean and it works.

And now comes the customer and over the years new features are added, let’s call it an inventory mode. You notice that these new inventory records have quite a lot in common with your orders, so you reuse them, but add some features.

Now full stop! It already happened. Why on earth are you reusing the old code and “just adding features”? That’s not the way to go. The correct solution would be to abstract away the common parts of your order and inventory records to something like TProductContainer (using Delphi naming conventions here) which has two descendants TOrder and TInventoryRecord.

But this comes at a cost: It requires time. It requires quite some steps:

  1. Think of a useful abstraction (just naming it is not easy. My TProductContainer above is stupid).
  2. Create the Interface
  3. Implement the new subclasses
  4. Change the application where appropriate (and if it’s just changing declarations, it still sucks as it’s time consuming)
  5. Test the whole thing

Now try to convince the project-manager or even your customer that implementing the required feature can be done in x days, but you’d like to do it in x*2 days because that would be cleaner. The answer would be another question like: “If you do it in x days, will it work?”. You’ll have to answer “yes”, in the end. So you will be asked “if you do it in x*2 days, will it work better than in x days?” and you’d have to answer “No” as the whole sense in cleaning up messy code is to keep it running just the same.

So, in the end those things will accumulate until it cannot be put away any longer and the refactoring has to be done no matter what, just because implementing the feature uses x days plus y days just for understanding the mess you have created over time. y being 2x or so.

The mean thing is: The longer you wait doing the inevitable, the longer you will have to fix it, so in the end, it should always be the x*2 way – if only those noneducated people would understand.