PHP Stream Filters

You know what I want? I want to append one of those nice and shiny PHP stream filters to the output stream.

I have this nice windows-application that recives a lot of XML-data that can be compressed with a very high compression factor. And as the windows application is for people with very limited bandwith, this seems to be the perfect thing to do.

You know, I CAN compress all my output already. By doing something like this:

<?php
ob_start();
echo "stuff";
$c = ob_get_clean();
echo bzcompress($c);
?>

The problem with this approach is that the data is only sent to the client once it’s assembled completely. bzip2 on the other hand is a stream compressor that is very well able to compress a stream of data and send it out as soon as a chunk is ready.

The windows client on the reciving end is certainly capable of doing that. As soon as bytes come in, it decompresses it chunk-wise and feeds it to a Expat based parser which will handle the extracted data. Now I want this to happen on the sending side aswell.

The following code does work sometimes:

<?php
  $fh = fopen('php://stdout', 'w');
  stream_filter_append($fh, 'bzip2.compress', STREAM_FILTER_WRITE, $param);
  fwrite($fh, "Stuff");
  fclose($fh);
?>

But sometimes it doesn’t and produces a incomplete bzip2-stream.

I have a certain idea of why this is happening (no sending out of data to the filter on shutdown), but I can’t prevent it. Sometimes the data is not put out which makes this method unusable.

I’m afraid to report this to bugs.php.net as I’m sure it’s something PHP was not designed for and it’ll get marked as BOGUS faster than I can spell ‘gnegg’.

So this means that the windows-client just has to wait for the data being extracted, converted to xml and compressed.

*sigh*

(thinking of it, there may be this option of outputting data to a temp-file (to which handle a filter is assigned to) and the read it out to the browser immediately afterwards. But come on, this can’t be the solution, can it?)

Update: I’ve since tracked the problem to a bug in PHP itself for which I found a fix. My assumption of writing to a temporary file could help was wrong as PHP itself does not check the return value of a bzlib function correctly and never writes out a half-full buffer on stream close. Neither to the output stream nor to a file.

Reporting Engines

There are quite many solutions available to you if you want to create printed (or PDFed) reports from your delphi application, but surprisingly, one one is really convincing.

The so called professional solutions like Crystal Reports or List&Label are very expensive, require a large runtime to be deployed and may even require you as the developer to pay royalties per delivered client

So for my particular problem, the only solution was to use a VCL based engine that can be compiled into the exe and does not need any additional components to be deployed.

Years ago, I was told to use ReportBuilder and maybe people were even right there: Quick Reports (as in included into delphi back then) had and still has a very bad reputation, and RB was the only other VCL based product available

RB has some problems though. Lacking Delphi 2006 support, limited report templates, field formating on the database access layer and last but not least: A nasty bug preventing barcodes from being correctly rendered to PDF-Files.

Then, I had a look at Fast Report and I tell you: That piece of software is perfect!

Granted, switching will come with a bit of work, though the paradigms of both engines kinda match. But once you’ve done the stupid rebuilding of the old templates, you’ll notice how wonderful Fast Report actually is. And you will notice immediately as it’s very, very intuitive to use – compared to RB. Things that required custom programming or even a little hacking here and ther in RB just work in FR. And they even work without forcing you to read through lots of documentation in advance

And everything – just everything is in the report template. Number formats, even small scripts for changing the report in subtle ways while it’s being printed. Just perfect for what I was doing.

So, if you are looking for a nice, powerful really easy to use reporting enginet that can be fully compiled into your EXE, you should really go with FR. It even costs less than RB.

Flattening Arrow Code

In an equally named article, the excellent (yes. Really. This is one of the blogs you HAVE to subscribe to) Coding Horror blog talks about flattening out deeply stacked IF-clauses in your code.

I so agree with the guy, though there seem to be two opinions in the matter of the points 1 and 4 in the list the article provides:

Replace conditions with guard clauses. This code..

Many people disagree. Sometimes because they say that Exceptions are a bad thing (I don’t get that either) and sometimes because they says that a function should only have one return point

Always opportunistically return as soon as possible from the function. Once your work is done, get the heck out of there! This isn’t always possible — you might have resources you need to clean up. But whatever you do, you have to abandon the ill-conceived idea that there should only be one exit point at the bottom of the function.

I once had to work with code a intern has written for us. It was exactly written as Coding Horror tells you not to. It was PHP code and all of it basically took place in a biiig else-clause around the whole page, with a structure like this:

if (!$authenticated){
   die('not authenticated');
else{
  // 1000 more lines of code, equally structured
}

This is a pain to read, understand and modify.

To read because the thing get’s incrediby wide requiring you to scroll horizontally, to understand because you sometimes find an }else{ not having the slightest idea where it belongs to, requiring you to scroll upwards for half a file to see the condition and to modify because PHP’s parser is inherently bad at reporting the exact position missing or spurious braces, which is bound to happen when you extend the beast.

But back to the quote: I talked to that intern about his code style (there were other things) and he mostly agreed, but he refused to change those deeply stacked IF’s. “A function must only have one single point of return. Everything else is bad design“, he told me.

Point is. I kinda agree. Multiple exit points can make it hard to understand the workings of a function. But if it’s a single, well definded condition that makes the function unable to continue or if the function somehow gets its result way early (like if it’s able to read the data from a cache of some kind), IMHO there’s nothing wrong with just stopping to work. That’s easy to read and understand and certianly does not have above problems.

And of course every function should be short enough to fit on one screen, so scrolling is never neccessary and it’s always obvious where that }else{ belongs to – at least without making you scroll.

Personally, I write code exactly as it is suggested in that article. And I try to keep my functions short. Like this, it’s very easy to understand the code (most of the time) and thus to extend it. Even by third parties.

Christoph, do you agree? And: No, I’m not talking about that sort-by-material-group-thing. That IS unclean. I know that (and so do you now *evilgrin*)

mp3act

When you have a home server, sooner or later your coworkers and friends (and if all is well even both in one person ;-) ) will want to have access to your library

Cablecom, my ISP, has this nice 6000/600 service, so there’s plenty of upstream for others to use in principle. And you know: Here in Switzerland, the private copy among friends is still legal.

Well, last sunday it was time again. Richard wanted access to my large collection of audiobooks and if you know me (and you do as a reader of this blog), you’ll know that I can’t just give him those files on a DVD-R or something. No. A webbased mp3-library had to be found.

Last few times, I used Apache::MP3, but that grew kinda old on me. You know: It’s a perl module and my home server does not have mod_perl installed. And I’m running Apache 2 for which Apache::MP3 is not ported yet AFAIK. And finally, I’m far more comfortable with PHP, so I wanted something written in that language so I could make a patch or two on my own.

I found mp[3]actmp3act which is written in PHP and provides a very, very nice AJAX based interface. Granted. It breaks the back-button, but everything else is very well done

And it’s fast. Very fast.

Richard liked it and Christoph is currently trying to install it on his windows server, not as successful as he wants to be. mp3act is quite Unix-Only currently.

The project is in an early state of developement and certainly has a rough end here and there, but in the end, it’s very well done, serves its need and is even easily modifiable (for me). Nice.

Evening leisure

You know what’s one of the greatest things to happen on the evening of a workday?

You come home, select ‘Play Movie’ on your Harmony Remote and watch the speedrun that downloaded itself while you were on the office. (using Windows Media Center. Sorry, but that has the advantage of just working).

This is the future. This is like watching TV with the full control of the program.

Why should I watch TV only to see programs I don’t want to for most of the time? Why should I cope with advertisments every 10 minutes? Why should I be forced to watch all the movies in the german synchronized version?

Not with me. This is what the internet is for. This is why I’m running a linux server at home.

Asterisk Extended

Playing around with Asterisk, it was inevitable for me to stumble upon AGI.

AGI is a protocol quite like CGI which allows third party applications to be plugged into asterisk, giving them full control over the call handling. That way, even non-asterisk-developers are able to write interesting telephony applications.

One thing I always wanted to do is to set the CallerID on incoming calls. Some numbers are stored in our customer database. There is no reason not to show the customer names on the phones displays instead of only the number.

The snom phones do have a little addressbook, but it’s very limited in both amount of memory and featureset, so it was clear that I’ll have to set the CallerID via Asterisk (SIP allows for transmission of a caller-id. And so does AGI)

Additionally, I thought, it would be very nice to use the swiss phone book at tel.search.ch or even the non-free ETV to try and guess numbers not already in our database.

That scenario is exactly what AGI is for.

As AGI works like CGI, it creates a new process for every call to AGI applications. This is not an option if you want to use interpreted languages. Well. it *is* an option considering our low amount of calls we are getting per time unit, but still. I don’t like to deploy solutions with obvious drawbacks.

Besides, launching a PHP interpreter (I’d have written this in PHP) can easily take a second or so – not acceptable if you want the AGI script to be mandatory on each call. Think of it. You don’t want the caller to wait for your application.

The solution to this is FastAGI, which works like FastCGI: A server keeps running and answers AGI-requests. Like this, you start the interpreter once and just serve the calls in the future. You save the startup-time of the interpreter.

Even better: It allows to run the AGI applications on a different machine than the PBX. This is good because you want the PBX to have as much CPU time slices as possible.

Unfortunately, this made PHP quite unsuitable for me: While it is possible to write a socket server in PHP (ext/posix does exist), I never managed to get it to work as I wanted to. It was slow, unstable and created zombies.

Then I found RAGI which was even better. For quite some time now, I have been looking for an excuse to do something with Ruby on Rails. With RAGI, I finally got it.

Getting the sample provided with RAGI to work was very easy (look at the README file). And reading through that sample file, I was very pleased to see the simplicity of writing a AGI-Application in Ruby (RAGI uses FastAGI, of course).

Now I can finally start hacking away in Rails to create my internal-customer-database / external-phone-lookup application (with some nice caching/timeout handling) to finally show the name behind the calling phone number on the displays of our SNOM phones.

Of course I’m going to provide the sourcecode here once I’m done.

Strangest JavaScript error ever

Let’s say you create some very nice AJAX-stuff for a web project of yours. With nice I mean: Not breaking the back-button where its functionality is needed, not doing something that works better without AJAX, and doing it while providing lots of useful visual feedback.

Let’s further assume that the thing works like a charm in every browser out there (not counting Netscape 4.x and IE in all versions – those are no browsers).

And then, IE throws this at you:

Unknown Runtime Error

Needless to say that the HTML output in question had a line count not even close to 370, so finding this thing easily was out of the question.

The solution: IE is unable to write to innerHTML of a TBODY element. But instead of providing an useful error message or even a link to the source with the line in question already highlighted (that’s what Firefox would do), it just bails out with completely useless error information.

*sigh*

(btw: That mix of fonts in the details section of the error message is just another indication of IEs great code quality)

The myth of XCOPY deployment

Since the advent of .NET, everyone is talking about XCOPY deployment.

XCOPY deployment means that the applications are distributabe without a setup routine. Just copy the file(s) where you want them and that’s it.

We are being told that this is much easier and safer than the previous non-.NET approaches which – as they continue – always required a setup program.

The problem with those statements is that they are all false.

First the ease of use: Think of it: Say you want to install Cropper (which made me write this entry. I found that screenshot utility via flow|state). What you are getting is a ZIP-File, containing 5 files and a folder (containing another 6 files). Nearly all the files are needed for the application to run.

XCOPY deployment in this case means: Create a folder somewhere (Windows guidelines advocate you create that in c:Program Files which is a folder windows does not want you to mess with and per default does not display its contents) and copy over all those files, being aware not to forget a file or the folder in the archive.

But it does not end there: As you have to launch the application and going all the way through those folders, you will want to have a shortcut in the start menu or on the desktop. With this new and “better” method of deployment, you’ll have to do that yourself.

This is a tedious task involving lots of clicks and browsing. An unexperienced user may not be able to do this at all.

What an unexperienced user will want to do is to copy that application right to the desktop. But in this case this does not work well as the whole application consits of multiple interdependant parts. Copying only the .EXE will break the thing.

Compare this with Mac OS X

In Mac OS X, application also consist of multiple parts. But the shell is built with XCOPY deployment (not called like this, of course. As a matter of fact, it does not have a name at all) in mind: In OS X, you can create a special kind of folder which is a folder only on the file system. The shell displays it to the user as a single file – the application.

Whenever you move that “file” around, OS X will move the whole folder. When you double click the “file”, the application will launch (the binary is a file somewhere in this special folder. The shell is intelligent enough to find and launch that). When you delete it, the shell will delete the folder including it’s contents (of course).

This makes XCOPY deployment possible as the applications become one piece. You want it on the desktop? Drag it there. In the Application folder (without warnings about not being allowed to mess with its contents, btw)? Drag it there? On an USB-Stick? Drag it there.

Well. There’s one other thing: It’s the users data and the applications data. Most of the applications will be used to create data with them. And all application somehow create their own data (for saving things like the window state or position for example). As all modern OSes are multiuser ones where a user does not necessarily have to have write access everywhere, there’s the concept of the home directory. That one is yours. You may store whatever you want in there.

So naturally, this is the place where the applications should store data to0.

User data goes to a specific folder of the users choice. Per default, applications should suggest some Documents-Folder. Like “My Documents” in Windows or “Documents” in Mac OS. In most of the cases you don’t want to delete that on uninstall.

Application settings are in Windows stored in the Registry (under HKEY_CURRENT_USER – a hive that belongs to the current user like his home folder does. And actually, the file behind that is stored in the home folder aswell (USER.DAT)) or in the Application Data folder below the users home folder.

Mac OS X Applications are advised to use the Preferences-Folder inside the Library Folder inside the users home directory<./p>

Now. Application data is something you want to remove when you uninstall the application (which means deleting a bunch of files in Windows or one “File” in Mac OS). Application data is created by the application, for the application. No need to keep that.

In Mac OS, you can do that by going into the folder I’ve described above and delete the files – mostly named after your application. There are no warnings, no questions, no nothing. Just delete.

In Windows, editing the registry is off-limits for end-users and very, very tedious to do for experienced users (due to the suboptimal interface of regedit and because the whole thing is just too large to navigate it easily), so you generally let the stuff stick there. Deleting the Application Data in the same-named folder is also impossible for the end user: That folder is hidden by default. Explorer does not display it. And it’s hard as hell to find, as you have to manually navigate into your home directory – there’s not easy GUI-access to that. So that sticks too.

All in all, this means that windows is – at least in its current state – very unsuited for XCOPY deployment:

  • It does not help at keeping together things that must be together
  • Its complex file system structure makes it hard to copy the application where windows wants it to be
  • Manually creating shortcuts is not feasible for an unexperienced user
  • Uninstallation of Application Data is impossible

So, we found out that XCOPY deployment is not easy at all. Now let’s find out how it’s not true that only .NET enabled you to do this.

Ever since there is Delphi, there theoretically is XCOPY deployment.

Delphi is very good at creating self-contained executables.

With delphi it’s a breeze to create one single .EXE containing all the functionality you need. That one single .EXE can be moved around as a whole (obviously), can be deleted, can even be put right into the start menu (if you want that). It can even create the start menu shourcuts, delete application data – basically configure and clean itself

It can even uninstall itself (embed an uninstaller, launch that with CreateProcess and set the flag to delete the .exe after it ran). And it can contain all it’s image and video and sound data it needs.

Just because nobody did it does not mean it was not possible.

Face it: Windows users are used to fancy installers. Windows users are not at all used to dragging and dropping an application somewhere. And currently Windows users are not even able to do so as dragging and dropping will break the application.

OS X and now Linux allow for true XCOPY deployment of desktop applications.

Well, you say… then maybe XCOPY deployment is just for those fancy ASP.NET web applications?

Maybe. But after XCOPY you need to configure your webserver – at least create a virtual directory or host. A good installer could do that for you – if you want it to.

Microsoft too has seen that this XCOPY thingie is not as great as everyone expected, so they added the new “One-Click Install” technology, which is not much more than a brushed-up MSI file which does a old-fashioned install.

To really make XCOPY deployment a reality (btw, I’m a big fan of depolying software like this), there must be some changes within Windows itself. Microsoft, copy that application bundle feature from OSX. That one works really, really good.

Btw: Am I the only one that thinks “XCOPY deployment” is a very bad term? What is XCOPY? Who the hell still uses XCOPY these days? And when we are using the command line: COPY would be enough.

PostgreSQL scales

Via zillablog, I was notified of FeedLounge switching to PostgreSQL

FeedLounge is just another in a serious of webbased services switching their RDBMS away from MySQL.

For one thing, it’s the features that’s driving this. Postgres just has more features and sometimes, you need to have them. Triggers? Views? Until very recently, those features were not available with MySQL.

And when they switch, they notice another thing: PostgreSQL scales very well.

While everyone says that MySQL is optimized for speed and that there’s no database system as fast as MySQL, this is only true for small setups.

In small setups MySQL scores with its ease of use and administration. But as soon as you want more (more features, more users accessing), you will run into MySQL’s limitations and – even more important: MySQL will slow down, it will use lots of RAM and disk space and it even will begin to corrupt it’s tables (a thing a RDBMS should never ever do – not even in case of broken hardware though that’s unavoidable).

PostgreSQL does not have these flaws. It may be a little bit slower under low load, but it speed and reliability scales with its users.

PostgreSQL scales.

Opera Mini

Today, Opera released Opera Mini, a browser written in Java for all the Java capable mobile phones out there

By the use of a special proxy server, they manage to both minimize the traffic a usual browsing session generates and to keep the application as performant as possible.

When I tried to download the application via WAP, all I got was an ‘Invalid Jad Request’-Error (wahtever that meant), but with some sneakyness, I found the download URL for the jar file none the less (the linked version is the high-memory version. There’s another for less advanced phones)

I copied the file over to my K750i via bluetooth which is cheaper than downloading it and it also had the advantage of actually working.

The browser is very nice. While it uses quite some time to finally launch, surfing is very quick. And the very good font rendering (of course operas small screen rendering is active aswell) makes this a pleasure to use and is the first justification why a phone should have a screen resolution as big as the K750i’s

And the most interesting thing: Opera uses the default internet GPRS profile. Not the WAP one. This makes surfing via opera a whole lot cheaper than via the built-in wap browser of my phone.

Congratulations, Opera. This rules!

(and thanks, Christoph, for pointing it out to me)