Uncategorized- gnegg

Jabber: Even file transfers work

Transferring large amounts of data is a problem to overcome for all the IM networks.

You see: Usually you don’t transfer files over the central IM server because it will use the IM providers bandwith which is better used sending out those small text messages. That’s why files are usually transferred in a P2P fashion.

The problem here is that usually there’s a NATing router or even a firewall working at one, or most of the time both ends of the communication. This usually forbids the making of a direct connection – at least without some serious trickery (warning: PDF link) going on

This is why I never expect a file transfer to work – even when using the native IM client.

And today, a guy sent me some image. Via Jabber. Via PyMSN-t.

Just when I wanted to write that it’s never going to work, I watched the bytes flow in.

I’m still unable to belive it: Wildfire/PyAIM-t/Psi succeeded in making a direct P2P file transfer happen from a friend in MSN to my PC running a jabber client.

pilif’s guide to jabber

I have been talking about jabber before on this blog (here and here). And each time the euphory was put back by one or another malfunctioning element. You know: I would never use a third party service for a fun-project if I can be my own provider aswell.

And being ones own provider is one of the biggest advantages of using jabber. Over the years (the experiment began in the winter of 2002/2003), I have been running a jabber-server here and then.

First it was after reading the jabber book (very interesting read) which ended with me installing a jabber server with transports for aim (aim-t) and icq (jit? I don’t remember). Installing jabber on debian was quite hard because there was no (useable) package (too old – as usual), but once I got it to work, it was fun.

The problems began with the advent of iChat and .Mac: aim-t was not able to detect the presence of .Mac users and thus I was unable to talk with them via the jabber server. Unfortunately, Richard is one of those .Mac guys, so I had to find another solution to talk with him.

For long, the only solution was original AIM itself, but in the fall of 2003 Trillian 2.0 was released with AIM/iChat-Support. This was the demise of my jabber-solution.

While I’ve always liked to have the whole client-configuration, contactlist and whatnot stored on the server, the advantage of actually being able to chat with richard made me switch to trillian and thus even pay for a IM solution – regardless of the many free alternatives.

Remember: At the time, Trillian was the only AIM client capable of talking with the .Mac guys. The original AOL client excluded of course, but who wants to be running a ton of IM clients at the same time (most of my buddies were on ICQ which was not compatible with AIM then)? And who wants to cope with advertising all over the place?

After that, I was keeping Trillian, where I used a Jabber-Plugin to still be connected to the Jabber-Server (which was completely pointless as I was and am the only user on that server and no-one I know is using jabber (any more)).

Then the Debian installation went away and Gentoo came. I’ve written about the pleasant experience with jabber on Gentoo. Still: As I was the only user, that jabber installation lived not very long either (I’ve never come around to have jabberd start automatically, so after a power outage, the service was gone. And I did not even notice *sigh*)

Only last week, my interest came back when I’ve seen that iChat provides jabber-support. Don’t ask me why. I just wanted to check the progress of the various projects once more.

I immediately noticed ejabberd which is what’s currently powering jabber.org

On their site I read about PyAIM-t. Finally a replacement for that old aim-t without .Mac support. And I checked the readme-file: Yes. PyAIM-t uses the oscar protocol which is what’s needed to get the presence info of those .Mac users

Installing ejabberd failed miserably though.

For one, the gentoo ebuilds are outdated(!) and I never managed to install the whole thing in a way that the command-line administration tool was able to access the (working) server. I admit: I’ve not invested nearly enough time to understand that erlang-thing. But why should I? It’s a for-fun-only project after all.

Via the installation instructions of that PyAIM-t transport I found out about Wildfire. Wildfire is GPL, but backed by a company with strict commercial interest. A bit like the MySQL-thing. For me it did not matter as I did not want to integrate the thing into a commercial solution. Heck! I did not even want to use the unmidified thing commercionally.

Installing wildfire was – even though it required Java – easy to do. Especially as Gentoo provides a current ebuild (hard masked though because Wildfire depends on Java 1.5). Getting the thing to work was a matter of emerge wildfire, /etc/init.d/wildfire start and rc-update add wildfire default as it’s the norm with Gentoo.

Then I read the documentation to learn how to add a SSL certificate (signed by our company’s CA) which was a bit hairy (note: the web interface does not work. if you use the web interface, you corrupt the certificate store).

Installing the transports (PyAIM-t, PyMSN-t, PyICQ-t) was a matter of untaring the archive, entering the right connections settings I’ve configured in wildfire and launching the scripts. Easy enough.

Then I went to select the right client (on windows this time around): I’ve already known jajc, new for me were Exodus and Psi and Pandion. I could have kept trillian, but the nicest thing about the jabber clients is that they can store their settings on the jabber server. Trillian can’t do that. So if I’m working on a new machine, I have to reconfigure Trillian where every pure jabber client will just fetch the settings from the server. Also, I wanted to have an OpenSource solution.

Now, that client-thing is a very subjective thing as functionality-wise, all three are identical – at least concerning the jabber-featureset (I’m not counting addons like RSS readers or whatever).

So here’s my subjective review:

Jajc is not open source, provides a ton of settings to tweak (too many for my taste) and does not look that attractive (UI-wise).

Exodus seemingly does not provide a way to make the different contacts on the list look differently depending on which transport they use and the chat window is very, very limited in featureset and looks. If you dislike good looking programs with tons of unimportant settings to tweak, go for Exodus (this was not meant with disrespect. I was one of those users myself).

What remains is Pandion and Psi.

What I like about Pandion is the nice contact list display. You know: With avatar display (which works cross-im-network with those python transports!) I also like the nice looking chat window. What I dislike is the limited amount of settings to tweak (hehe… It’s hard to make it right for me. Isn’t it?).

I like the space-economic, yet still nice looking contact list in PSI. I also like the design of the chat window and the count of settings to tweak.

Personally, I can’t decide between Psi and Pandion, so I’m running both of them currently. One day I will sure as hell know which of them I want to use.

So finally I’m up to speed with jabber again: Nice opensource client, working server and – finally – the .Mac AIM-Users on my contact list, while even able to chat with them.

So, you may ask: Why go through all this? Why not just stick with trillian?

Easy!

A pure Open Source solution. No strange subscription model
As settings are stored on the server, equal configuration wherever I am.
Jabber has inherent support for multiple connections with the same account.
Jabber works on many mobile phones. That way I can IM with my mobile phone while not being locked into a specific service
It was fun to set up!

*happy*

A praise to VMWare Server

This is putty, showing the output of top on one of our servers. You may see that there are three processes running which are obviously VMWare related.

What’s running there is their new VMWare Server. Here’s a screenshot of the web-interface which gives an overview over all running virtual machines and allowing to attach a remote console to anyone of them:

As you can see, that server (which is not a very top-notch one) has more than enough capacity to do the work for three servers: A gentoo test machine and a Windows 2003 Server machine doing some reporting work.

Even under high load on the host machine or the two virtual machines, the whole system remains stable and responsive. And there’s so much work needed to even get the VM’s to high load, so that this configuration could even be used in production right now.

Well… what’s so great about this, you might ask.

Running production servers in virtual machines has some very nice advantages:

It’s hardware independant. You need more processing power? More ram? Just copy the machine to a new machine. No downtime, no reinstallation.
Need to move your servers to a new location? Easy. Just move one or two machines instead for five or more.
It’s much easier to administer. Kernel update with the system not booting any more? typing “shutdown -h” instead of “shutdown -r” (both happened to me)? Well… just attach the remote console. No visiting the housing center anymore
Cost advantage. The host-server you see is not one of the largest ones ever. Still it’s able to handle real-world-traffic for three servers and we still have reserve for at least two more virtual machines. Why buy expensive hardware?
Set up new machines in no time: Just copy over the template VM-folder and you’re done.

And in case you wonder about the performance? Well, the VM’s don’t feel the slightest bit slower than the host (I’ve not benchmarked anything yet though).

We’re currently testing this to put a configuration like this into real production use, but what I’ve seen so far looks very, very promising.

Even though I don’t think we’re going to need support for this (it’s really straight-forward and stable), I’m more than willing to pay for a fine product like this one (the basic product will be free, while you pay for the support).

Now, please add a native 64bit edition ;-)

Powerbook runs XP

It’s done. The contest has been won.

Windows XP is installable on a MacBook Pro and it even boots from there after the installation.

This solves a big problem I’m having: In the office, I’m using a 30″ cinema display connected to a Windows XP box which I had to custom-build because no out-of-the box systems have graphics cards with dual-link DVI ports which is needed for all resolutions bigger than 1920×1080 (technically, the limit is even 1600×1200, but 1920 still works somehow).

Now, custom-built boxes are nice, but they have two flaws: The first is the noise. Even though I got it to run very quiet despite the kick-ass graphics card I had to put in, it’s louder than your average laptop (it was even louder before I unplugged the chipset fan. As it’s working stable since more than a year now, I guess it doesn’t matter). The second problem is the problem of data redundancy:

If you prefer and have the ability, like I do, to both work at home and in the office, you are dependant of having current data at both places. Version control systems help a lot here, but they don’t solve all the problems (unfinished revisions which I don’t like to commit and binary files). In the end, the only way to have your data where you need it is to maintain it only at one place at a time.

This is the main advantage I’m seeing in laptops (beside the quietness).

Looking at the current laptop pc market, it’s even worse in respect of dual-link DVI outputs: Usually, you don’t even get single-link DVI. And custom-building one is no option unfortunately. There are some barebones, but what comes out of such an operation is a loud, badly manufactured “thing” with short battery life.

So, hardware-wise, powerbooks were always perfect because all of them have my direly needed dual-link DVI port.

The problem was the software. I’m dependant on Delphi for my daily work. Even if days can pass without me actually using it, it still happens that I need it. And when I do, it must be quick.

The other thing is multimedia. No matter what you are now going to tell me: Nothing on the Mac matches the perfect architecture of DirectShow which allows media players and codecs to be developped independently. Core Media Player with the right codec pack is unbeatable in performance, usability (at least for me) and versatility. Sorry Quicktime (only quicktime and maybe DivX). Sorry MPlayer (always uses 100% CPU, awkward GUI). You just don’t beat that.

Also multimedia related is my passion for speedruns and superplay movies in particular. For the latter ones, you need the emulator and the original ROM and especially the emulators (some of which are patched for the movies to work) don’t run on the Mac Platform and if they do, they only do with some limitations (like pausing the emulation stopping the playback of the movie in Snes9x).

If I want a single computer to work both at home and in the office, it must do both: Provide an environement to work with and an environement to play with.

OS X allows me to do some developement (TextMate comes to mind), but not all. It allows me to do some multimedia (XVid works sometimes), but not all. Thus, OS X is currently not a solution for me. At least not the single one (I’d love to work in TextMate for PHP and Ruby, eclipse for Java, but I can’t do Delphi or Windows CE).

So what I need is Windows (where the software does everything I need) on a Macintosh Laptop (where the hardware does everything I need).

Up until today, this was not possible.

Now it is. This wonderous hack (which is not completely disclosed yet, but I have a very good idea how it works) solves my problem in allowing me to combine the optimal software (for me. I know lots of people who can be perfectly happy with OS X) with the optimal hardware (for me).

Needless to say that I’ve ordered a MacBook Pro at our hardware distributor. They even had 13 on stock, so I’ll be getting mine tomorrow.

I hope the hack gets disclosed shortly, so I can do that nice dual-boot configuration :-)

Or maybe Virtual PC just works good enough for delphi and the speedruns…

Update: A howto with needed tools is now ready to be donwnloaded

Sure. Just dump your trash right here!

Boy was I pissed when I read my mail today:

Dear Spammer. What do you think to get out of this? All links your post will be masked, so no page rank for you. And I will almost certainly not overlook something like this (400 comments in one night), so no chance in it persisting either.

I’m sick of cleaning up after you guys. Dump your trash somewhere else. /dev/null sounds like a nice alternative.

Oh, and MT: Why did your Spam filter not catch this?

PHP Stream Filters

You know what I want? I want to append one of those nice and shiny PHP stream filters to the output stream.

I have this nice windows-application that recives a lot of XML-data that can be compressed with a very high compression factor. And as the windows application is for people with very limited bandwith, this seems to be the perfect thing to do.

You know, I CAN compress all my output already. By doing something like this:

<?php
ob_start();
echo "stuff";
$c = ob_get_clean();
echo bzcompress($c);
?>

The problem with this approach is that the data is only sent to the client once it’s assembled completely. bzip2 on the other hand is a stream compressor that is very well able to compress a stream of data and send it out as soon as a chunk is ready.

The windows client on the reciving end is certainly capable of doing that. As soon as bytes come in, it decompresses it chunk-wise and feeds it to a Expat based parser which will handle the extracted data. Now I want this to happen on the sending side aswell.

The following code does work sometimes:

<?php
  $fh = fopen('php://stdout', 'w');
  stream_filter_append($fh, 'bzip2.compress', STREAM_FILTER_WRITE, $param);
  fwrite($fh, "Stuff");
  fclose($fh);
?>

But sometimes it doesn’t and produces a incomplete bzip2-stream.

I have a certain idea of why this is happening (no sending out of data to the filter on shutdown), but I can’t prevent it. Sometimes the data is not put out which makes this method unusable.

I’m afraid to report this to bugs.php.net as I’m sure it’s something PHP was not designed for and it’ll get marked as BOGUS faster than I can spell ‘gnegg’.

So this means that the windows-client just has to wait for the data being extracted, converted to xml and compressed.

*sigh*

(thinking of it, there may be this option of outputting data to a temp-file (to which handle a filter is assigned to) and the read it out to the browser immediately afterwards. But come on, this can’t be the solution, can it?)

Update: I’ve since tracked the problem to a bug in PHP itself for which I found a fix. My assumption of writing to a temporary file could help was wrong as PHP itself does not check the return value of a bzlib function correctly and never writes out a half-full buffer on stream close. Neither to the output stream nor to a file.

Reporting Engines

There are quite many solutions available to you if you want to create printed (or PDFed) reports from your delphi application, but surprisingly, one one is really convincing.

The so called professional solutions like Crystal Reports or List&Label are very expensive, require a large runtime to be deployed and may even require you as the developer to pay royalties per delivered client

So for my particular problem, the only solution was to use a VCL based engine that can be compiled into the exe and does not need any additional components to be deployed.

Years ago, I was told to use ReportBuilder and maybe people were even right there: Quick Reports (as in included into delphi back then) had and still has a very bad reputation, and RB was the only other VCL based product available

RB has some problems though. Lacking Delphi 2006 support, limited report templates, field formating on the database access layer and last but not least: A nasty bug preventing barcodes from being correctly rendered to PDF-Files.

Then, I had a look at Fast Report and I tell you: That piece of software is perfect!

Granted, switching will come with a bit of work, though the paradigms of both engines kinda match. But once you’ve done the stupid rebuilding of the old templates, you’ll notice how wonderful Fast Report actually is. And you will notice immediately as it’s very, very intuitive to use – compared to RB. Things that required custom programming or even a little hacking here and ther in RB just work in FR. And they even work without forcing you to read through lots of documentation in advance

And everything – just everything is in the report template. Number formats, even small scripts for changing the report in subtle ways while it’s being printed. Just perfect for what I was doing.

So, if you are looking for a nice, powerful really easy to use reporting enginet that can be fully compiled into your EXE, you should really go with FR. It even costs less than RB.

mp3act

When you have a home server, sooner or later your coworkers and friends (and if all is well even both in one person ;-) ) will want to have access to your library

Cablecom, my ISP, has this nice 6000/600 service, so there’s plenty of upstream for others to use in principle. And you know: Here in Switzerland, the private copy among friends is still legal.

Well, last sunday it was time again. Richard wanted access to my large collection of audiobooks and if you know me (and you do as a reader of this blog), you’ll know that I can’t just give him those files on a DVD-R or something. No. A webbased mp3-library had to be found.

Last few times, I used Apache::MP3, but that grew kinda old on me. You know: It’s a perl module and my home server does not have mod_perl installed. And I’m running Apache 2 for which Apache::MP3 is not ported yet AFAIK. And finally, I’m far more comfortable with PHP, so I wanted something written in that language so I could make a patch or two on my own.

I found mp[3]actmp3act which is written in PHP and provides a very, very nice AJAX based interface. Granted. It breaks the back-button, but everything else is very well done

And it’s fast. Very fast.

Richard liked it and Christoph is currently trying to install it on his windows server, not as successful as he wants to be. mp3act is quite Unix-Only currently.

The project is in an early state of developement and certainly has a rough end here and there, but in the end, it’s very well done, serves its need and is even easily modifiable (for me). Nice.

Flattening Arrow Code

In an equally named article, the excellent (yes. Really. This is one of the blogs you HAVE to subscribe to) Coding Horror blog talks about flattening out deeply stacked IF-clauses in your code.

I so agree with the guy, though there seem to be two opinions in the matter of the points 1 and 4 in the list the article provides:

Replace conditions with guard clauses. This code..

Many people disagree. Sometimes because they say that Exceptions are a bad thing (I don’t get that either) and sometimes because they says that a function should only have one return point

Always opportunistically return as soon as possible from the function. Once your work is done, get the heck out of there! This isn’t always possible — you might have resources you need to clean up. But whatever you do, you have to abandon the ill-conceived idea that there should only be one exit point at the bottom of the function.

I once had to work with code a intern has written for us. It was exactly written as Coding Horror tells you not to. It was PHP code and all of it basically took place in a biiig else-clause around the whole page, with a structure like this:

if (!$authenticated){
   die('not authenticated');
else{
  // 1000 more lines of code, equally structured
}

This is a pain to read, understand and modify.

To read because the thing get’s incrediby wide requiring you to scroll horizontally, to understand because you sometimes find an }else{ not having the slightest idea where it belongs to, requiring you to scroll upwards for half a file to see the condition and to modify because PHP’s parser is inherently bad at reporting the exact position missing or spurious braces, which is bound to happen when you extend the beast.

But back to the quote: I talked to that intern about his code style (there were other things) and he mostly agreed, but he refused to change those deeply stacked IF’s. “A function must only have one single point of return. Everything else is bad design“, he told me.

Point is. I kinda agree. Multiple exit points can make it hard to understand the workings of a function. But if it’s a single, well definded condition that makes the function unable to continue or if the function somehow gets its result way early (like if it’s able to read the data from a cache of some kind), IMHO there’s nothing wrong with just stopping to work. That’s easy to read and understand and certianly does not have above problems.

And of course every function should be short enough to fit on one screen, so scrolling is never neccessary and it’s always obvious where that }else{ belongs to – at least without making you scroll.

Personally, I write code exactly as it is suggested in that article. And I try to keep my functions short. Like this, it’s very easy to understand the code (most of the time) and thus to extend it. Even by third parties.

Christoph, do you agree? And: No, I’m not talking about that sort-by-material-group-thing. That IS unclean. I know that (and so do you now *evilgrin*)

Evening leisure

You know what’s one of the greatest things to happen on the evening of a workday?

You come home, select ‘Play Movie’ on your Harmony Remote and watch the speedrun that downloaded itself while you were on the office. (using Windows Media Center. Sorry, but that has the advantage of just working).

This is the future. This is like watching TV with the full control of the program.

Why should I watch TV only to see programs I don’t want to for most of the time? Why should I cope with advertisments every 10 minutes? Why should I be forced to watch all the movies in the german synchronized version?

Not with me. This is what the internet is for. This is why I’m running a linux server at home.