More disk space needed

Can somebody explain me, why my Mac OS X needs 4 TB of diskspace to encrypt my home directory which currently is about 15 GB in size?

Before I got this message, it wanted me to free another 1KB, btw. When I did that and retried, this message popped up. Unfortunately, I can’t reproduce that other message though.

Computers under my command (2): marle

While everyone keeps calling her Marle, she is actually the princess Nadia of the Kingdom of Guardia in what many people are calling the best console RPG ever made, Chrono Trigger

Chrono Trigger was one of the last RPGs Squaresoft ever did for the SNES and it’s special in many ways: Excellent Music (by Yasunori Mitsuda), excellent graphics, smooth game play, really nice story and: Excellently done characters.

Robo, Frog, Lucca, Marle, Crono, Magus and Ayla – every one of them has its very own style and story. Aside from Crono which is quite the ordinary guy, every one of them is special in its own kind.

The server marle is special on its own way too.

It’s not as outstanding as shion, but it’s special in its own way: It was the first 64Bit machine running a 64Bit OS I’ve ever deployed.

The OS was Gentoo linux (as usual) and the machine itself is some IBM xSeries machine equipped with a 3Ghz Xeon processor and 2GB of RAM, so basically nothing you need 64 Bit for.

It still was an interesting experiment to get the machine to work with a 64 Bit OS, though all that went completely uneventful.

Ever since deployed, marle is running at a customers site without crashes or other problems.

marle ~ # uptime
     11:56:13 up 265 days, 44 min,  2 users,  load average: 0.00, 0.01, 0.00

Not much happening there currently I guess. Also, it’s amazing how quickly time passes – installing that machine feels like it was only yesterday.

Computers under my command – Issue 1: shion

Picture of the "real" Shion Uzuki

After yesterdays fun with one of my servers, I thought I could maybe blog about some of them – especially when they are kind of “special” to me.

Of course, the first machine I’m looking at is my PowerPC Mac mini which I called “Shion”, after the girl Shion Uzuki of the Xenosaga trilogy.

I don’t really have a very advanced naming scheme for my servers, but the important ones get names I tend to remember.

First it was people from Lord of the Rings (with Windows servers having names belonging to the evil people). Then, after I ran out of names, it was places in LotR and after I run out of those too, I began naming (important) servers after girls in console RPGs.

And of all the names, I guess shion is a very fitting name for a server. In the game, Shion is a robotics engineer and the inventor of that android called KOS-MOS.

And in my network, shion has a special place:

I initially bought the machine to run a SlimServer on it as my previous NSLU solution was not really usable as hardware for the heavy, perl-based slim server.

After I replaced the slim-server, I obviously installed a samba server on shion to serve the non-music files as-well. Back then, I only had one external drive connected to the server.

Next thing to get installed was OpenVPN which I used for quite a nice configuration allowing me transparent access from and to the office.

Shortly after that, I finally found a USB ethernet adapter which made shion replace my ZyAir access point. I also had to buy a USB hub back then and I decided to use the remaining two ports of that to plug in additional hard drives, leading to shion’s current disk space capacity of roughly 1.2 TB.

Then I installed mp[3]act (I’ve also blogged about it) and shortly after replaced it with Jinzora due to mp[3]act being quite bug-ridden and not in development any longer. (update 2013: links removed – mp[3]act is now pointing to a porn site and Jinzora is gone)

In all that time (one year of operation), shion never crashed on me. Overall, the stability of my home network went through the roof since switching all tasks over to her: No more strange connection losses. No more rebooting router and cable modem when lots of outgoing connections are active. No more inexplicable slowness in the internal network.

Shion does a wonderful job for me and I would never ever go back to any less flexible or stable solution.

Lately, I thought about maybe ditching her for a more powerful intel-based Mac Mini, but in the end shion is fast enough for my current purpose and I could never ditch a machine as nice as this one.

Flexible, Stable, Fast, Quiet and quite inexpensive. A machine worthy of being referred to with a name and a female pronoun.

Linux, PowerPC, gcc, segmentation fault

If you ask of me me to name the one machine in my possession I love the most, that’ll be my Mac Mini.

It’s an old PPC one, I bought a bit more than a year ago with the intention of installing Linux on it and using it as home-server/router. It’s not the quickest machine there is, but it’s the most quiet and it does its job like no other machine I ever had: Its Samba file server, OpenVPN Gateway, bittorrent client, mp3 streaming server, SlimServer, just all you could ever use a home server for.

From the beginning, it was clear to me: The distribution I’m going to install on the beauty was to be Gentoo Linux. This decision was based on multiple reasons, from hard facts like always current software to soft facts like nice command-prompts.

Basically, the machine just sat there after I installed it, doing its job. Until this week when I wanted to install some software on it – mainly the unrar command to extract some file right on one of the external HDs I plugged in (shion – that’s what the machine is called – is connected to about 1TB worth of external HDs).

Unfortunately, emerge unrar failed.

It failed hard with a SIGSEGV in gcc (or its cousin cc1).

Naturally I assumed there to be some bug in the gcc I originally installed (3.3 something – as I said: I did not touch the installation for a year now) and I tried to reemerge gcc.

… which ALSO failed with a segmentation fault.

I had no interest what so ever in reinstalling the box – I invested much too much time in its configuration. Cron jobs here, certificates there, home grown scripts everywhere. Even with all the backups I had in mind – I did not want to do that kind of job. Besides: Who tells me if it’s really a software problem? Maybe the hardware is at fault which would mean that my work was in vain.

Searching for “gcc segmentation fault ppc” in google is… interesting… but not really something you can do if you actually want a solution for this problem.

In the end, I mentally prepared myself to go on with the reinstallation – still hoping it’d be a software problem.

And by accident, I came across the Gentoo PPC FAQ which I more or less read out of pure interest while waiting for the ISO to be burned.

To my biggest delight, said FAQ was really helpful though as it had a question that went “Why does gcc keep segfaulting during ebuilds?

So it is a kernel problem! Of course I had preemption enabled! And that option – while working perfectly on all my x86 boxes – causes cache corruption on PPC.

Now that I knew what the problem was, I had two possible ways to go on: Quick and dirty or slow, but safe and clean:

  1. Recompile the kernel on the obviously defective machine, hoping the cache corruption would not hit or at least would not lead to a non-bootable kernel to be compiled.
  2. Boot from a Gentoo live-CD, chroot into my installation, recompile the kernel.

Obviously, I took the option 1.

I had to repeat the make command about 20 times as it continued to fail with a segmentation fault here and then. Usually I got away with just repeating the command – the cache corruption is random after all.

I was unable to get past the compilation of reiserfs though – thank god I’m using ext3, so I could just remove that from the kernel and continue with my make-loop.

Rebooting that kernel felt like doing something really dangerous. I mean: If the cache corruption leads to a SIGSEGV, that’s fine. But what if it leads to a corrupted binary? And I was going to boot from it…

To my delight, this worked flawlessly though and I’m writing this blog entry behind the rebooted MacMini-router. This time, even compiling the all new gcc 4.1.1 worked as expected, so I guess the fix really helped and the hardware’s ok.

Personally, I think fixing this felt great. And in retrospect, I guess I was lucky as hell to have read that FAQ – without it, I would have gone ahead with the reinstallation, compiling yet another kernel with preemption enabled which would have led to just the same problems as before.

Maybe the (very talented) Gentoo Hanbook guys should add a big, fat, red (and maybe even blinking) warning to the handbook to tell the user not to enable preemption in the kernel.

I know it’s in the FAQ, but why is it not in the installation handboook? That’s the place you are reading anyways when installing Gentoo.

Still: Problem solved. Happy.

SQLite on .NET CF – Revisited

Another year, another posting.

Back in 2004, I blogged about Finisar.SQLitem which at the time was the way to go.

Today, I am in quite the same situation as I was back then, with the difference that this time, it’s not about experimenting. It’s a real-world will-go-life-thing. I’m quite excited to finally have a chance at doing some PocketPC / Windows Mobile stuff that will actually be seen by someone else than myself.

Anyways: The project I blogged about is quite dead now and does not even support the latest versions of SQLite (3.2 is the newest supported file format). Additionally, it’s a ADO.NET 1.0 library and thus does not provide the latest bells and whistles.

Fortunately, someone stepped up and provided the world with
ADO.NET SQLite, which is what I’m currently trying out. The project is alive and kicking, supporting the latest versions of SQLite.

So, if you, like me, need a fast and small database engine for your PocketPC application, this project is the place to look I guess.

My task: RemoveTempHxDs

Le’ts say, you want to inform your user about what’s going on (which is a nice thing to do).

This is an example of how not to do it:

What exactly is that “RemoveTempHxDs”, the installer is doing right there? And why is the progress bar at 100% for more than three minutes when I made the screenshot?

If you are unable to provide meaningful progress information, don’t provide it at all. Make your program display a “neutral” progress bar (some spinning wheel or something like that) and make it tell the user it’s “Doing stuff…”. Why expose useless internals?

While I see some value in displaying information like that if it’s to have more information when you are trying to support the application. But in that case, a log file of some kind is much more valuable as it both gives YOU as developer the information you need and does not confuse your user.

Programatically generating XML

If you have to generate XML, it’s usually considered good style to use one of these defined APIs like DOM or XMLWriter.

Just writing out a string to the line is considered bad practice because… why, actually?

Jeff Atwood once more wrote down what I have been thinking for quite some time now.

In many cases, just dumping out XML with sprintf or whatever your language provides you with is faster, independent of bugs in the libraries you use and easier to read.

There are five characters that need to be treated with caution in XML: the &, the <, the >, the " and the '.

Quoting even is straight forward and you usually don’t run into niceties like quoting backslashes in regular expressions you are passing to perl -e inside a double quoted string on your shell (I don’t even want to count the ‘s needed to actually get the regex parser in perl to see just one of them).

And even if you screw up, you can still rely on the XML parser to bail out if something is wrong.

The time you waste learning your library, coping with its bugs and finally working with the usual bloat of todays OOP interfaces (interface as in “user interface”) far outweighs the occasional quoting problem which should not happen anyways.

And don’t make me get started on trying to understand the structure of the XML code like Jeff posted is going to create:

System.Text.StringBuilder sb = new System.Text.StringBuilder();

XmlWriterSettings xs = new XmlWriterSettings();
xs.ConformanceLevel = ConformanceLevel.Fragment;
xs.Indent = true;

XmlWriter xw = XmlWriter.Create(sb, xs);
xw.WriteStartElement("status");
xw.WriteAttributeString("code", "1");
xw.WriteEndElement();
xw.WriteStartElement("data");
xw.WriteStartElement("usergroup");
xw.WriteAttributeString("id", "usr");
xw.WriteEndElement();
xw.WriteEndElement();
xw.Flush();
return sb.ToString();

If you are seeing this in code you have to maintain (but you have not written), how would you tell what XML it generates? How does the readability of that compare to this?

string s =
@"<status code=""{0}"" />
<data>
<usergroup id=""{1}"" />
</data>";
return String.Format(s, "1", "usr");

Note that I’m not that much of a .NET guy, but I’m quoting Jeff’s code here

Summary in one word: Jeff’s Article: ACK!

One day with Serendipity

Here we go: Everything migrated. Every link (hopefully) fixed. Worked around (I think) some problems with images uploaded from MT clashing with Serendipity’s (s9y from now on) mod_rewrite handling and re-categorized every entry: the new gnegg.ch is up and running.

So, how is life with s9y?

Fist of all: I got no single comment SPAM. This is due to the better SPAM countermeasures and due to all URLs changing. I’ll have to see how good the SPAM prevention will work, though I have an idea it can’t be that bad (see below).

While s9y is slower than MT in delivering pages (understandable considering MT is generating static pages), it’s more feature-rich compared to MT – at least if you consider s9y to be a blogging engine, not a framework to create blogging-engine-like tools.

I love the plugin system: There’s nothing you can’t write a plugin for and people seem to have noticed that – at least considering the wealth of plugins available for you to download and install (directly from the administration interface).

Also, because I’m using a premade template and because s9y is a bit more intelligent in reusing templates, the whole site finally has a consistent look. No more usage of outdated templates when commenting or displaying error messages.

The most interesting thing though is the SPAM prevention: When you post a comment, it will go through the following procedure:

  • Is it exactly the same comment as another posted before? If so, reject it. This prevents a spammer that got through once from getting through again. And it prevents you from double-posting by accident.
  • Is your IP-Address posting a comment within 2 minutes after posting another one, the comment will be rejected. I know proxy servers and NAT routers exist and I will tweak the time if I should ever get more popular. A cookie-based approach obviously doesn’t work to flood-protect the blog from malicious spammers.
  • Does the comment point to an URL listed on SURBL, it’ll be rejected. I’m sorry, but this is a sacrifice I must ask for.
  • If you post a comment to an entry older than 30-days, it’ll be insta-moderated. I promise to activate it as soon as possible.
  • If you post to a comment older than 7 days, you’ll have to solve a captcha, just to be sure. If you cannot solve it, feel free to contact me via Email
  • After you post a comment with more than 3 links, I’ll have to approve it first. If you post more than 20 links, it’ll be rejected.
  • A word-filter is active aswell, though I think all these measures stop the spam before even getting here.
  • If all this fails, I’m sure the SPAM will be detected by Akismet

While I know that some restrictions may hurt you, please believe me that the restrictions are in place to both increase the overall quality of content here and to make my life a bit easier.

Serendipity really is a nice blogging engine. Go ahead and try it!

New face, new engine, new everything

Management Summary of this longer entry: 1) Comments are back, 2) I’m using Serendipity instead of Movable Type and 3) This layout – though premade – is going to stay.

But now my reasoning:

As I’ve stated earlier today, I had enough comment spam arriving on gnegg.ch. Not only the blog was filled up with junk, but also my mailbox was hit (MT was sending mails for every comment).

To underline how BAD it was, notice this: During last weekend I was off the internet most of the time. In the two nights (friday to saturday and saturday to sunday), gnegg.ch was hit by 683 SPAM comments, of which MT only classified 4 as spam.

For each of these 683 comments, I got an email message. Which was especially bad as I was checking mail from my mobile phone (that was the most expensive mail checking process in my life I guess – imagine the sheer size of only the headers)

Even worse was the interface for comment removal: The biggest page size I could select was 50 comments, so I had to delete the comments in groups of 50, each time waiting for the affected pages to be rebuilt over and over again.

There is a mutliselect option in MT, but it always affects all comments per page, so chosing to display all comments and then using the “Select All” feature would not have helped as it would have deleted the legit comments too.

This just so you understand why I had to do something. I did not want to have another “fun” comment removal session next sunday evening (most of the comments get posted on the weekends – probably in the hope they will remain unnoticed for a while longer – which they did).

At first, I just wanted to turn off the comments and keep it at that.

But what is a blog without comments? Yeah. right… not much.

So I went ahead and installed Serendipity because I knew that it had some really nice SPAM-countermeasures included.

As I currently don’t have the time needed to port the old MT template over, I selected a template that comes with s9y and I have to say: It looks great, IMHO. I think I’ll keep it at this.

I’m no web designer and even if I could convince Richard to create a new layout for me (thinking that the old one just is a bit too dark and grey for my current mood), it would take AGES for me to create a Smarty version out of it, so I decided to go with premade templates.

And this one (Perun Blue) is really nice – IMHO even better than the old, custom made, one. So, I hope, you can live with this.

While the import process worked flawlessly, many links inside the site are broken and I’m currently in the process of fixing them.

Comments disabled

Ok. this is it. I have enough.

While I value the legit comments of my visitors, I’m deleting over 200 spam comments per day lately. This must stop. NOW.

Unfortunately, no technical measure currently available really prevents comment spam at least not without serious disadvantages.

Let me go into this:

  • Use a catpcha: Captchas can be broken and in fact ARE broken all over the place. No point in placing another hurdle that’s easily overcome by machines, but can’t be overcome at all by some humans. True: I could decrease the readability to make OCRing the thing harder, but what’s the point? Once the captcha is unreadable, it can’t be broken by machines, but it can’t be solved by humans either.
  • Use a service like TypeKey to authenticate users and let only authenticated users post: Easy to implement, but unfortunately, noone seems to trust MT (neither do I – fully), so noone is using the service. Unfortunately, it doesn’t solve the problem either as machines are well able to create TypeKey accounts (I doubt their captcha is so much better – and even if it currently is: Above problems apply to them aswell).
  • Create your own authentication service: While this may be more liked than TypeKey, it means a lot of work to integrate it into MT and has the same drawbacks (machines can create accounts unless you use a captcha, where my first point applies again).
  • Use a SpamAssassin-like system to get rid of the SPAM. MT has such a system, but it doesn’t really work. Neither seem the blacklists to do their job.

So I come to the only tool that really works to take care of all comment spam: Turn off comments. No discriminating against visually impaired people, no possibility for even the smartest algorithm to sneak a comment into the system. Problem solved.

Personally, I think MT is lacking in terms of counter-spam measures and I will once more have a look at Serendipity which provides more fine-grained control. Until then, I’m sorry, but I have to disable comments on this site.

Spammers: 1, Freedom: 0