Tag: solutions

Linux, PowerPC, gcc, segmentation fault

If you ask of me me to name the one machine in my possession I love the most, that’ll be my Mac Mini.

It’s an old PPC one, I bought a bit more than a year ago with the intention of installing Linux on it and using it as home-server/router. It’s not the quickest machine there is, but it’s the most quiet and it does its job like no other machine I ever had: Its Samba file server, OpenVPN Gateway, bittorrent client, mp3 streaming server, SlimServer, just all you could ever use a home server for.

From the beginning, it was clear to me: The distribution I’m going to install on the beauty was to be Gentoo Linux. This decision was based on multiple reasons, from hard facts like always current software to soft facts like nice command-prompts.

Basically, the machine just sat there after I installed it, doing its job. Until this week when I wanted to install some software on it – mainly the unrar command to extract some file right on one of the external HDs I plugged in (shion – that’s what the machine is called – is connected to about 1TB worth of external HDs).

Unfortunately, emerge unrar failed.

It failed hard with a SIGSEGV in gcc (or its cousin cc1).

Naturally I assumed there to be some bug in the gcc I originally installed (3.3 something – as I said: I did not touch the installation for a year now) and I tried to reemerge gcc.

… which ALSO failed with a segmentation fault.

I had no interest what so ever in reinstalling the box – I invested much too much time in its configuration. Cron jobs here, certificates there, home grown scripts everywhere. Even with all the backups I had in mind – I did not want to do that kind of job. Besides: Who tells me if it’s really a software problem? Maybe the hardware is at fault which would mean that my work was in vain.

Searching for “gcc segmentation fault ppc” in google is… interesting… but not really something you can do if you actually want a solution for this problem.

In the end, I mentally prepared myself to go on with the reinstallation – still hoping it’d be a software problem.

And by accident, I came across the Gentoo PPC FAQ which I more or less read out of pure interest while waiting for the ISO to be burned.

To my biggest delight, said FAQ was really helpful though as it had a question that went “Why does gcc keep segfaulting during ebuilds?”

So it is a kernel problem! Of course I had preemption enabled! And that option – while working perfectly on all my x86 boxes – causes cache corruption on PPC.

Now that I knew what the problem was, I had two possible ways to go on: Quick and dirty or slow, but safe and clean:

  1. Recompile the kernel on the obviously defective machine, hoping the cache corruption would not hit or at least would not lead to a non-bootable kernel to be compiled.
  2. Boot from a Gentoo live-CD, chroot into my installation, recompile the kernel.

Obviously, I took the option 1.

I had to repeat the make command about 20 times as it continued to fail with a segmentation fault here and then. Usually I got away with just repeating the command – the cache corruption is random after all.

I was unable to get past the compilation of reiserfs though – thank god I’m using ext3, so I could just remove that from the kernel and continue with my make-loop.

Rebooting that kernel felt like doing something really dangerous. I mean: If the cache corruption leads to a SIGSEGV, that’s fine. But what if it leads to a corrupted binary? And I was going to boot from it…

To my delight, this worked flawlessly though and I’m writing this blog entry behind the rebooted MacMini-router. This time, even compiling the all new gcc 4.1.1 worked as expected, so I guess the fix really helped and the hardware’s ok.

Personally, I think fixing this felt great. And in retrospect, I guess I was lucky as hell to have read that FAQ – without it, I would have gone ahead with the reinstallation, compiling yet another kernel with preemption enabled which would have led to just the same problems as before.

Maybe the (very talented) Gentoo Hanbook guys should add a big, fat, red (and maybe even blinking) warning to the handbook to tell the user not to enable preemption in the kernel.

I know it’s in the FAQ, but why is it not in the installation handboook? That’s the place you are reading anyways when installing Gentoo.

Still: Problem solved. Happy.

More Asterisk stuff

I thought I’d give a little update on what’s going on in my Asterisk installation as some of the stuff might be useful for you:

Speed Dial

If you have Snom Phones and want to program the function keys to dial a certain number, be sure to select “Speed Dial” and not “Destination” when entering the number.

Destination was used in earlier firmwares but it now used to not only make the phone dial that number, but also subscribe to the line to make the LED light up when the line is used.

This obviously makes no sense at all with external numbers and requires some configuration for internal ones (see below). The additional benefit is that buttons with “Speed Dial” assigned don’t turn on the LED.

Dial by click

You can dial a number from the Mac OS X address book aswell. Asterisk will make your phone ring and redirect the call once you pick up (just like AstTapi on Windows). I had the best experience with app_notify. I don’t quite like the way how it notifies clients of incoming calls (hard-coding IP-Addresses of clients is NOT how I want my network to operate), but maybe there will be a better solution later on. Currently, I’m not using this feature.

Dialing works though.

You don’t have to modify manager.conf, btw, if you already have the entry for the AstTapi-Solution. app_notify will ask for username (manager context) and password when it launches the first time.

Subscription

As noted above, your Snom Phone can be advised to monitor a line. The corresponding LED will blink (asterisk 1.2+) when it’s ringing and light up when the line is busy.

Snom-wise, you’ll have to configure a function key to a “Destination” and enter the extension you like to monitor.

Asterisk-wise you have to make various changes:

sip.conf

  • Add subscribecontext=[context], where context is the context in extensions.conf where the corresponding SNOM phone is configured in. I’ve put this to the [general]-Section because all phones are sharing the same context (internal).
  • Add notifyringing=yes if you have Asterisk >= 1.2 and want to make the LEDs blink when the line is ringing.

extensions.conf
This is a bit hacky: In the sip-context add a notify extension for every line you want to be allowed to be monitored. Unfortunately, you can’t use macros or variables here, so it’s messy.

On my configuration it’s:

[internal]
exten => 61,hint,SIP/61
exten => 62,hint,SIP/62
exten => 63,hint,SIP/63
exten => 64,hint,SIP/64
exten => _6[1-9],1,Dial(SIP/${EXTEN},,tWw)

While I would have preferred

[internal]
exten => _6[1-9],hint,SIP/${EXTEN}
exten => _6[1-9],1,Dial(SIP/${EXTEN},,tWw)

Though this may have been fixed with 1.2.2, but I’m not sure just yet.

You may have to reboot your phone after making the configuration change there. To check the registration in asterisk use SIP show subscriptions.

You should get something like this:

asterisk*CLI> SIP show subscriptions
Peer User Call ID Extension Last state Type
192.168.2.152 62 3c26700b57e 61 Idle dialog-info+xml
1 active SIP subscription

This is not quite tested as of yet because the guy at extension 61 is currently in his office and I don’t want to bother him ;-)

Update while editing/correcting this text: It works. They guy has left and I checked it.

Evening leisure

You know what’s one of the greatest things to happen on the evening of a workday?

You come home, select ‘Play Movie’ on your Harmony Remote and watch the speedrun that downloaded itself while you were on the office. (using Windows Media Center. Sorry, but that has the advantage of just working).

This is the future. This is like watching TV with the full control of the program.

Why should I watch TV only to see programs I don’t want to for most of the time? Why should I cope with advertisments every 10 minutes? Why should I be forced to watch all the movies in the german synchronized version?

Not with me. This is what the internet is for. This is why I’m running a linux server at home.

Strangest JavaScript error ever

Let’s say you create some very nice AJAX-stuff for a web project of yours. With nice I mean: Not breaking the back-button where its functionality is needed, not doing something that works better without AJAX, and doing it while providing lots of useful visual feedback.

Let’s further assume that the thing works like a charm in every browser out there (not counting Netscape 4.x and IE in all versions – those are no browsers).

And then, IE throws this at you:

Unknown Runtime Error

Needless to say that the HTML output in question had a line count not even close to 370, so finding this thing easily was out of the question.

The solution: IE is unable to write to innerHTML of a TBODY element. But instead of providing an useful error message or even a link to the source with the line in question already highlighted (that’s what Firefox would do), it just bails out with completely useless error information.

*sigh*

(btw: That mix of fonts in the details section of the error message is just another indication of IEs great code quality)

No topic-based help system installed

Recently I had to do some Delphi-work again. To my surprise, the online help seemed to have stopped working. I always got this error message:

No topic-based help system installed

Programming without an online help is very tedious and sometimes nearly impossible.

When I had to look up in which Unit TWinControl is declared, I had two possibilities: Either look it up in the source code (Borland ships the full source code to their class library) or fix the help system once and for all.

I deceided to do the latter (searching after TWinControl is no fun).

Googling in the web turned out nothing. In Groups most of the time, the suggestion was to reinstall the whole thing

I absolutely did not have time for this, so I dug deeper.

The problem is caused by the installation of the VS2005 Beta which resets some AppID-GUUID. Afterwards delphi crashes while loading the IDE-package htmlhelp290 which in the end causes delphi to think that there’s no help installed

I fixed it doing the following:

  1. Reset the help-viewer-appid. In the registry under HKEY_CLASSES_ROOTAppIDDExplore.exe, set AppId to {4A79114D-19E4-11d3-B86B-00C04F79F802}
  2. In HKEY_CURRENT_USERSoftwareBorlandBDS3.0Disabled IDE Packages remove the entry for htmlhelp290 that has been created.
  3. Start Delphi and use the help again

What I don’t know is if this has a negative effect to VS, but this does not matter for me: I need Delphi to work.

The whole thing is a consequence of the .NET orientation of delphi: Earlier, Delphi was as self-contained as the executables it can build: Drop it into a directory and run it. No problems, no questions asked.

With Delphi integrated into .NET and using .NET-Components, problems begin to rise: First there was a bug in D8 causing it to stop working after .NET 1.1 SP1, now this.

Hopefully, they find a way back to both .NET (for the acceptance in the buzzword-centered world where you can’t have a dev-tool not .NET capable) and self-containment.

Once more: PHP and SOAP

I can’t reist: I made my third attempt at getting a SOAP-Server in PHP to work (I only documented my first try here on the blog).

My first try was a little more than two years ago. That one failed miserably.

The next try was last november. I came somewhat further than I did my first time, but Visual Studio was unable to import the WSDL correctly as soon as I was passing arrays of structs around

And now I tried again – this time with PEAR SOAP 0.9.1

This time all looks so much better. First of all, I do this because I really have to: For one of our PopScan customers, we are accessing their IBM DB2 database – currently using a Perl-based server that’s nearing the end of its maintainability, so I deceided to redo it with PHP (PHP-code is somewhat cleaner than Perl code and I’m more fluent in PHP than in Perl)

The DB2-client (especially the one needed for that old 7.1 database) is clumsy, a bit unstable and really not something I want to link into our Apache-Server that serves all our clients.

So the idea was to compile another apache, run it on another port, bound to localhost only. Add PHP with the DB2-client. Access this combo via some way of RPC with the nice DB2-free standard-installation.

Well. And instead of once again designing a custom protocol (like I did for the Perl-Server), I though: Maybe give SOAP another shot.

In contrast to previous experience, this time, it was the Server that worked and the client that was failing. Using PEAR SOAP 0.9.1, creating the server (which creates the dreaded WSDL) went without flaw. This time I was even able to import the WSDL into VS 2003, which I tried just for fun.

Passing around arrays of structs of structs was no problem at all. After building the self::$__typedef and self::$__dispatch_map arrays, passing around those data types has become really intuitive: Just create arrays of arrays in PHP and return them. No problem.

Well done, PEAR team!

This time I’ve had problems with the PEAR SOAP Client. It insisted in passing around ints as strings which the server (correctly) did not like.

Instead of using lots and lots of time debugging that, I went the pragmatical way and used PHP5’s build in SoapClient functionality. No problems there.

And then it suddenly broke

My test-client was written for the CLI version of php which was version 5.0.4. The apache-module of the live-server was 5.0.3.

All I got with 5.0.3 was a HTTP Client Error (SoapFault exception: [HTTP] Client Error).

Whatever I did, it did not go away, but to my delight I have seen that PHP did not even connect to the server to fetch the WSDL. This was good as I was able to debug much quicker that way.

In the end it was the URL of the WSDL. Every version of PHP5 (even the 5.1 betas) – besides 5.0.4 – does not like this:

http://be.sen.work:5436/?wsdl

it prefers this

http://be.sen.work:5436/index.php?wsdl

I ask now: Why is that this way? The first version is a valid URL aswell. The served WSDL is correct – it’s the same file that gets called and it returns totally the same content. This is so strange.

After all, I have to say. SOAP with PHP – after two years – still is not ready for prime time. It’s still in the state of “sometimes working – sometimes not”. But as I now have an environement where it’s known to be working and as I’m in total control of said environement, I will go with SOAP none-the-less. It’s so much cleaner (and more secure: more people than just me are looking at the SOAP-code) than designing yet another protocol and server.

Oh. And the bottom line is: Never trust protocols that call themselves “simple” or “lightweight” ;-)

Lots of fun with OpenVPN

OpenVPN may seem to you as being “just another VPN solution”. And maybe you are right.

However, OpenVPN has some distinct advantages over other VPN-solution that makes it quite interesting for deployment:

  • NAT traversal. OpenVPN uses plain old UDP-Packages as a transport medium. Every NAT router on this world can forward them correctly out-of-the-box. If not, create the usual port-forwarding rule and be done with it. If that fails too (whyever it could fail), use the TCP-protocol.
  • Ease-of-use: Install, create two certificates, use the VPN. It’s as easy as 1-2-3
  • Designed with small installations in mind. OpenVPN is not a big slow beast like IPSec for example. While it may not be as secure, it does not have all the problems associated with IPSec.
  • User-Space. OpenVPN runs completely in userspace (while using the TUN device provided by the kernel). This way the installation is non-critical and does require no reboots. Updates in case of security problems do not require reboots either.

So after this unexpected praise: What brings me to writing this posting?

Well. I’ve just deployed one of the coolest things on earth: Using OpenVPN, I have connected my home network to the network in the office. Both ends see each other and allow for direct connections. I’m not only able to print on the offices printers from home (which admittedly is as useless as it is cool), but I’m also able to – for example – stream music from home to the office over a secured channel. All using straight IP connections without any NAT-trickery or other things.

Actually not even one port is forwarded through my NAT-gateway (a ZyAir B-2000 – as the Airport-Basestation does not allow for static routes (see below), I was forced to cross-grade).

I already had some of this functionality using my previously deployed PPTP-setup, though this had some disadvantages:

  • Flacky support in Linux. Maintaining the beast across windows- and mac versions was not easy as something always broke on new versions.
  • Suboptimal security. You know: PPTP has flaws – quite like WEP. Though I’ve tried to work around them by using very very long passwords.
  • Suboptimal usability: When I wanted to connec to the office, I had to dial into the VPN, so user interaction was needed. Additionally, the default-gateway was redirected (I could have turned that off), so all open TCP connections got disconnected when I dialled.

My current solution does not have any of those problems (I don’t know about the security of course – no one does. For now, OpenVPN is said to be secure): No dialling is required, no problems with changing software-versions are to be expected (as it runs on a dedicated router which I don’t intend on changing), and I don’t have to dial in. The default gateway is not changed either of course, so the usual internet-connections go out directly. This way I’m unaffected from the office’s suboptimal upstream of 65KBytes/s (unless I use services from the office of course – but this is unavoidable).

So. What did I do?

At the very first, I had to recompile the kernel on the server side once. I have not included TUN-support when I created my .config last year. After this, emerge openvpn was all that was needed. I kept the default configuration-file somewhat intact (install with the “examples” USE-flag and use the example-server.conf), but made some minor adjustments: 

local x.x.x.x
push "route 192.168.2.0 255.255.255.0"
client-config-dir ccd
route 192.168.3.0 255.255.255.0
#push "redirect-gateway"

(just the changed lines)

and the /etc/openvpn/ccd/Philip_Hofstetter:

iroute 192.168.3.0 255.255.255.0

Now, what does this configuration do?

  • Bind to the external interface only. This has only cosmetical reasons
  • Push the route to the internal network to the client. Using the default configuration, all OpenVPN-Addresses are in the 10.8.0.0 network which allows me for nice firewall-settings on the server-side. The 192.168.2.0/24 network is our office-network
  • Tell OpenVPN that there are some client-specific configuration options to reach the 192.168.3.0/24 net which is my home network
  • Comment out the option to let OpenVPN set the default gateway. We really don’t want all the traffic in my home net going through the office

Then we create this client-configuration file. It’s named after the CN you use in the SSL-certificate, while replacing spaces with underscores. You can see the correct value by setting up everything and then connecting to the server while watching the logfile.

In the client specific configuration-file we confirm the additional route we want to create.

The configuration file on the client router is unchanged from the default.

The only thing you need now is the SSL-certificate. Create one for the server and more for each client. I won’t go into this in this article as it’s somewhat complicated on itself, but you’ll find lots of guides out there.

I used our companies CA to create the certificates for both the server and the client.

After this, it’s just a matter of /etc/init.d/openvpn start on both machines (the path to the certificates/keys in the configuration files must match your created files of course).

Just watch out for the routing: On the server I had to change nothing as the server was already entered as default gateway on all the clients in the office network.

In the client network, I had to do some tweaking as the default-gateway was set to the Airport Basestation, which (understandably) knew nothing about the 192.168.2.0/24 network, so was unable to route the IP-packets to the VPN-gateway in the internal network (my Mac Mini).

Usually you solve that by installing a static route on the default gateway in your network. Unfortunately, this is not possible on an airport basestation. A problem I have solved by replacing it with a ZyAir B-2000 from Zyxel which allows for setting static routes.

On that new access-point I created a route equivalent to this unix-command:

route add -net 192.168.2.0 netmask 255.255.255.0 gw 192.168.3.240

Where 192.168.3.240 is the address of my Mac Mini on which OpenVPN was running as client.

Then I issued “echo 1 > /proc/sys/net/ipv4/ip_forward” on the Mac Mini to allow the packets to be forwarded.

So whenever I send packets to one of the offices computers – let’s say 192.168.2.98, this is what happens:

  1. The client uses it’s IP and netmask to find out that the packet cannot be delivered directly. It sends it to the default gateway (my ZyAir)
  2. The ZyAir consults its routing table to watch for the route to 192.168.2.0/24 and finds 192.168.3.240 as gateway for that network (every other address would have been routed thorugh my cable modem)
  3. 192.168.3.240, shion, watches it’s own roting table where OpenVPN has created a route thorugh the VPN-interfaces (10.8.0.x) to the 192.168.2.0/24 network. It delivers the packet there.
  4. On the other end of the tunnel, the OpenVPN-Server delivers the packet to the destination server.

The path of the reply-packets is the same – just from the bottom to the top.

After getting the routing as I wanted it (verifyable by pinging petween computers in both networks), the next step was pure cosmetics:

  • Create an internal DNS-server. Use it as a slave for the office’s DNS-server to allow for DNS-lookups to work without crossing the VPN each time
  • Use said DNS-server to create entries for the computers in my home network
  • Make the office DNS-server a slave for that home-zone (to reach my computers by name)

All of this was most interesting to implement and went much more smootly than anything else I’ve tried so far VPN-wise. Finally, I have the optimum solution concering connectivity to my office.

And besides that: It was fun to implement. Just worthy of a “Supreme nerd” – the title I got here for my 92 points.

WoW: Language Hacking

As I explained in my previous posting, I very much like to play World of Warcraft in the english version.

Now I got my hands on the US-version and installed it (after uninstalling the german version).

The problem came after patching to the current version: My account was not recognized anymore – no wonder: The game was connecting to the US servers while my account is on the european ones.

A bit searching for worldofwarcraft.com in the games directory revealed the string set realmlist [something] in base.mpq

As always, google was my friend and showed me the solution: Add

SET realmlist "eu1.wow.battle.net"

to the file config.wtf in the directory WTF of your WoW installation.

This lets you login to the european servers where your account is recognized.

Works well (at least until the next patch is released ;-)

Update: if you have a file called realmlist.wtf in the main installation directory, change that one, not the config.wtf as it will get overwritten on every launch. And additionally, you should set the server to eu.logon.worldofwarcraft.com instead – the older one was for the beta.

Why I love the command line

Today I had the task to join together quite some mp3-files.

I had about 100 radio plays, each devided in three to six files which I wanted to have joined to one file per play so I can better organize them on my iPod

There are tools out there doing exactly that. mp3surgeon being one of them. All these tools a) have a non-scriptable GUI (meaning lots and lots of clicks) and b) cost money

b) would not be a pronlem if those tools would work for me, but because of a) they do not.

Then I found mpgedit a command line tool capable of joining MP3’s (respecting VBR-headers, but without recoding the new file)

As it’s usable from the command line, I could write a small script doing exactly what I wanted:

<?

$dir = dir(".");
while (false !== ($entry = $dir->read())) {
	if (preg_match('/^.+$/', $entry)) continue;
	$path = '.\'.$entry;
	if (is_dir($path))
 	    doJoin($path);
}

function doJoin($dir){
	echo "Looking in $dirn";
	$of = escapeshellarg("..\".basename($dir).".mp3");
	chdir($dir);
	$files = array();
	$d = dir(".");
	while (false !== ($entry = $d->read())) {
	   if (!preg_match('/.mp3$/', $entry)) continue;
	   $files[] = $entry;
	}
	$d->close();
	sort($files);
	$files = array_map('escapeshellarg', $files);
	system("c:mp3mpgedit_nocurses.exe -o $of -e- ".implode(' ', $files));
	chdir("..");
}
?>

Note that it’s written in PHP as this is the language I currently do most of my work with. And note that it’s very customized to just my needs. None the less it works very well and saves me from about 200’000 clicks

Now this is exacltly why I love the command line.

AirPort basesation and external DHCP server

Recently, I bought an airport basestation.

I wanted to use it as a NAT router and a wireless access point. DNS and DHCP I wanted to do via a fully-fledged BIND/dhcpd combination running on my iMac.

DNS I need because I’m doing some work for the office from home. As much of it is web based, I need virtual hosts on my server and I certainly don’t want to go back to stone age and move around hosts files. DNS was invented for something, so please, let me use it.

DHCP I wanted because sometimes, I’m using applications on my notebook that require some ports forwarded to them (bittorrent for example). Forwarding ports without fixed IP-adresses can be difficult (especially if changing the forwarding address requires a restart of the router), so I wanted the possibility to give the MAC-adress of my notebooks NIC a fixed IP-address. This is not possible with airports built-in DHCP server (and I don’t blame them for this – it’s quite a special feature)

Now, imagine how disappointed I was seing, that this is not possible when using Apples configuration program:

They tie NAT and DHCP together: Either you turn off both NAT and DHCP, NAT only, or none of them. Turning off DHCP only is not possible.

Looking around on the web, I came across Jon Sevys Java Based Configurator again.

With this tool my configuration certainly is possible:

  1. Configure your basestation using Apples utility. Tell it to enable NAT and distribute IP-Adresses
  2. Update the configuration and exit Apples utility.
  3. Run the Java Based configurator.
  4. On the “DHCP Functions”-Tab, unckeck the Checkbox
  5. On the “Bridging Functions”-Tab uncheck “Disable bridging between Ethernet and wireless Lan”
  6. Save the configuration.

    The last step is important if you want the Basestation to continue working as an usable wireless access point. I forgot to do this the first time I tried and did not get an IP-Adress and could not connect to the wired lan after setting one manually either. Logical, but disturbing if you think you got the solution but it still does not work as expected…