To embed a vimeo video on some page, I had a look at their different methods for embedding and the easiest one seemed to be what is basically JSONP – a workaround for the usual restriction of disallowing AJAX over domain boundaries.
But did you know, that JSONP not only works around the subdomain restriction, it basically is one huge cross site scripting exploit and there’s nothing you can do about it?
You might have heard this and you might have found articles like this one thinking that using libraries like that would make you save. But that’s an incorrect assumption. The solution provided in the article has it backwards and only helps to protect the originating site against itself, but it does not help at all to protect the calling site from the remote site.
You see, the idea behind JSONP is that you source the remote script using <script src=”http://remote-service.example.com/script.js”> and the remote script then (after being loaded into your page and thus being part of your page) is supposed to call some callback of the original site (from a browsers standpoint it is part the original site).
The problem is that you do not get control over the loading let alone content of that remote script. Because the cross-domain restrictions prevent you from making an AJAX request to a remote server, you are using the native HTML methods for cross domain requests (which should not have been allowed in the first place) and at that moment you relinquish all control over your site as that remotely loaded script runs in the context of your page, which is how you get around the cross domain restrictions – by loading the remote script into your page and executing it in the context of your page.
Because you never see that script until it is loaded, you cannot control what it can do.
Using JSONP is basically subjecting yourself to an XSS attack by giving the remote end complete control over your page.
And I’m not just talking about malicious remote sites… what if they themselves are vulnerable to some kind of attack? What if they were the target of a successful attack? You can’t know and once you do know it’s too late.
This is why I would recommend you never to rely on JSONP and find other solutions for remote scripting: Use a local proxy that does sanitization (i.e. strict JSON parsing which will save you), rely on cross-domain messaging that was added in later revisions of the upcoming HTML5 standard.
Today in the office, a discussion about the merits of Ruby compared to Python and the other way around (isn’t it fun to have people around actually willing to discuss such issues?) lead into us making fun of different programming languages by interjecting some sore points about them into their names.
The Skype conversation went roughly as follows (I removed some stuff for brevity but all the language names are intact):
thepilif: ja-long variable names and no function pointers-va really sucks thepilif: though there’s always C(*^~**<<)++ thepilif: and then there’s alyways Del-Access violation at address 02E41C10. Read of address 02E41C10-phi thepilif: or P-false==true-HP Coworker: ok so for the sake of it i should add py thon thepilif: or java-everything is global-script thepilif: too bad it doesn’t work for C thepilif: C-sigsegv thepilif: they know why they just chose one letter Coworker: exactly, k&r are smart Coworker: has-how the fuck do i do a print-skell? Coworker: pe/(^$^)/rl thepilif: or pe-module? object? hash? what’s the difference-rl Coworker: so we could say pe/$^/rl thepilif: and ru-lets rewrite our syntax on the fly-by Coworker: l(i(s(p))) thepilif: can’t you wrap this into another pair of ()? thepilif: (l(i(s(p)))))) Coworker: yes even better thepilif: and add the syntax error thepilif: one too many ) Coworker: it’s impossible to match them just by looking thepilif: totally impossible. yes Coworker: the human brain is no fucking pushdown automata Coworker: but maybe the lisp people are Coworker: vb! vb needs one thepilif: visual-on error resume next-basic thepilif: and of course brain-<<<<<******<<<>>>>-fuck thepilif: c-tries to be dynamic, but var just doesn’t cut it-# thepilif: c-not quite java nor c(++)?-# thepilif: though the first one feels better thepilif: oh.. and of course HT-unknown error-ML thepilif: as a tribute to IE6 thepilif: and of course la-no bugs but still not usable-tex thepilif: sorry, Knuth thepilif: and send-$*$_**^$$$-mail
So the question is: Do you have anything to add? Do you feel that we were overly unfair?
We all know how beneficial to the performance of a web application it can be to serve assets like CSS files and JavaScript files in larger chunks as opposed to smaller ones.
The main reason behind this is the latency incurring from requesting a resource from the server plus the additional bandwidth of the request metadata which can grow quite large when you take cookies into account.
But knowing this, we also want to keep files separate during development to help us with the debugging and development process. We also want the deployment to not increase too much in difficulty, so we naturally dislike solutions that require additional scripts to run at deployment time.
And we certainly don’t want to mess with the client-side caching that HTTP provides.
The only thing (besides a one-time configuration of the plugin) you have to do during development is to wrap all your <link>-Tags with {asset_compile}….{/asset_compile} and the plugin will do everything else for you, where everything includes:
automatic detection of actually linked files
automatic detection of changed files
automatic minimizing of linked files
compilation of all linked files into one big file
linking that big file for your clients to consume. Because the file is still served by your webserver, there’s no need for complicated handling of client-side caching methods (ETag, If-Modified-Since and friends): Your webserver does all that for you.
Because the cached file gets a new URL every time any of the corresponding source files change, you can be sure that requesting clients will retrieve the correct, up-to-date version of your assets.
sacy handles concurrency, without even blocking while one process is writing the compiled file (and of course without corrputing said file).
sacy is released under the MIT license and ready to be used (though it currently only handles CSS files and ignores the media-attribute – stuff I’m going to change over the next few days).
Now, I’m no Python programmer by any means. Sure. I know my share of Python and I really like many of the concepts behind the language. I have even written some smaller scripts in Python, but it’s not my day-to-day language.
That chapter about string handling really really impressed me though.
In my opinion, handling Unicode strings they way python 3 is doing is exactly how it should be done in every development environment: Keep strings and collections of bytes completely separate and provide explicit conversion functions to convert from one to the other.
And hide the actual implementation from the user of the language! A string is a collection of characters. I don’t have to care how these characters are stored in memory and how they are accessed. When I need that information, I will have to convert that string to a collection of bytes, giving an explicit encoding how I want that to be done.
This is exactly how it should work, but implementation details leaking into the language are mushing this up in every other environment I know of making it a real pain to deal with multibyte character sets.
Features like this is what convinces me to look into new stuff. Maybe it IS time to do more python after all.
As I said earlier this month: While Java applets are dead for games and animations and whatever else they were used back in the nineties, they still have their use when you have to access the local machine from your web application in some way.
There are other possibilities of course, but they all are limited:
Flash loads quickly and is available in most browsers, but you can only access the hardware Adobe has created an API for. That’s upload of files the user has to manually select, webcams and microphones.
ActiveX doesn’t work in browsers, but only in IE.
.NET dito.
Silverlight is neither commonly installed on your users machines, nor does it provide the native hardware access.
So if you need to, say, access a bar code scanner. Or access a specific file on the users computer – maybe stored in a place that is inconvenient for the user to get to (%Localappdata% for example is hidden in explorer). In this case, a signed Java applet is the only way to go.
You might tell me that a website has no business accessing that kind of data and generally, I would agree, but what if your requirements are to read data from a bar code scanner without altering the target machine at all and without requiring the user to perform any steps but to plug the scanner and click a button.
But Java applets have that certain 1996 look to them, so even if you access the data somehow, the applet still feels foreign to your cool Web 2.0 application: It doesn’t quite fit the tight coupling between browser and server that AJAX gets us and even if you use Swing, the GUI will never look as good (and customized) as something you could do in HTML and CSS.
But did you know that Java Applets are fully scriptable?
Per default, any JavaScript function on a page can call any public method of any applet on the site. So let’s say your applet implements
If you do that, you have to remember though that any applet method called this way will run inside the sandbox regardless if the applet is signed or not.
So how do you access the hardware then?
Simple: Tell the JRE that you are sure (you are. aren’t you?) that it’s ok for a script to call a certain method. To do that, you use AccessController.doPrivileged(). So if for example, you want to check if some specific file is on the users machine. Let’s further assume that you have a singleton RuntimeSettings that provides a method to check the existence of the file and then return its name, you could do something like this:
Now it’s safe to call this method from JavaScript despite the fact that RuntimeSettings.getInterfaceDirectory() directly accesses the underlying system. Whatever is in PrivilegedAction.run() will have full hardware access (provided the applet in question is signed and the user has given permission).
Just keep one thing in mind: Your applet is fully scriptable and if you are not very careful where that Script comes from, your applet may be abused and thus the security of the client browser might be at risk.
Keeping this in mind, try to:
Make these elevated methods do one and only one thing.
Keep the interface between the page and the applet as simple as possible.
In elevated methods, do not call into javascript (see below) and certainly do not eval() any code coming from the outside.
Make sure that your pages are sufficiently secured against XSS: Don’t allow any user generated content to reach the page unescaped.
The explicit and cumbersome declaration of elevated actions was put in place to make sure that the developer keeps the associated security risk in mind. So be a good developer and do so.
Using this technology, you can even pass around Java objects from the Applet to the page.
Also, if you need your applet to call into the page, you can do that too, of course, but you’ll need a bit of additional work.
You need to import JSObject from netscape.javascript (yes – that’s how it’s called. It works in all browsers though), so to compile the applet, you’ll have to add plugin.jar (or netscape.jar – depending on the version of the JRE) from somewhere below your JRE/JDK installation to the build classpath. On a Mac, you’ll find it below /System/Library/Frameworks/JavaVM.framework/Versions/<your version>/Home/lib.
You need to tell the Java plugin that you want the applet to be able to call into the page. Use the mayscript attribute of the java applet for that (interestingly, it’s just mayscript – without value, thus making your nice XHTML page invalid the moment you add it – mayscript=”true” or the correct mayscript=”mayscript” don’t work consistently on all browsers).
In your applet, call the static JSObject.getWindow() and pass it a reference to your applet to acquire a reference to the current pages window-object.
On that reference you can call eval() or getMember() or just call() to call into the JavaScript on the page.
This tool set allows you to add the applet to the page with 1 pixel size in diameter placed somewhere way out of the viewport distance and with visibility: hidden, while writing the actual GUI code in HTML and CSS, using normal JS/AJAX calls to communicate with the server.
If you need access to specific system components, this (together with JNA and applet-launcher) is the way to go, IMHO as it solves the anachronism that is Java GUIs in applets.
There is still the long launch time of the JRE, but that’s getting better and better with every JRE release.
I was having so much fun last week discovering all that stuff.
Modern languages like JavaScript or Ruby provide the programmer with an option to “reopen” any class to add additional behavior to them. In the case of Ruby and JavaScript, this is not constrained in any way: You are able to reopen any class – even the ones that come with your language itself and there are no restrictions on the functionality of your extension methods.
Ruby at least knows of the concept of private methods and fields which you can’t call from your additional methods, but that’s just Ruby. JS knows of no such thing.
This provides awesome freedom to the users of these languages. Agreed. Miss a method on a class? Easy. Just implement that and call it from wherever you want.
which is lots of small (but terribly inconventiently named) classes wrapped into each other to provide the needed functionality. In this example, what the author wanted is to read a file line-by-line. Why exactly do I need three objects for this? Separation of concern is nice, but stuff like this make learning a language needlessly complicated.
In the world of Ruby or JS, you would just extend FileInputStream with whatever functionality you need and then call that, creating code that is much easier to read.
And yet, if you are a library (as opposed to consumer code), this is a terrible, terrible thing to do!
We have seen previous instances of the kind of problems you will cause: Libraries adding functionality to existing classes create real problems when multiple libraries are doing the same thing and the consuming application is using both libraries.
Let’s say for example, that your library A added that method sum() to the generic Array class. Let’s also say that your consumer also uses library B which does the same thing.
What’s the deal about this, you might ask? It’s pretty clear, what sum does after all?
Is it? It probably is when that array contains something that is summable. But what if there is, say, a string in the array you want to sum up? In your library, the functionality of sum() could be defined as “summing up all the numeric values in the array, assuming 0 for non-numeric values”. In the other library, sum() could be defined as “summing up all the numeric values in the array, throwing an exception if sum() encounters invalid value”.
If your consumer loads your library A first and later on that other library B, you will be calling B’s Array#sum().
Now due to your definition of sum(), you assume that it’s pretty safe to call sum() with an array that contains mixed values. But because you are now calling B’s sum(), you’ll get an exception you certainly did not expect in the first place!
Loading B after A in the consumer caused A to break because both created the same method conforming to different specs.
Loading A after B would fix the problem in this case, but what, say, if both you and B implement Array#avg, but with reversed semantics this time around?
You see, there is no escape.
Altering classes in the global name space breaks any name spacing facility that may have been available in your language. Even if all your “usual” code lives in your own, unique name space, the moment you alter the global space, you break out of your small island and begin to compete with the rest of the world.
If you are a library, you cannot be sure that you are alone in that competition.
And even if you are a top level application you have to be careful not to break implementations of functions provided by libraries you either use directly or, even worse, indirectly.
If you need a real-life example, the following code in an (outdated) version of scriptaculous’ effects.js broke jQuery, despite the latter being very, very careful to check if it can rely on the base functionality provided:
Interestingly enough, Array#call wasn’t used in the affected version of the library. This was a code artifact that actually did nothing but break a completely independent library (I did not have time to determine the exact nature of the breakage).
Not convinced? After all I was using an outdated version of scriptaculous and I should have updated (which is not an option if you have even more libraries dependent on bugs in exactly that version – unless you update all other components as well and then fix all the then broken unit tests).
Firefox 3.0 was the first browser to add document.getElementByClassName, a method also implemented by Prototype. Of course the functionality in Firefox was slightly different from the implementation in Prototype, which now called the built-in version instead its own version which caused a lot of breakage all over the place.
So, dear library developers, stay in your own namespace, please. You’ll make us consumers (and your own) lives so much more easier.
Way way back, I was talking about java applets and native libraries and the things you need to consider when writing applets that need access to native libraries (mostly for hardware access). And let’s be honest – considering how far HTML and JavaScript have come, native hardware access is probably the only thing you still needs applets for.
Java is slow and bloated and users generally don’t seem to like it very much, but the moment you need access to specific hardware – or even just to specific files on the users filesystem, Java becomes an interesting option as it is the only technology readily available on multiple platforms and browsers.
Flash only works for hardware Adobe has put an API in (cameras and microphones) and doesn’t allow access to arbitrary files. .NET doesn’t work on browsers (it works on IE, but the solution at hand should work on browsers too) and ActiveX is generally horrible, doesn’t work in browsers and additionally only works under windows (.NET works in theory on Unixes and Macs as well).
Which leaves us with Java.
Because applets are scriptable, you get away with hiding the awful user interface that is Swing (or, god forbid, AWT) and writing a nice integrated GUI using web technologies.
But there’s still the issue with native libraries.
First, your applet needs to be signed – no way around that. Then, you need to manually transfer all the native libraries and extension libraries. Also, you’ll need to put them in certain predefined places – some of which require administration privileges to be written into.
And don’t get me started about JNI. Contrary to .NET, you can’t just call into native libraries. You’ll have to write your own glue layer between the native OS and the JRE. That glue layer is platform specific of course, so you better have your C compiler ready – and the plattforms you intend to run on, of course.
So even if Java is the only way, it still sucks.
Complex deployment, administrative privileges and antiquated glue layers. Is this what you would want to work with?
Fortunately, I’ve just discovered two real pearls completely solving the two problems leaving me with the hassle that is Java itself, but it’s always nice to keep some practice in multiple programming languages, as long as it doesn’t involve C shudder.
The first component I’m going to talk about is JNA (Java Native Access) which is for Java what P/Invoke is for .NET: A way for directly calling into the native API from your Java code. No JNI and thus no custom glue code and C compiler needed. Translating the native calls and structures into what JNA wants still isn’t as convenient as P/Invoke, but it sure as hell beats JNI.
In my case, I needed to get find the directory corresponding to CSIDL_LOCAL_APPDATA when running under Windows. While I could have hacked something together, the only really reliable way of getting the correct path is to query the Windows API, for which JNA proved to be the perfect fit.
JNA of course comes with its own glue layer (available in precompiled form for more plattforms than I would ever want to support in the first place), so this leads us directly to the second issue: Native libraries and applets don’t go very well together.
This is where applet-launcher comes into play. Actually, applet-launcher’s functionality is even built into the JRE itself – provided you target JRE 1.6 Update 10 and later, which isn’t realistic in most cases (just today I was handling a case where an applet had to work with JRE 1.3 which was superseded in 2002), so for now, applet-launcher which works with JRE 1.4.2 and later is probably the way to go.
The idea is that you embed the applet-launcher applet instead of the applet you want to embed in the first place. The launcher will download a JNLP file from the server, download and extract external JNI glue libraries and finally load your applet.
When compared with the native 1.6 method, this has the problem that the library which uses the JNI glue has to have some special hooks in place, but it works like a charm and fixes all the issues I’ve previously had with native libraries in applets.
These two components renewed my interest in Java as a glue layer between the webbrowser where your application logic resides and the hardware the user is depending upon. While earlier methods kind of worked but were either hacky or a real pain to implement, this is as clean as it gets and works like a charm.
And next time we’ll learn about scripting Java applets.
Last summer, I was into making git commits look nice with the intent of pushing a really nice and consistent set of patches to the remote repository.
The idea is that a clean remote history is a convenience for my fellow developers and for myself. A clean history means very well-defined patches – should a merge of a branch be neccesary in the future. It also means much easier hunting for regressions and generally more fun doing some archeology in the code.
My last post was about using git add -i to refine the commits going into the repository. But what if you screw up the commit anyways? What if you forget to add a new file and notice it only some commits later?
This is where git rebase -i comes into play as this allows you to reorder your local commits and to selectively squash multiple commits into one.
Let’s see how we would add a forgotten file to a commit a couple of commits ago.
You add the forgotten file and commit it. The commit message doesn’t really matter here.
You use git log or gitk to find the commit id before the one you want to amend this new file to. Let’s say it’s 6bd80e12707c9b51c5f552cdba042b7d78ea2824
Pick the first few characters (or the whole ID) and pass them to git rebase -i.
% git rebase -i 6bd80e12
git will now open your favorite editor displaying your list of commits since the revision you have given. This could look like this.
pick 6bd80e1 some commit message. This is where I have forgotten the file
pick 4c1d210 one more commit message
pick 5d2f4ed this is my forgotten file
# Rebase fc9a0c6..5d2f4ed onto fc9a0c6
#
# Commands:
# p, pick = use commit
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#
The comment in the file says it all – just reorder the first three (or how many there are in your case) to look like this:
pick 6bd80e1 some commit message. This is where I have forgotten the file
squash 5d2f4ed this is my forgotten file
pick 4c1d210 one more commit message
Save the file. Git will now do some magic and open the text editor again where you can amend the commit message for the commit you squashed your file into. If it’s really just a forgotten file, you’ll probably keep the message the same.
One word of caution though: Do not do this on branches you have already pushed to a remote machine or otherwise shared with somebody else. git gets badly confused if it has to pull altered history.
Isn’t it nice that after moths you still find new awesomeness in your tool of choice?
I guess I’ll have to update my all-time favorite tools list. It’s from 2004, so it’s probably ripe for that update.
is probably not a sensible thing to do (at least not if you intend to actually put something into that space the pointer points to).
At least it began breaking very, very soonish and consistently once TMyRecord got enough members – too bad though that I first looked at the completely wrong space.
Nothing beats the joy of seeing a very non-localized access violation go away after two hours of debugging though.
Remember last week when I was ranting about nothing working as it should?
Well – this weeks feels a lot more successful than the last one. It may very well be one of the nicest weeks I’ve had in IT so far.
The plugin system I’ve written for our PopScan Windows Client doesn’t just work, it’s also some of the shiniest code I’ve written in my life. Everything is completely transparent and thus easy to debug and extend. Once more, simplicity lead to consistency and consistency is what I’m striving for.
Yesterday, we finally managed to kill a long standing bug in a certain PopScan installation which seemed to manifest itself in intermittently non-working synchronization but was apparently not at all working synchronization. Now it works consistently.
Over the weekend, I finally got off my ass and used some knowledge in physics and and a water-level to re-balance my projector on the ceiling mount making the picture fit the screen perfectly.
Just now, I’ve configured two managed switches at home to carry cable modem traffic over a separate VLAN allowing me to abandon my previously whacky setup wasting a lot of cable and looking really bad. I was forced to do that because a TV connector I’ve had mounted stopped working consistently (here’s the word again).
The configuration I thought out worked instantly and internet downtime at home (as if somebody counts) was 20 seconds or so – the TCP connections even stayed all up.
I finally got mt-daapd to work consistently with all the umlauts in the file names of my iTunes collection.
If this week is an indication of how the rest of the year will be, then I’m really looking forward to this.