Tailscale on PFSense

For a bit more than a year, I’m a user of Tailscale which is a service that builds an overlay network on top of Wireguard while relying on OAuth with third party services for authentication

It’s incredibly easy to get going with Tailscale and the free tier they provide is more than good enough for the common personal use cases (in my case: tech support for my family).

Most of the things that are incredibly hard to set up with traditional VPN services just work out of the box or require a minimal amount of configuration. Heck, even more complicated things like tunnel splitting and DNS resolution in different private subnets just work. It’s magic.

While I have some gripes that prevent me from switching all our company VPN connectivity over to them, those are a topic for a future blog post.

The reason I’m writing here right now is that a few weeks ago, Netgate and Tailscale announced a Tailscale package for PFSense. As a user of both PFSense and Tailscale, this allowed me to get rid of a VM that does nothing but be a Tailscale exit node and subnet router and instead use the Tailscale package to do this on PFSense.

However, doing this for a week or so has revealed some very important things to keep in mind which I’m posting about here because other people (and that includes my future self) will run into these issues and some are quite devastating:

When using the Tailscale package on PFSense, you will encounter two issues directly caused by Tailscale, but both of which also seen in other reports when you search for the issue on the internet, so you might be led astray when debugging it.

Connection loss

The first one is the bad one: After some hours of usage, an interface on your PFSense box will become unreachable, dropping all traffic through it. A reboot will fix it and when you then look at the system log, you will find many lines like

arpresolve: can't allocate llinfo for <IP-Address> on <interface>
I’m in so much pain right now

This will happen if one of your configured gateways in “System > Routing” is reachable both by a local connection and through Tailscale by subnet router (even if your PFSense host itself is told to advertise that route).

I might have overdone the fixing, but here’s all the steps I have taken

  • Tell Tailscale on PFSense to never use any advertised routes (“VPN > Tailscale > Settings”, uncheck “Accept subnet routes that other nodes advertise.”
  • Disable gateway monitoring under “System > Routing > Gateways” by clicking the pencil next to the gateway in question.

I think what happens is that PFSense will accidentally believe that the subnet advertised via Tailscale is not local and will then refuse to add the address of that gateway to its local ARP table.

IMHO, this is a bug in Tailscale. It should never mess with interfaces its exposing as a subnet router to the overlay network.

Log Spam

The second issue is not as bad but as the effect is so far removed from the cause, it’s still worth talking about it.

When looking at the system log (which you will do for above issue), you will see a ton of entries like

sshguard: Exiting on signal
sshguard: Now monitoring attacks.
this can’t be good. Can it?

What happens is that PFSense moved a few releases ago from a binary ring-buffer for logging to a more naïve approach to check once a minute whether a log file is too big, then rotating it and restarting daemons logging to that file.

If a daemon doesn’t have a built-in means for re-opening log files, PFSense will kill and restart the daemon, which happens to be the case for sshguard.

So the question is: Why is the log file being rotated every minute? This is caused by the Tailscale overlay network and the firewall by default blocking Tailscale traffic (UDP port 41641) to the WAN interface and also by default logging every dropped packet.

In order to fix this and assuming you trust Tailscale and their security update policies (which you probably should given that you just installed their package on a gateway), you need to create a rule to allow UDP port 41641 on the WAN interface.

much better now

This, too, IMHO is a bug in the Tailscale package: If your package opens port 41614 on an interface on a machine whose main purpose is being a firewall, you should probably also make sure that traffic to that port is not blocked.

With these two configuration changes in place, the network is stable and the log spam has gone away.

What’s particularly annoying about these two issues is that Googling for either of the two error messages will yield pages and pages of results, none of which apply because they will have many more possible causes and because Tailscale is a very recent addition to PFSense.

This is why I decided to post this article in order to provide one more result in Google and this time combining the two keywords: Tailscale and PFSense, in the hope of helping fellow admins who run into the same issues after installing Tailscale on their routers.

Fun* with SwiftUI Beta 5 and 6

After a lot of momentum in Beta 4 where I finally got my rogue smide.ch client for the watch to work, Beta 5 and 6 were a bit of a letdown with regards to real-world usability.

Already with Beta 4, Apple has deprecated @ObjectBinding and BindableObject, both of which total staples of SwiftUI and totally required for you to do any kind of meaningful application because they provide the glue by which you hook your UI up to you actual application.

With this, even the last pieces of sample code shown at WWDC sessions about SwiftUI were now invalidated. What a breakneck speed of development.

On the other hand, this was also a case of parallel evolution inside Apple because all the old pair provided was also provided by the Combine framework with latter having the advantage of actually be usable not just in SwiftUI but anywhere in your applications.

So I can completely understand the reasoning behind the deprecation. If you are willing to clean this up, then the beta period is the time to do it.

However in Beta 4 on watchOS, while the old method was deprecated, the new way only worked partially: If you changed your class that was previously inheriting from BindableObject to now inherit from the correct Combine.ObservableObject, none of your publishes would actually be picked up by SwiftUI and the UI would remain static.

So in Beta 4, even though it was deprecated, I kept using BindableObject because that’s what was working. @ObjectBinding on the other hand, I could replace with @ObservedObject

But then, Beta 5 happened and the Beta 4 app crashed on startup.

Trying to compile it lead to a linker error because BindableObject now was gone for good. Note that the compiler was still just complaining about it being deprecated, but at link time, the symbol was missing and linking failed. This would also explain the crash at startup of the old Beta 4 app.

I’ve quickly replaced BindableObject with Combine.ObservableObject which made the app build again and run fine – on the simulator

On the real hardware, it would continue crashing on launch.

Even after installing the logging profile on the watch in order to get some information via the Console, all I got was a single log line entry from Carousel complaining about the launched process shutting down.

As this is just a fun project after all, this is where I stopped again, waiting what further betas will bring.

After a while, Beta 6 came and went. It brought no change.

Then, Beta 7 happened, but I didn’t even bother trying to recompile without an updated Xcode which finally happened today and, spoiler alert, my App is back in a running state. No further changes were required.

So it all wasn’t my fault after all.

Next time I’ll talk about the changes I’ve done since Beta 7 and Xcode Beta 6

Fun with SwiftUI – Beta 4

After hitting yet another wall with Beta 3, the moment Beta 4 was released I’ve updated all my devices, recompiled the application and looked at it on the watch.

This time, without any change on my part, things just fell into place:

Location permission worked correctly. I had my list of Bikes sorted by location

What I didn’t like as much though were all the deprecation warnings I was already accumulating. During this years Beta Period, none of the APIs presented at WWDC were finalized. SwiftUI and Combine are still very much in flux and the APIs change on a biweekly basis.

At the time of this writing, Beta 5 has already removed all older deprecations and has added even more deprecations. We’re now at a point where most of the videos from WWDC sessions that are explaining SwiftUI and combine are not applicable to the real world any more, but that’s for another post.

Some lipstick on the pig

With things mostly working right, there was one thing that was bugging me in the list of bikes you’re seeing in the screenshot above: The distances to my current location were manually formatted in meters, but I knew that Apples platforms come with very good locale dependent unit formatters, so I wanted to fix that.

MeasurementFormatter has a .naturalScale unit option that’s supposed to pick scale and unit automatically dependent on the user’s locale. In case of distances, here in Switzerland, I would expect a distance to be shown in meters up until 1 km at which point I would expect a distance to be shown in km with one fractional digit of accuracy.

But that’s now what MeasurementFormatter does: It insisted on using km and it insisted on three fractional digits of accuracy. That’s why I’ve decided to format on my own. But I knew there must be a proper solution.

It tuns out, there is – but it’s part of MapKit, not of Foundation: There’s MKDistanceFormatter to use for this and while MeasurementFormatter has a unitOptions property, the MKDistanceFormatter has a unitStyle property which you can set to .abbreviated to get to the proper effect.

So I have added that and also used battery icons based on SFSymbols to display the bikes battery levels like so:

we’ll never know why there’s no battery.50 image. Only .100, .25 and .0

Reactive warts in SwiftUI

Remember when I said that the whole UI hierarchy was going to be built from one parent SwiftUI view that would decide on a state object? That’s the design I totally went with:

My main ContentView is just a big switch statement, completely exchanging its subview hierarchy dependent on what the global state handler object thinks should be currently active.

As you can see in above code, there’s only .ListingBikes – there’s no state to list a single bike. That’s fine, because that’s up to that BikeList view to decide to instead of listing a list of bikes it wants to show a single bike.

I did this using a NavigationLink nee NavigationButton setting its destination to the detail view:

What’s nice is that you get a free sliding animation and a free back button out of this. However, what’s not as nice is that if you do this, the detail view gets pushed as another native view on top of the existing view.

Which means that even when the big switch statement from the screenshot above causes a different sub-view to be rendered (and it does get rendered), then the additional view pushed by the NavigationLink remains shown on top and does not get closed.

In the end, here on Beta 4, I went with a NavigationDestinationLink so I could first close the detail view and then tell the state handler I wanted to create a booking.

At the time of this writing (Beta 5 has just been released), NavigationDestinationLink is already deprecated again and the state whether the destination is showing or not can be passed as a $binding, however, also at the time of this writing, this currently messes with the free back button

Another thing that falls in the same bucket of re-painting the whole hierarchy does not in fact repaint the whole hierarchy is a SwiftUI View’s navigationBarTitle modifier: if you set that once on any subview, it will persist even through you removing that subview and replacing it by another which doesn’t use the navigationBarTitle modifier.

Meaning that setting a property on a subview as an effect on global state.

This feels wrong to me.

First booking

Anyways – enough complaining. With all of this in place, I did my very first commute with a smide bike using nothing but my watch. Here you see me at the end of the trip, ready to end the booking:

That felt great.

What doesn’t feel great is the mess Beta 5 made out of my nicely layouted UI. But that’s for another day.

Fun with SwiftUI – Beta 3

After being abruptly stopped last time by a compilation step that would not complete in a finite amount of time, I let the project rest until Beta 3 was released.

I could probably have found my mistake, but I was also willing to give it another two weeks and then see whether the compiler would just tell me what I was doing wrong.

Which, when Beta 3 was released, it actually did. Trying to recompile the project would immediately be stopped at a clear error message (I don’t actually remember the details any more), but the fix was very easy after all.

Motivated to finally move on, I finished hooking up my state handler to the UI itself and I was finally at a point where I would run this skeleton in the simulator.

Simulator says: nope.

First the good news: Apple was right when they proudly said that they have improved the watch simulator workflow: Launching the simulator in stand-alone mode finally is a sub-second endeavor and so is actually launching your app in the debugger.

Working this way is actual honest-to-god fun.

Yes, things should just work like this out of the box, but until now, they never did: Running the watch simulator meant also running the phone simulator and proxying all debugger operations through the phone simulator, including breaking connections and horrible, horrible lag.

But none of this still happens in WatchOS 6: The simulator can run on its own and it launches instantly. No connection issues.

At least not to talk to Xcode…

My initial excitement about things working so well was abruptly dampened by the fact that all network access I was trying to do in the simulator ended up failing with a generic error and in the log output some backend component would complain about losing connection to the background transfer service.

Of course I first assumed to be the source of the problem and I spent two hours trying to find out what I was doing wrong.

I shouldn’t have, because my last resort was to check the Apple Beta forums and there, I didn’t even have to bother posting a question: Others had the issue too and the solution is to just use a real device.

Onwards to the real device

Updating a watch to a Beta OS is a tricky proposition: There is no (official) way to ever downgrade and stuff is known to be shaky.

Also, my watch is the one single computer I use that produces data for which I have no backup and re-creating the data is a (literally) painful experience.

I’m talking about workout data.

For two years now I’m running 10ish km every day and while I know rationally that the actual act of running is what counts, unfortunately, my subconscious only accepts a run as having happened when it’s also tracked in the Activity app and when the rings are closed.

So would I dare updating to a beta version knowing that I can’t downgrade and that the watch is producing irreplaceable data that I heavily rely on?

Of course I would. 🤓

But only after checking our local electronics retailer to make sure that they had a replacement watch on stock if worse comes to worst. Yes. I know that you can ask Apple to downgrade a bricked/unsuable watch, but that would mean days without the watch and days without my runs being tracked.

Inacceptable.

Anyways. Updating to watchOS 6 went fine and a small test walk around my house has shown that tracking workouts was still generally working fine. So I was all set to try it on the real device.

Moving forward on the device

The good news: While not as fast as the simulator, deploying and debugging on the watch still is considerably quicker and more reliable than it ever was on any previous combination of Xcode, iOS and watchOS.

Debugging still involves proxying though the phone, but now it’s reliable. Over the course of 4 weeks of doing this (spoiler alert), I only had one or two instances where Xcode wouldn’t talk to the watch and more and I had to restart my computer, my phone and my watch to get connectivity restored.

Judging by other people’s prior experiences, this is a huge step forward.

The other good news: Network requests to indeed work on the real device. My client could fetch a JWT token from the smide.ch service and it could get a list of currently available bikes.

Impressive rendering speed

I have chosen the most naïve implementation possible and just fed the whole list of ~200 bikes directly into the UI framework. No dealing with cell reuse, no limiting the size of the list, nothing. Just “hey SwiftUI, please render this list of 200 bikes”.

And render it does: It’s quick and scrolling through the whole list is buttery smooth without doing any kind of optimization work. And once the next roadblock is fixed (see below) and the list gets dynamically re-sorted as my location changes, that too is buttery smooth.

I’m getting away with telling the framework that the list has changed and needs repainting and I just pass it a new updated list of bikes. The change is instantaneous. Even though it’s a new list of 200 items.

This is so much fun. I shouldn’t need to care about minimizing updates. I shouldn’t need to care about cell reuse. I shouldn’t need to deal with this. And with SwiftUI, I don’t have to.

Location roadblocks

Excited, I moved forward to asking for location access and using location to sort the list of bikes.

And this is where things ground to a halt.

Whenever my independent watch app extension would be launched, I would be calling CLLocationManager.authorizationStatus() which would tell me that my status was .notDetermined, so I would ask for permission.

My delegate callback would be called with .authorizedWhenInUse, but CLLocationManager.authorizationStatus() would still return .notDetermined and all attempts at calling location specific API would be ignored.

As this was my first strides into CoreLocation, I assumed this to be my fault and spent a lot of time debugging this, moving code around and trying out things, but not matter what I did, the effects didn’t change.

Then I tried Apple’s Sample Code from 2016 which of course worked fine even after I changed the integrated watch app to be usable independently.

After a few more hours of trial and error, I finally was able to pin it down though: In Beta 3 (and presumably earlier Betas too), the CoreLocation permission management is broken if your watch app is a completely independent watch app.

If it has a companion iOS app, then requesting location permission is fine, but when you have a watch app without any iOS app which has a plist that looks like this:

<key>WKWatchOnly</key>
<true/>

Then requesting location permission would trigger a race condition where your permission is simultaneously granted and not granted.

I could have caved and made an empty iOS companion app at this point, but I decided to report this issue using Feedback Assistant and call it another two weeks.

The relief I felt when I’ve seen Apple’s official code sample to fail the same way as my sample code did the moment I set that WKWatchOnly flag was one hell of a feeling.

I wasn’t doing it wrong. I wasn’t losing my mind.

Next time, things will finally fall into place, but only after dealing with deprecations.

Fun with SwiftUI – Beta 2

After spending the first two weeks of the beta period to get a foundation going, I was eager to start working on the actual watch app.

This was right about the time when Beta 2 hit, so first I’ve upgraded to that and then started with the Watch project.

Building the UI

Eager to play around with SwiftUI, the first thing I have done was to actually just create a skeleton UI:

What immediately sprang to my mind as I was working on this was the fact that the built-in preview feature of SwiftUI forces you to keep your views self-contained and to keep the dependencies small and to keep your data easily mockable.

Otherwise you will suddenly be in the position where your Xcode preview requires working network connections and a lot of application state.

I’ve also learned that Beta 2 was still on very shaky grounds before running the actual code even once: My attempts to display a map view caused Xcode to crash completely the moment it tried to paint the UI preview, so I’ve stubbed that out to just be a rectangle

But overall, designing (if you can call it that) the skeleton UI went very quickly (a matter of a few hours) and I was eager to hook everything up.

A hard stop

After working on the UI, the next step was to produce a backend that orchestrates the actual application state. This single class is the only thing that keeps track of state in the application and based on which the UI decides what to paint and how and where the UI will call into in order to change the overall state (for example when the user logs in or when they start a booking)

This is what (at the time) you would use @ObjectBinding and BindableObject for.

My next step, thus, was to create what I called the ApplicationStateHandler which I had implement the BindableObject protocol.

That handler itself would expose a state property which could have one of various values of an ApplicationState enum. The main SwiftUI view would basically be a huge select statement over that state property and then decide what actual view to render based on the state.

This was my plan, but no matter what I did, the moment I had ApplicationStateHandler implement the BindableObject protocol, I would put Xcode 11 Beta 2 in a state where it was using 100% of each of my 8 CPU cores while trying to compile my code.

So in the end, I wasn’t stopped by incomprehensible error messages (I got my share of those too), but by a compilation run that did not seem to want to complete in finite time.

Instead of solving the halting problem, I decided to wait another two weeks because I already had other non-project related things on my plate.

Stay tuned for next time to see what stopped me hard in Beta 3

Fun with SwiftUI – Beta 1

As explained before, I’ve decided to scratch my own itch and write an independent Apple Watch client for the smide.ch bike sharing service.

The first step to getting from the idea to the final watch app wasn’t actually involving the Watch at all: Before I could get started, I needed to know how the existing smide clients actually work and how to talk to their server.

Then I wanted to have a unit-tested library that I could use from the Watch Frontend.

On top of that library, I wanted to have a command-line client for easier debugging of the library itself.

And only then would I start working on the frontend on the watch.

Preliminaries

So as the Developer Beta 1 for XCode 11, WatchOS 6 and Catalina rolled out, the first few days of development I spent reverse-engineering the official Smide Client.

As always, the easiest solution was to just de-compile their Android Client and lo and behold, they are making use of retrofit to talk to their server which lead to a very nice and readable interface documentation right in my decompiler

Armed with this information, a bit of grepping through the rest of the decompiled code and my trusty curl client, I was able to document the subset of the API that I knew I was going to need for the minimal feature-set I wanted to implement.

In order to have a reference for the future, I have documented the API for myself in the OpenAPI format

This is useful documentation for myself and if I should ever decide to make the source code of this project available, then it’ll be useful for anybody else wanting to write a Smide client.

Moving to XCode: SmideKit

Now that I had the API documentation I needed, the next step was to start getting my SmideKit library going.

Even though there are tools out there that generate REST clients automatically based on an OpenAPI spec, all the tools I looked at produce code that relies on third-party libraries, often Alamofire. As XCode 11 was in a very rough shape already on its own, I wanted to minimize the dependencies on third-party libraries, so in the end, I’ve opted to write my own thin wrapper on top of URLSession

The SmideKit library

SmideKit is a cross-platform (by Apple’s definition) library in that the code itself works across all of Apple’s OSes, but there are individual targets for the individual OSes

But by manually setting the Bundle Name to $(PRODUCT_NAME) in the individual Info.plist files, I can make sure that all projects can just import SmideKit without any suffixes.

As this library is the most crucial part of the overall project, I have written unit testes for all methods to make sure we correctly deal with expiring tokens, unresponsive servers and so on.

The command line client

The first user of SmideKit would be a macOs command-line frontend called smidecli. It would offer various subcommands for listing bikes, booking them and ending bookings.

Here’s a screenshot of me booking a bike

Going from nowhere to the working command-line client has taken me the whole period of Beta 1. Two weeks is a long time but between my actual day job and my newly put upon me parenting duties, my time was a bit limited.

Still. It felt good to go from nowhere to writing a library, writing a command-line frontend and then actually using it to book a bike. On the other hand: None of the code written at this point had anything to do with the announcements of WWDC. All work done could just as well have been done on the old SDKs. But still: Having a good foundation to stand on, I was sure was going to pay off.

Next time: Adventures in Beta 2

Fun project in SwiftUI

This year’s WWDC really shook the Apple Ecosystem with probably the most announcements ever happening at a single conference.

Three of the announcements when put together finally pushed me over the line to scratch a personal itch of mine that I was having for a bit more than a year: I’m a very happy customer of the smide.ch bike sharing service here in Zürich, Switzerland: Their electric bikes are well maintained, extremely fun to use and readily available to the point that they have become my main means for transport for my daily commute.

On the other hand, as the Apple Watch has become more and more capable over time between updates to the OS and to the hardware itself to the point where I can now theoretically leave my phone at home and just rely on the watch.

The last remaining hard stopping block was the smide client which so far is only available on the phone, but not on the watch.

So there you see my itch that needed scratching.

In this context, this year’s WWDC was the perfect storm for me: independent Watch apps, SwiftUI and especially SwiftUI on the watch as a real native UI framework unshackled from the constraints of the purely Storyboard based hacks in WatchKit.

The moment I dug through the announcements, I knew: I need to make myself a Smide client and after 3 Beta releases from Apple, that’s what I have done:

After launching the App and assuming you’re logged in, it lists the bikes around you, sorted by distance to your current location (they have both black (smaller) and white (bigger) bikes – hence the coloring):

Tap any bike and you get a detail view including a map

Start the booking and you get some booking information

This feature-set is very limited compared to the official client:

  • Logging in only works for accounts that were created with an email address and a password. There is no way for my third-party client to possibly work with any oAuth provider
  • There is no way to report an issue with a bike
  • The client doesn’t currently take into account the free 10 minute reservation
  • The official client does some additional user interface activity reporting which my rogue client doesn’t do.
  • There is zero payment related functionality: As I personally have a 3 year subscription, I don’t need it and besides, this is a rogue client and I can’t and don’t want to deal with their payment system.

Still. This was a fun experience to develop and to keep up-to-date between the various Beta releases, all of which deprecated some essential functionality that was also shown off during conference sessions.

Over the next few days I’m going to write down a development diary like I have done for tempalias back in the days.

To put a bit of a damper on your expectations: As this work is not sanctioned by Smide themselves and as it’s based on reverse-engineering their existing client and because this year’s API for SwiftUI and Combine is still very much in flux, I’m reluctant to release the source code of this.

Tracking runs with the Apple Watch

When I started running this summer, I also wanted to make use of my Apple watch to keep track of my routes and my speed over time, so I looked into the various apps and services around that.

Generally, there are two parts to tracking a run: One part is the actual data gathering that happens while you’re running and the other part is the analysis and comparison other other runs afterwards.

Unfortunately, of all the applications I looked at, none excelled at both, so in the end what I’ve ended up with is writing custom code to give me the best of both worlds.

Here’s what I’ve looked at.

Apple Workouts.app

The built-in Workout app of the watch, being a watch-native app made by Apple, is more equal than other apps: It’s the only app that allows you to trigger the screen lock while in the app and with WatchOS 4 it’s also the only app that gives you very easy access to the media controls. And finally, it’s the only app that can log its tracked workout and movement Activitiy in the Activitiy app in their actual colors instead of just gray (yes. very important this).

It offers very readable data on-screen while it’s running, it can send timely notifications as you pass another kilometre and it never crashes.

Looking at the Map of the run in the Activity app, it also collects very accurate location data.

As good as it is for collecting data, as bad it is for analysis of the data though: The best you can do is have a look at a single workout. There’s no way to compare two – unless you take screenshots and do that manually.

There is also no way to export the data: In the workout details there is a share button, but that just exports a corny text and a useless picture. No detail is included.

Screen Shot 2017-09-04 at 08.51.13.png
Yes. This is all you get from exporting a run. I love the picture. So useful.

So for any analysis you want to do based on runs recorded with the Workouts app, you have to first manually transfer data from screenshots to some other machine readable form and even then: The screenshots alone don’t provide nearly enough useful data.

Strava

This is the other extreme in the list of apps I looked at: It provides excellent analysis and it has an extremely motivating high-score list for user-provided segments of a run. You don’t have to match routes exactly – the moment you run through an existing previously created segment, you’ll be able to compare your effort to others.

Screen Shot 2017-09-04 at 08.47.09.png
Segment rankings (this segment is uphill)

It’s also great at automatically matching previous runs over the same route, so you can compare your runs over time.

Screen Shot 2017-09-04 at 08.44.32.png
Getting faster over time (this mostly uphill too)

The other social features it offers don’t interest me, so I can’t really talk about them.

However: As good as the analysis is, as bad its recording feature is: Of all the apps I looked at it provides the least amount of detail during the run and, what’s worse, its GPS tracking is extremely inaccurate and unreliable.

I’m always running having my phone with me – mainly for easy access to all my media and to Overcast and also because most of my runs I do on my way home from the office where I need the phone anyways. Strava doesn’t make use of this but instead solely relies on the watches GPS which is much less accurate than the phones.

I can understand this: The device is smaller, so it’s harder to put in powerful antennas, it has way less battery and a much weaker CPU than the phone, so it just can’t be as good. It’s totally ok for when you only have the watch with you, but when you have the phone with you, it’s a shame if the app can’t use it.

Runkeeper

Runkeeper uses both the watch and the phone for location tracking and it provides a great UI while the workout is ongoing.

Its analysis features aren’t as good as the ones from Strava though. It doesn’t do the automated segment high-scoring and it’s not as good at comparing runs over the same route with each other.

And finally, the UI of the site doesn’t look as polished as does Strava’s – but that’s just a matter of taste I guess.

… master of none

For all of July and August, my mode of operation was to use Runkeeper to acquire the data during the run and then to export a .gpx file from their site and to import it into Strava.

This gave me the best of both worlds: Very good data gathering and very good data analysis.

However, I wasn’t entirely happy with this either as the process was somewhat cumbersome and, lately, unstable.

Probably caused by iOS 11 Beta, I’ve seen various failure modes related to Runkeeper, all of wich are very annoying:

  1. The workout might start on the Watch but it will not manage to also start it on the phone. This way, the workout will be tracked, but no route data will be saved.
  2. Runkeeper on the phone will crash after about 10 minutes. There’s no indication of this happening, but the result will be that a 10 minutes run is logged instead of the real data on the watch. If this happens, there is no way to even just get to the data without the route.

Issue 1) I could work around often by launching Runkeeper manually on the phone, then starting the workout on the watch and then making sure that the workout would also start on the phone.

If that happened, then route data was tracked correctly.

Unfortunately, sometimes, this stopped working all-together and the only way for the watch to talk to the phone again was to completely uninstall and reinstall Runkeeper on both the Phone and the Watch. This is annoying when you want to start running, but you can’t because the Software-gods have put 20 minutes of fiddling with the App Store in front of you (also, Runkeeper is bigger than the App Store’s 3G download limit, so you better have wifi  available).

Issue 2) is much worse though: There’s no indication of it happening. You’d think that the blue bar “Runkeeper is actively using your location” on the phone would be a good indicator, but it isn’t: When the crash happens, the bar stays there until you unlock your phone. Then it goes away.

So there’s no way to be sure unless you periodically unlock your phone which is very annoying and distracting during the run – especially as you’re sweaty and TouchID won’t work most of the time (I use a strong 25 character password).

I know – even if it isn’t tracked, a run is a run. But it certainly doesn’t feel that way and how it feels is very important to keep motivated to doing this – especially under bad weather conditions.

let’s just hack it

Now, admittedly, these are very likely beta-woes that will eventually solve themselves. We’re pretty far into the beta cycle though (Beta 9 at the time of this writing), so I’m suspicious that these issues won’t be fixed come release but will have to wait for a future update to either Runkeeper or the OS.

Losing about a third of my runs to software issues felt really unacceptable to me, especially considering that Runkeeper still wasn’t offering some features that the workout app was (like enabling the screen lock – which is important when running in the rain).

However, when I looked again at the WWDC sessions this year, I found out that IOS11 will finally offer an API to read and write route data for workouts. This means that the data you track using Apple’s built-in app will finally be available to other apps to read.

This would give me the best of all worlds: Use the best data-gathering app and export it to the best analysis app; side-stepping the stability issues.

IMG_1324.png
Such Wow. Much UI.

So, this weekend, I hacked together a quick solution (MIT licensed) that does exactly this. It lists you all your workouts and if you tap one, it will eventually show you a share-sheet, allowing you to select a location to store a .gpx file to.

That file contains all the information required for Strava to do its analysis.

In a perfect world, this app would of course upload directly to Strava. And it would not block the UI thread while it’s exporting the gpx file. And it would actually have some UI to speak of.

But this was a quick-hack that solved an issue for me – and who knows – maybe it will fix it for you.

If you need a real solution for this, Twitter user @dwlz is apparently working on a real app that will be usable for normal people and I’ll definitely switch to that when it’s ready. But until then, I can finally track my runs with the peace of mind of having a crash-free solution that still provides the best analysis possible.

Screen Shot 2017-09-04 at 08.54.14.png
A run tracked with the Workout app and uploaded to Strava using my hack

another fun project: digipass

As a customer of digitec, I often deal with their collection notices which I get via email and which invite me to go to their store and fetch my order (yes. I could have the goods delivered, but I’m impatient and not willing to pay the credit card surcharge).

Ever since Passbook happened on iOS 6, I wished for these collection notices to be iOS Passes as they have a lot of usability benefits:

  • passes are location aware an pop up automatically when you get close to the location
  • Wallet automatically turns the screen brightness all the way up
  • passes could potentially be updated remotely
  • once added to the Wallet, passes don’t clutter your mailbox and you’ll never lose them in the noise of your inbox.

So my latest fun project is digipass.

Next time you get a digitec collection notice, just forward it to

digipass@pilif.me

After a few seconds, you will get the same collection notice again, but with the PDF replaced by an iOS Wallet pass that you can add to your Wallet.

I have slightly altered the logo and the name to make it clear that there’s no affiliation to digitec.

The pass will be geo-coded to the correct store, so it will automatically pop up as you get close to the store.

As I don’t want access to your digitec account and because digitec doesn’t have any kind of API, I unfortunately can’t automatically remove the pass when your fetch your order – that’s something only digitec can do.

The source code for the server is available under the MIT license.

Disclaimer:

  • I’m not affiliated with digitec aside of being a customer of theirs. If they want me to shut this down, I will.
  • I am not logging the collection notices you’re forwarding me. If you don’t trust me, you can self-host, or redact the notice to contain nothing but the URLs (I need these in order to build the pass).
  • This is a fun project. If it’s down, it’s down. If it doesn’t work, submit a pull request. Don’t expect any support
  • The LMTP daemon powering this is running in my home. I have a very good connection, but I also have not signed an SLA or anything. If it’s down, it’s down (the message will get queued though).
  • The moment I see this being abused, it will be shut down. Just like my previous email based fun project

AV Programs as a Security Risk

Imagine you were logged into your machine as an administrator. Imagine you’re going to double-click every single attachment in every single email you get. Imagine you’re going to launch every single file your browser downloads. Imagine you answer affirmative on every single prompt to install the latest whatever. Imagine you unpack every single archive sent to you and you launch ever single file in those archives.

This is the position that AV programs put themselves on your machine if they want to have any chance at being able to actually detect malware. Just checking whether a file contains a known byte signature has stopped being a reliable method for detecting viruses long ago.

It makes sense. If I’m going to re-distribute some well-known piece of malware, all I have to do is to obfuscate it a little bit or encrypt it with a static key and my piece of malware will naturally not match any signature of any existing malware.

The loader-stub might, but if I’m using any of the existing installer packagers, then I don’t look any different than any other setup utility for any other piece of software. No AV vendor can yet affort to black-list all installers.

So the only reliable way to know whether a piece of software is malware or not, is to start running it in order to at least get it to extract/decrypt itself.

So here we are in a position where a anti malware program either is useless placebo or it has to put itself into the position I have started this article with.

Personally, I think it is impossible to safely run a piece of software in a way that it cannot do any harm to the host machine.

AV vendors could certainly try to make it as hard as possible for malware to take over a host machine, but here we are in 2016 where most of the existing AV programs are based on projects started in the 90ies where software quality and correctness was even less of a focus than it is today.

We see AV programs disabling OS security features, installing and starting VNC servers and providing any malicious web site with full shell access to the local machine. Or allow malware to completely take over a machine if a few bytes are read no matter where from.

This doesn’t cover the privacy issues yet which are caused by more and more price-pressure the various AV vendors are subject to. If you have to sell the software too cheap to pay for its development (or even give it away for free), then you need to open other revenue streams.

Being placed in such a privileged position like AV tools are, it’s no wonder what kinds of revenue streams are now in process of being tapped…

AV programs by definition put themselves into an extremely dangerous spot on your machine: In order to read every file your OS wants to access, it has to run with adminitrative rights and in order to actually protect you it has to understand many, many more file formats than what you have applications for on your machine.

AV software has to support every existing archive format, even long obsolete ones because who knows – you might have some application somewhere that can unpack it; it has to support every possibly existing container format and it has to support all kinds of malformed files.

If you try to open a malformed file with some application, then the application has the freedom to crash. An AV program must keep going and try even harder to see into the file to make sure it’s just corrupt and not somehow malicious.

And as stated above: Once it finally got to some executable payload, it often has no chance but to actually execute it, at least partially.

This must be some of the most difficult thing to get right in all of engineering: Being placed at a highly privileged spot and being tasked to then deal with content that’s malicious per definitionem is an incredibly difficult task and when combined with obviously bad security practices (see above), I come to the conclusion that installing AV programs is actually lowering the overall security of your machines.

Given a fully patched OS, installing an AV tool will greatly widen the attack surface as now you’re putting a piece of software on your machine that will try and make sense of every single byte going in and out of your machine, something your normal OS will not do.

AV tools have the choice of doing nothing against any but the most common threats if they decide to do pure signature matching, or of potentially putting your machine at risk.

AV these days might provide a very small bit of additional security against well-known threats (though against those you’re also protected if you apply the latest OS patches and don’t work as an admin) but they open your installation wide for all kinds of targeted attacks or really nasty 0-day exploits that can bring down your network without any user-interaction what so ever.

If asked what to do these days, I would give the strong recommendation to not install AV tools. Keep all the software you’re running up to date and white-list the applications you want your users to run. Make use of white-listing by code-signatures to, say, allow everything by a specific vendor. Or all OS components.

If your users are more tech-savy (like developers or sys admins), don’t whitelist but also don’t install AV on their machines. They are normally good enough to not accidentally run malware and the risk of them screwing up is much lower than the risk of somebody exploiting the latest flaw in your runs-as-admin-and-launches-every-binary piece of security software.