Using your ISP’s built-in VPN in Japan

It’s a little known fact that the routers in Japanese ISPs are incredibly full featured when compared to the gateways you get abroad. One feature that is standard in most of them is VPN. In this post I’ll show you how I set-up my home network that allows me to access my GPU desktop from anywhere.

The first step is to look at the bottom of your router to find the default username and password. Then log in to the router which is typically the .1 address in your subnet. In my home it’s

NTT’s Hikari Router home screen

Expand the 詳細設定 (Advanced Options) menu on the left to show the VPN server settings link.

VPN Settings page

Check the 有効 checkbox to enable your VPN and take a note of the pre-shared key by clicking the 表示 button. Next make a VPN account by clicking one of the edit (編集) account slots and giving it a username and a (very strong) password.

To connect to the VPN on Mac open up the Network settings in the Setting app and click the add button to create a new network. Select VPN and choose the L2TP over IPSec type and give it a name.

Mac OS Network Settings

The Server Address is your public IP address which you can get by Googling for “Whats my IP address”

Authentication Settings

Click the Authentication Settings to enter your password and Shared Secret, click OK then Apply and you’ve finished the set-up. To test it you have to go outside your LAN but if everything is correct it’s as simple as clicking Connect.

Next would be to connect to your GPU machine. If you have a Linux machine you should be able to simply SSH into it using its local IP address. I use Windows for my desktop machine so I use Remote Desktop. You only have to enable it in the Remote Desktop Settings under System in the Settings Window like so:

If you use Jupiter (which I don’t) you can start it up and log into it via your browser as long as you’ve set the listen address to so that it accepts remote connections.

Hello ND4J

I haven’t blogged in over 4 years so I’ve decided to kick-things off again by doing a series on DeepLearning4J which I’m using more and more of recently. DL4J is great because there’s no need to translate the models to something else for production. You deploy them to production by just using them inside a Tomcat web app or inside Camel pipeline right along side what you are deploying today. It’s also very ops friendly, since they can monitor the apps with JMX like they are
used to.

Neural networks are a different kind of beast than the other machine learning algorithms. If you’ve ever played around with them, chances are that they almost always failed compared to other algorithms. There’s a lot of technique that goes into training a neural network effectively. Luckily Deep Learning frameworks have been giving us access to these techniques by name. When you know what the name of the thing you are looking for it’s much easier than doing the matrix math by hand but it’s daunting to newcomers. The basics of the matrix math isn’t too bad however, and it’s worth being familiar with it.

Let’s start this series in the spirit of hello-tensorflow (and also borrowing heavily from this great tutorial). I’m going to make a single neuron and try and train it to output 1 when I input 0 and output 0 when I input 1. Only instead of numpy or tensorflow, I’m going to use ND4J (the matrix library that DL4J is built on top of) so that I can use Scala.

Deep Learning is almost synonymous with GPUs but since we have only a single neuron it’ll run best on a CPU. First you need to create a build.sbt file and put it in an empty directory. It should look something like the following:

name := "hello-nd4j"

version := "1.0"

scalaVersion := "2.11.8"

libraryDependencies ++= Seq(
  "org.nd4j" % "nd4j-native-platform" % "0.7.2",

Next create a main class. You can just put a scala file in the same folder as build.sbt:

object HelloND4J extends App {
  println("hello world")

Now run sbt run and make sure it runs.

Next add some imports:

import org.nd4j.linalg.api.ndarray.INDArray
import org.nd4j.linalg.factor.Nd4j._
import org.nd4j.linalg.ops.transforms._

Now you should be able to do the same matrix operations that you can with Octave/Matlab and Numpy! Time to make some artificial neurons. First, lets create the input data and the outputs we want to learn. (Note: I’m embedding the bias into the input arrays as a constant 1.0 in the last position.)

val input1 = create(Array(0.0, 1.0))
val input2 = create(Array(1.0, 1.0))
val inputs = Array(input1, input2)

val label1 = create(Array(1.0))
val label2 = create(Array(0.0))
val labels = Array(label1, label2)

Again, in this trivial example, we are trying to learn the NOT function using only multiply and add instead of boolean instructions or if statements. Let’s make the computer figure out what values it needs to make it work.

A neuron is usually defined as some a function that takes an input, multiplies it by a set of weights adds a bias and outputs the value of applying the result through some sort of activation function. Lets define the weights and biases:

var W = create(Array(0.8, 0.5))

Since there is only one input, we only need a single weight and a single bias. I’ve initialized them to something random.

Next lets define a function that does the computation I mentioned above:

def forward(input: INDArray, w: INDArray) =
  sigmoid(input mmul w.transpose())

I’m using sigmoid as the activation function here but it could be something else. I encourage you to give it a try. You can now use this to make predictions like so:

val prediction1 = forward(input1, W) // 0.62
val prediction2 = forward(input2, W) // 0.79

It gives us 0.62 when we input a 0, which is closer to 1 than 0 but 0.79 when we input a 1, which is not what we want. To get the computer to figure out what it needs to change we need to measure how wrong it is. There are many
different ways to do this but for this example I’m going to use simplest; the difference between the right answer and the prediction:

def loss(prediction: INDArray, label: INDArray) =
  label sub prediction

Now we need a way to use the information about how wrong it is to find out what direction we need to change W to get a more correct answer. Just like you can use derivatives in physics to get the acceleration of a ball from a start and an end point, you can use the derivative of our forward function to get the acceleration between the right answer and our prediction. Tensorflow can figure out the derivatives for you, but since we are doing things manually we’ll just define it in a function which for sigmoid it’s simply x * (1 – x):

def backward(out: INDArray) =
  out mul (out add (-1))

Calling backward will give you a delta for W that you can multiply by the right answer to get a better W. This delta is only for a single sample. The W for the current input might not be the right W for the other inputs. For that reason, you usually multiply the delta by a small learning rate, so that you inch closer to the right answer for all the inputs.

Now you can put everything together to make a training routine by wrapping it in a for loop like so:

val learningRate = 0.1
for (i <- 0 until 10000) {
  val pred = forward(inputs(i % 2), W)
  val error = loss(pred, labels(i % 2))
  val deltaW = backward(pred)
  val updateW = error mul deltaW

  W = W sub ((inputs(i % 2).transpose() mmul updateW) mul learningRate)

After running through each example about 5000 times calling forward on should be really close to the answer we want:

println(s"0.00 => ${forward(input1, W)}") // 0.00 => 0.94
println(s"1.00 => ${forward(input2, W)}") // 1.00 => 0.05

Now if we look at W we can see the values you need to do NOT without an if statement:


Which should show you -5.67 and 2.72 if you’ve been following along.

In the next part of the series I’ll go over how to use DL4J’s higher level API to define more complex neural networks.

How Cappuccino handles pass-by-reference APIs in Cocoa.

I’ve been using Cappuccino at work recently and I came across a corner of Objective-J that I previously didn’t know about: how to use enumerateObjectsUsingBlock.

One of the nice things about Objective-C is that it has blocks.  Blocks allow you to make functions that take other functions as arguments, all while preserving scope. One of the most popular block-using APIs is the NSArray method enumerateObjectsUsingBlock. It takes a block that has three arguments: the current value of the iteration, the current index and a reference to a Boolean stop variable. If you set the variable to YES the loop will stop (much like the break keyword, which only works to break out of while and for loops.)

This is a problem for JavaScript because it always passes by value. Since it’s very cheap to create objects in JavaScript you could just pass an object with a property you can set, but that means that you have to remember the property name to set and the caller has to remember to copy that value back out. The Cappuccino developers thought of using a function that closes over an object in the parent scope via a macro called AT_REF.

To use a cappuccino AT_REF you simply call it and pass in the new value you want to set. Like so:

[array enumerateObjectsUsingBlock:function (item, i, stop) { 

In the implementation of enumerateObjectsUsingBlock you’ll see something like the following:

var stop = NO; 
for (var i = 0; i < length; i++) { 
    block(arr[i], i, BY_REF(stop)); 
    if (stop) 

The AT_REF macro expands to something like:

(function(val) { 
    if (arguments.length) 
        return (stop = val); 
    return val; 

That way you can both set the value by calling it with an argument and get the value by calling it with no arguments.

It’s brilliant if you ask me. But then again, pretty obvious when you think about it.

In the future the devs would like to integrate this into the language by using @ref() and @deref(), (hence the name of the macro).

To define methods that use AT_REF in your own code you can simply @import <Foundation/Ref.h>.  Clients of your API just need to know to call the argument to set and get it’s value.

What worries me about Scala

I’ve recently been rediscovering podcasts.  I’m particularly fond of the NodeUp podcast, but I also listen to podcasts about Rails, Clojure, and Scala.  The Scala podcast is by far the most academic; taking a shot every time they say the word “type” makes for a pretty nice drinking game.

I must admit I like academic and theoretical discussion as much as the next guy, but I worry about how little attention is paid to practical matters.  NodeUp on the other-hand is almost completely about practical things: libraries, frameworks, scaling, deployment, etc.  I’ve found them all to be pretty informative.  The Rails podcast is also fairly practical, but they do tend to cover object-oriented design patterns from time to time.  I admit that JavaScript and Ruby aren’t exactly paradigm-shifting languages, but still, programming is about more than just the mathematics.

For example, the Scala world has two great web frameworks: Lift and Play.  It would be nice to hear how they feel, not just about how type safe they are.  Granted type safety is one of the main reasons I’m interested in Scala over node/ruby, but programming is about tradeoffs.  Things like programmer productivity anecdotes are really worth something to new-comers and are a great fit for a discussion-centric medium like a podcast.

If Scala is going to become a mainstream language, there needs to be more discussion on these touchy-feely and practical subjects. If you know any, please let me know in the comments.  Podcasts about Scala are really hard to find.

Apple’s JIS Keyboard layout and the ADM3A

A few weeks ago a blog post on showed pictures of the old terminal that Bill Joy used when he created VI.  Terminals back then didn’t have arrow keys so if you wanted to move the cursor around you held down some kind of modifier and hit the H, J, K or L keys.  And they were labeled as such.  This news didn’t really interest me because I’ve read that somewhere before but when someone on my twitter stream posted a picture of the keyboard I noticed how much it resembles my Mac’s keyboard layout.

Since I live in Japan I use a Mac with a Japanese keyboard layout. The standard Japanese keyboard layout is a little hard to get used to at first because while it is a QWERTY keyboard some characters are in different places.  The placing of the double quote character and the left and right parenthesis eventually grew on me, but I always had a nagging feeling that the Japanese keyboard arbitrarily changed the locations of these very important for programming keys for no good reason.

When I saw the keyboard layout of the ADM3A I realized that it was in fact kind of the opposite.  The US keyboard layout has changed significantly since the 70s, but the Japanese keyboard didn’t.  Apple even keeps the position of the CTRL key where it should be (To the left of A).

The legendary ADM3A keyboard layout. (credit. Wikipedia)

The Apple Japanese keyboard. The Standard Japanese Keyboard layout has the exact same number row keys as the ADM3A.







I think this might be one of the reasons that I’ve grown to like the Japanese keyboard layout.

Password Maker

With all the recent database compromises going on lately, I’ve decided to step my passwords up a notch.

My old “password scheme” was pretty simple, I had 4 different levels of passwords so that if a lower one got compromised the more sensitive passwords would still be safe.  The flaw in this scheme is that if one of your sensitive passwords are exposed then all your other sensitive sites are.  I used to think that my banking websites would have pretty hardened systems, but that’s not necessarily the case.

The only safe way to handle passwords online is to have a unique password for every site you use.  A lot of people complain that memorizing passwords is hard, but I find it actually pretty easy (I memorized thousands of Kanji after all). The real hard part is remembering which password you used for what site.  When you have 4 passwords, you can just iterate through them without getting locked out but it doesn’t work when you have tens or hundreds of passwords to try.

Luckily there are a few companies with products to solve this problem. I looked at services like LastPass and 1Password. 1Password was way to expensive and LastPass was hard to use, ugly, and for some inexplicible reason stored your passwords on a server.  No matter what encryption they use today, it’ll be completely inadequate in 5 or 10 years.

Enter PasswordMaker, it’s an open-source extension for Firefox that generates passwords for websites based on the URL of the site you are visiting hashed with a master password.  This is probably the only scalable solution to the password problem.  It also does this without storing your password anywhere which makes it the most secure solution I’ve found yet. Plus the algorithm for generating passwords is simple so if I’m ever in some kind of a pinch, I can regenerate my passwords by hand (with the help of the md5 utility).

The main problems with PasswordMaker is that it doesn’t work very well on mobile phones.  There’s an Android port, but it’s about as bare as can get.  It could use some UX love. For example, It could take advantage of Android’s Share intent to get rid of the whole URL input dance that you have to do now.

Although this setup is working for me without too much pain, I still think that the whole username/password system is way too complicated.  The fact that I need a program to manage them definitely makes it a smell.  I loved OpenID, but apparently it’s also too complicated for users, so now I’m really rooting for Mozilla’s recently announced BrowserID.  As more and more software and services move to the cloud, BrowserID, or something like it, is pretty much going to be required to keep login-creep at bay. It should help reduce the anxiety that people like my wife have when thinking about whether or not they should sign-up for some new site.

My first Arduino

Yesterday my wife decided to buy yet another sewing pattern book and I decided to tag along.  As a Kindle user, there’s usually very little for me to do at a bookstore.  I usually just look around the IT section of the magazine rack and thumb through the amazing Ubuntu magazine, or Nikkei Linux.  This time though, a magazine called “Otona no Kagaku” caught my eye.



“Otona no Kagaku” is magazine that comes with some kind of sience related device inside that you can put together.  There were all kinds of projects from synthesizers to airplanes, all sorts of fun.  Kids today are so lucky, I wish I had a magazine like that when I was a kid.

Anyway I’m too old for most of them to be interesting but one of them came with an Arduino board (branded as “Japanino”).  Since the magazine cost about as much as a new pre-assembled Arduino + shipping I decided to buy it.  The magazine is filled with a bunch of projects other than the one on the cover and explains how to go about doing them.  It’s amazing all the cool things people are doing with the Arduino. One guy uses an array of Arduinos to electricute himself in the face it seems.  To each his own I guess.


The Persistence of Vision project was really simple to set up.  Just a few screws and the website had all the software you needed already.  Here’s what the finished product looks like.


The IDE is really simple.  It’s amazing how low the bar is to get code onto these things.  Way back when (circa 2005) I still had to build a cross-compiler environment.  The language is like a C version of Processing, which is really intuitive and gets rid of having to write a boiler-plate event-loop in the main function.

Now I’m thinking about what to do next.  At work I’ve been experimenting with a presence system based on bluetooth which automatically locks the computer when I’m out of range.  Unfortunately Bluetooth has a pretty long range.  I wonder if there’s any NFC shields for the Arduino.  It’d be pretty cool if you could use your phone as a smart-card via NFC.



Announcing Better Mail

About a year ago I bought the first Android phone available in Japan for the sole reason of making apps. Unfortunately I’ve had very few itches that needed scratching.

About a month ago that changed, because the built-in Gmail app has a fatal flaw.

The Problem

If you temporarily lose connectivity while sending an e-mail, the e-mail gets stuck in a “Sending…” limbo.  There’s no way to re-send the message.  It’s not even possible to edit the message.  It’s not in the outbox, and not in the sent folder.

When OhLife first started I spent 30 minutes writing my first journal entry.  Unfortunately I’ll never get to read that post again because it never finished sending.

Also the native app doesn’t support Japanese emoji.  Which means that sometimes I get what looks like blank e-mail from my wife or my friends which were actually just full of emoji.

I decided to do something about it.  So I started using the Gmail mobile web app which doesn’t have this problem. Pretty soon however I started to miss the integrated experience of the native app.

My Solution

Better Mail is a native android app with a WebView hardcoded to  But that isn’t all, it also listens to the same new e-mail broadcast as the Gmail app.  It will notify you of new e-mail without a constantly running background process.  It also includes a native menu so that you can navigate without having to scroll all the way to the top. Also, since Better Gmail is a separate application it stays in memory longer than a web-page would which makes it much less painful to use.  I also took the time to fix some of my pet-peeves with Gmail’s notification style. For example, I added a “Quiet Mode” feature that automatically turns off sounds and vibrations when you get new e-mail at 2 in the morning.

The Price

A buck.  Making this application wasn’t trivial, since I had to do some reverse-engineering.  While the die-hard Free Software nut inside me has some reservations about using DRM, it does improve performance a bit.  And performance is the biggest problem.

I have a lot of ideas in the pipeline that I could work on if there’s enough interest.  Eventually I would like to try injecting my own javascript code into the page to provide an even deeper level of integration.


I’ve decided to work on bigger and better things.  The code is now Open-Source (GPL) and available for free!

The Future

This app is definitely a minimum value product, there’s plenty of room to expand.  If anyone has a feature request let me know via e-mail or in the comments.

How To Get etags Working in Emacs

You’ve been using Emacs for how long and you haven’t figured this out yet?

Yup.  Up until now I’ve never really needed to actually get etags working in Emacs.  I remember trying to figure it out long ago as a newbie and giving up.  It’s initially difficult because it’s not as simple as just saying “etags my-src-dir”, though once you have a couple of UNIX tools on your utility belt though, it’s practically as simple as “etags my-src-dir”.

Just show me the code.

Although there might be other more correct ways to do this, the following works fine for PHP:

    cd ~/src-code-top
    find . -name '*.php' -print | xargs etags

Woah find and xargs?

etags (unfortunately) only works on files and not directories.  Those files also have to be arguments and not stdin.  So we need to use find to give us a list of file names and xargs to convert each filename into an argument for etags.

Explain find.

find is a swiss army knife of file search but mostly you can just memorize “find . -name ‘*.filetype‘ -print”.  find needs 3 arguments, the directory to start looking in, some condition, and some action. The “.” tells it to look starting from the current directory. The “-name” part means find matching filenames (We use the single-quotes to keep bash from expanding). Finally -print tells it to print what it found one line at a time.

Explain xargs.

xargs takes lines from stdin and supplies it as the arguments to some other program.  Just what we need for etags. Although xargs is as versitile as find we don’t need to supply it with anything, the default behavior is fine.

Ok, now how do I use it in Emacs?

Just use M-x visit-tags-file and point it to your newly minted TAGS file.  After that you can easily find out where the hell that class/function is hiding just by doing “M-.” (You can also return to the file you came from by pressing M-*).  Of course “M-.” and “M-*” work in a stack like manner so you can keep “M-.”ing to dig as deep as you need to and easy get back by pressing “M-*” and equal number of times.