Closures in Clojure in five minutes

I created the following for a coaching excercise. If you step through the code one part at a time you actually may have learned something about closures and Clojure.

If you do the exercise, I suggest running it in Light Table. This is an IDE being built around the concept of live editing. That is, the code you're writing is running is you're writing it. So you get to immediately see what you're doing. (I set up the exercise for Light Table 0.1; there is an 0.2 out now; not sure if everything still works as expected.)

;; Function application
(+ 3 4)
(- 10 20)

;; Function definition
(defn add [a b]
  (+ a b))

(add 3 45)

;; Local binding
(let [i 0]
  i)

(let [i 0]
  (add i 1))

;; Everything is a constant.
;; For mutable state we have to use references.
;; Following shows how to declare and dereference a reference.
(let [i (ref 0)]
  (add (deref i) 1))

;; Updating of reference values requires transactions.
;; 'dosync' evaluates its argument inside of a transaction.
;; 'alter' takes a function which does the updating.
(let [i (ref 0)]
  (dosync (alter i (fn [x] (add x 1))))
  (deref i))

;; Functions are also values...
(defn inc [i] (add i 1))

;; ... and so can be passed as arguments...
(let [i (ref 0)]
  (dosync (alter i inc))
  (deref i))

;; ... and returned as values.
(defn foo []
  (let [i (ref 0)]
    (fn []
      (dosync (alter i inc))
      (deref i))))

;; Global bindings.
(def f (foo))
(def g (foo))

;; Add these as needed/wanted...
(f)
(g)

;; Congrats! You just used closures in Clojure.

Bonus points if you understand what the following piece of code is actually doing!

(defn new-calculator []
  (let [i (ref 1)]
    (fn [msg]
      ;; 'cond' is an if-elsif kinda thing...
      (cond
        (= msg "inc")
          (fn [d]
         (dosync
              (alter i
                     (fn [x] (+ x d))))
         (deref i))
        (= msg "dec")
          (fn [d]
         (dosync
              (alter i
                     (fn [x] (- x d))))
         (deref i))
       ))))

(def o (new-calculator))

((o "inc") 4)
((o "dec") 1)

Six stages of debugging

Found this quote by Ergo^ at bash.org:

Six Stages of Debugging
1. That can't happen.
2. That doesn't happen on my machine.
3. That shouldn't happen.
4. Why does that happen?
5. Oh, I see.
6. How did that ever work?

 Very true to life. :-)

What I do all day (reprise)

I was lucky enough to get my paper, "Automated Architectural Reviews with Semmle", accepted at ICSM 2012, and got to present it last week in Italy. It's actually about a good part of my daily work. And it's just plain interesting. :-) So go read it already !

(If anyone has a good tip for where I might host the paper I'll upload it and link to to it from here. Or you can always just e-mail me and ask for a copy.)

Koopa WIP: Cobol Preprocessor

I just added an experimental Cobol preprocessor to Koopa. So far all it does is expand copy books. There is no support for replacements of tokens in copy books, or of any other preprocessing statements. Still, even this basic support was enough for Koopa to be able to parse another three of the five remaining test files which are part of the testsuite.

If you're interested in playing with this (or in expanding it) grab the source code from the project page and start with the Cobol85PreprocessingTest class.

Everything's a remix

Maybe you've seen this TED talk by Kirby Ferguson on how everything's a remix ? If you did, did you also spontaneously think of the following video ?


This is Gotye remixing remixes by other netizens of what was originally his own song. And as much as I loved the original there is something much more incredible about this remixed remixes version...

I have said it before and I'll say it again: the web knows how to bring out the best in people by connecting their passions.

Now Software geeks can have Dremels too!

If this article on Wired has got it right the Google's Dremel will be nothing short of amazing. Complex queries on petabytes of data with results in seconds ? Yes please!

Curiosity

This has got to be the most amazing picture I ever saw. It's Curiosity descending towards Mars, being slowed down by its parachute.

Congrats to the team for the successful landing!

Koopa 2000+

It's been a little over a year since Koopa got its 1000th download, and now the counter has gone well past 2000. Yay!

Thanks again to everyone who downloaded Koopa and tried it out. And especially thanks to all the amazing contributors who actually helped Koopa grow!

Game of Blocks

Prepare to be blown away by another of the web's incredible creations:

Id Code Reviews

Ever wondered about the ins and outs of Id's 3D game engines ? Then Fabien Sanglard's website is a small treasure trove of knowledge waiting for you. There are code reviews for all of the engines they've open sourced, all documented with nice code snippets, easy diagrams, and even some historical context. Just wonderful!

Failure to predict...

This question on Stack Overflow has an excellent explanation of branch prediction as well as an example of the slowdown you can get when it fails.

Beam me up!

When it comes to projects involving existing software/platforms people never want to walk from one version to the next. Instead they keep looking for a transporter beam which instantly displaces them to where they want to be... Sorry to have to be the one to tell you, but we just barely learned to walk.

Holy Leaping Seconds, Googleman!

Here is an interesting post from the Google blog on how they handle leap seconds: Time, technology and leaping seconds. To spoil the surprise: their NTP servers basically spread the extra second over the course of the day rather than dealing with a jump in time in the back-end software.

Pinterested

Thanks to my wife I am now back on the hip side of the intertubes: yes, I have a Pinterest page. I was using this blog to link to some items I thought cool. I'll start tracking these on Pinterest instead. See you there!

Koopa 1up

For those of you keeping track: the Cobol grammar in Koopa just got a major update thanks to a contribution by Adrian Noguero. Koopa now parses more files in the testsuite than before, and overall coverage of the parsed files has gone up to 98.92%! There are lots of source files which score a perfect 100% coverage. That means that for those files Koopa is able to categorize all tokens.

Adrian: thanks man!

Oracle vs. Java

Quoting from Ars Technica:

If the verdict that Android infringed copyrights stands, it could put programmers in a difficult situation. Java is an open source language, but now it's not clear how free programmers are to use it, since Oracle has said that anyone following the Java APIs—which are basically sets of instructions about how to use Java—needs a license.

"Other people will be deterred from using Java as the basis for some non-standard thing in the future," said Tyler Ochoa, a professor of copyright law at Santa Clara University. "It could be the case that Oracle wins the battle and loses the war."

Anyone else following the Java soap ? "This week Jay's evil stephfather is trying to take over his inheritance, possibly squandering it in the process. Tune in!" Or that's what it feels like anyway...

This is why I love xkcd.

Scertify Code

I got a message in my mailbox today from Armel Gouriou from Tocea, about their Scertify Code product. They added support for auditing Cobol code. And guess what they chose as a basis for doing those analyses ? Koopa! They're even so nice to give my little project a great shout-out on their blog!

Coolest thing to happen to Koopa so far! :-)

What I do all day

If you're a software geek you were probably asked at some point what it is you do all day. You then do your best to try to describe the code you write in a way which makes sense to you, and which you hope can be grokked... sorry, understood by the other person. But in the end they'll usually just walk away thinking you're doing "something with computers"...

If you're a software geek you will likely "get" what that something is. If only because you've been writing code since you were eight years old. But that still doesn't help you explain this to your friends.

I build cars

This is the earliest comparison I remember, and it comes from a joke where Bill Gates supposedly compares the computer industry with the automotive one: "If GM had kept up with the technology like the computer industry has, we would all be driving $25.00 cars that got 1,000 miles to the gallon." You can probably guess the comeback to that one.

I'm not a big fan of this comparison, mostly because I don't really like cars (I watch Top Gear for the explosions and crashes, not for the reviews :). But it's a comparison that works, to some extent.

So I'm a technical architect. What is my contribution to this car we're building ? Well, I suppose I would be designing the engine and drive train. The brakes. Crumple zones. Etc. What might be surprising is that I don't really care about the body work or what it looks like. That's a job for functional analysts, usability experts and GUI designers. What I need to know is how many passengers it should be able to move, or how much weight it should be able to pull. What speeds and accelerations are you looking for ? What mileage ? At what cost ?

The software developers would be building the car. But, and this is counterintuitive if you've never built software, the developers are really only building one car. Just one. You can't think of developers as the factory workers on a manufacturing line. The manufacturing line is itself a piece of software which can create others. That car you own ? That's just a copy of the one at the factory.

I build houses

My dad was an engineer. People would hire him to make sure that the house they wanted wouldn't collapse when they opened a window. Or that twenty years down the line they wouldn't have to tear it all down and rebuild it just to make it safe again. That's pretty much how I look at my job. Now, because I'm an architect you might think that I actually design the house. Again, this is something which is for the functional analysts, usability experts and GUI designers. My responsibility is in designing the foundations, structural skeleton, the plumbing, etc. --My dad always wanted me to be an engineer; he may not realise just how much that wish came true.--

The foundation of a house is actually a useful analogy. You don't build a skyscraper on the foundations for a residential home. The same is true for software. I can create something simple which is useful to you and the data you have. But don't expect that this same software will still work if you multiply any of those numbers by ten. It will most likely fail.

The physics can also be very different. If you have a roof floor which is supported by two columns then each column will carry half the load. But if you have a software subsystem which is supported by two other subsystems, then each of those subsystems may still be carrying the full load... Spreading the processing load over two CPUs instead of one also doesn't necessarily halve the load on each CPU. This is very much dependent on the algorithms you're using, the communication overhead which is required, etc. Unlike my dad I can't calculate my way out of this. My job is mostly based on experience and experimentation.

I build cities

The analogy of a house works best for software which is stand-alone. This kind of software is rapidly on the decline. Most software you get today will have some form of online services integrated. Or maybe it is fully online, running straight in your browser. The point is that you're not living in one house anymore. You're really moving around a town or city. And these kinds of systems need similar infrastructure: roads, sewers, transit systems, etc.

This is very much the playground of a technical architect. The houses themselves may still require analysts and GUI experts, but the infrastructure itself is mostly a technical matter (if all goes well you'll never see it). But is also very difficult to design because it has a highly dynamic nature. We have to be able to cope with the digital equivalent of traffic jams, for instance. But while you could set up bigger roads, these are also very expensive when they're going unused. Plus you need to provide for possibly varying amounts of policing and fire-fighting. If you're working on, say, a financial district you have to make sure that no thieves get into the offices.

This is also why I find it very exciting to see what Google, Amazon and Facebook are doing with their platforms. They are not just building cities. On the scale they are working you could say that they're building whole countries. And on that scale you need a huge amount of manpower, or you need to start thinking outside the box (and maybe both)...

I build ecosystems

This is, by far, my favourite analogy. The idea that software is something which is (relatively) static has been proven wrong a long time ago. In today's interconnected world this is less and less true. Software grows and it evolves. It spreads. Much like ecosystems. In the same way that we don't want an ecosystem to go out of whack we want our software-ecosystems to be healthy as well. But this takes work and care.

This is how I prefer to see my job: to design and help create a sustainable software ecosystem in which we can all live, work and play.

So...

So I'm not really sure any of this has helped to clarify anything more about what it is I do. If you're still left thinking that I do "something with computers" that's ok. In the end it doesn't make much difference anyway. :-)

First bug ever found

I know why software bugs are called bugs, but I never realized that it was Rear Admiral Grace Murray Hopper who found the first one!

I caved...

So, year, after two years I finally bought a Playstation 3. I'm playing Uncharted 3, and am loving it. Finally a grown-up game. (No more Zelda for me.) My only gripe so far is that there are way too many climbing puzzles. But I guess that's part of the Tomb Raider legacy these games have to live up to...

It's a Smallworld

Yeah, I was rooting for Day9, whose great Starcraft commentaries I have posted about before.

Looking forward to episode 2.

An Immediate Connection

You can skip the first minute or so to get to the really good part. There are some ideas here which I think are worthwhile exploring. Some of them are not really new. The live editing, for instance, is something you can see in Notch's Ludum Dare videos. But Bret takes this idea several steps further.

Oh, and here is Bret's site.

007

Best rendition of the Bond theme. Ever. (It's all the way at the end though.)

blprnt

Also check out Jer's blog.

How to put twenty clowns in a Volkswagen

Context: here.

public class You {

      private static Volkswagen vw;

      private static final class Popo extends Clown {

            private static int clowns = 0;

            private boolean unpacked = false;

            public Popo() { clowns++; }

            @Override

            public int hashCode() {

                  if (!unpacked && clowns < 20) {

                        unpacked = true;

                        vw.add(new Popo());

                  }

                  return super.hashCode();

            }

      }

      public static void main(String args[]) {

            vw = new Volkswagen();

            vw.add(new Popo());

            vw.done();

      }

}

Yes, I know, this can be made shorter. Don't really care. :-)

Mojam

Maybe you saw it, maybe you didn't, but this last week-end the Humble Bundle Mojam took place. (I'm sorry, I would have said something before the week-end, but I only found out myself on Sunday. Yeah, I'm tuned in to all the cool happenings on the interweb, I am...) In short: you could watch Mojang create a brand new game in only sixty hours. Something like a Ludum Dare, but then done by a whole team and with the proceeds going to charity. Apparantly you could even vote on the theme which, naturally, turned out as a steam-punk Egyptian themed multiplayer shooter.

I didn't get to watch much of the creation itself, but you can check out a previous post of mine on Notch taping his Ludum Dares to get an idea of the process. The difference that a bigger team brings ? Better art and multiplayer goodness, I would say.

Best of all, if you donated something, you not only get to download and play the game, you also got a copy of the final source code! Now that's a big bonus to a software geek like myself. I have only looked at it briefly, but the structure seems very similar to what I have seen Notch code in his Ludum Dares. In particular they are still juggling all the pixels themselves; no real hardware acceleration is used. It's basically writing to a back buffer and then blitting it to screen. Easy. (Right?) :-)

Julia

The Julia programming language seems really interesting. Julia has a combination of cool features: multimethods, coroutines, macros, meta-programming, generics, distributed computing (with cloud-computing in mind), etc.

She can even call Fortran code. What ever was wrong with Cobol as a target ? :-)

Make install: Rasp's game

Rasp is working on a 2D isometric roguelike using Slick, a Java 2D game library. And he's blogging about it. It's still in the early stages, but the graphics look good. His approach to lighting, with colour blending, makes for some nice screenshots. And he shows you how it's done.

Go check it out. And if you know of other programmer blogs please let me know!

Being a geek dad...

Is watching my two-and-a-half year old daughter using an iPad like she's never done anything else in her life. Now, to be clear, I didn't buy the iPad for her. I originally bought it for my wife. But it's my daughter who seems to get the most out of it. And it's not just simple games either (Toca Boca games are superb by the way); she starts up and changes apps like it's the most natural thing to do.

My own first encounters with technology were goofing around on an Apple II clone, getting it to print something to the screen. Followed by a Game Boy (the original one) when I was ten or so. My little girl has exponentially more power under her fingertips, and she uses it without thinking twice about it. To me that's just amazing. It shows not only how far computers have come in power, but also in accessability.

It also makes me wonder what she could be doing in another year's time...

ScummVM

There's in an excellent article over at Ars Technica which tells the story of ScummVM. Great read about a fun open source project.