Process and Product

Yesterday I came across a post entitled “default behavior” by Ben Augarten about how important it is to put your products out in the world. It starts with him talking about how the code he’s writing is for an internal tool and hence will never see the light of day. Few people will use it and he’ll lose ownership of it once his internship ends. He implores programmers to release their code as products into the world and seems more than a little disdainful of programmers who don’t release their own products. He seems to be under the impression that if you’re not working on your very product that you’re dead inside.

He’s wrong.

It’s great to love to your product. It’s great to build something, to show it to the world, to have people use it, get feedback and make it better. If that’s what motivates you and makes you happy then by all means go ahead and release products. Life is too short to be unhappy (though people are working on that). What it doesn’t mean is that not releasing products makes you unhappy.

There’s a distinction to be made between process and product. Product is the fruit of your labor, process is the labor itself. In the rush to create startups, launch products, have dozens of repos on your Github account, we tend to lose sight of the process itself.

While I certainly understand (and have felt) the allure of releasing a product, there’s a lot of joy to be gained in the process itself. While many of us got into computers and programming because of the things we could create, many of us (including myself) got into it because of the joy of coding itself. I love learning new technologies, working through problems, crafting solutions. I love the mental strain of thinking my way to answer, I love the physical feel of my fingers on a keyboard and the seeing the characters of the screen. I love the process of seeing something being created as much I like the feeling of having created something. The same goes for writing or drawing or cooking – how you make something is just as important as what you make.

I suspect that this joy for the process itself is why so many incredibly talented and hardworking people are happy working for closed source companies (financial and material renumeration aside). If you are fulfilled by the labor itself then you care less about the fruits of your labor. This isn’t particularly new thinking – it predates open source, computers, software, even modern science and technology. As far as I know, the earliest well-known expression of such a notion is from the Bhagavad Gita where the god Krishna tells Prince Arjun:

“To action alone hast thou a right and never at all to its fruits; let not the fruits of action be thy motive; neither let there be in thee any attachment to inaction”

I don’t entirely agree with this: I do believe in property rights and I think that as a creator you deserve some say in how your creation is used. However I do agree with the underlying idea that you can find motivation in the actions themselves and not just their results. Getting too invested in the result can detract from joy you recieve from the process (and vice-versa). I’m not going to claim that process is better or more meaningful than product. Humans and their motivations and emotions are far more complex than such simple comparisons. But I am going to claim that finding fulfillment in the process itself is a perfectly valid way of living your life.

It’s great that Ben and so many other programmers and entrepreneurs want to create products and see them in action. We need that. They don’t have to become process-oriented. But they do have to understand that not all people will value products as much as they do and they have to respect that. Sometimes the journey is more important than the destination.

Sunday Selection 2012-07-01

Around the Web

Hackers and Fighters

When it comes to the debate between computer science and programming I’m of the opinion that knowing some computer science will make you a better programmer. Being a regular programmer will give you a better appreciation of the problems scientists need to solve. I’d love to see a proper scientific test of this hypothesis rather the jumble of anecdote and posturing that exists on the web today. This article is one of the more balanced ones and gives a worthwhile metaphorical veiw of the matter.

How One Hacker Quit the Programming Life for Bluer Skies

As much as I love writing code for my day job and wouldn’t be doing anything else, I don’t think that need be true for everyone. For a lot of programmers, writing code is just a day job and that’s ok. I find the notion that “real programmers” program day-in-day-night to be rather silly. You should do what makes you happy. Peer pressure that makes you produce more is still peer pressure. How good you are as programmer is determined by the quality of your code – not the hours you put in and not the number of open-source projects you maintain. And when you decide that you’ve had enough, don’t be afraid to go do something more interesting.

A much higher education

These are interesting times for academia and education in general. While I’m all for free (both as in beer and in freedom) digital distribution and decentralized, on-demand self-education, you can pry my expensive liberal arts education from my cold dead hands. I don’t think the higher-education system is unsustainable, I think there’s increasing pressure to commercialize and corporatize it and we’re unwilling to explore other options. Education is the foundation of a free society. Academic freedom is one of the pillars of modern science and technology. It’s about time we took that seriously.

Video

Jon Moore on Hypermedia APIs

Programming for the web is still one of my blind spots. That being said, I find the idea of programmatic exploration of APIs very interesting. I don’t fully understand all the implications and applications and I don’t think I will until I write a hypermedia application. But if you’re interested in knowing what programming for the modern web is like, I think this is a good start.

My Emacs Config

I’ve released parts of my emacs configuration on Github. Fork away! I first started really getting into Emacs after reading Steve Yegge’s blog posts on the matter. Since then I’ve pretty much fallen in love with Emacs, waxed eloquently (or maybe not so eloquently) about why it’s a power tool and despite all the cool modern editors and IDEs you can pry my Emacs from my cold dead hands.

I add to my Emacs configuration every time I embark on a project in a new language or using a new toolset. Currently I have a good 15MB worth of Elisp code for programming in C, Python, JavaScript and Ruby, dealing with XML and Git, blogging with WordPress etc. etc. All that stuff is not in the repository (though the names of some of the packages are if you want them). What is on Github are my general Emacs configurations, custom keybindings and custom Elisp functions for moving around the buffer and relocating files.

A note on keybindings

Most of my keybindings are different from the standard Emacs ones. I rebound them to be easier for me to remember, especially the movement keybindings. The mnenomic I use is:

  • `f` key moves forward
  • `b` key moves backward
  • `d` key kills (deletes) forwards
  • `w` key kills (deletes) backwards

I use the Ctrl and Alt modifiers to make these actions work on words or individual letters. The Ctrl key makes each of the above operate by word and the Alt or Meta modifier makes them work by letter. The Ctrl-m combination is a prefix key for various functions to reposition the current open buffer in the window. The functions are defined in `functions.el`.

A note on tools

I’m a great believer in customizing and shaping your tools to fit your work and your workflow. However, I believe more strongly that the code is not the point and that your tools are definitely not the point. In keeping with that I rarely tinker with my Emacs setup nowadays: I’ve found a configuration that works for me and is acceptably efficient. If I move to new tools and language (Haskell and Go are on the horizon) then I might make more additions.

The Code is Not the Point

There’s this meme in the programming community of thinking of our code as art. This is not a new phenomenon – it dates back at least to The Art of Computer Programming in 1968. More recently we have Paul Graham to thank for drawing the comparison between Hackers and Painters. With the rise of languages like Ruby and Processing and personalities like _why the lucky stiff programming has been gaining a reputation as an art form and a source of creative joy. And it should be. I love programming, it makes me feel good and I feel much better on days I’ve written code and produced something than on days I haven’t. However, I think the “code is art” or “programmers are artists” meme can be misleading because the code is not the point.

Miyamoto Musashi was a medieval Japanese swordsman and Ronin, widely regarded as the best swordsman of all time. He was also the author of a book titled “The Book of Five Rings” – a book on swordsmanship, strategy, tactics and philosophy that is still relevant today. There is one passage in the book that is relevant to our discussion:

The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means. Whenever you parry, hit, spring, strike or touch the enemy’s cutting sword, you must cut the enemy in the same movement. It is essential to attain this. If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him.

The primary thing when you take an editor in your hands is your intention to solve the problem, whatever the means. Whether you write a script, a unit test, a function, or a library, you must move towards the solution in the same movement. It is essential to attain this. If you think only of scripts, tests, functions or libraries, you will not be able to actually solve the problem.

Because the code and it’s surrounding artifacts, including your sense of beauty, is not the point. It does not matter how much code you’ve written, what your test coverage is, how much you’ve learned in the process, if you haven’t not solved the problem (or solved it inadequately, or solved the wrong problem). Anything that does not bring you closer to your solution is suspect.

Does this mean that clean code, good comments, test coverage are not necessary or important? Of course not. If your code is not well-written and clear, are you sure you’ve solved the problem? If you have no tests, are you sure you’ve solved all aspects of the problem? If you have no documentation, how will others use and improve on your solution?

Does this mean that your code is not art? Does this you mean you should not carry an artists’ sense of elegance and aesthetics? Of course not. By all means take pride in your work. Make it a point of honor that others can understand your code without sending a dozen emails. Please aim for the solution that is not just correct, but also elegant, concise and efficient.

But keep in mind that the code is not the point. Beauty, elegance and pride are no substitutes for correctness. We are scientists and engineers first, artists second. If the theory does not fit the facts, if the code does not solve the problem, it must be discarded no matter how beautiful it is.

I don’t think of myself as an artist any more. My code is not art. I take pride in my work but it is the pride of an engineer. I want my code to have more in common with a Rolls Royce engine than it does with Sunflowers. I try to do the cleanest, most elegant job I can. But whenever I write code, the intention is to cut the enemy, whatever the means.

Python as a glue language

I’ve spent the better part of the past few weeks redoing much of the architecture for my main research project. As part of a programming languages research group I also have frequent discussions on the relative merits of various languages. Personally I’ve always liked Python and used it extensively for both professional and personal projects. I’ve used Python for both standalone programs and for tying other programs together. My roommate likes Bash for his scripting needs but I think Python is a better glue language.

My scripting days started with Perl about 6 years ago but I quickly gave up Perl in favor of Python. I don’t entirely remember why, but I do remember getting the distinct impression that Python was much cleaner (and all-round nicer) than Perl. Python has a lot of things going for it as a scripting language – a large “batteries included” standard library, lots of handy string functions for data mangling and great interfaces to the underlying OS.

Python also has decent features for being a general purpose language. It has a good implementation of object-orientation and classes (though the bolts are showing), first class functions, an acceptable module system and a clean, if somewhat straitjacketed syntax. There’s a thriving ecosystem with a centralized repository and a wide variety of libraries. I wish there were optional static types and declarative data types, but I guess you can’t have everything in life.

Where Python shines is at the intersection of quick scripting and full-fledged applications. I’ve found that it’s delightfully easy to go from a bunch of scripts lashed together to a more cohesive application or framework.

As an example I moved my research infrastructure from a bunch of Python and shell scripts to a framework for interacting with virtual machines and virtual networks. We’re using a network virtualizer called Mininet which is built in Python. Mininet is a well engineered piece of research infrastructure with a clean and Pythonic interface as well as understandable internals. Previously I would start by writing a Python script to instantiate a Mininet virtual network. Then I would run a bunch of scripts by hand to start up virtual machines connected to said network. These scripts would use the standard Linux tools to configure virtual network devices and start Qemu virtual machines. There were three different scripts each of which took a bunch of different parameters. Getting a full setup going involved a good amount of jumping around terminals and typing commands in the right order. Not very tedious, but enough to get annoying after a while. And besides I wouldn’t be much of a programmer if I wasn’t automating as much as possible.

So I went about unifying all this scripting into a single Python framework. I subclassed some of the Mininet classes so that I could get rid of boilerplate involved in setting up a network. I wrapped the shell scripts in a thin layer of Python so I could run them programmatically. I could have replaced the shell scripts with Python equivalents directly but there was no pressing need to do that. Finally I used Python’s dictionaries to configure the VMs declaratively. While I would have liked algebraic data types and a strong type system, I hand-rolled a verifier without much difficulty. OCaml and Haskell have definitely spoiled me.

How is this beneficial? Besides just the pure automation we now have a programmable, object-oriented interface to our deployment setup. The whole shebang – networks, VMs and test programs – can be set up and run from a single Python script. Since I extended Mininet and followed its conventions anyone familiar with Mininet can get started using our setup quickly. Instead of having configurations floating around in different Python files and shell scripts it’s in one place making it easy to change and remain consistent. By layering over and isolating the command-line interface to Qemu we can potentially move to other virtualizers (like VirtualBox) without massive changes to setup scripts. There’s less boilerplate, fewer little details and fewer opaque incantations that need to be uttered in just the right order. All in all, it’s a much better engineered system.

Though these were all pretty significant changes it took me less than a week to get everything done. This includes walking a teammate through both the old and new versions and troubleshooting. Using Python made the transition easier because a lot of the script code and boilerplate could be tucked into the new classes and methods with a few modifications. Most of the time was spent in figuring out what the interfaces should look like and how they should be integrated.

In conclusion, Python is a great glue language. It’s easy to get up and running with quick scripts that tie together existing programs and do some data mangling. But when your needs grow beyond scripts you can build a well-structured program or library without rewriting from scratch. In particular you can reuse large parts of the script code and spend time on the design and organization of your new applcation. On a related note, this is also one of the reasons why Python is a great beginner’s language. It’s easy to start off with small scripts that do routine tasks or work with multimedia and then move on to writing full-fledged programs and learning proper computer science concepts.

As a disclaimer, I haven’t worked with Ruby or Perl enough to make a similar claim. If Rubyists or Perl hackers would like to share similar experiences I’d love to hear.