It’s occurred to me more than once that I really don’t know many great programmers. I go to a small school and most of my fellow students are about the same level as I am (or lower). Actually I don’t really know what level they are on because I don’t get to work with them as much as I would have to in order to find out. There is only one person whom I can say for certain is a better programmer than I am and I’m currently doing a project with him. My professors are good computer scientists, but I have no idea how good at programming they are.

But from what I’ve read (and what I can attest to from personal experience, to some extent) achieving excellence in any field (especially programming) requires a lot of dedication and hard work — about 10,000 hours of it. I’ve thought about whether or not it really is worth it to invest so much time in one activity, even if you really love it. I love spinning code, but I don’t want to be sitting in front of a bright screen all the time. Watching the dancers the other night, I got to see what dedicated practice can bring. They put on a great performance and everyone enjoyed it (including myself) and creating something beautiful and wondrous is a worthy cause. But I’m also interested in getting to know about what they dancers themselves think about their work. I’d love to know their feelings and emotions as they dance and have people cheer and whether they think it’s fair compensation for their commitment. And I don’t mean compensation in only the monetary sense, though that is important too.

Admittedly there is nothing in the programming world that is quite the same as a great dance performance. Our victories are more personal and what people see (and sometimes applaud us for) is often a small sliver of everything that we do. But that’s fine by me. When I solve a hard problem after a long time (my personal record is 3 hours hunting a pointer bug) or make something that I think is really cool (a recursive-descent parser for a little language), I think I feel some of the elation, satisfaction and relief that I think the dancers would have felt too. Yes, it does feel really good. As each year goes by I get better at doing what I love doing. But I rarely ever think about all the practice and experience that has gone into making me capable of whatever it is I am doing. Not too long ago, I would never have imagined myself capable of writing a UNIX shell or designing a programming language, but know I am doing both those things and it feels almost natural.

Ok, that last sentence was a lie. It doesn’t feel natural, but it feels like it’s just outside the range of being natural. When I’m doing things like that, I’m on the edge of incompetence. It was hard and it was painful, but now that I know I can do it, I feel much better. In some ways, I wish such chances came more often (I think the education system for computer-related studies needs to be revamped significantly, but that’s another matter) and I know that each such experience leaves me just a little bit better. Do flamenco dancers feel the same way? Maybe. It would be interesting to find out.

Excellence is a rather strange thing in that it’s hard to achieve and the return on investment on its pursuit can be very little until you get to a certain tipping point. And then there all the people who seem to be trying really hard without getting anywhere. I’m not surprised that many people choose not to put in the investments that it takes to be excellent. As a girl I liked once told me, there are a lot of people leading average lives who are very happy about it. I guess that’s true. I’m not clear about where I stand on excellence myself. I do want to be really good at what I want to do and I fully understand that it won’t be easy. But I also don’t want to give up everything on the quest for excellence. “No sacrifice, no victory” sounds very noble and all, but there’s a tinge of recklessness that I really don’t like.

At this point, the word “balance” might seem appropriate. But that’s bullshit too. I don’t think people who are great at something got there by seeking balance. The better option is breaking the rules, or at least fracturing them. The prime example is 37signals. They’re a small company, with little VC funding who don’t give away their products for free and still make millions of dollars. And they didn’t do it by working round the clock either. They broke “rules” like working 80 hours a week and making free products and other such things. But they also knew what rules to break. They didn’t break rules about being thrifty or having a solid business plan. They might not be the paragon of excellence and they’re certainly arrogant, but they’re doing well so far.

As someone seeking excellence myself, I’m trying to bend my own set of rules. I bend rules by taking courses out of sequence, doing independent studies where I can write lots of code and meet interesting people and actively trying to talk to people I admire. I need to put in 10,000 hours, so I build my life to provide opportunities to do just that. I really wish that there were a lot more people doing the same.

Happy hacking.

Ars Longa, Vita Brevis

As I’ve pondered before, life is short and it takes a fair amount of dedicated effort and time to come up with something beautiful and useful. With the rise of the Internet and instantaneous communications, we’re becoming a culture that is very much used to continuous streams of small information packets. The essay is becoming a holdover from the old days when having long periods of times to do nothing but sit and read was common. However, there are a number of really good essayists alive today, and a lot of them are on the Internet. There’s Paul Graham, whose essays are practically the stuff of legend for programmers. There is also Steve Yegge who seems to have retired, but left behind a fairly large collection of essay-length material (including an article on why you should write a blog). Outside the Internet there is Warren Buffet who has written long detailed letters to shareholders for the last 32 years each of which is an education in and of itself (and I can’t help but wonder how many shareholders actually read through them all).

I don’t think I’m making a mistake when I say that the essay is still alive and well today, albeit in somewhat modified forms. But the fact remains that putting out something of such length and depth takes up a lot of time and energy (not to mention the countless hours that go into accumulating the knowledge and organizing the thoughts that must flow into such a work). In many ways, writing an essay is similar to a software project. There is planning and preparation that must happen upfront, but nothing is really for certain until you sit down and start writing. Writing a good essay that other people will want to read and tell their friends about is no easier than writing good software that others will want to use.

Blog meet Essay

The blog and the essay are fundamentally different things. A blog is a magazine compared to an essay’s book. The blog as a format is great for some things: without easy blogging I probably wouldn’t be writing at all. But the rise of blogs (and accompanying software) has left the long form essay in the dark. You could simply write long articles and put them on your blog like Steve Yegge. But reverse chronological ordering really isn’t the best format for a collection of essays. For small numbers, a simple list of titles, maybe with a blurb is probably the best. Once you get to a larger number (Paul Graham for example), a simple list doesn’t cut it any more.

There is also the actual writing experience. Whenever you write a longer piece over the course of many days, you start to go back and visit the old parts. Part of it is for editing, but you also want to read what you’ve read before so that you know you’re keeping your essay coherent. Blog software doesn’t easily let you do this. I know WordPress stores revisions, but there doesn’t seem to be an easy, upfront way to see diffs of different versions against each other. I suppose a wiki could be better as an essay platform. Dokuwiki has excellent visual diff function and Writeboard also lets you compare versions.

Perhaps we do need some sort of specialized software for writing essays. Something that puts drafting editing at the center as opposed to at the edges. Personally I’ve been using Emacs with Git to get some of the same result, but I would really like to see a webapp that can do something like that. After all, there isn’t much use in writing an essay if no one is going to read it (and how better to get people to read it than to put it out on the Internet).

I, Essayist

Even though there may be no quick-and-easy publishing solution like WordPress for essays, writing an essay is far less dependent on tech tools than most other things today. Like I said before, Emacs and Git do a fairly good job together. I would like to be able to put all my drafts online with some sort of commenting system so that people can see the evolution of my essays, but I’ll settle for just being able to show a final product.

Separate from showing the essay is the mental exercise of actually sitting down and writing the essay (and then revising and editing). That’s something that I’ll have to get back into the habit of doing and will probably take time. Subject matter is also an issue, but a good starting point would be to simply expand on the themes that I cover in this blog, while making sure that people who read my blog can read my essays without getting bored (and vice versa). Expect my first essay to be on essays, sometime in the next few weeks.

Lindenmayer systems or L-systems are pretty awesome. You start with a string of symbols and a bunch of rules describing how those symbols transform into other symbols. And then you just keep reapplying those rules over and over. L-systems can easily generate a wide variety of interesting forms and patterns including many plant morphologies.

Two years ago I made a  implementation of L-systems in Python using the Python Image Manipulation library. But one of the goals of that project was to create a tool that non-programmers can use. It was an interesting experiment, but after a few months I was no longer sure if it was a good idea to create a “non-programming” interface to something that is so computationally powerful. This time around I’ve eschewed the whole “pretty interface” idea in favor of dealing with real code.

I chose to use Processing so that I could work a few steps removed from nitty-gritty image manipulation. Processing is based on Java, so I can use Java with all it’s libraries if I need to (and to some extent, I did). I would have preferred it if I could have Python underneath, but I’m not complaining too much. There is also a version of Processing running on Scala and there is a port in the works to JavaScript and Canvas which I’m interested in. It might be an interesting future project to port it to Jython.

How it works

L-systems in general work by generating a string of symbols that work as driving instruction to a LOGO-style turtle (think of it as a pen you can move with simple instructions like “forward”, “left”, “backward” etc.). The lowermost layer is a simple turtle implementation that takes a bunch of movement commands and draws them to a screen (or canvas).

On top of that is a class that represents Lindenmayer systems in general. A class stores the symbols, rules and the mappings from symbols to turtle instructions. It also implements the basic rewriting algorithm and allows to provide an external turtle object (in case you want to process the image outside of the actual L-system). It also takes the generating string of symbols and passes it to the turtle to draw.

The top of the stack is functional Processing code where you have a setup() function that sets up the image canvas and initializes the L-System object. The draw() function executes repeatedly and controls how many generations the L-system goes through. Each L-system is two functions: a setup function that creates the L-system object with symbols and rules and another function that gets called on every draw(). This allows you to do extra processing before or after the L-system’s actual generation.

Go Get It

I’m calling the resulting system Lisip: LIndenmayer Systems In Processing. It’s released under a GNU GPL and is available right now on Github. There are just three source files (the Turtle, the L-System class and the Processing code driver which includes a number of L-systems). I think the code is fairly self-explanatory (thanks to a large extent to how much Processing cleans up the usual Java syntax). I do plan to add some proper documentation within the week.

I’ll continue to add L-systems to it and probably add to the Turtle’s functionality in the weeks to come. Feel free to fork it and add your systems, but I’d appreciate it if you dropped me a line if you do fork it.

Happy Hacking.

What led me to flesh out this idea was Martin Fowler’s observation that the three most popular version control systems: Subversion, Git and Mercurial are all completely open source. With just a little bit of thought, I realized that there’s a company that does exactly what I propose: Apple. Apple’s dev tools like XCode and the Objective-C compilers and libraries are open source are free for everyone to download and use. The core of OS X is the open source Darwin kernel. But layered on top of this is a closed and really high quality system of interfaces and applications that make for a very appealing user experience.

First let me say that what I’m about to say isn’t a commentary on the pay-for-support model that Red Hat and Canonical use. I think that’s a perfectly legitimate model that has it’s advantages for consumers, companies and open source hackers. But I want to propose an alternative that focuses on making money from selling the software itself.

Opening Doors

Why does this make sense? Leaving aside any moral imperatives, one of the biggest reasons for open source is that you can tinker and fix things that are broken. If you see something that is not right, you can easily pop open the hood and dig around in the internals. Now, the only way that you’ll know if something is broken is if you either use it yourself or have someone tell you that it’s broken. However, in my experience it’s much easier to fix something when you see it broken for yourself. It also gives you a better idea of what the fix should be like. It’s also more likely that you’ll change a program for the better if you’re using it day in and day out. Eating your own dog food has its advantages. If you’re a programmer, you’re most likely to find bugs and missing features in tools that you use every day. So it makes sense for you as a developer to use open source tools so that there’s an easy way for you to fix or change things that in turn will help your own developer experience.

If you’re a corporation (or organization) like Apple or Microsoft then it makes sense to open source your developer tools as well. For a corporation, the benefit to open sourcing a product is that you get feedback both in terms of ideas, comments and bug reports and also in terms of real working code. What you end up with is a positive feedback loop: you make tools and release them to developers for free who then help you make even better tools. I love the lesson that 37signals has taught the world in terms of tools and basic infrastructure: Ruby on Rails is part of their basic infrastructure, but they’ve decided to open source. As DHH claims, it’s hard to make money off of basic infrastructure (unless maybe if you’re Microsoft with a huge already installed base). Better to release it into the world and benefit from the improvements that other people make to it.

Making Money

So that’s all nice and dandy for developers, but what about the poor corporation that wants to make a decent buck? The basic idea is that the number of people who aren’t developers far outweighs the number of people who are. Also the number of people who just want something that works far outweighs the number of people who want something that they can tinker with ad infinitum. Even most of us hacker types occasionally break down and just want something that we can shut off our brains and use (for me it’s Excel in my engineering labs and Photoshop for image manipulation). People will pay for convenience, quality and polish. That should be the motivating factor behind businesses who want to sell software.

To make money from selling software you first need to hire good developers and give them good tools that they customize to their needs. Fortunately, you already have great free dev tools. Now pay these people well and give them a concrete goal so that they put real polish into the products, especially in the areas that open source hackers might tend to neglect. Since you’re going to be charging money, you can afford to put some money into hiring professional designers and UI experts and make sure that your app really looks good as well as works good (something that is still sorely lacking in a lot of open source software). Of course you need other people on team too: at least a fair number of testers who are the same people who will be using your for-pay product day in, day out. You end up with a product that has been crafted by motivated developers using great tools with feedback from your target user base. Admittedly you can still end up with a crappy product that no one wants to pay for, but that doesn’t mean the idea itself doesn’t work.

Though I have no real experience making products like that, the cases in front of us are pretty clear to see. Apple and Adobe both make lots of money selling high quality consumer software and I personally consider Microsoft Office to still be a better user experience that OpenOffice.org (especially with the ribbon interface).

One of the questions I haven’t fully resolved is whether your for-pay app should be open sourced. I am tempted to say no because I can’t think of a business model where you can open source your product (code and design elements) and still expect people to pay a decent price for it. If someone with more experience than me can come up with a working model, do let me know. However, one compromise that might be fair is to open up your app a few years later once your company has a newer product. Under this model, Apple would open source all their code prior to OS X. I’m less familiar with Adobe, but I think prior to the Creative Studio editions would be a good time point. Microsoft might consider releasing the pre-NT Windows code and maybe Office prior to 2000 or 2003. That way, your business still makes money today while making a community contribution in the future. It also helps users who can get their hands on a copy of a program if they need to open a file from 10 years ago that the current version no longer supports. It frees up the company to go ahead while worrying a little less about backward compatibility. It also keeps companies on their toes because they know that any market advantage they have will be effectively erased in a few years.

The Plan Isn’t Perfect

No plan survives contact with the enemy

– General Field Marshall Helmuth von Moltke

And this plan will definitely have to be modified if and when a company adopts it. One immediate problem I can see is that inevitably there will be calls to open source all the company’s product. I don’t have a suitable response to that apart from saying that would probably cause the business plan to collapse. Also if your product’s core algorithms don’t vary much from one generation to the next, open sourcing older versions might be equivalent to shooting yourself in the foot (though if you’re expecting a handful of algorithms to bankroll your company till the heat death of the Universe, you’re probably doing it wrong).

Again, I don’t have any real world business experience (though I would like to, someday). I also don’t know any company that has a policy like this at its core. If you’re interested in starting a company like this (or know of one already), do drop me a line. I’d really like to play a part in making a world where people can get paid for writing their code and then releasing it for other people to use and improve. A business that makes money but still open sources its code (even if its a few years late) would be a great step in that direction.

All I did last night was recreate an project I’d done a few semesters ago with Lindenmeyer Systems. In some respects, our previous work was a bit misdirected and we should have been building on top of Processing all along. Last night I was reminded firsthand of the importance of using good tools suited for the task at hand. I didn’t write much in terms of code, but I did manage to build up a fair bit of functionality (a much better measure of progress I think). But it’s still not done and I suspect at least an hour or two more of steady work before I get to something that I can show off. This morning as I was trawling in the interwebs I came across this essay written by my creative writing professor and her quotation of “Ars longa, vita brevis” rang out as so true. The art is long and life is so short.

I’m not really an artist, though I like pretty things. I’m a hacker at heart. More important than the actual beauty of the object is the joy I feel in actually creating it. As  a coder, I guess I’m half decent by now. I’d call myself a really really advanced beginner (close to intermediate). But in terms of art, I’m pretty much a greenhorn. What’s more, the art that I’m doing is in code. I thought that would be fun and easy. It’s not easy and it remains to be seen if it’s fun. Though I love writing code and can concentrate better on writing code than on anything else, when it comes to art, I’m a bit lost. It’s been a while since I’ve done any drawing or painting, I prefer using my words to create images in people’s heads. Also without the use of hands and real physical paints and paper, it’s a bit harder to play around. Admittedly, it’s easier to tinker, redo and recreate with computerized tools but there is more of an upfront investment and the learning curve is significantly steeper (at least in the beginning).

In some ways, you could say that I’m painting entirely with mind. It’s liberating: I don’t have to worry about drawing a perfect circle or making sure that the sides of my squares are all the same length, the machine does it for me and I can work at a higher level. At the same time, I can’t just splash some paint on campus and see what it looks like. I have to look up an API reference instead of stroking away extra paint with a brush and I sometimes I have to get my hands dirty debugging (including dealing with Java’s broken type systems). If real artists did this all the time, we’d never progress beyond stick figures. That’s not to denigrate Processing, the people behind it or computational art. The gains in productivity and expression we get from tools like them far away my pain. Don’t mind me. I’m just bitching.

The art is long and life is short. It takes some 10,000 hours to become an expert in anything and it takes continued practice to keep up that level of expertise. And life is short. Sometimes I wish that I could sever all human contact and just sit and write code (or stories or whatever) and the next moment I realize that it’s stupid because it’s meaningless to live completely in isolation. There’s no point in my writing code or making art if no one uses and appreciates what I create. It’s rather ironic that it takes antisocial devotion to a task to create something that others can appreciate. If only our brains could really multitask, things would be so much easier.

Well, that’s enough bitching and moaning for one day. I’m going to get back to my code/art.

I hopped on the C channel at Freenode and within minutes I had not only found out why program was misbehaving, but I also got two separate suggestions on how to fix it. I may have bent the rules a little bit, but I trust my professor will understand that I actually did learn something and not just steal code of the Internet. I still don’t really know how to implement one of the suggested fixes, but I implemented the other. The fix I didn’t understand is probably the cleaner of the too and I plan on asking my professor more about it.

I repeated the experience yesterday, but with JavaScript and the canvas element (while trying to use Processing.js). I did IRC in the past, but never really to solve problems. I went on channels (mostly #archlinux and #python on Freenode) mainly to interact with the community. Whenever I had problems, I generally googled my way to a solution. In some ways, asking on IRC could be considered the easier path: instead of searching for material, you simply go ask someone who knows. While that may be true to some extent, it’s certainly not what happens all the time. When asking the C question, the other channel members made it perfectly clear to me that I needed to go read more (which I totally agree with). They also helpfully pointed me to resources. IRC channels aren’t “cheatsheets” by any stretch of the imagination. I got the feeling, on both the C and Processing.js channels, that the people on them are very well-versed in what they do. They’re willing to help you, but only if you help yourself (some make this point more forcefully than others).

IRC is a good example of a way to learn Computer science (or at least programming) in a master-apprentice fashion. It’s different from learning from a tutorial or reading the man page. I hesitate to call it “teaching” because it’s more of a free form Q&A. Instead of having someone give you the information up front (a typical classroom setting) or going out and hunting it down yourself, you get to pick the brains of people who have already internalized the knowledge that you’re after. To get to it, you have to ask the right questions and that means knowing the problem well enough to figure out what questions are worth asking. And that requires some thought.

It’s not exactly master-apprentice because you aren’t learning under the tutelage of a single master. I think a more apt analogy would be spending some time at a monastery. You don’t have a very strong connection to the people there, but you share some things in common. You can go with a purpose or without one. But you will benefit more if you do have one. There are some basic rules, mostly concerning respect and etiquette. If you break them, you will be asked to leave and in extreme circumstances, removed. The experience you have is very much up to you and could leave with a profound sense of enlightenment and a strong desire to return and learn some more.

Ok, so I’m embellishing a bit and programmers aren’t really monks (except maybe if you’re Richard Stallman). IRC certainly isn’t solution to all questions and you could easily come away being more confused than when you started, but that could happen with a book or a normal class. IRC is another resource, but it’s a good one. No matter how good search technology may be or how well-written technical docs may be, sometimes it’s good to be able to be walked through problems by a human being. And yes, I do still have questions for my professor.

Though the book is about children who are fighting a war for humanity’s survival, it’s also about soldiers learning to be generals and all-round leaders. At first read, it doesn’t seem to have anything to do with programmers. But a few months ago I read an OSNews article about the problem with design and implementation of software. After sitting in the back of my head and germinating for the past few months the one paragraph that came back to me last morning is:

The major problem with this is that ALL of software is design. 100% of software is design from the high level architect-like design to the low-level design of a for-loop. The implementers of software are not human! I knew you suspected as much given how odd many programmers are. No, the implementers of software are actually ‘perfect’ machines. They are the compilers (interpreters, preprocessors… are all included in the generic use of the this word). For almost all purposes, the compiler is perfect. I’ve yet to run into a situation where I’ve written code and the compiler has not followed my instructions and that is the reason something broke. It hasn’t happened yet.

One of the core themes of Ender’s Game and Ender’s Shadow is how the main character, Ender learns to become the commander of an elite fighting force. This force is special because unlike other commanders, Ender trusts his soldiers and gives them the independence to develop their own strategies and techniques. What I realized is that just as Ender is the commander of his fighting force programmers are the commanders of their own programming forces. Instead of having snipers or demolition experts, we have our compilers and source control systems. And like Ender, we must learn how to use your teams well and get the work done.

As programmers, especially novice programmers, we tend to anthropomorphize. A lot. Often enough we say things like “My program isn’t doing what I tell it to do” or “My program is misbehaving again”. In many ways, anthropomorphizing is a coping mechanism: it gives us a comfortable way of thinking about something that is very different from most other things we experience in life. While it might help us to think about computers and programs in human-like terms, it can also be a pretty limiting point of view.

The OSNews article above is about how thinking in terms of separate implementation and design phases is detrimental to writing quality software. The “perfect machines” idea that he talks about is an argument against the way we humanize computers. Because we think about our programs as people, we assume that they make mistakes just like people do. We assume that our programs aren’t doing what we want because they don’t like us or that they’re just misbehaving. I’ve been programming for a good few years now and I find myself falling into this trap more often than I would like. However, the truth is that we’re not dealing with people: we’re dealing with perfect machines. A mistake in our program is much more likely to be our own fault than it is to be a problem in the compiler or libraries.

Just as Ender learned to trust his soldiers to act on their own under battle situations, we programmers need to trust our machines. We need to understand that our machines are right, more often than not. Sure compiler errors aren’t perfect and it sucks when GCC spits out three pages of errors to tell you that you missed a semicolon, but that doesn’t mean that your program doesn’t “want” to compile. It’s not easy, because the world of computers is really very different from the world of people and thinking of programs in “people terms” helps us form a bridge. But like all such bridges, this one is ultimately a set of training wheels which we need to lose. And you can’t ride a mountain bike if you’re not ready to lose the training wheels.

Personally, I’m going to try to make a conscious effort to not blame the machine when something goes wrong. Problem-solving is what we do as programmers and it would really help to look in the right place. The right place is in my program because I can trust that my troops have done what I’ve told them what to do and if things are going wrong, it’s because I’m telling them to do the wrong thing. I could go on waxing eloquent about how we need to take responsibility for our programs, but I think I’ve made my point. Our implementors are perfect machines, but we’re certainly not perfect and we need to keep that in mind.

As the years go by, it seems like we keep on piling layer on top of layer while the results aren’t proportional to what we have to learn to get things done. Now, I’m not proposing that we all start writing down-to-the-metal code or force everyone to become a programmer, but things are starting to look like a mess. Web programming is an interesting development, but it adds yet another layer on top of the existing kernel, operating system, libraries and GUI toolkits. Add to that the fact all browsers are still a bit different from each other and you can start to understand why I’ve yet to make a serious foray into web programming.

But even without the web and the many formats and barely interoperating systems out there, there’s enough on the desktop to get you depressed. Start with the fact that there are currently three major operating systems out there and if you want to write a program that runs on all three of them, you don’t have an easy task. You either embrace three different toolkits and programming methodologies and maintain 3 very different codebases, or you use something like Java which works on all three, but screams non-native on each one. Even though there are languages like Python that run on all of three, it really puts me off that there is still no top-notch multiplatform GUI library. wxWidgets tries pretty hard, but if you look at the screenshots you can pretty easily that they don’t look quite right. It’s not very surprising that lots of smart developers are flocking to the web, where things in comparison are a lot smoother.

There is also the fact that programming languages, like all other pieces of major software, suck more than others. I still stand by what I said in my last post, that it’s an exciting time for language enthusiasts, but I also feel that there are some lessons we really need to learn. I’m starting to have concerns that there may not be any true general purpose language, simply because there are so many different types of problems to be solved. I think we need to start creating broader categories: a set of systems languages similar to C going in the direction of D and Go. A set of hyper-optimizing VM-based languages designed for long-running, parallel server applications (the current JVM is a good example). A set of languages for writing end-user apps that are significantly high-level, but are still compiled to pretty fast native code (maybe not C or even optimized VM fast, but better than todays Python or Ruby). I’m thinking Python in its Unladen Swallow incarnation might fill this gap.

As a programmer, the state of tools that we have to use is really quite depressing. Tools like Emacs and Vi are powerful and all, but let’s face it: we could really be having much more powerful IDE technology. We should be having full blown incremental compilation with autocompletion and support for rendering documentation for every major language out there. We should also have seamless version control with granularity down to the undo level. Every change I make should be saved and I should be able to visually browse all these changes, see what they are and restore to an older state (or commit them if I want to). We have the raw computing power needed to do all this, but yet we remain stuck doing mostly batch-style edit-compile-debug cycles and mucking around in plain text. Eclipse with its incremental compiler makes things much easier, but there’s so much more we could be using our machines for.

As a user, what irritates me is the amount of manual labor we still have to do on a daily basis. We still have to carefully name and place files so that we can file them later. I have to manually hit the save button (see version control bit above). Even with the Internet collaboration is a mess with most people throwing around emails with increasingly larger attachments. Add to that the fact that most email clients are pretty dumb pieces of software. Google Wave is a step in the right direction, if enough people get around to actually using it (and if it can integrate to some extent at least with the desktop). Also I think the web and the desktop need to be brought closer together. Ideally I would be able to sit down on any computer with a live Internet connection and have my full custom work environment (or at least the most important parts of it).

I’m fully aware that none of the things I’ve mentioned are trivial. In fact, they’re probably very hard projects that will take expert teams a good few years to complete. One day I would like to seriously work on some of the programmer-related issues, especially the IDE part. I love Emacs, but there are some parts of Eclipse I really like too. For the time being I’m going to have to make do with what I have, but I’ll be sure to keep an eye for interesting things and movements in the right direction.

I guess it all started with the rise of scripting languages like Python, PHP, Ruby and the poster boy of scripting: Perl. Personally, these languages with their dynamic typing, “batteries included” design and interesting syntax provided a breath of fresh air from the likes of C++ and Java. Not that C++ and Java are necessarily bad languages, but they aren’t the most interesting of modern languages. In the early years of this decade computers were just getting fast enough to write large scale software in scripting languages. Things have changed a lot since then.

Dynamic languages aren’t just reserved for small scripts. Software like Ruby on Rails has proved that you can write really robust back end infrastructure with them. The languages for their part have kept on growing, adding features and making changes that keep them interesting and downright fun to use. Python 3.0 was a brave decision to make a break from backwards compatibility in order to do interesting things and it goes to show that these languages are far from ossifying or degrading.

Then there is JavaScript which was supposed to die a slow death by attrition as web programmers moved to Flash or Silverlight. But we all know that didn’t happen. JavaScript has stayed in the background since the rise of Netscape, but it’s only recently with advances in browser technology and growing standards support that it has really come into its own. I’ve only played with it a little, but it’s a fun little language which makes me feel a lot of the same emotions I felt when discovering Python for the first time. Thanks to efforts like Rhino, you can even use JavaScript on the client side for non-web related programming.

Of course, if you want to do really interesting things with these languages, then performance is not optional. Within the last year or two there’s been a strong push in both academia and industry to find ways to make these languages faster and safer. Google in particular seems to be in the thick of it. Chrome’s V8 JavaScript engine is probably the fastest client side JavaScript environment and their still experimental Unladen Swallow project has already made headway in improving Python performance. V8 has already enabled some amazing projects and I’m waiting to see what Unladen Swallow will do.

While we’re on the topic of performance, mentioning the Java Virtual Machine is  a must. The language itself seems to have fallen from grace lately, but the JVM is home to some of the most powerful compiler technology on the planet. It’s no wonder then that the JVM has become the target for a bunch of interesting languages. There are the ports of popular languages — JRuby, Jython and Rhino. But the more interesting ones are the JVM-centric ones. Scala is really interesting in that it was born of an academic research project but is becoming the strongest contender to Java’s position of premier JVM language. Clojure is another language that I don’t think many people saw coming. It brings the power of LISP to a modern JVM unleashing a wide range of possibilities. It has it’s detractors, but it’s certainly done a fair bit to make Lisp a well known name again.

Academia has always been a hot bed when it comes to language design. It’s produced wonders like Lisp and Prolog and is making waves again with creations like Haskell (whose goal is ostensibly to avoid popularity at all costs) and the ML group of languages. These powerful functional languages with wonderful type inference are a language aficionado’s dream come true in many ways and they still have years of innovation ahead of them.

Almost as a corollary to the theoretically grounded functional languages, systems languages have been getting some love too. D and now Go are both languages that acknowledge that C and C++ have both had their heyday and it’s time to realize that systems programming does not have to be synonymous with bit twiddling. D has gotten some flak recently for not evolving very cleanly over the last few years, but something is better than nothing. Also a real shift towards eliminating manual memory management is a welcome addition.

As someone who intends to seriously study language design and the related concepts in the years to come, it’s a really great time to be in getting involved in learning about languages. At the moment I’m trying to teach myself Common Lisp and I have a Scala book sitting on the shelf too. One fo these days, I plan on sitting down and making a little toy language to get used to the idea of creating a language. Till then, it’s going to be really interesting just watching how things work out in an increasingly multilingual world.

The book is the brainchild of Peter Seibel, the author of the very well written Practical Common Lisp which I can safely recommend as a good starting point for people starting to learn Lisp. Seibel’s interviewing style is very effective, the questions are always short and to the point but they help guide the person in question into providing a lot of very interesting and useful information. They are also fairly open ended allowing the interviews to flow in interesting directions, unique to each person. He asks some general questions to all 15, things such as how they learned to program, what were their hardest problems or bugs, how they go about debugging and what their idea of the current state of programming is. But he also asks them things about their particular area of work. For example, with Jamie Zawinski he talks a lot about Netscape and Mozilla while with Donald Knuth the obvious topics are his masterpiece The Art of Computer Programming and literate programming. There are also some questions that you wouldn’t always think about, such as how to spot programming talent.

What makes the book very accessible is that it keeps technical jargon to a minimum. Anyone who has programming computers for a significant amount of time will have no problem in understanding most of the content. For areas that are more esoteric the programmers themselves provide very lucid explanations making it quite easy to follow along. A lot of the matter is about things that each programmer encounters: how to read someone elses code, how to debug programs, how to collaborate and pass on projects, how to improve as a programmer, what educational/personal qualifications are most helpful, etc. The book isn’t just about how awesome these people are: it’s about understanding how masters of the craft rose to that position and how us mere mortals might follow in their footsteps. Seibel does a pretty good job of asking the same question that many of us programmers would like to ask them if we had the chance.

Beyond the structure of the interviews, the book is made amazing by the stellar choice of programmers. Not only are they master programmers, they have very diverse experiences and interests resulting in lots of very interesting and powerful opinions. I particularly enjoyed Guy Steele’s interview which contains a fair amount of philosophy regarding programming and computer science and Fran Allen’s call for computing to become more socially relevant. At the same time, Zawinski and Thompson embodied what can be best described as the classic hacker mindset: coding for the pure joy of creation and problem solving. Many of the interviews are humorous in some way or the other and none of them degenerated into holy wars or religious diatribes.

There is absolutely no reason why you should not read this book. There are some areas for improvement, but these are of a presentational nature (I wish the interviews were broken in segments so that I didn’t have to read a full one at a sitting). Content wise there is nothing I have to say. Will this book help you become a master programmer? Well, not in any tangible, measurable sense. This book is not a howto or a handbook. Each person had their own ideas of how to do things and many of them disagree with what could be called ‘industry standards’ (in particular how code should be formatted or structured). What this book will do is give a window into the minds of some of the legends of our culture. I’ve found it most useful to compare what these people have done and how they think to what I do and think myself. Essentially, read the book the same way you would read Benjamin Franklin’s autobiography. You won’t learn how to become a great statesman, inventor or Founding Father of a nation, but you can figure out some ways to live a better, more impactful life.

Coders at Work tells the stories of people who might be very well be considered heroes of our times. If you have an interest in becoming a better programmer, go buy this book, read it cover to cover, think about everything they say about their programming practices and experiences and see how you can use that to make your own experience better. And above all else, remember and act on what you read.