Integrated Development Environments are one of the primary tools of our trade. But personally I think an IDE can only be a good product if it has a robust text editing component. The king of text editors is Emacs. No offense to Vi users, but the Emacs + Elisp combo is unbeatable in my opinion, for a number of different reasons, programmability being the main one. Of course Emacs does have its problems. Steve Yegge discusses these problems quite thoroughly in his post on XEmacs. What’s interesting is that he claims that if Emacs doesn’t get its act together and fix its problems, it won’t be long before a browser sprouts enough editing capabilities to eclipse Emacs. I think that time is close at hand.
Enter Bespin. It’s a Mozilla project that is still under heavy development, but there’s a public demo already available. Bespin is an online text-editing environment written entirely in open source JavaScript. It’s provides syntaz highlighting and some project management and looks really slick by making use of the HTML5 canvas element. The best way to understand what Bespin can do would be to watch their introductory video:
It may not be immediately obvious what all the fuss is about if you’re used to using a modern IDE like Eclipse. After all, it’s just a text editor, right? Yes, it is just a text editor and that’s where the beauty of it lies. Firstly, t’s programmable in the Emacs tradition. That means that you are allowed and encouraged to write your own bits of JavaScript to adapt it to work the way you want to. Secondly, it’s sports an open server API which means that you could potentially use Bespin as an editor-like interface to anything you want. This has already led to Bespin running on top of the Eclipse IDE. Eclipse acts as the server and Bespin acts as the editor interface to it. Here’s a screenshot of Bespin+Eclipse in action showing Eclipse communicating errors in the Java code that Bespin is editing.
Keep in mind that Bespin is still in a very experimental stage. However, the example above is clear evidence that Bespin is going to be a force to reckon with in the IDE sphere soon enough. I think that bringing a client-server model to IDEs is going to offer a significant productivity improvement to programmers especially as developers and companies have to wrangle increasingly complex, heavy weight codebases.
Development machines need to be real monsters nowadays to efficiently handle the demands of the software being developed. Using Bespin, developers could invest in industrial strength servers which serve as both code repositories and build farms. The developer’s machine then act simply as terminals or can run local lightweight server instances for the Bespin frontends to connect to. This means that developers can afford having cheaper machines themselves, but don’t need to wait for long compile times as they’re being outsourced to the powerful servers. Code is also centrally stored meaning that it can be pulled up a moment’s notice and worked on collaboratively. This could mean more efficient workflows for software companies.
But the biggest win would be for the developers themselves. Here is how I can see myself using Bespin: I’d have a machine sitting in a safe, well connected location running a Bespin server. This would be my main development machine so I’d trick it out with as much RAM and processor speed as I could afford. The server itself might be some form of headless Eclipse or something equivalent for whatever language I happen to be progamming in. The cool thing is that since there is an implementation agnostic server API, I could even write my own server that lashes together compilers and build tools for whatever language I happen to be using and displays results via Bespin. The server would also use a version control system like Git seamlessly as part of the storage infrastructure. I could then connect to this server either from any modern browser meaning that I wouldn’t have to drag a machine with me to write some code. Anywhere hacking would be a boon for open source developers and students. I can also imagine a Github-like service that hosts both files and their development tools so that developers can avoid even running their own Bespin servers.
So is the age of always-available, online IDEs upon us? I think not quite yet. It’s still untested territory to a large degree. Bespin itself is coming along quite nicely and is implementing some very interesting ideas. But a lot is going to depend on how well the Bespin servers work. Headless Eclipse is fine for Java, but there needs to be similarly capable systems evolving for other languages and platforms as well. The API does a good job of keeping things minimal and implementation agnostic, but it also puts the onus on server developers to make quality systems targeting their respective platforms. I’m looking forward to some good work coming out of the Bespin and related projects in the near future. I’m even seriously considering learning some JavaScript so that I can better understand how the whole thing works. For the time being though I’ll stick to Emacs, though I’ll certainly keep an eye on the Bespin mailing list.
Programmers and hackers are often an introverted bunch. But the things we create have a considerable and measureable impact on the world and conversely our work is affected and influenced by things around us. Today’s selection is an attempt on my part to highlight those facts.
Reading
What Open Source shares with Science I’m sure this analogy has been made numberous times and I think it’s a pretty good one. This post is an interesting read because it goes quite a bit in depth regarding the history of science.
Media
The Last Lecture Carnegie Mellon Computer Science Professor Randy Pausch was one of the world’s prominent experts in virtual reality but he’s probably best known for his Last Lecture. I think everyone on the planet should watch this, especially if they’re young. There’s some computer science in it, but it’s the holistic experience that it’s worth watching for.
Books
I’ve decided to drop the software part for this installment in favor of something more 20th century.
Rebel Code This is the story of Linux and the Open Source Revolution and how it played an important part it creating the world we see around us today.
After four months away I’m now back to hosting The ByteBaker at WordPress.com. I would have liked to stay on paid hosting and experimented some more with custom themes and widgets, but unfortunately my hosting experience at Fatcow has not made that possible. Things started out well but have recently deteriorated to below acceptable levels in terms of speed. I still have 6 months of hosting left and I have some ideas for a purely static site that I might want to put up, but I won’t make any progress on that until later in the summer.
For now it’s back to The ByteBaker as my main platform. I’ve decided to go with a clean, minimal theme with just a small amount of customization. I’ve always wanted to have something that was a bit more permanent than a day to day blog. With that in mind I’ve decided to have a static front page for new readers who are looking to have a quick overview of the most important articles that I’ve posted. I’m working on a longer post on why a pure blog isn’t the most suited to what I want, but for now the best I can do is to refer you to Steve Yegge’s blog posts on the matter.
When I talked about starting a life recording experiment, I mentioned how I wanted to have a wiki that I could use as a staging area for possible blog posts and as a catch-all for any interesting ideas I might have. I wasn’t sure about what service or tool I wanted to use because I had some conflicting requirements. At this point the main choices I’m considering is using an online wiki hosted at PBWorks or writing text files which get turned into web pages on Github Pages. I’m going to start trials on both of them and make a decision by the end of the summer.
I’m looking forward to writing some good posts with a simpler, more efficient setup on hand.
So for the past week or so my host seems to have been having serious troubles with their MySQL servers which means that and WordPress-based system (such as this blog) is running painfully slowly. The experience hasn’t been a good one for me and I’ve seriously been considering switching hosts (especially since the only reply I can get is that the engineers are aware of the issue and are looking into it). I don’t enjoy the whole process of selecting and then moving to a new web host, but I’d take that pain if it meant my site ran smoother.
My search for a good hosts was rather depressing. Being a starving college student, I really didn’t want to spend a lot of money on hosting. I found a number of good deals online, but almost all the hosts I looked at had a fair amount of negative reviews about them. I understand that if your operations grow beyond a certain size you’re going to have some disgruntled customers no matter what. But I just couldn’t bring myself to actually deciding on another host. I’m a little reluctant because I’m only about 4 months into the year that I’ve paid for and I would really like not to lose that money.
Four main factors influenced my not making a decision. Firstly, I want to give my current hosts some time to solve their current problems and see if they actually do anything. Secondly, I wanted to look for cheaper alternatives. This blog has always been running on WordPress and I really like the WordPress platform. Admittedly the only other platform that I’ve really used is Blogger and that too, years ago. For most of last year this blog was hosted on the free WordPress.com site with a custom domain name (that cost me only $10 a year).
WordPress.com gives a really robust hosting solution at the cost of limited customization. Though I first moved away from WordPress.com because I wanted to have more control, I’ve since come to realize that I don’t do all that much customization. In fact, I prefer to have a more content driven-site and so adding widgets and such is really not that important. The main issue is that since I moved to dedicated hosting my trafiic has almost tripled. I’m not sure whether that’s actually a result of moving away from WordPress or simply that I’ve been putting in more effort. I’d like to think it’s the latter but I’m not sure. I also just read a very detailed article on how much hosting bandwidth a website needs as the readership grows. Though I still have a rather small daily hit rate, I have had the occassional spikes. WordPress.com has handled theses spikes without flinching, but I’m pretty sure that my current host wouldn’t do as well. This is an important consideration for me as readership has been increasingly steadily over the past few months and I want to keep encouraging it. A slow or unresponsive site would really derail those plans.
The third option is more expensive: using a more expensive solution such as MediaTemple’s Grid Service or one of the plans from FutureQuest or Pair (both of which have almost no negative reviews). They both sound like great options, but at about $10-$20 a month, I’m not sure I can justify the investment, especially since it will certainly be a good few months since I actually have so much traffic. Since I have no plans to monetize this site all that money would be coming straight of my pocket. Again, not something that I would enjoy doing.
While being a student may have some disadvantages in terms of financial strength, the good thing is that I get my school’s great network services all for free. I already have a small server on the network which I use for file backups and I could just host The ByteBaker on its very own server (if the network people are fine with it). The only downside of this is that I would then have to do all the server administration on my own. Having never actually administered my own open-to-the-world server I don’t know how much of a challenge this would be. I would have total control and wouldn’t really have to think much about bandwidth or excessive CPU strain (unless the site suddenly gets hundreds of thousands of hits a day). On the flip side, I would be totally on my own if something were to ever go wrong and that’s rather unsettling, even for a do-it-yourself guy like me. Once again, I would really like to be able to just write this blog and not fiddle with the technology stack behind it.
One of the reasons I like to write blog posts is that they’re a great way for me to think out loud. In the process of writing this post I’ve come to realize that simplicity is really what’s important for me. The more I’ve been thinking about it, the more I realize that WordPress.com is actually the sweet spot for me at this point. For just $10 a year I get a really robust platform which should be able to scale very well. And for just $15 more I get custom CSS for my site which combined with their Sandbox theme means that I can have most of what I can get from a custom theme. The only thing that I don’t get is the ability to use my own plugins and themes and I’m stuck with a very basic stats tracking package, but that’s something I’m willing to accept. In retrospect, I think I may have jumped the gun somewhat by moving to paid hosting without seriously exploring everything that WordPress.com had to offer.
If it weren’t for the fact that I’d spend a considerable amount on my current host, I would move back to WordPress.com right now. But considering the circumstances, I’ll wait till the end of the week to see if my host resolves its SQL issues. I don’t have much hope because after reading around on the internet, it seems like other people have been having the same problem for a while with no solution. At the moment it seems like by Monday I’ll be back at my old home. The domain name and the RSS feed will still say the same, so regular readers have no reason to worry. Here’s looking to a faster, simpler ByteBaker.
Those of you who have visited the site in the last few days have probably noticed that page loads are going slow. I’ve been talking to my host and it seems like they have a problem with their CGI/MySQL servers causing websites that are backed by databases (such as this one) to run slowly. They tell me that they are working to resolve the problem and I hope they come up with a solution quickly. If the issue doesn’t get resolved soon I might seriously consider moving to another host.
I’m sorry for the inconvenience this is causing to my readers and I really hope that things get resolved soon.
When I first started to get really interested in computers in the late 90s, most of the computers I saw where still boring beige boxes and they had rather prominent reset buttons. This was probably good because almost all the computers I saw run Windows (either 98 or 95) and had to be restarted rather often. Since then the reset button seems to have fallen from grace. My family’s first computer in 2001 had a large power button, but the reset button was tucked sort of underneath it and doubled as the disk activity indicator. My current laptop which I’ve had for about two years now doesn’t even have a reset button. If I want to do a hard reboot I have to hold down the power button for a few seconds to turn it off and then start it again.
To be honest, I really don’t need to do a hard reboot all that much nowadays. It’s mostly when I’ve royally screwed up X configuration and nothing responds anymore. So I can’t say that I really miss it all that much and I’m not too concerned with its demise. However computers do need to be restarted every now and then, though I hope we’re getting to the point where that becomes an extremely rare necessity. But hardware isn’t the only thing that needs restarting. Software projects occassionally need a restart too.
At the recent EuroLisp Symposium there was talk of scrapping Common Lisp and starting over. The Firefox browser descends from the Mozilla which in turn originates from the Netscape code base that was open-sourced in 1998. But in the process, almost all the original Mozilla was rewritten in a better form (especially the core Gecko layout engine). The Unix concept has been reimplemented numerous times over the last decades. I’ve been working on a research project for the past year to explore applying formal grammars to describe complex patterns. In the process I’ve become the toolmaker for the group — I’ve been building a set of Python programs that the others can use to do experimentation. I’m currently working on the 5th rewrite of this toolset and each time the tools get better and I get faster at bringing things back to their previous level of usefulness (and then some). In case you’re wondering, I’d like to open source these tools in the future if the rest of the group agrees.
Hitting the reset button on a software project may seem like a hard decision to make, but I’ve come to think of it as a natural part of the software development project. Like it or not, software continually grows. Once a version goes out to the public people start finding things they don’t like or features they’d like to have. All this goes on the drawing board for the next version. As time goes by, the software pushes against the boundaries of what it was supposed to do. Layers are added and parts are written to support the new additions. In a healthy project with good developers and strong leadership (or community consensus), it’s possible to make the additions smoothly, at least for a long time. But if you add too much to a particular piece of software, the pieces gradually start to lose cohesion. Competing subsystems and inconsistent interfaces begin to emerge and it becomes harder and harder to add non-trivial functionality or even fix outstanding bugs. At some point, the codebase reaches a sort of critical mass: it’s easier to scrap the whole darn mess and rebuild it from scratch.
I took a software engineering class last semester and learned all about agile development and the iterated development model. There are some good ideas in there and it’s certainly better than waterfall models, but I think that it misses one important idea: users and there software form a feedback loop which can be essentially unbounded. Agile models help developers build better software and encourage maintainability, but there’s still a clear delivery date after which the developers role is reduced to mainly bug fixes. We don’t have a model for how to build organic software that can grow with users’ needs over the years (though I think we’re getting closer with the success of large scale open source user-centric projects like Firefox). Till we do have such a model, I think rewrites should be considered an essential part of a software project’s life-cycle.
Restarting a software project isn’t a decision to be taken lightly, but it’s sometimes the best way to go. The problem is, it takes a good amount of will power from strong leaders to make a rewrite work properly and it’s easy to mess up the process. From my personal experience of resetting a project (and from what I’ve read) perhaps the hardest part is letting go of what’s been done before. It’s tempting to just reuse components from previous generations if they worked right (even if they were stopgap hacks at the time). This is tempting because not only do developers not have to actually rewrite from scratch, it can make things much faster. A rewrite by definition means that you won’t have a fully working product for a while and that feeling can be quite frustrating especially when you know that there actually is a working version in cold-storage. If only you could just borrow a few parts, life would be much better. Of course doing that defeats the very purpose of a rewrite. I’ll admit that there are some circumstances where code reuse can work, but these situations need to be carefully thought out and the code itself needs to be scrubbed clean so that it doesn’t reintroduce any of the problems that prompted the rewrite.
As this post approaches the thousand word mark, I think I should spare a word for the users of software under rewrite (whether they be end-users or other developers). Users need to have software that works, not something that’s broken and clearly experimental. Software under rewrite shouldn’t be pushed out until the rewrite is done and the older version should be kept available. Even more important is that the rewrite should actually produce something that the users want. The new product should be sufficiently similar to the old that the learning curve is mostly negligible (at least for everyday tasks) and old bugs should be fixed. If you’re going to do a rewrite, do it properly and fix the bugs while you’re at it.
The people who are most likely to get shafted from a rewrite are developers who are using your API. If the rewrite requires substantial changes to the API it will break a lot of client software and make a lot of people very unhappy. It that seems to be the case it’s time to slam on the brakes and see if the existing API can be preserved. This isn’t as much of a problem if the API is well-designed to start with, but that’s the topic for another post. At this point I’d just like to point out one of Jamie Zawinski’s classic articles which you should read before starting any software project (including a rewrite) and at various points during the process.
I used this post as a way to organize some of the thoughts I’ve had while during my own rewrite. I’m trying very hard to keep the external interface the same, but I might have to make some sacrifices. I’m also going to be exposing a public API for the first time so that there can be multiple independent interfaces written so that’s something that will take some thought. I also have a buglist from the last version in front of me because I think that allowing bugs to propagate across generations is just lazy. I’m hoping that by the end of it I’ll have a cleaner system as well as a project that I can cleanly extend without a rewrite for a longer time. I don’t envision any more rewrites for a while to come. But then again, the only guarantee of life is death, so I’m not placing any bets. And besides, rebuilding from scratch is kinda fun too.
It’s a lovely day here in South Virginia, I’m back from a very enjoyable 4-mile hike and now it’s time for me to put together another round of readings media and software from around the web.
Reading
Unix turns 40 I’ve been a Linux user for about three years now and a Mac user for about a year and a half. Both of these are Unix variants and I feel very indebted to the makers of Unix and all it’s variants and improvements over the decades for giving me an excellent learning tool and a rock-stable programming platform
Media
Google Wave Preview The world’s first look at what promises to be a very interesting technology and collaboration platform in the years to come. This is definitely something I wish for everytime I work on a group project (which is increasingly often.) I think most team-workers will play the same way.
Software
Chromium This is the open source core around which Google Chrome is built. Keep in mind that this is a developer beta for Linux and OS X users and not ready for prime time at all. But it would be great if you take a look and fill out bug reports.
Yesterday I decided to get a GitHub account. I don’t have anything to really put up on GitHub as of now, but I decided to get an account just the same. I wanted the username ‘basu’ because that’s usually the name people use to call me (and it’s the username I use on my own machines). However, that name was taken and so I settled for something just a little different: ‘basus’. Just out of curiosity, I decided to look up the profile of user ‘basu’. It turned out to be someone who had signed up in April, but didn’t seem to have actually done anything on Github. I felt a little surge of anger that someone had taken my name and then done absolutely nothing with it. I guess I really do like my name after all.
The thing is, it’s getting increasingly difficult to find a good name on the internet. And I’m not just talking about usernames. It took quite a bit of searching before I found a domain name that was free and that I wanted to use. When it comes to usernames, I try to get names that have ‘basu’ in them, generally with a letter or number at the end. That’s true for all my email addresses (I use about 3 full time for various things) and for my Twitter account. If it’s a new enough service, I can generally get just what I wanted. (If I was at GitHub a few months earlier, I would have gotten my name too)
It might seem a bit vain that I’m making a fuss about getting the name I want, but I have a reason. For this purpose, let’s divide the web services that I use in to two groups. First are the public ones that I want to give out to people. These are things like my email, my website and various social network/professional things. I want a username that’s easy for people to remember and is as close to my real name as I can get it. I don’t like having pseudonyms or handles because I think they’re just a drag (though I have nothing against people who do have them). It’s easy for people to get in touch with me if all they have to remember is my name and initials. Things get harder if they have to remember some non-obvious combination of my name and other letters or numbers (even if they make perfect sense to me). My personal email has my birthday in it, which is ok for family and close friends, but I don’t give it out to anyone else.
The second type of service is private. These are services I use where the name is used exclusively for login and verification purposes. I probably don’t want other people to know my username, but my passwords are generally pretty strong. What’s more important for me is that I would rather not have a bunch of different usernames and passwords for all the services that I use. Nothing irritates me more than when I just need to check something quick and I end up needing three attempts just to get the correct combination password. It’s a bit easier for services which require that I have an email address as login, I have a separate email account set aside for just that purpose.
While the bad news is that it can be too late to get the name you want, the good news is that there are other people who feel the same way and are doing something about it. The OpenID project is aimed at addressing the issue of having a long list of various login combinations and the security and usability problems associated with that. The idea of OpenID is simple: you create a single account with a traditional username and password with a service that is an OpenID provider. There are some dedicated OpenID providers like ClaimID or myOpenID, but a bunch of services you might already use like AOL, MySpace and WordPress.com are also providers. Once you’ve signed up, you get a URL which is your actual OpenID. Now when logging into a service that supports OpenID you just use your URL and tell your provider that you want that particular service to use your OpenID. This allows you to login to a bunch of different services with a single authentication system. It’s easier on your brain cells and safer too. There are a bunch of services that use OpenID with more joining all the time.
So I’m still a bit disgruntled about not being able to get my name on GitHub, but I’ll get over it eventually. In the long run, I suppose it won’t matter all that much. In some ways, I wish that people got some sort of unique identifier that worked uniformly accross all services everywhere. I think OpenID is a step in the right direction, though its not going to be a solution for everything. This is one of the cases where the problem requires a solution that is social as well as technical: how does everyone get the name/identity they want without stepping on other peoples’ toes. Like most other problems of that sort, there is no cut and dried answer, just a number of possible options and we have to choose the one we’re most comfortable with. For now the solution I’v chosen is a combination of OpenID and separate usernames, but like so many other things, I’ll keep my eyes open for something better.
I just started work at the KnowledgeWorks building at Virginia Tech’s Corporate Research Center yesterday. The CRC is a nice place with lots of wide open spaces between buildings, lots of greenery and a quite lovely little lake in the middle of it all. The place I’ll be working is a floor that houses a large portion of Tech’s Computer Science department. The floor has an interesting layout. There seem to be faculty offices around the outer walls of the building and the central space is dedicated to grad students. They aren’t quite cubicles, but it’s not open space either. There are numerous curving partitions with about 3-4 workstations along each partition. There’s also a small area with couches for a more relaxed setting. It’s different from the large open computer labs that I’m used to working in, but I think it’ll grow on me. Each individual work area is just enough for one person to work. It’s quite possible to ignore everyone else (especially with a pair of headphones) but there’s ample opportunity for interaction. I only wish there were more whiteboards.
Yesterday was also the first day I actually got my Linux laptop to use an external monitor correctly. I’ve tried hooking up an external monitor to my laptop on numerous occasions, but I don’t think I ever got it to work quite right. There were a number of things working together to stop me everytime: drivers that didn’t work quite right, window managers that didn’t handle multiple monitors quite how I wanted them to, etc. I think I came close on a number of occasions, but I wanted something that ‘just worked’ and none of the solutions I tried were quite like that. However, a combination of XRandR, the open source radeonhd driver and the Xmonad window manager managed to let me hook up an external monitor in a way I really liked. Xmonad lets each monitor act as a viewport onto a different virtual screen, which I think is the best way to do multiple monitors. By comparison, the Mac Spaces approach of treating both monitors as a single huge desktop is something I can’t stand. I only wish that the Xmonad documentation were updated to say that their support worked with XRandR instead of the deprecated Xinerama.
The previous two paragraphs might seem unrelated, but they both have to do with creating with comfortable, efficient computer-centric workspaces. An increasing number of people are spendingsignificant amounts of time in front of computers (for business and for pleasure) and so it’s important to have a space that is enjoyable to be in. After coming back from work I remembered that I had read about a really amazing home office setup which I thought was a very good place for a computer professional to work out of. Some googling around turned up three such well thought-out office layouts that I think are good examples of what a comfortable workspace should be like. They are:
- Stefan Didak’s home office
- Mitch Haile’s home office FAQ (with some good layout diagrams)
- Dennis Klein’s home office
The spaces these people have made are pretty close to a computer geek’s dreams. Not only do they have an amazing amount of power under the hood, they also look good. These are all spaces that I would love to work in. Though I would probably run Linux with a tiling window manager of some sort.
Since I started college, I’ve learned to appeciate the value of a good working space more and more. I’ve also learned that building a good space isn’t easy, especially if you work with other people. It’s hard to get the right combination of openness and privacy. Back at Lafayette I live in a single room. It’s nice to be able to shut the door, but my desk is pretty small and the chair really sucks. The labs have much more space and better chairs but it can get a bit distracting if there are a lot of people in the room. The library is well lit and spacious and a good environment on the whole, but again too many people are a problem.
A lot of these problems are solved by simply having a personal office (with a door) with coworkers within walking distance. Beyond that, the problem becomes one of setup, equipment and space utilization. I think the setups described above are really good at serving the owner’s needs. At this point, a lot of the descriptions are subjective. I personally would have no idea what to do with 5 or 6 monitors, but I could definitely get used to 2 or 3. In fact, I’m starting to think that two large monitors is the bare minimum for anyone who does serious work with a computer. A comfortable chair and a decently ergonomic setup are essential unless you want to shell out a fair amount of cash for medical treatment later. I don’t have much of an opinion on tables and desks, but I’m fairly picky about chairs and keyboards.
On a bit of a tangent, I’ve been hearing about this new trend called coworking where freelance workers (mostly software people) get together to work in a semi-public space. Now, I love to have people near me to bounce ideas off, but I wouldn’t be comfortable working in a semi-public setting all the time. I love my laptop, but don’t like to be working off it all the time. If I ever become that sort of a freelance/work-from-home worker, I might cowork once or twice a week, but I’ll still want a decently tricked out home office.
On an ending note, let me just reiterate how important I think it is to have a good working environment. I still have a long way to go before I can put together anything as neat as the spaces you can find online, but till then I’m going to keep my eyes open for interesting and comfortable setups and environments. Though one thing is certain, I’m definitely going to have a bunch of monitors.
Over the past year and a half I’ve come to realize that writing documentation for your programs is important. Not only is documentation helpful for your users, it forces you to think about and explain the workings of your code. Unfortunately, I’ve found the tools used to create documentation (especially user-oriented documentation) to be somewhat lacking. While we have powerful programmable IDEs and equally powerful version control and distribution systems, the corresponding tools for writing documentation aren’t quite at the same level.
Let me start by acknowledging that there are different types of documentation for different purposes. In-code documentation in the form of comments are geared toward people who will be using your code directly, either editing it or using the API that it exposes. In this area there are actually good tools available. Systems like Doxygen, Epydoc or Javadoc can take formatted comments in code and turn them into API references in the form of HTML or other formats. Having the API info right in the code, it’s easier to make sure that changes in one are reflected in the other.
User-oriented documentation has slightly different needs. As a programmer you want a system that is easy to learn and is fast to use. You also want to be able to publish it different formats. At the very least you want to be able to create HTML pages from a template. But you also want the actual source to be human-readable (that’s actually a side-effect of being easy to create) because that’s probably what you, as the creator, will be reading and editing the most.
Then there are documents that are created as part of the design and coding process. This is generally wiki territory. A lot of this is stuff that will be rewritten over and over as time progresses. At the same time, it’s possible that much of this will eventually find its way into the user docs. In this case, ease of use is paramount. You want to get your thoughts and ideas down as quickly as possible so that you can move on to the next thought. Version controlling is also good to have so that you can see the evolution of the project over time. You might also want some sort of export feature so that you can get a dump of the wiki when necessary.
Personally, I would like to see the user doc and development wikis running as two parts of the same documentation system. Unfortunately, I haven’t found tools that are quite suitable. I would like all the documentation to be part of the same repository where all my code is stored. However, this same documentation needs to be easily exported to decent looking web pages and PDFs and placed online with minimal effort on my part. The editing tools also need to be simple and quick with a minimal learning curve.
There are several free online wiki providers out there such as PBworks and WikiDot which allow the easy creation of good looking wikis. But I’m hesitant to use any of them since there isn’t an easy way to easily tie them into Git. Another solution is to make use of Github’s Pages features. Github lets you host your git repositories online so that others can get them easily and start hacking on them. The Pages features allows you to create simple text files with either the Textile or Markdown formatting systems and have them automatically turned into good looking HTML pages. This is a good idea on the whole and the system seems fairly straightforward to use, with some initial setup. The engine behind Pages, called Jekyll is also free to download and use on your own website (and doesn’t require a Git repository).
In addition to these ‘enterprise-quality’ solutions, there are also a number of smaller, more home-grown solutions (though it could be argued that Jekyll is very homegrown). There’s git-wiki, a simple wiki system written in Ruby using Git as the backend. Ikiwiki is a Git or Mercurial based wiki compiler, in that it takes in pages written in a wiki syntax and creates HTML pages. These are viable solutions if you like to have complete control of how your documentation is generated and stored.
Though each of these are great in and of themselves, I still can’t help feeling that there is something missing. In particular, there is lack of a common consensus of how documentation should be created and presented. Some projects have static websites, others have wikis, a few have downloadable PDFs. Equally importantly there isn’t even a moderately common system for creating this documentation. There are all the ways I’ve noted above, which seem to be the most popular. There are also more formal standards like DocBook. Finally lets not forget man and info pages. You can also create your own documentation purely by hand using HTML or LaTex. Contrast this to the way software distribution works (at least in open source): there are binary packages and source tarballs and in many cases some sort of bleeding-edge repository access. There are some exceptions and variations in detail, but by and large things are similar across the board.
Personally, I still can’t make up my mind as to how to manage my own documentation. I like the professional output that LaTex provides and DocBook seems like a well-thought-out standard, but I’d rather not deal with the formatting requirements, especially in documents that can change easily. I really like wikis for ease of use and anywhere editability, but I must be able to save things to my personal repository and I don’t want to host my own wiki server. I’ve previously just written documentation in plain text files and though this is good for just getting the information down, it’s not really something that can be shown to the outside world. For the time being, I’ll be sticking to my plain text files, but I’m seriously considering using Github Pages. For me this offers the dual benefit of easy creation in the form of text files as well having decent online output for whatever I decide to share. I lose the ability to edit from anywhere via the internet, but that’s a price I’m willing to pay. I can still use Google Docs as a emergency temporary staging area. I’m interested in learning how other developers organize their documentation and would gladly hear any advice. There’s a strong chance that my system will change in some way in the future, but that’s true of any system I might adopt.
