The role of software engineering in research

As my research project continues our software is gradually growing more and more features, and we’re adding new functions almost everyday. While it’s certainly a very good feeling to be adding functionality, it can also get ungainly very quickly. About two weeks ago, Slashdot posted a very interesting query by a person interested in doing a PhD on software design principles. Since then, I’ve been giving more and more thought to the role that industry-style software engineering principles play (or should play) in computer science research.

Research in science has been mostly held to be separate from engineering. Chemistry research is not performed in quite the same way that chemical engineering is. Even in cases where there is significant amounts of high technology equipment is used (the LHC is a good recent example), the engineering teams are separate from the scientists. However, computer science is not really a typical science at all. Research in computer science faces many of the same problems that software development faces:

  • Multiple people working simultaneously on large and growing codebases (team management).
  • Adding or removing features without requiring major rewrites (extensibility).
  • A way to collect large amounts of data and detect failures (testing and error logging)
  • Having multiple copies of the experiment running, with different parameters (scalability and version control).
  • Last, but not least, the experiment should actually work and produce results (in corporate terms, the software should ship).

Keeping the similarities in mind, what lessons can we as researchers learn from industry techniques and practices (by industry, I mean open source as well)? I’m not sure what other researchers do, or if there is some sort of standard procedure, so I’ll just talk about what my small team has done to make our work easier.

Divide and conquer:

The first thing we did was to carve up the various parts of the project. While my two teammates took up the core engine and the output system, I chose to deal with the human interaction part. At the same time. we had regular meetings where we planned out how our parts should interact as well making high-level descriptions of what each part of the code should do. This meant we could each develop our code largely independent of the others while still having a fair idea of where the whole project was going.

Throwing away version 1:

Though our first version was a working program, we made it keeping in mind that we would probably have to change a lot of things, possible even rewrite it. As a result, we developed it quickly, and focused on making it “just work” without adding too many features. This allowed to get a good feel for what the rest of the project would be like and let us make important decisions before we had invested a lot of time into it. In research you often don’t know what you’re looking for until you start looking, and this was the case for us. Our throwaway version gave us a better idea of which direction our research was heading, what concepts had been explored before, what was new and how difficult various aspects were.

Strict Version control:

Right after we started working on our post-throwaway system, we moved all our code to a Subversion repository on our college’s research cluster. We do are own work in our working commits and perform daily commits and updaes. Our repository is also divided into trunk, branch and tag directories to keep separate the most recent copy, older releases and other milestones. This way we can see the evolution of our project at a glance more easily than having to check out older versions. This comes in particularly handy, since being a research project we tend to be looking at what we did before and what results we got rather regularly.

Coding conventions:

Another post-throwaway decision was regarding coding styles. We fixed general naming and commenting styles, decided on tabs vs. spaces (4-space indent, in case you’re interested) as well as a number of minor things that weren’t project-threatening, but could have led to unnessecary confusion later on. We also keep detailed records of how we’ve changed our codes and any bugs we’ve introduced/fixed on a common wiki meaning that everyone is always up-to-date on changes.

Personal freedom:

By standardizing some aspects of our project, we’ve been able to have greater freedom as individual researchers. All our code is in platform-independent Python. Besides that our only other common tool is Subversion, which is also freely available and easy to use. By choosing open source and platform-independence, we have been able to have three developers work on three different platforms using different editors, IDEs and SVN clients. We can work equally well on our own machines sitting in our rooms, on the computers in our CS department’s labs, or even from the Mac Pro computers in the Arts buildings via SSH. It’s hard to quantify how much easier this independence has made our work. This has meant that we didn’t have to rush to the CS Lab’s whenever we had a thought or problem and we could whenever we wanted, however we wanted as long as we stuck to some simple rules. Scientists and programmers both need a certain amount of freedom and we’ve been able to secure this freedom by a proper choice of tools and techniques.

Things to do:

It’s been almost four weeks since we started using software engineering practices for our research work. We don’t know how things would have turned out had we not implemented the practices that we have. However we have come to rely heavily on the framework and guidelines that we have built for ourselves. At the same time, we’ve come to realize that there are parts of our project that could benefit from adopting some more industry-standard practices. I’ll end with a brief list of the things we plan to implement in the next few weeks:

Robust error handling: We’re currently using some very ad-hoc methods to handle errors and track down bugs. But we’re in the process of building a flexible, yet robust error handling and reporting system into our project. Since we intend our tool to be used by non-programmers, we need to implement a system that won’t mean looking at the code everytime something goes wrong.

Flexibility via a strong framework: Our work has turned out to have more applications than what we originally intended for it. As a result, we were considering forking our code into two separate branches. Though we still haven’t reached a proper decision regarding this, I’ve personally been trying to redesign our project into a “framework + dynamic modules” system. This is purely a software engineering effort, since it will mean a lot of refactoring and restructuring, but will not have much of an external effect rather than making our job easier.

Unit testing and other automation: Till now, an important part of our work has been getting user feedback, partly because our system is very visual intensive and also because we weren’t quite sure what we were looking for. Now that we have a better idea of where our project should be headed and what we should be looking for, we can begin to automate some parts of our work. However we won’t just be checking for bugs, but rather generating a large number of results and then further improve our system to get what we want.

I’d love to hear from anyone who’s been involved in computer science research projects and has opinions of how to use software engineering principles to make things easier/more effiicient.

Evolve to Email

Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

Jamie Zawinski’s Law of Software Envelopment

Every program in development at MIT expands until it can read mail.


Both of the above were supposed to be humorous criticisms of software bloat: adding so many unnecessary features to a piece of software that it’s real functionality gets buried deep under everything else. However, I was recently working with one of our ‘clients’ for our summer research projects and email came up. She’s marked out a number of places on Google Earth which would serve as models for the urban development aspect of the project. When she wanted to give us the placemark information, the easiest thing for her to do was to export them as a .KMZ file and attach it to an email. As we were trying to figure out the whole export-email-import process, I realized that email is a really big part of our lives.

Being a college student, sending files to other people, whether it’s classmates, writing assistants, or the professor, is something that I have to do on a regular basis. Even though there are public and private network drives which are meant to be used for moving files between a group or simply from a lab computer to a personal one, most students simply prefer emailing (to others or to themselves). A program having the ability to read mail might be unnecessary and a sign of bloat, but being able to send email is certainly a feature that would be very handy to most people. (Of course, if you’re a Emacs fan, then you can read/write email and browse the web from your text editor.)

Unfortunately there are two problems that make ubiquitous integrated email a difficult proposition. Firstly, even though many applications, such as Microsoft Office to include the ability to attach documents to email from from within the application itself, most people aren’t really aware of this feature. And so they fire up an email client to be able to create a new email and attach the document in question. Secondly, and perhaps more importantly, desktop email clients are no longer the norm. In fact, if you’re emailing a document to yourself, chances are you’re on a public computer and will be using webmail. Ideally whatever program you’re using should have a button that opens up your webmail with a new message and starts attaching your file automatically. I don’t know of any software that does this, and I don’t even know if this would be possible using current technologies. Perhaps as Google Gears and similar offline web technologies become more popular, you’ll someday be able to attach straight to webmail. Till then you’re stuck doing the whole cycle manually.

But why email, I here you ask. Yes, email shouldn’t be used for file transfer, that’s not what it was built for. In the old days you had dirt simple FTP. Nowadays you have all sorts Web2.0 backup, share, transfer services. But everyone still uses email to transfer files. Why? Once again two reasons. One, it’s easy. Seriously, what would you prefer, have your files drop into your inbox with no action on your part, or would rather open your browser (or another tab if you’re in webmail), go to a website, maybe type in a log in and password, click a link, get rid of some dialog warning you that you might be downloading the digital equivalent of leprosy, and then finally see your file? We programmers may deride our users for being lazy, but sometimes that laziness makes perfect sense. Moving on to number 2, (and again this is probably more important than 1), emailing is what people have been doing and most people really don’t want to learn a new way of doing something that gets them the same result.

So email is here to stay. In fact, let me clarify that, webmail is here to stay. Sure every big corporation has some fancy heavy duty email client which all it’s employees have to use, whether it be Outlook, Entourage or something else. And there are a bunch of old-timers clinging onto the last release of Eudora and Linus Torvalds running Alpine. But webmail is gradually becoming what every normal person uses. So if you’re out to write the next killer app then you’ll almost certainly be one step up if you can figure out a way to directly export to webmail attachment (at least the big ones like Gmail, Hotmail and Yahoo). Of course, if you’re app is some social web thingy with “sharing” as the main keyword, you might as well just latch onto Facebook or Twitter.

Realizing that email is one of the most powerful forces on the internet (never mind that most web traffic is BitTorrent), leads us to understand one of the reasons why Google has in a few years become such a powerful player in the digital world. Let us never forget that their first major product outside of search was Gmail. And Gmail was nice, a gigabyte of space, no glaring graphic ads, no deactivation if you don’t log in regularly. Throw in the whole beta-invites thing to make forbidden fruit more sweet and you have people lining up. And once they had people pulled in by what was arguably a superior product to anything else available at the time, they could leverage it. People don’t need a gigabyte to store email, they need it store attachments. So give them quick virus scanning of attachments, give them image-thumbnails so they don’t sit around forever waiting to download something they didn’t want to see. Add an MP3 player for people who still don’t know about P2P. Acquire a small startup building a web-based word processor and offer to open up people’s documents without them having to download. Let them be able to make changes, save and most importantly email back. Etc Etc, you get where I’m going with all this. Sure this isn’t the whole story of Google’s rise to power, but it’s important to realize the same thing that Google did: that if you structure your business model around something that everyone uses, like email, you’ll have an easier time drawing them to what other stuff you want them to use.

So the moral of today’s rant is: Email is big, it will be, and it’s not controlled by a monopoly. If you can leverage, then by all means do so. I’ll end with a succinct quote from another Jamie Zawinski article regarding Groupware:

And if it doesn’t work with webmail, you’ve lost before you’ve even begun, so don’t do something dumb like requiring a plugin.

Cocoa, Python and the quest for platform independence

For the past few days, I’ve been looking into Apple’s Cocoa infrastructure in some detail. The reason once again stems from my research projects. All our code is in platform-independent Python meaning that we can develop on any of the 3 popular operating systems and then run on any others. This has been a great advantage since the three members of our team each run different OSes. For most of the time our project was purely command-line and text input based, using graphics only for the final display. However we have now come to the point where we need to fork our system into two different systems: one for architects and one for artists. How each program will develop is something that we will work on over the next few weeks, but one thing is clear: for the artists at least, a purely textual interface will simply not do.

Till now our system had been accepting an instruction file as input. While this method of using a instruction was very powerful (we could essentially embed our own programming language), this not the direction we want to go with the artists. And besides, not every artist wants to sit down and right pages of instruction to draw a picture, we might as well ask them to write the program themselves. It would be best to create a GUI for them to use. Even before this had been clear, I had written a simple interface that was essentially a text editor containing a hook to our Python backend. It was written in Tk, was exceptionally ugly, but got the job done.

My Tk Interface running under Leopard

Now I think the Tk toolkit is capable of letting me create the entire GUI interface that I have in mind and it will be cross platform as well, but there’s one problem: it’s ugly, especially on the Mac. I think this is because the Python Tkinter version doesn’t use Aqua (like Tcl/Tk), but rather uses X11. Though I spend a lot of my time in Vim and at the command line, I’m still a fan of good looking user interfaces (see my last post). As a result I would feel rather guilty about making something that I know would look bad on the platform it would be used most. So like a good programmer off I went to find a solution.

I thought I would be able to use Cocoa to create a front end, especially since Xcode now has support for the PyObjC bridge. Unfortunately this has turned out to be much harder than it at first appeared. Cocoa is Apple’s very powerful object oriented API. It is almost everything you will ever need to create a Mac application. While it allows to create native applications by leveraging the powerful infrastructure Apple provides you, it also doesn’t allow you to perform the sort of rapid duct-tape styling programming that Python and UNIX encourages. In Python it’s very easy to write your core application in Python with any number of GUI front-ends that are easily interchangeable. Cocoa, on the other hand, is not another GUI toolkit that you can simply add to the mix. If you want to use, you’ll have to invest a certain amount of time in learning your way around.

Python can’t communicate with Cocoa directly. To¬† have your Python programs leverage Cocoa, you need to go through the PyObjC bridge. Though the bridge is certainly effective and I could use it, I’ve been having troubles finding decent documentation. Apple’s own tutorial is the sort of document I’m looking for, but it is dangerously out of date. Lacking a proper tutorial, I would have to figure out a lot of things on my own. Once again, a large time investment.

Considering that I have only two weeks left to make real progress on my project and that learning Cocoa or Objective-C was not my original goal, I’ve decided not to pursue that avenue at the moment. I would certainly love to learn Cocoa at some point in the future and PyObjC seems like it could become a very powerful tool. But I simply don’t have the time at the moment. I am going to start working on a proper GUI interface in Tkinter. Perhaps sometime later this year, I’ll get back to some Cocoa.

OS X Leopard, Firefox 3 and native UI

I installed Firefox 3 yesterday on both my old G4 Mac and my Windows laptop. Arch linux, which I normally use doesn’t have a package for it yet. Firefox 3 has a number of new features, which are being talked about on almost any random technology site, so I won’t bore you with them. I’ll just focus on one thing that I like: the changes to the user-interface.

Firefox 1 and 2 sported a UI that seemed to be uniform across platforms. However this uniformness across platforms also meant that Firefox didn’t quite look like the other applications for the platform. This isn’t really a big thing, especially considering the horrible UI inconsistencies introduced in Windows Vista. But at the same time, it’s little touches like this that can make or break an app. Sure there were skins, but you really want something like that to builtin. One of my favorite pet peeves was regarding Firefox 2 on OS X. The buttons for various web apps like Gmail or the Google search page’s search button would show up as ugly Windows 95-ish rectangular boxes rather the sleek Mac buttons. It was something that really irritated me, since I spent a lot of time in Gmail. I’ve been glad to see that the new Firefox fixes that and adopts a very native look and feel.

I also upgraded my G4 Mac to OS X Leopard today. I had been putting it off since it has only a 1.25GHz processor and less than a gigabyte of RAM. However, it turns out that my fears were mostly unfounded. It took about 20 minutes to install and it runs as fast as Tiger did. The only noticeable difference is that the boot and login times are somewhat longer. But considering that I rarely logout or shutdown, that isn’t really an issue at all. I’m surprised by how Apple’s development takes into account older machines and actively works to extend their lifetimes. This stands in sharp contrast to the effect that Windows has as the Great Moore’s Law Compensator. In fact I will go as far as to say, that some things actually run better on Leopard than on Tiger, the Leopard-Firefox 3 combo in fact seems to very capable.

On Tiger I always experienced Flash videos being somewhat choppy irrespective of what browser I used. That problem no longer exists on Leopard. Furthermore, the UI changes to Firefox fit right in with the overall darker look that Leopard sports. I’m really looking forward to using Spaces, which should come in really handy as I start doing Cocoa application development. I was also impressed by how little impact Spotlight scanning the disk had on performance. I would never have known it was going on if I hadn’t tried to use it. There are some features of Leopard which I won’t be using such as Time Machine ( I have a homegrown SVN backup system), and that will probably mean my system won’t be under a full load anytime. If it turns out that my system can’t handle (considering that I’ll be using Xcode fairly heavily), adding a gigabyte of RAM might help speed things up. I had been considering buying a new Macbook to last me until the end of college, but it seems like my current machine will be all the Mac I’ll need for a few more years (or at least until Snow Leopard hits the stage).

Back to the drawing board

I’ve just returned from meeting one of our clients for my summer projects: a professor in my college’s art department. What we have developed over the last three weeks is essentially a way to stitch together fixed images into probability-controlled tilings. However, it turns out that what our artist friends really need is more fine grained image control. They want to be able to create much more interesting patterns combining both images and geometric patterns. As I was explaining how our program worked, I realized that our instruction language model was rather complicated and not really suited for what the artists needed. So it seems, that our program is really not a very good match to what the artists need.

At this point, there are a number of things we could. We could go back to the drawing board, start from scratch and rewrite our system according to what the artists need. We could also try to build in top of our existing program and try to add and modify it to bring its functionality close to what we need. However, we need to keep in mind, that we had also planned to use this system to create a automated urban planner. We still haven’t met with the architect yet, but it seems like our system might be a better fit for what she needs. But that doesn’t answer the question: what do we do?

Now if this were a real software project with a delivery deadline, this state of affairs would probably be a very bad thing. Luckily this is still very much a research project so there’s quite a lot of leeway. Unfortunately, we’re still not quite sure of what exactly we have to do. It seems that our general architecture is still valid and will work, however the details will have to be remade. We’ll need a new way to generate the patterns from the instructions, simply stitching together images won’t do. It is possible that we will need multiple drawing systems working on top and parallel to each other. The instruction language also needs to be changed to make it more suitable to the artists’ need. We currently have a system that is geared towards placement of individual large scale elements, and nothing that deals with simple pattern generation. Ideally we would need a blend of both.

At this point, it’s easy to get disheartened. After all, we’ve spent almost 3 weeks creating something that we just learned is pretty close to useless. On the flip side, we are certainly not going to get bored. From a personal standpoint, this has still been a success, I’ve learned about parsing methods and language design. The experience that I’ve gained will make it easier for me to help develop a language that is powerful enough to express everything that the artists want to, while not requiring them to spend days learning it. It also means that the next version will also hopefully be developed much faster and more smoothly than the previous two versions. I am going to make it my personal project to see how fast and efficiently we can develop subsequent versions.