Lindenmayer Systems in Processing

Processing is a really powerful language and environment for programming images, animations and visual interactions. For the past few years I’ve had a passing interest in creating art using computation (thanks in no small part to a few great teachers). This semester I decided to buckle down and get to work doing some cool stuff with computational art. Just to get a feel of Processing and see what might be possible, I decided to revisit a project I had done a few years ago: Lindenmayer systems.

Lindenmayer systems or L-systems are pretty awesome. You start with a string of symbols and a bunch of rules describing how those symbols transform into other symbols. And then you just keep reapplying those rules over and over. L-systems can easily generate a wide variety of interesting forms and patterns including many plant morphologies.

Two years ago I made a  implementation of L-systems in Python using the Python Image Manipulation library. But one of the goals of that project was to create a tool that non-programmers can use. It was an interesting experiment, but after a few months I was no longer sure if it was a good idea to create a “non-programming” interface to something that is so computationally powerful. This time around I’ve eschewed the whole “pretty interface” idea in favor of dealing with real code.

I chose to use Processing so that I could work a few steps removed from nitty-gritty image manipulation. Processing is based on Java, so I can use Java with all it’s libraries if I need to (and to some extent, I did). I would have preferred it if I could have Python underneath, but I’m not complaining too much. There is also a version of Processing running on Scala and there is a port in the works to JavaScript and Canvas which I’m interested in. It might be an interesting future project to port it to Jython.

How it works

L-systems in general work by generating a string of symbols that work as driving instruction to a LOGO-style turtle (think of it as a pen you can move with simple instructions like “forward”, “left”, “backward” etc.). The lowermost layer is a simple turtle implementation that takes a bunch of movement commands and draws them to a screen (or canvas).

On top of that is a class that represents Lindenmayer systems in general. A class stores the symbols, rules and the mappings from symbols to turtle instructions. It also implements the basic rewriting algorithm and allows to provide an external turtle object (in case you want to process the image outside of the actual L-system). It also takes the generating string of symbols and passes it to the turtle to draw.

The top of the stack is functional Processing code where you have a setup() function that sets up the image canvas and initializes the L-System object. The draw() function executes repeatedly and controls how many generations the L-system goes through. Each L-system is two functions: a setup function that creates the L-system object with symbols and rules and another function that gets called on every draw(). This allows you to do extra processing before or after the L-system’s actual generation.

Go Get It

I’m calling the resulting system Lisip: LIndenmayer Systems In Processing. It’s released under a GNU GPL and is available right now on Github. There are just three source files (the Turtle, the L-System class and the Processing code driver which includes a number of L-systems). I think the code is fairly self-explanatory (thanks to a large extent to how much Processing cleans up the usual Java syntax). I do plan to add some proper documentation within the week.

I’ll continue to add L-systems to it and probably add to the Turtle’s functionality in the weeks to come. Feel free to fork it and add your systems, but I’d appreciate it if you dropped me a line if you do fork it.

Happy Hacking.

Sunday Selection 2010-02-28

Reading:

Unlink Your Feeds – A Manifesto is a passionate appeal for not diverting all your social information to everywhere else. It makes a good case for keeping individual social networks separate. I suggest you follow the links at the end of the post too.

Media:

Unlearn Your MBA by David Heinemeier Hansson is something that you really need to listen to if you want to start your own company any time soon. DHH is quickly becoming one my heroes, both in terms of business and software and if I start a software business, I’m definitely going to taking a few pages out of his books.

Software:

FreeMind is a free, cross platform mind-mapping tool. Mind mapping is a great way to brainstorm new ideas. Though I love plain text as much as the next guy, the visual impact of mind mapping can be helpful. I’ve only just started using at the recommendation of a professor and will probably have an article on it sometime.

Combining free and proprietary software

I was reading Martin Fowler’s unscientific agglomeration of opinion on version control tools yesterday when I came up with an idea about finding a compromise between open and closed source tools. Thinking about it, I realize that I’ve come close to getting this idea a number of times before, but never really quite reached the tipping point. Anyways, if you’re in a hurry, the basic idea is this: make developer tools (compilers, IDEs, version control tools etc.) completely free and open source. Also make underlying architecture free and open source as well (like the Linux kernel or Ruby on Rails). But you also want to make some money from software, right? To do that, anything that is not developer focused stays closed source and is charged for.

What led me to flesh out this idea was Martin Fowler’s observation that the three most popular version control systems: Subversion, Git and Mercurial are all completely open source. With just a little bit of thought, I realized that there’s a company that does exactly what I propose: Apple. Apple’s dev tools like XCode and the Objective-C compilers and libraries are open source are free for everyone to download and use. The core of OS X is the open source Darwin kernel. But layered on top of this is a closed and really high quality system of interfaces and applications that make for a very appealing user experience.

First let me say that what I’m about to say isn’t a commentary on the pay-for-support model that Red Hat and Canonical use. I think that’s a perfectly legitimate model that has it’s advantages for consumers, companies and open source hackers. But I want to propose an alternative that focuses on making money from selling the software itself.

Opening Doors

Why does this make sense? Leaving aside any moral imperatives, one of the biggest reasons for open source is that you can tinker and fix things that are broken. If you see something that is not right, you can easily pop open the hood and dig around in the internals. Now, the only way that you’ll know if something is broken is if you either use it yourself or have someone tell you that it’s broken. However, in my experience it’s much easier to fix something when you see it broken for yourself. It also gives you a better idea of what the fix should be like. It’s also more likely that you’ll change a program for the better if you’re using it day in and day out. Eating your own dog food has its advantages. If you’re a programmer, you’re most likely to find bugs and missing features in tools that you use every day. So it makes sense for you as a developer to use open source tools so that there’s an easy way for you to fix or change things that in turn will help your own developer experience.

If you’re a corporation (or organization) like Apple or Microsoft then it makes sense to open source your developer tools as well. For a corporation, the benefit to open sourcing a product is that you get feedback both in terms of ideas, comments and bug reports and also in terms of real working code. What you end up with is a positive feedback loop: you make tools and release them to developers for free who then help you make even better tools. I love the lesson that 37signals has taught the world in terms of tools and basic infrastructure: Ruby on Rails is part of their basic infrastructure, but they’ve decided to open source. As DHH claims, it’s hard to make money off of basic infrastructure (unless maybe if you’re Microsoft with a huge already installed base). Better to release it into the world and benefit from the improvements that other people make to it.

Making Money

So that’s all nice and dandy for developers, but what about the poor corporation that wants to make a decent buck? The basic idea is that the number of people who aren’t developers far outweighs the number of people who are. Also the number of people who just want something that works far outweighs the number of people who want something that they can tinker with ad infinitum. Even most of us hacker types occasionally break down and just want something that we can shut off our brains and use (for me it’s Excel in my engineering labs and Photoshop for image manipulation). People will pay for convenience, quality and polish. That should be the motivating factor behind businesses who want to sell software.

To make money from selling software you first need to hire good developers and give them good tools that they customize to their needs. Fortunately, you already have great free dev tools. Now pay these people well and give them a concrete goal so that they put real polish into the products, especially in the areas that open source hackers might tend to neglect. Since you’re going to be charging money, you can afford to put some money into hiring professional designers and UI experts and make sure that your app really looks good as well as works good (something that is still sorely lacking in a lot of open source software). Of course you need other people on team too: at least a fair number of testers who are the same people who will be using your for-pay product day in, day out. You end up with a product that has been crafted by motivated developers using great tools with feedback from your target user base. Admittedly you can still end up with a crappy product that no one wants to pay for, but that doesn’t mean the idea itself doesn’t work.

Though I have no real experience making products like that, the cases in front of us are pretty clear to see. Apple and Adobe both make lots of money selling high quality consumer software and I personally consider Microsoft Office to still be a better user experience that OpenOffice.org (especially with the ribbon interface).

One of the questions I haven’t fully resolved is whether your for-pay app should be open sourced. I am tempted to say no because I can’t think of a business model where you can open source your product (code and design elements) and still expect people to pay a decent price for it. If someone with more experience than me can come up with a working model, do let me know. However, one compromise that might be fair is to open up your app a few years later once your company has a newer product. Under this model, Apple would open source all their code prior to OS X. I’m less familiar with Adobe, but I think prior to the Creative Studio editions would be a good time point. Microsoft might consider releasing the pre-NT Windows code and maybe Office prior to 2000 or 2003. That way, your business still makes money today while making a community contribution in the future. It also helps users who can get their hands on a copy of a program if they need to open a file from 10 years ago that the current version no longer supports. It frees up the company to go ahead while worrying a little less about backward compatibility. It also keeps companies on their toes because they know that any market advantage they have will be effectively erased in a few years.

The Plan Isn’t Perfect

No plan survives contact with the enemy

— General Field Marshall Helmuth von Moltke

And this plan will definitely have to be modified if and when a company adopts it. One immediate problem I can see is that inevitably there will be calls to open source all the company’s product. I don’t have a suitable response to that apart from saying that would probably cause the business plan to collapse. Also if your product’s core algorithms don’t vary much from one generation to the next, open sourcing older versions might be equivalent to shooting yourself in the foot (though if you’re expecting a handful of algorithms to bankroll your company till the heat death of the Universe, you’re probably doing it wrong).

Again, I don’t have any real world business experience (though I would like to, someday). I also don’t know any company that has a policy like this at its core. If you’re interested in starting a company like this (or know of one already), do drop me a line. I’d really like to play a part in making a world where people can get paid for writing their code and then releasing it for other people to use and improve. A business that makes money but still open sources its code (even if its a few years late) would be a great step in that direction.

Ars Longa, Vita Brevis

I occasionally go and do something crazy, something completely unbecoming an engineer. Last semester I took an Creative Writing course with a most wonderful teacher. This semester I’m doing an independent study in art with an equally wonderful teacher. Last night I was up till about midnight getting the grip of Processing — a programming language and environment for creating stunning visuals. It’s a pretty sweet environment and I’ve only scratched the surface of what’s possible, but boy, was it hard.

All I did last night was recreate an project I’d done a few semesters ago with Lindenmeyer Systems. In some respects, our previous work was a bit misdirected and we should have been building on top of Processing all along. Last night I was reminded firsthand of the importance of using good tools suited for the task at hand. I didn’t write much in terms of code, but I did manage to build up a fair bit of functionality (a much better measure of progress I think). But it’s still not done and I suspect at least an hour or two more of steady work before I get to something that I can show off. This morning as I was trawling in the interwebs I came across this essay written by my creative writing professor and her quotation of “Ars longa, vita brevis” rang out as so true. The art is long and life is so short.

I’m not really an artist, though I like pretty things. I’m a hacker at heart. More important than the actual beauty of the object is the joy I feel in actually creating it. As  a coder, I guess I’m half decent by now. I’d call myself a really really advanced beginner (close to intermediate). But in terms of art, I’m pretty much a greenhorn. What’s more, the art that I’m doing is in code. I thought that would be fun and easy. It’s not easy and it remains to be seen if it’s fun. Though I love writing code and can concentrate better on writing code than on anything else, when it comes to art, I’m a bit lost. It’s been a while since I’ve done any drawing or painting, I prefer using my words to create images in people’s heads. Also without the use of hands and real physical paints and paper, it’s a bit harder to play around. Admittedly, it’s easier to tinker, redo and recreate with computerized tools but there is more of an upfront investment and the learning curve is significantly steeper (at least in the beginning).

In some ways, you could say that I’m painting entirely with mind. It’s liberating: I don’t have to worry about drawing a perfect circle or making sure that the sides of my squares are all the same length, the machine does it for me and I can work at a higher level. At the same time, I can’t just splash some paint on campus and see what it looks like. I have to look up an API reference instead of stroking away extra paint with a brush and I sometimes I have to get my hands dirty debugging (including dealing with Java’s broken type systems). If real artists did this all the time, we’d never progress beyond stick figures. That’s not to denigrate Processing, the people behind it or computational art. The gains in productivity and expression we get from tools like them far away my pain. Don’t mind me. I’m just bitching.

The art is long and life is short. It takes some 10,000 hours to become an expert in anything and it takes continued practice to keep up that level of expertise. And life is short. Sometimes I wish that I could sever all human contact and just sit and write code (or stories or whatever) and the next moment I realize that it’s stupid because it’s meaningless to live completely in isolation. There’s no point in my writing code or making art if no one uses and appreciates what I create. It’s rather ironic that it takes antisocial devotion to a task to create something that others can appreciate. If only our brains could really multitask, things would be so much easier.

Well, that’s enough bitching and moaning for one day. I’m going to get back to my code/art.

Another discovery: DokuWiki

Looks like this is going to be a semester of discoveries for me. While working on my data visualization project,  I discovered IRC. While working on my parallel programming project, I discovered a great lightweight wiki software called DokuWiki. We’re a small team (2-4 people depending on how you count) and we needed a place where we could gather our findings and record our work. We tried to use Google Wave in the start, but it didn’t quite work out. Wave is nice when you need to collaborate in real time and what you’re doing is mostly brainstorming. But if you’re working at a slower pace and need more structure, it just doesn’t feel right. More on this later.

What we wanted was a solution that was quick and easy to set up, was resource-light and could be hosted locally. We thought about larger wiki systems like MediaWiki but we settled on DokuWiki. First off, it’s flat files: no database or SQL required. It makes a point of being lightweight, so there’s no WYSIWYG. Instead there’s a simple wiki syntax instead that suits us hacker types just fine. Looking at the configuration page you can easily tell that it’s focused on functionality rather than looking pretty. But that doesn’t mean that it short on features or good design. It tracks revisions (with short commit messages as well) and has a really nice visual diff tool. Using Git for the better part of a year has made me really love good diff tools.

The wiki syntax is also well thought out. There’s little redundancy but strong features. And CamelCase is optional. Links are enclosed in double square brackets in the form of [[url|link text]] and you can make new pages just by putting in an empty link. It supports multiple level of headings and generates an automatic table of contents for any page with more than 3 headings. Though the design is simple and minimal, it does look pretty good. It’s clean and standardized so you won’t get lost and the edit controls are neatly tucked into bars at the top and bottom. It looks more polished than a default MediaWiki setup, for example. It’s easy to tell that DokuWiki is meant to be a clear documentation tool as opposed to another “build my website quickly” software and in our case, that’s a good thing.

And did I mention the sourcecode support? It uses the GeSHI syntax highlighter, meaning that it supports a wide variety of languages (including some fairly obscure ones like Lotus Formulas and LocoBasic). You can also turn snippets of code into downloadable files by just specifying a filename. You can tell that this was a tool made for hackers by hackers. And of course I just love it. Uploading images is easy, but if you want to add files with other extensions, you need to edit a config file which shouldn’t be a problem for most people who’ll be using this sort of software.

There are a number of other goodies in the bag that I haven’t had time or cause to investigate (including plugins). The fact that I’m not administrating the installation means that I won’t be playing around with it as much as I could, but I think I can turn it into a user point-of-view experience. I wish there was some easy theming support, but I can live without it. This is the first time that I’ve used a self-hosted wiki and though I’ve had experience with other wikis in the past (I like PBWorks) I think I’ll definitely turn to DokuWiki if I need a simple but strong wiki for a code-focused project in the future. I might consider hosting it on my personal server in the near future too and get some more chance to play with it. Right now I’m perfectly willing to keep it simple and focus on my real work (more on that later too).