Flamenco dancers, programmers and excellence

Tuesday night I went to see a Flamenco performance by the Flamenco Vivo Carlota Santana company. It was a wonderful performance, especially the second half (at least that’s what I think, knowing nothing of flamenco). It must have them days of practice to get the choreography and music down right and years to get to the point where they could actually move the way did. Not to mention, the continuing time and effort investment in keeping in physical shape and on top of their game. Being the nearly obsessed student of programming that I am, I of course thought about programming as well.

It’s occurred to me more than once that I really don’t know many great programmers. I go to a small school and most of my fellow students are about the same level as I am (or lower). Actually I don’t really know what level they are on because I don’t get to work with them as much as I would have to in order to find out. There is only one person whom I can say for certain is a better programmer than I am and I’m currently doing a project with him. My professors are good computer scientists, but I have no idea how good at programming they are.

But from what I’ve read (and what I can attest to from personal experience, to some extent) achieving excellence in any field (especially programming) requires a lot of dedication and hard work — about 10,000 hours of it. I’ve thought about whether or not it really is worth it to invest so much time in one activity, even if you really love it. I love spinning code, but I don’t want to be sitting in front of a bright screen all the time. Watching the dancers the other night, I got to see what dedicated practice can bring. They put on a great performance and everyone enjoyed it (including myself) and creating something beautiful and wondrous is a worthy cause. But I’m also interested in getting to know about what they dancers themselves think about their work. I’d love to know their feelings and emotions as they dance and have people cheer and whether they think it’s fair compensation for their commitment. And I don’t mean compensation in only the monetary sense, though that is important too.

Admittedly there is nothing in the programming world that is quite the same as a great dance performance. Our victories are more personal and what people see (and sometimes applaud us for) is often a small sliver of everything that we do. But that’s fine by me. When I solve a hard problem after a long time (my personal record is 3 hours hunting a pointer bug) or make something that I think is really cool (a recursive-descent parser for a little language), I think I feel some of the elation, satisfaction and relief that I think the dancers would have felt too. Yes, it does feel really good. As each year goes by I get better at doing what I love doing. But I rarely ever think about all the practice and experience that has gone into making me capable of whatever it is I am doing. Not too long ago, I would never have imagined myself capable of writing a UNIX shell or designing a programming language, but know I am doing both those things and it feels almost natural.

Ok, that last sentence was a lie. It doesn’t feel natural, but it feels like it’s just outside the range of being natural. When I’m doing things like that, I’m on the edge of incompetence. It was hard and it was painful, but now that I know I can do it, I feel much better. In some ways, I wish such chances came more often (I think the education system for computer-related studies needs to be revamped significantly, but that’s another matter) and I know that each such experience leaves me just a little bit better. Do flamenco dancers feel the same way? Maybe. It would be interesting to find out.

Excellence is a rather strange thing in that it’s hard to achieve and the return on investment on its pursuit can be very little until you get to a certain tipping point. And then there all the people who seem to be trying really hard without getting anywhere. I’m not surprised that many people choose not to put in the investments that it takes to be excellent. As a girl I liked once told me, there are a lot of people leading average lives who are very happy about it. I guess that’s true. I’m not clear about where I stand on excellence myself. I do want to be really good at what I want to do and I fully understand that it won’t be easy. But I also don’t want to give up everything on the quest for excellence. “No sacrifice, no victory” sounds very noble and all, but there’s a tinge of recklessness that I really don’t like.

At this point, the word “balance” might seem appropriate. But that’s bullshit too. I don’t think people who are great at something got there by seeking balance. The better option is breaking the rules, or at least fracturing them. The prime example is 37signals. They’re a small company, with little VC funding who don’t give away their products for free and still make millions of dollars. And they didn’t do it by working round the clock either. They broke “rules” like working 80 hours a week and making free products and other such things. But they also knew what rules to break. They didn’t break rules about being thrifty or having a solid business plan. They might not be the paragon of excellence and they’re certainly arrogant, but they’re doing well so far.

As someone seeking excellence myself, I’m trying to bend my own set of rules. I bend rules by taking courses out of sequence, doing independent studies where I can write lots of code and meet interesting people and actively trying to talk to people I admire. I need to put in 10,000 hours, so I build my life to provide opportunities to do just that. I really wish that there were a lot more people doing the same.

Happy hacking.

Unlinking your feeds and the impermanence of Twitter

About a week I stumbled on this interesting manifesto by Tim Maly on why we should unlink our feeds. I recommend you read the full article, but the heart of the matter is that you’re making a terrible mess of things by sending your feeds from one social network to all the others. You do a disservice to people who are following you on one network (by making them see everything else on all your other networks) and you spoil the mood and general atmosphere that you’re dumping into.

While I agree with the theory in general, I can’t bring myself to go the whole nine yards and completely disconnect everything. A related article by Alexis Madrigal argues that the unlinking doesn’t work with Twitter. Twitter has no memory and Twitter is most useful when you have other meatier services (like a blog, website or even just Facebook) that give people a better idea of who you are. The author argues that Twitter’s relative impermanence means that it’s worth piping your Twitter stream into something more permanent. For my part however, I see things the other: I want Twitter to be my catch-all because it is impermanent.

The New Old Deal

Here’s the deal: I accept that cross-linking feeds leads to some amount of pollution and that’s not something I should be subjecting my friends too. My friends on Last.fm don’t care about how many billions of floating point calculations I’m running at the moment and the readers of this blog probably don’t care about my thoughts on modern instrumental music. But as Madrigal puts it, Twitter is different. I feel uncomfortable calling Twitter a social network. To me it’s  more like a broadcast service. You send out little snippets and anyone connected can read it. Of course you could have a private feed and carefully control your followers, but I feel like that’s just a holdover from email (where spam is a clear and present danger). Also, Twitter is not email. It takes far less overhead to skip something that you don’t care about and personally at least, I don’t feel the same pressing urgency for my Twitter inbox as I do for my email.

When someone follows me on my Twitter account, I want people to understand that they’re getting the whole deal. They are getting my 140-character updates (which make up the bulk of my stream) but they are also getting my regular tech-related articles as well my discoveries online. Tim Maly notes that time is precious and accounts are cheap and it seems that he was talking primarily about other peoples time. While that is true, my time is also precious and so is my mental overhead. As an example, I take myself.

Account Overload?

I could, right now, split my Twitter stream into at least three separate accounts: one for updates only, one for my blog and tumblelog and one for my Last.fm feed. Thanks to Twitterfeed, I can set things up automatically to post to whichever account I want. That’s all fine and dandy and I’m really tempted to do it. But what happens when I have a thought about music that isn’t engendered by Last.fm? Does it go into statuses, music or both? Do I really want to tell my close friends that they now have to follow me on three different accounts to get everything (not to mention the overhead of @-reply conversations that could easily start crossing accounts)? Should I have a fourth account that pulls everything in for those who want it? I’ll be the first to admit that my example is somewhat contrived and probably a worst case scenario, but it deserves some thought. I would rather have Twitter collect everything with a disclaimer that people might be getting more than they bargained for.

As for pulling out of Twitter to somewhere else, I’ll agree that’s just a bad idea. Twitter has grown it’s own syntax with @replies and hashtags and the like which really make no sense elsewhere. The only place that you should even consider piping Twitter to is your Facebook status. As a friend of mine said when he dabbled in Twitter briefly: “It was like setting my Facebook status, except that’s all I could do”. Point taken. Even then, it’s a good idea to sanitize your stream to remove all the Twitter-speak. I use the Tweeter application which gives you some good filtering abilities.

In conclusion

  • Cross-linking your social networks is a bad idea.
  • Except for Twitter. It makes a certain amount of sense to pipe your feeds to Twitter.
  • Exporting Twitter to elsewhere is also a bad idea, because of Twitter-speak, except maybe for your Facebook status, if properly sanitized.

As an addendum, if you do decide to use Twitter as your catch-all, I suggest you standardize on a solution. Many services give you the option of piping to Twitter from within the service yourself. That may be fine if you have one or two services and want your posts to appear immediately, but the overhead grows as the number of services grow (and each service has the options in a different place). I recommend using RSS as your glue and piping things through Twitterfeed. There will be a short delay, but I don’t think that will much of an issue for most people.

On Essays

I’ve been thinking about essays on and off for the past few days. It all started when I was in the process of updating my static HTML website that I call Basu:shr. I have a section called essays which is currently populated mostly with papers that I wrote for various courses at college. Looking over some of my older work I realized that I didn’t really write longer pieces anymore. This blog is my primary writing activity at the moment and most of my posts are in the 700 to 1000 word range. I’m perfectly happy writing short articles because I’ve always admired brevity and conciseness (which is why I like Twitter as well). But at the same time, I’m slightly worried that I might be losing the ability of writing longer, more detailed pieces.

Ars Longa, Vita Brevis

As I’ve pondered before, life is short and it takes a fair amount of dedicated effort and time to come up with something beautiful and useful. With the rise of the Internet and instantaneous communications, we’re becoming a culture that is very much used to continuous streams of small information packets. The essay is becoming a holdover from the old days when having long periods of times to do nothing but sit and read was common. However, there are a number of really good essayists alive today, and a lot of them are on the Internet. There’s Paul Graham, whose essays are practically the stuff of legend for programmers. There is also Steve Yegge who seems to have retired, but left behind a fairly large collection of essay-length material (including an article on why you should write a blog). Outside the Internet there is Warren Buffet who has written long detailed letters to shareholders for the last 32 years each of which is an education in and of itself (and I can’t help but wonder how many shareholders actually read through them all).

I don’t think I’m making a mistake when I say that the essay is still alive and well today, albeit in somewhat modified forms. But the fact remains that putting out something of such length and depth takes up a lot of time and energy (not to mention the countless hours that go into accumulating the knowledge and organizing the thoughts that must flow into such a work). In many ways, writing an essay is similar to a software project. There is planning and preparation that must happen upfront, but nothing is really for certain until you sit down and start writing. Writing a good essay that other people will want to read and tell their friends about is no easier than writing good software that others will want to use.

Blog meet Essay

The blog and the essay are fundamentally different things. A blog is a magazine compared to an essay’s book. The blog as a format is great for some things: without easy blogging I probably wouldn’t be writing at all. But the rise of blogs (and accompanying software) has left the long form essay in the dark. You could simply write long articles and put them on your blog like Steve Yegge. But reverse chronological ordering really isn’t the best format for a collection of essays. For small numbers, a simple list of titles, maybe with a blurb is probably the best. Once you get to a larger number (Paul Graham for example), a simple list doesn’t cut it any more.

There is also the actual writing experience. Whenever you write a longer piece over the course of many days, you start to go back and visit the old parts. Part of it is for editing, but you also want to read what you’ve read before so that you know you’re keeping your essay coherent. Blog software doesn’t easily let you do this. I know WordPress stores revisions, but there doesn’t seem to be an easy, upfront way to see diffs of different versions against each other. I suppose a wiki could be better as an essay platform. Dokuwiki has excellent visual diff function and Writeboard also lets you compare versions.

Perhaps we do need some sort of specialized software for writing essays. Something that puts drafting editing at the center as opposed to at the edges. Personally I’ve been using Emacs with Git to get some of the same result, but I would really like to see a webapp that can do something like that. After all, there isn’t much use in writing an essay if no one is going to read it (and how better to get people to read it than to put it out on the Internet).

I, Essayist

Even though there may be no quick-and-easy publishing solution like WordPress for essays, writing an essay is far less dependent on tech tools than most other things today. Like I said before, Emacs and Git do a fairly good job together. I would like to be able to put all my drafts online with some sort of commenting system so that people can see the evolution of my essays, but I’ll settle for just being able to show a final product.

Separate from showing the essay is the mental exercise of actually sitting down and writing the essay (and then revising and editing). That’s something that I’ll have to get back into the habit of doing and will probably take time. Subject matter is also an issue, but a good starting point would be to simply expand on the themes that I cover in this blog, while making sure that people who read my blog can read my essays without getting bored (and vice versa). Expect my first essay to be on essays, sometime in the next few weeks.