Process and Product

Yesterday I came across a post entitled “default behavior” by Ben Augarten about how important it is to put your products out in the world. It starts with him talking about how the code he’s writing is for an internal tool and hence will never see the light of day. Few people will use it and he’ll lose ownership of it once his internship ends. He implores programmers to release their code as products into the world and seems more than a little disdainful of programmers who don’t release their own products. He seems to be under the impression that if you’re not working on your very product that you’re dead inside.

He’s wrong.

It’s great to love to your product. It’s great to build something, to show it to the world, to have people use it, get feedback and make it better. If that’s what motivates you and makes you happy then by all means go ahead and release products. Life is too short to be unhappy (though people are working on that). What it doesn’t mean is that not releasing products makes you unhappy.

There’s a distinction to be made between process and product. Product is the fruit of your labor, process is the labor itself. In the rush to create startups, launch products, have dozens of repos on your Github account, we tend to lose sight of the process itself.

While I certainly understand (and have felt) the allure of releasing a product, there’s a lot of joy to be gained in the process itself. While many of us got into computers and programming because of the things we could create, many of us (including myself) got into it because of the joy of coding itself. I love learning new technologies, working through problems, crafting solutions. I love the mental strain of thinking my way to answer, I love the physical feel of my fingers on a keyboard and the seeing the characters of the screen. I love the process of seeing something being created as much I like the feeling of having created something. The same goes for writing or drawing or cooking – how you make something is just as important as what you make.

I suspect that this joy for the process itself is why so many incredibly talented and hardworking people are happy working for closed source companies (financial and material renumeration aside). If you are fulfilled by the labor itself then you care less about the fruits of your labor. This isn’t particularly new thinking – it predates open source, computers, software, even modern science and technology. As far as I know, the earliest well-known expression of such a notion is from the Bhagavad Gita where the god Krishna tells Prince Arjun:

“To action alone hast thou a right and never at all to its fruits; let not the fruits of action be thy motive; neither let there be in thee any attachment to inaction”

I don’t entirely agree with this: I do believe in property rights and I think that as a creator you deserve some say in how your creation is used. However I do agree with the underlying idea that you can find motivation in the actions themselves and not just their results. Getting too invested in the result can detract from joy you recieve from the process (and vice-versa). I’m not going to claim that process is better or more meaningful than product. Humans and their motivations and emotions are far more complex than such simple comparisons. But I am going to claim that finding fulfillment in the process itself is a perfectly valid way of living your life.

It’s great that Ben and so many other programmers and entrepreneurs want to create products and see them in action. We need that. They don’t have to become process-oriented. But they do have to understand that not all people will value products as much as they do and they have to respect that. Sometimes the journey is more important than the destination.

Advertisements

Python as a glue language

I’ve spent the better part of the past few weeks redoing much of the architecture for my main research project. As part of a programming languages research group I also have frequent discussions on the relative merits of various languages. Personally I’ve always liked Python and used it extensively for both professional and personal projects. I’ve used Python for both standalone programs and for tying other programs together. My roommate likes Bash for his scripting needs but I think Python is a better glue language.

My scripting days started with Perl about 6 years ago but I quickly gave up Perl in favor of Python. I don’t entirely remember why, but I do remember getting the distinct impression that Python was much cleaner (and all-round nicer) than Perl. Python has a lot of things going for it as a scripting language – a large “batteries included” standard library, lots of handy string functions for data mangling and great interfaces to the underlying OS.

Python also has decent features for being a general purpose language. It has a good implementation of object-orientation and classes (though the bolts are showing), first class functions, an acceptable module system and a clean, if somewhat straitjacketed syntax. There’s a thriving ecosystem with a centralized repository and a wide variety of libraries. I wish there were optional static types and declarative data types, but I guess you can’t have everything in life.

Where Python shines is at the intersection of quick scripting and full-fledged applications. I’ve found that it’s delightfully easy to go from a bunch of scripts lashed together to a more cohesive application or framework.

As an example I moved my research infrastructure from a bunch of Python and shell scripts to a framework for interacting with virtual machines and virtual networks. We’re using a network virtualizer called Mininet which is built in Python. Mininet is a well engineered piece of research infrastructure with a clean and Pythonic interface as well as understandable internals. Previously I would start by writing a Python script to instantiate a Mininet virtual network. Then I would run a bunch of scripts by hand to start up virtual machines connected to said network. These scripts would use the standard Linux tools to configure virtual network devices and start Qemu virtual machines. There were three different scripts each of which took a bunch of different parameters. Getting a full setup going involved a good amount of jumping around terminals and typing commands in the right order. Not very tedious, but enough to get annoying after a while. And besides I wouldn’t be much of a programmer if I wasn’t automating as much as possible.

So I went about unifying all this scripting into a single Python framework. I subclassed some of the Mininet classes so that I could get rid of boilerplate involved in setting up a network. I wrapped the shell scripts in a thin layer of Python so I could run them programmatically. I could have replaced the shell scripts with Python equivalents directly but there was no pressing need to do that. Finally I used Python’s dictionaries to configure the VMs declaratively. While I would have liked algebraic data types and a strong type system, I hand-rolled a verifier without much difficulty. OCaml and Haskell have definitely spoiled me.

How is this beneficial? Besides just the pure automation we now have a programmable, object-oriented interface to our deployment setup. The whole shebang – networks, VMs and test programs – can be set up and run from a single Python script. Since I extended Mininet and followed its conventions anyone familiar with Mininet can get started using our setup quickly. Instead of having configurations floating around in different Python files and shell scripts it’s in one place making it easy to change and remain consistent. By layering over and isolating the command-line interface to Qemu we can potentially move to other virtualizers (like VirtualBox) without massive changes to setup scripts. There’s less boilerplate, fewer little details and fewer opaque incantations that need to be uttered in just the right order. All in all, it’s a much better engineered system.

Though these were all pretty significant changes it took me less than a week to get everything done. This includes walking a teammate through both the old and new versions and troubleshooting. Using Python made the transition easier because a lot of the script code and boilerplate could be tucked into the new classes and methods with a few modifications. Most of the time was spent in figuring out what the interfaces should look like and how they should be integrated.

In conclusion, Python is a great glue language. It’s easy to get up and running with quick scripts that tie together existing programs and do some data mangling. But when your needs grow beyond scripts you can build a well-structured program or library without rewriting from scratch. In particular you can reuse large parts of the script code and spend time on the design and organization of your new applcation. On a related note, this is also one of the reasons why Python is a great beginner’s language. It’s easy to start off with small scripts that do routine tasks or work with multimedia and then move on to writing full-fledged programs and learning proper computer science concepts.

As a disclaimer, I haven’t worked with Ruby or Perl enough to make a similar claim. If Rubyists or Perl hackers would like to share similar experiences I’d love to hear.

Still Alive

Rumors of my demise are greatly exaggerated. To be honest, I always feel a tinge of guilt and regret whenever I write a post along the lines of “I’m alive” or “I’m back”. Generally it means that I’ve spent the preceding few weeks in some form of pseudo-business, done some work, probably watched far too much TV and generally spent more time consuming than creating. Oh well, what’s done is done.

As I said, I am very much still alive (not quite kicking, but I’m working on that). It’s summer which means it’s warm outside and so I’m increasingly inclined to spend time inside. I don’t like the heat very much. While I’ve been gone I’ve been thinking about a number of things of varying levels of importance to me (and to you, my dear readers). So here’s a quick brain dump, in no particular order.

It’s been a while since I learned a new programming language. I’ve been using C and Python extensively and while I like good systems and scripting languages as much as the next hacker, it’s a bit like eating only whole wheat bread with Nutella. Not that there’s anything wrong with whole wheat bread or nutella, but I am craving a bit more variety. I’ve been looking at more exotic sources of nutrition, mainly Haskell and Clojure (nothing like a good Lisp to spice things up). Real World Haskell has been sitting idle on my desk for far too long – I stalled at chapter 4 and it’s about time I picked it up again. I’m considering grabbing a copy of Joy of Clojure too.

While writing code is certainly fun, I do missing writing words. And no, I don’t mean words meant for publication in a scientific journal (see whole wheat bread and Nutella above). In particular, I’ve been wanting to tell some good stories. I’ve always loved books and television and movies but it took some binge-watching of Doctor Who to help me realize that what I really like is a good story. And I like telling them as much as I like hearing (or seeing, or reading) them. It’s been entirely far too long since I wrote any sizeable amount of fiction and I can feel my storytelling muscles atrophying. If I put it off for much longer all I’m going to be capable of is Michael Bay-esque explosive blockbusters (though that might nicely supplement my starving grad student’s salary). I’ve been thinking about doing NaNoWriMo this year. NaNoWriMo sounds like a writing marathon and like all sporting events, training helps. I’ve always considered my blog to be training for non-fiction writing (and it’s helped) so maybe a similar fiction-writing training routine might help. Speaking of blogging, I miss it too. Writing really is a great way to sort out my ideas and think aloud (so to speak). So more of that too.

In other news, this blog runs off WordPress.com and I pay for mapping the Bytebaker.com domain to their servers. Apparently this will expire in about two weeks. I’ve been barking on and off about the need for next-generation content management systems that allow for more than a blog/static-site dichotomy. Given that it’s summer and I have a solid deadline this might be a good time to bite the bullet and roll my own. I did a half-hearted attempt a few years ago and round two has been long overdue. I’m still working out just what I want from this new system, but I have some cool ideas (mostly culled from other places). More on that later.

On a mostly unrelated note I tried out a new coworking space a few blocks from my place. It’s called the Popshop and seems to be run by a bunch of Cornell seniors. It’s pretty decent, they have lots of markers, whiteboards, air conditioning, a couch and a 3D printer. But the chairs suck so I may not be back too often. I’m also more in favor of quiet and isolated working spaces. I do spend most days working in an open plan setup, but we each have a lot of space to ourselves and there isn’t a whole lot of social interaction. I suppose coworking spaces are great if you’re looking for people to bounce ideas off but I prefer isolation when it comes to actually pumping out the code (or the words).

To recap: I’m alive. I’m looking forward to lots more hacking and writing in the near future. Coworking might be cool, but I’ll stick to my office for now. See you all tomorrow.

Generation Flux

A few months ago Fast Company run a multipart piece on “Generation Flux”. The piece had two intertwined themes. The first is the idea that we’re living in age of constant (and perhaps accelerating) change and that to stay competitive businesses and institutes have to ride this wave of change and go with the flow. The second idea is the notion that the most successful people are those who are intimately familiar with this state of flux and can craft their lives to take advantage of it. As part of the piece they profiled several members of Generation Flux – technologists, businesspeople and researchers like danah boyd and DJ Patil.

Though the piece focused on the tech industry and business, I think the basic ideas apply to all fields including (especially?) academia. In fact I think that the best researchers and scientists have always been those who have been spread out over a number of areas. While focus and diligence are necessary for productive research, I’m starting to think that it’s important (if not fundamental) to have a wider halo of interests and knowledge surrounding your core area.

As a new graduate student this is a question of great personal interest: I have a limited amount of time and energy in grad school (and later) and it’s in my best interest to make the most of it. As with many important things there’s a dilemma: if I spend too much time and effort on one thing I’ll miss out on everything else and that can be very limiting. I already know this first hand: I know a good amount about programming languages, but I’ve been scrambling to teach myself about networks and know next to nothing about AI. But on the other hand if I don’t dig deep enough into one relatively narrow area I’ll never have the knowledge or the insight to know what the important problems and come up with appropriate solutions.

So what are the lessons of Generation Flux and how do they apply? Accepting and adapting to change is definitely a big part of it. As danah boyd tells us: “We all have to learn new skills. Being able to live on one set of skills over a career is not realistic. Change is going to happen, not all of it good, in serious ways.” But simply being able to ride the wave is not enough. And it’s certainly not advisable to jump ship to the next shiny thing at the first sign of trouble. DJ Patil has more personal advice to offer: “At the end of the day, you have two things: your energy and your intellectual curiosity. If you’re willing to apply them, try to add value to the world, the possibilities are so endless.”

Patil, boyd and the other Gen Fluxers seem to be able to strike a balance between change and constancy. In times of perpetual change the key to success seems to lie in two complementary values: first is the ability to live on the edge of chaos and move fluidly from one spot to another. But second (and just as important) seems to be the ability to be tenacious, diligent and sometimes downright stubborn. Patil for example taught himself mathematics and worked midnight to morning to get computer access. While he’s worked on amazing projects he’s also turned down lucrative offers because they didn’t fit his vision of what he wanted to do.

Viewed through the lense of graduate school the lessons become: Explore broadly and lightly across areas related to what you’re interested and then buckle down, dive deep and keep going until you get to something novel. Of course the timing is critical and to some extent they have to happen in parallel. As Matt Might puts it: going rogue too early or too late can be fatal. Luckily that’s what advisors, mentors and colleagues are for.

Personally I’m still in stage 1: I’m still taking classes and exploring the broad regions of computer science but I’m also making forays deep into some areas (particularly programming languages and datacenter networks). Looking further ahead I think it’s great that we’re going to be living in a time where being a member of Gen Flux is a good thing. Gen Flux is perhaps just a modern term for Renaissance Men (or Women) – people with a breadth of knowledge and skills but also with singular and far-reaching accomplishments in some of those fields. And that seems like a goal worthy of a lifetime worth of time and energy.

Graduate School Semester 2

I’m about a month into my second semester of graduate school at Cornell’s excellent Computer Science department. (Shameless plug: If you applied and got admitted you should definitely come visit, we’re awesome. If you’re thinking of applying contact me with questions.) The first semester involved a fair amount of getting used to grad school life. It’s pretty different from undergrad and I covered my initial impressions before. There’s a lot of autonomy (even in the first few semesters) but that means it’s all that much easier to screw up. The most important lessons I’ve learned are to start early, make plans and schedules and set up routines and environments that make getting the work done the default (this applies to both school work and research). I’m definitely not perfect at it, but I try to suck a little less each day.

While last semester was a good learning experience I got a lot less done than I could have. While I don’t want to cry over spilt milk I certainly don’t want to make the same mistakes again. My class load and TA work are about the same as last semester. However I have a better idea of how much time each takes so I can schedule blocks of time more effectively to get large chunks of work done at a time (and not worry about it otherwise). That in turn means that I can have more time for research (which is something I definitely want to do more of this time around).

I’m really happy about the choice of classes I have for this semester. I’m taking Advanced Programming Languages and Datacenter Networks – both are areas in which I have an interest but I know less than I would like to. I have great professors in both and so far the material has been very interesting (and useful). I’m the TA for a class on functional programming which is turning out to be a good learning experience as well. I’ve done some amount of functional programming but not a lot and not in a structured way. I’m working through the exercises and homeworks myself so that I can better help out the students and learning a lot in the process. Since I’m going to be doing a lot of functional programming in the future (Haskell programming in particular) this a good way to level up as well as get my TA duties done.

Last semester I had a small research project which was more of way to get familiar with the concepts and tools I’ll be using later. I am a little disappointed in that my final deliverables weren’t as complete as I would have liked but the experience will come in handy. This semester I have a more concrete (and more ambitious) project. I’m also starting sooner and thanks to last semester I have a far better idea of the challenges I’ll face and how much work it will take to get around them. My main interest in programming languages and right now the project isn’t very language-oriented. But there is a lot of cool systems-hackery involved and once the foundations are laid I can move on to the more higher-level language-oriented parts of it. I’m still taking baby steps (figure out build systems, building testbenches and having rather intense discussions with my compiler) but within a week or two I want to progress to the real meat of the project.

Aside: In case you’re wondering, it involves networks and trusted computing, but more on that in a future post.

Apart from school and research work I’m hoping to do some more exploring. Cornell has a really nice campus but I only saw a small fraction of it last semester (and probably spent a bit too much time in my apartment). I’d like to be able to get out more and take advantage of everything that Cornell has to offer. That’s a bit easier said then done in winter, but that’ll change as things get warmer.

I’m still trying to work out the best “work life balance”. While things like Cal Newport’s fixed schedule productivity seem appealing it might be unworkable for me right now. More importantly, I’m still not sure how separate my work and my life should be, or even what constitutes “work”. I haven’t decided if I consider my writing or my on-the-side hacking (which I’ve been doing far too little of recently) to be work, play or something else. Part of me would like to think that the work-life distinction is only applicable to a more Industrial Age setting where you don’t like your job and want to spend as little time doing it as possible. Ideally you should do work you love (which I’m gradually approaching) and have no need to draw a distinction. While that seems appealing I’m afraid it might lead to sitting (or standing) in front of my machine all day which is not what I want to do. Luckily these aren’t questions I have to answer definitely right now, but I can keep refining my answers over time.

I’m hoping that the rest of the semester will have lots of great learning, cool hacks and maintaining some semblance of a life away from my machines. I know that graduate school can easily become a drag and very stressful and I’m determined to not let myself end up in such a position. Luckily I’m in a good department with great support from friends, family and professors. I’d like to see this semester be more productive and a step on the way to deciding exactly how I want my grad school experience to turn out.