Going on an Internet diet

Living on a college campus for the past year has meant that I’ve become used to having an always-on high speed internet connection. It’s certainly very convenient to have the internet available at a moment’s notice. Furthermore campus-wide wireless means that I’m not limited to working at my desk. On the flip side though, it’s a challenge to make sure that I stay on the internet only as long as I need (or want to). The internet, for all it’s uses (and perhaps because of them) can be very addictive.

I’m leaving for India today and I’ll be away from an internet connection for at least a day, perhaps two. I do have a decent internet connection at home (256Kbps DSL), it doesn’t really compare to what I’m used to. And there’s certainly no wireless. Last time I was home, this situation was sometimes frustrating. However this time, I’ve decided to do a little experiment. I’ll be home for just over a month and I’m going to try to be keeping my internet usage to under an hour a day. ‘Internet usage’ in this case means email, browsing, reading my blogs and feeds and checking Facebook. I can’t feasibly cram in writing posts along with other activities into a hour time slot. But since I plan on writing regularly, I’ve decided that the best thing to do would be to write my posts offline and then copy/paste them into WordPress. Any time I spend looking up links will count towards the one hour quota though.

There are a number of things that I’m looking to accomplish with this diet. Firstly, the Internet is a great information source, but the signal to noise ratio can be dangerously low at times. I’m hoping that under time pressure I’ll learn to be more discerning about what I choose to pay attention to and what I don’t look at. I hope that I’ll be able to use the extra time to do something productive. I haven’t read as much over the last semester as I should have and I think that this experiment will be a good way to catch up and maybe make a new habit. At the same time, I’m going to be looking closely at whether or not having the internet affects other computer tasks that don’t need connectivity. In particular I’m looking to make substantial headway on my research project which will involve a lot of coding. I’m familiar enough with Python that I don’t need to constantly look up the reference docs, so it’ll be interesting to see if not being online all the time let’s me write code faster (or slower).

There are some details that I need to work around. For example, if I get into a conversation with a friend do I need to disconnect at the one hour mark or can I keep talking if I don’t have anything better to do? I’m also certain that I will ocassionally need to look up Python documentation and I’m not sure whether or not that should count towards my internet usage. I’ll be starting the diet from the start of next week after I’ve reached home and had a chance to recover from the trip. I should have the above questions resolved in a week and after that it’ll be up to me to actually enforce my diet. Whatever happens, I should have some interesting conclusions at the end of it.


Creating extensible programs is hard

As part of my research work I’m building a program that needs to have a pluggable visualizer component. I would like users to be able to create and use their own visualization components so that the main output from the program can be viewed in a variety of ways: simple images, 3D graphics, maybe even sounds. My program is in Python, but I would rather node limit my users to having to write Python visualization code. Ideally, they should be able to user any language or toolkit that they prefer. The easiest way to do this (as far as I can see) is to have the visualizers to be completely separate standalone programs. The original program would save its output as a simple text file and the visualizer would then be responsible for reading the file and performing its actions. However at the same time I would like to make it easy to write Python visualizers and that wouldn’t have to supply their own file reading operations. These goals means that I need to design and implement a stable framework that can handle all this extensibility.

I only really started working on this last night, but in that time I’ve realized that this is harder than it sounds. Here are some the things that I’ve figured that I have to do:

  1. Determine whether the visualizer is a Python module or a standalone program
  2. If it’s a standalone program, save the output as a file and and then call the program with the output file as a parameter
  3. If it’s a Python module, there should be a class (or a number of classes) correspond to visualizers.
  4. The visualizer objects should have a clean API that the main program should be able to use.

While none of these tasks are impossible or require advanced computer science knowledge, they do require a considerable amount of care and planning. Firstly, the mechanism to detect whether the visualizer is a Python module should be robust and accept user input, which means that there has to be error checking and recovery. There also needs to be a stable interface between the main program and modules that are loaded.  There should be clear communication between the parts but also the modules should not be able to interfere with the main program.

Allowing third party (or even second party) code to run inside your framework is not something to be considered lightly. Malicious, or even sloppily written code can have very dangerous effects on your own code. In my case, I’ll be directly translating my programs output to API calls on the Python visualizer objects. Calling non-existent methods would throw exceptions and at the very least I need to make sure these exceptions are caught. I also need to make decisions regarding just how much information the visualizers should get. Luckily for me, I won’t be allowing the modules to be sending back any information, so that makes my job easier.

Writing an extensible program like the one I am now is an interesting experience. I’ve been interested in software engineering for a quite a while now and though I’ve written large programs before, this is the first time that I’ve made one specifically geared towards extensibility. Extensibility brings to the forefront a number of issues that other types of development can sweep under the carpet. Modularity, security, having a clean API, interface design, everything is a necessity for making a properly extensible system. Furthermore, having an extensible system means that you are never quite sure what is going to happen or how your software will be used. This being the first time that I’m making such a system, I’m going to be very careful. I’m putting more time into the design phase because I don’t want to do a rewrite partway through the project. Let’s hope that this experience proves to be a good one.

Software is Forever Beta

Word on the web is that Google just pulled the Beta label off it’s Chrome browser. As Google Operating System has noted, it’s not that Chrome is fully ready for mass consumption but rather that it’s just good enough to enter the fray with Firefox and Internet Explorer and that Google is in the process of sealing bundling deals with some OEMs. There is still work to be done, there are still bugs and their  are important features in the works (including an extension system). But the event raises a question that I don’t think has ever been convincingly answered: when is the right time to take the Beta label off a piece of software?

Wikipedia says that a beta version of a software product is one that has been released to a limited number of users for testing and feedback, mostly to discover potential bugs. Though this definition is mostly accurate, it’s certainly not the complete definition. Take for example Gmail which is open to anyone and everyone, isn’t really out there for testing, but is still labeled beta years after it was first released. You could say that in some ways Gmail changed the software culture by being ‘forever beta’. On the other hand Apple and Microsoft regularly send beta versions of their new products to developers expressly for the purpose of testing.

Corporate branding aside, everyone probably agrees that a piece of software is ready for public release if it generally does what it claims to do and doesn’t have any show-stopping bugs. Unfortunately this definition isn’t as clear cut as it seems. It’s hard to cover all the use cases of a software product until it is out in the real world being actively used. After all, rigorous testing can only prove the presence of bugs, not their absence. It’s hard to tell what a showstopping bug is until the show is well under way. Also, going by the definition, should the early versions of Windows have been labeled beta versions because they crashed so much? With exam week I’ve seen college library iMacs choke and grind to a halt (spinning beachball of doom) as student after student piles on their resource intensive multimedia. Is it fair to call these systems beta because they crack under intense pressure?

Perhaps the truth is that any non-trivial piece of software is destined to be always in beta stage. The state of software engineering today means that it is practically impossible to guarantee that software is bug-free or will not crash fatally if pushed hard enough. As any software developer on a decent sized project knows, there’s always that one obscure that needs to be fixed, but fixing it potentially introduces a few more. However, that being said, the reality of life is that you still have to draw the line somewhere and actually release your software at some point. There’s no hard and fast rule that says when your software is ready for public use. It generally depends on a number of factors: what does your software do? Who is your target audience? How often will you release updates and how long will you support a major release? Obviously the cut-off point for a game for grade-schoolers is very different from that for air traffic control software. Often it’s a complicated mix of market and customer demands, the state of your project and the abilities of your developers.

But Gmail did more than bring about the concept of ‘forever beta’. It introduced the much more powerful idea that if you don’t actually ‘ship’ your software, but just run it off your own servers, the release schedule can be much less demanding and much more conducive to innovation. Contast Windows Vista with it’s delayed releases, cut features, hardware issues and general negative reaction after release, with Gmail and it’s slow but continuous rollout of new features. Looking at this situation shows  that Gmail can afford to be forever beta whereas Windows (or OS X for that matter) simply cannot. The centralized nature of online services means that Google doesn’t need to have a rigid schedule with all or nothing release dates. It’s perfectly alright to release a bare-bones product and then gradually add new features. Since Google automatically does all updates, that means that early adopters don’t have to worry about upgrading on their own later. People can jump on the bandwagon at any time and if it’s free, more people will do so earlier, in turn generating valuable feedback. It also means that features or services that are disliked can be cut off (Google Answers and Browser Sync). That in turn means that you don’t have to waste valuable developer time and effort in places where they won’t pay off.

In many ways the Internet has allowed developers to embrace the ‘forever beta’ nature of software instead of fighting it. However even if you’re not a web developer, you can still take measures to prevent being burned by the endless cycle of test-release-bugfix-test. It’s important to understand that your software will change in the future and might grow in unexpected directions. All developers can be saved a lot of hardship by taking this fact into account. Software should be made as modular as possible so that new features can be added or old ones taken out without need for drastic rewrites. Extensive testing before release can catch and stop a large number of possible bugs. Use dependency injection to make your software easier to test (more on that in a later post).  Most importantly however, listen to your users. Let your customers guide the development of your products and don’t be afraid to cut back on features if that is what will make your software better. After all, it isn’t important what you label your software, it matters what your users think of it. People will use Gmail even if it stays beta forever because it has already proved itself as a reliable, efficient and innovative product. Make sure your the same can be said of your software.

Semester end wrap up

Finals week starts from tomorrow and that means another semester has come to an end. I’ve done a number of interesting things this semester, and I think it would be a good idea to tie them together at the end.In many ways this semester has been a continuation of things I started earlier and a starting point for new things. I cna’t say that I wrapped up many things, but I don’t think that’s a bad thing. In fact it’s probably a sign that I have things to keep me busy and interested. So here goes with the end of semester wrap up:

1. My research work

Over the summer I started doing work with a professor on using formal grammars to understand pattern formation. Summer was very much a ‘testing-the-waters’ phase. We worked on two separate projects: automated urban design and auto-generated art in the hopes of being able to find some underlying rules for creating patterns. Though we produced some good work (and a number of good looking graphics), there was still a lot to be done. This semester we decided to drop the urban design aspect and instead refocus on a more basic topic: representing general patterns as grammar rules for artificial languages. To do this I’ve been working on building a simple language to describe languages and then generating samples of those languages, which can then be visually displayed. I hope to have a working version ready by the end of January. This project has been interesting because I got to learn a lot about language design and it was my first time writing a lage piece of software as part of a team. It also fit in very well with my programming languages course, speaking of which…

2. Programming Languages Course

I was supposed to take an algorithms course, but due to a scheduling conflict ended up taking a programming languages course as an independent study instead. I’ve always had an interest in programming languages and working with formal grammars over the summer furthered that interest. Taking the course gave me a chance to formally study about my interests and introduced me to a number of new and interesting ideas.  I learned about a number of new languages on the way, including ML, Prolog and Smalltalk. The course also helped to clarify some earlier ideas I had about what constituted good language design. I realized that syntax can be very impotant and that a lot of times appreciating the full power of a language involves a steep learning curve, but it can be well worth it. And this course led me to one of my best discoveries of the semester, namely…

3. Scala

I’ve liked the Java platform, but never had much love for the language. However, I recently discovered Scala, a really nice statically-typed, multiparadigm language. It feels a lot like Java, but is much cleaner. Its features of note include a good merging of object-oriented and functional paradigms, a clean way to reuse code without the problems of multiple inheritance, static typing and good interaction with existing Java code. I’ve started a series evaluating Scala as an alternative to Java. I’m planning on using more Scala next semester, so we’ll see how that goes.

4. Beginning my electrical and computer engineering major

I’m pursuing an electrical and computer engineering major at college and this semester was the first course in the sequence: a basic digital circuits course. I enjoyed this course, learned a lot and made some good friends. I like dealing with low level logic gates and pushing bits around as much as I do slinging functions all over the place. I’m taking the second course in the sequence next semester alongside a computer organization course. I’m looking forward to having another exciting semester coming up.

5. Web design work

I’ve been making websites for a good few years now, but I’ve only started takin it seriously this semester. I’ve started to design websites for a number of student groups on campus as well as preparing templates for ePortfolios for foreign language students at our college. I started using Dreamweaver and despite my normal scepticism for WYSIWYG tools, I quite enjoy using it and it beats manually managing links. Most of my work isn’t online yet, but it should be within the next few months  (once students start using it). This is certainly I’ll be pursuing and considering that I plan to move to paid hosting early next year, I might just end up making my own WordPress theme.

6. A new focus on blogging

I might have put this last on the list, but it’s certainly not the last thing on my mind. I’ve realized that if you’re doing things that are fun and interesting, there are probably a lot of people out there who would like to know about it. I think being able to communicate your work is as important as actually producing it. I’m looking forward to many more semesters of fruitful blogging ahead.

That basically wraps it up for things I’ve done this semester. There’s a bunch of things that I have planned for next year, but I think I’ll save that post until the start of next year.