Filed under Operating Systems …

Nov 25 2008
1 Comment
Hardware, Operating Systems, Productivity, Software

Making Linux boot faster

I’ve been a committed Linux user for the past 3 years now (though I’ve developed a liking for the Mac platform too) and currently I dual boot Vista and Arch Linux on my laptop. It’s a pretty rare ocassion for me to actually boot into Windows, mostly it’s just a game of SimCity 4. But whenever I do I’m really hate how much slower it is than my Linux install. What I hate even more is how even after I log in my system is still unusable for a good 20 seconds while everything finishes loading in the background.

By comparison, my linux install takes just 30 seconds to get to a usable desktop. Admittedly, what I boot into isn’t really a ‘desktop’ in the traditional sense of the world. I run Stumpwm, a lightweight tiling window manager, I don’t have fancy 3d graphics, I don’t even have a taskbar or desktop icons. That being said, I do boot into a fully usable system with an active wireless network connection.I was quite happy with the 30 second boot time until I came across an article about a group of researchers who managed to make a linux system boot in 5 seconds. Needless to say a 5 second boot time is really impressive and makes my boot time look extremely sluggish by comparison. 

With Thanksgiving Break around the corner, I’m considering putting some time into a building an experimental system in order to see just how fast a boot I can get. Chances are I’ll be using an older Dell machine, but I’m not sure about the specs yet. There are two main steps I’ll be taking to cut down on boot time:

1. Cutting down on unnecessary processes

A standard linux boot often starts up a number of processes and tasks that aren’t really needed on a normal single user system. Namely, I don’t need a Cron scheduler, nor do I need any type of mail server to be started. Since I never print, I can get rid of CUPS too. I use a lightweight graphical login manager called Slim, but if I wanted to shave another second or two, I could boot straight to a logged in desktop.

2. Making changes to the kernel

Since I’ll know exactly what hardware I’ll be running, I can build a custom kernel with appropriate modules for the devices built right in. However, I’ll still need to start the Hardware Abstraction Layer and udev so that I can use hotplugged devices (USB drives and the like). Since this is an experimental machine, I probably could remove the need for hotplugging, but I think in the real world, that crosses the usability line in the wrong direction. 

Adding things directly into the kernel has a two-fold effect: firstly I no longer have to wait for the modules to be loaded separately later in the boot process. Secondly, I’m trying to figure out if this will allow me to get rid of the initial ramdisk. Arch linux boots up by having the kernel load up a compressed disk image into the RAM which contains the basic modules needed to get on with the boot: mostly hard drive modules. This then lets the kernel mount the root filesystem and get on with the boot process. Removing the need for a ramdisk might be a significant time boost and is something I will definitely be looking into. 

I doubt whether I’ll be able to get a boot time under 10 seconds, especially since I won’t be making any source level changes. I’ve been looking for information to see if a different filesystem (other than the popular ext3) would have a noticeable impact, but I haven’t been able to find any concrete information yet. I might be looking around at some other kernel patches. Unlike the researchers mentioned above I won’t be making any changes to X or implementing readahead (which lets the kernel look ahead to see which filesystem blocks need to be loaded next). As always, I’ll be keeping a log of my activities and I should have some interesting results.

Sep 09 2008
1 Comment
Internet, Operating Systems, Software

Is Chrome really so important?

So the intertubes are resounding with word of Google’s latest offering: a next-generation browser called Chrome. Chrome certainly embodies some really cool ideas and could just pave the way for a new generation of browsers that are designed to support rich web apps and not just the text and images of web 1.0. But honestly, how far is Chrome going to go and how soon?

Chrome is being touted by some as being a full-blown web operating system that will soon supercede Windows. Ahem. Allow me to respectfully disagree. Sure the cloud is becoming an important part of everyone’s computing experience, but the desktop isn’t going anywhere soon. Let’s keep in mind the fact that the majority of computer users aren’t really tech savvy and aren’t continuously on the move. The most common use that people have for computers is word-processing, spreadsheets, maybe presentations, email and Facebook. Let’s face it: your grandmother doesn’t really want or need her cookie recipes to be kept on remote servers using Amazon S3.

Though personally I do quite like Google Chrome, there are some things that really trouble me. First up is memory usage. Google Chrome takes up about 267MB of memory which is more than IE8 which in turn is more than Windows XP. Seriously, all that for a browser so that I can run webapps which for the most part have features I could find in mid-90s software? Wasn’t it the promise of cloud computing that we would have trillion of clock cycles and terabytes of storage at our fingertips just for the asking? Webapps still have quite some way to go before I can justify a quarter of a gigabyte just to run a browser. Let’s not even talk about things for which there aren’t any webapps yet. I’ve recently begun moving away from word processors to Latex for my writing. There isn’t an online Latex environment. Nor are there full-scale online development environments (though CodeIDE is pretty cool). As a programmer, I’m not quite ready to move onto the cloud full time.

So the desktop is here to stay. After all it doesn’t really make sense to scale everything to thousands of processors just because you can. All that being said and done, I do think Google Chrome brings some interesting ideas. Separating the workload across multiple processes is interesting and the V8 javascript virtual machine could prove to be a good step forward. Only time will tell if Google Chrome really does have a substantial impact on the state of the web. But a web operating system is still some time away.

Tagged ,
Jul 29 2008
1 Comment
Hardware, Operating Systems, Software

Save everything

The hard drive in my first computer was a mere 20GB. When we had to replace it three years later, we got one with a 40GB capacity. My current laptop (which was the first computer I bought on my own) has a 160GB hard drive. Even my pocket sixed iPod classic sports 80GB . Today you can get terabyte hard-drives for a few hundred dollars. So the question is: with all this massive storage easily available, why is the user still prompted to save their work? Why not just save everything?

In a way, even though we have 21st century storage capacity, we still have 20th century storage techniques. Filesystems of today while certainly efficient at doing their job don’t quite do the right job. After all the bulk of today’s personal data isn’t in the form of documents that can be neatly sorted into hierarchical categories. Instead most of our data is in the form of pictures, music and video, for which the easiest interface is some sort of multi-property based search rather than directory-style storage. As our data grows, metadata will become increasingly important.

But even if we have rich metadata, that’s still only going to take up a small amount of space. What I would really like to see is ubiquitous versioning. Any changes that I make to any files (including moving and renaming) should be easily viewable and I should be able to roll back to previous versions without any difficulty. Software developers have already been using robust versioning systems for decades, but I would like to see it become an inherent property of the file-storage system itself. Versioning goes hand in hand with backups, and while Apple’s Time Machine is a step in the right direction, its still got a while to go.

Another twist in the storage tale is that though local storage in the form of hard drives and flash drives is becoming dirt cheap, online bulk storage is cheaper still (and in some cases free). Unfortunately there is often quite some work to be done to get reliable online storage working seamlessly with your local machine. Like versioning, the technology is already out there, it just needs to be packaged into a convenient, always-available form.

So where do we start? I think Google Docs has shown a good starting point: instead of making the user explicitly save something, applications should just go ahead and save it anyway. If the user decides to actually keep it she can then rename it to something meaningful and move to somewhere else. Perhaps there should be some sort of garbage collection where files that were autosaved and then untouched are deleted after a certaing amount of time (after asking the user, of course). Or you could just save everything forever and only run garbage collection if disk space gets dangerously low.

Once you have a basic save-everything system, you could add versioning on top of that. I was hoping to find a versioning filesystem already in existence, but the closest to a fully operational one that I could find were Wayback and CopyFS, not quite what I’m talking about yet. ZFS shows some promise with its copy-on-write and snapshot features. Hopefully it will only be a few more years before one of the major OS makers (or an Open-source initiative) decide to bake version control into the filesystem (or at least tie them together closely). Once we have the capability to store such massive amounts of versioned data seamlessly, we need a way to find it all. WinFS would have gone a long way to solving this problem, if it had ever gotten finished. I’ve personally come to see the shelving of WinFS as one of the greatest tragedies our industry has faced in recent times. The hierarchical file structure is being pushed to the limit and WinFS would have given a good way forward. However, as personal data gets into the terabyte range, we will absolutely need filesystems that can work with rich metadata. Hopefully WinFS will be pulled out of the mothball or Apple will come up with a working solution post Snow Leopard.

Right now I’m stuck with my large hard-drives hopelessly underutilized. I’ve started trying some home-grown solutions such as putting all my documents under version control. Over the next semester at college I’ll also be experimenting with S3 and trying to run a personal backup server. Hopefully I’ll be able to put all those gigabytes to work.

Tagged , , , ,
Jul 14 2008
3 Comments
Books, Operating Systems, Productivity, Software

Book Review: Beginning Ubuntu Linux Third Edition

Ubuntu and desktop Linux have come a long way in the past few years. Ubuntu is currently one of the most popular, if not the most popular distro for desktop linux users. It was my first distro and though I no longer use it, I’ve always acknowledged to be a well-polished piece of work and I always recommend it to people who are just starting on their personal Linux journey. Like most other things in computers, getting used to a new operating system is made easier if there is a good source of documentation available. Beginning Ubuntu Linux, published by Apress is a particularly good example of documentation geared towards to the new user. I’ve reviewed the previous versions of the book and I find that the books have kept improving just Ubuntu itself.

One of the things that makes this book particularly appealing for me is that it starts out with a brief but informative review of the philosophy and history surrounding Linux and Ubuntu. I personally believe (and I think that many other Linux users share this) that there is much more to Linux and open source software than simple technical excellence. It is a way of thinking and acting that I find very appealing and which I wish others to understand. This book does its part in helping new users understand the culture that gave rise to the software that they will soon be using.

The book continues the practice of understanding that most of the people reading it will be Windows users. As a result the chapters dealing with installation also tell users how to properly back up their data and how to smoothen the transition. The guide through the actual installation process is also very in-depth and well written. Partitioning is often the most confusing part of the installation for a new user. I’m glad to see that partitioning has been dealt with very well with all the options in the install process carefully explained and the pros and cons weighed carefully. The chapter dealing with common installation problems is as good as before but now includes information on more than just installation problems. I particularly liked the section on how to deal with resolution and other common graphics problems since these can be very frustrating if not dealt with properly.

Once installation is complete the book goes on to describe with an equal amount of care how to perform various day to day tasks and how to customize your system. The section that deals with Linux equivalents is also comes in very handy for new users who just want to be pointed quickly in the right direction. The book geos beyond describing simply the core operating system and the user interfaces. Of particular note are the sections devoted to how to use multimedia systems. You’d be hard-pressed to find a computer user who doesn’t have a substantial collection of various music and video files and this book helps newbies get up and running with minimal effort. This new edition keeps the sections on using OpenOffice and the BASH shell but adds substantial material regarding the new automatic multimedia setup, the 3D graphics effects that have the Ubuntu desktop much more visually appealing and also on security and encryption. There is also a mini-tutorial on using the GIMP for basic image manipulation which I think shutterbugs will find handy.

The last part of the book is devoted to slightly more advanced topics such as package management, backups and automation and remote access. Personally I feel that package management deserves a more central place, right alongside installation, but the book’s modular structure means that this isn’t much of a problem. Overall the last few chapters act as a springboard from where newbies can start another journey to the level of power user and beyond.

The book as a whole is well laid out and material is clearly separated. The use of sidebars and small tips and warning sections means that a good amount of extra information is presented without interrupting the main flow of the text. There are also lots of links to other information sources where the interested reader can go to for more in-depth information. Most of these are freely available online sources and the full URL is often provided resulting in minimum effort for the reader. For Windows and Mac OS X users there are also pointers to third party tools that might make migration easier. The book is replete with high-quality black-and-white screenshots which add to the complete guide experience that the book provides. The third edition updates everything to be in sync with Ubuntu 8.04 and comes with a double-sided DVD containing a ready-to-install image on one side and various ISO images of the Ubuntu derivatives on the other. In essence the book has everything that a user would need to get up and running with Ubuntu.

The book is at a reasonable price of $39.99 and I think it’s a good investment for anyone looking to jump into the world of Linux. Even though you’ll get the most from this book in the first few weeks after installing Ubuntu for the first time, the later parts of the book will serve as a handy quick reference for those types you find yourself needing to dig under the hood. There is certainly a large amount of information online which means that books of this type are not strictly necessary, but at the same time it can make things a lot easier to have a quick reference close at hand. My litmus test for this sort of this sort of product is generally: would I give it to my mom? This time the answer is yes.

Tagged , ,
Jun 21 2008
Leave a comment
Operating Systems, Programming, Software

Cocoa, Python and the quest for platform independence

For the past few days, I’ve been looking into Apple’s Cocoa infrastructure in some detail. The reason once again stems from my research projects. All our code is in platform-independent Python meaning that we can develop on any of the 3 popular operating systems and then run on any others. This has been a great advantage since the three members of our team each run different OSes. For most of the time our project was purely command-line and text input based, using graphics only for the final display. However we have now come to the point where we need to fork our system into two different systems: one for architects and one for artists. How each program will develop is something that we will work on over the next few weeks, but one thing is clear: for the artists at least, a purely textual interface will simply not do.

Till now our system had been accepting an instruction file as input. While this method of using a instruction was very powerful (we could essentially embed our own programming language), this not the direction we want to go with the artists. And besides, not every artist wants to sit down and right pages of instruction to draw a picture, we might as well ask them to write the program themselves. It would be best to create a GUI for them to use. Even before this had been clear, I had written a simple interface that was essentially a text editor containing a hook to our Python backend. It was written in Tk, was exceptionally ugly, but got the job done.

My Tk Interface running under Leopard

Now I think the Tk toolkit is capable of letting me create the entire GUI interface that I have in mind and it will be cross platform as well, but there’s one problem: it’s ugly, especially on the Mac. I think this is because the Python Tkinter version doesn’t use Aqua (like Tcl/Tk), but rather uses X11. Though I spend a lot of my time in Vim and at the command line, I’m still a fan of good looking user interfaces (see my last post). As a result I would feel rather guilty about making something that I know would look bad on the platform it would be used most. So like a good programmer off I went to find a solution.

I thought I would be able to use Cocoa to create a front end, especially since Xcode now has support for the PyObjC bridge. Unfortunately this has turned out to be much harder than it at first appeared. Cocoa is Apple’s very powerful object oriented API. It is almost everything you will ever need to create a Mac application. While it allows to create native applications by leveraging the powerful infrastructure Apple provides you, it also doesn’t allow you to perform the sort of rapid duct-tape styling programming that Python and UNIX encourages. In Python it’s very easy to write your core application in Python with any number of GUI front-ends that are easily interchangeable. Cocoa, on the other hand, is not another GUI toolkit that you can simply add to the mix. If you want to use, you’ll have to invest a certain amount of time in learning your way around.

Python can’t communicate with Cocoa directly. To  have your Python programs leverage Cocoa, you need to go through the PyObjC bridge. Though the bridge is certainly effective and I could use it, I’ve been having troubles finding decent documentation. Apple’s own tutorial is the sort of document I’m looking for, but it is dangerously out of date. Lacking a proper tutorial, I would have to figure out a lot of things on my own. Once again, a large time investment.

Considering that I have only two weeks left to make real progress on my project and that learning Cocoa or Objective-C was not my original goal, I’ve decided not to pursue that avenue at the moment. I would certainly love to learn Cocoa at some point in the future and PyObjC seems like it could become a very powerful tool. But I simply don’t have the time at the moment. I am going to start working on a proper GUI interface in Tkinter. Perhaps sometime later this year, I’ll get back to some Cocoa.

Tagged , , ,
← Older posts
Newer posts →
Follow

Get every new post delivered to your Inbox.

Join 331 other followers