I come to work this morning and the intertubes are shaking with Google’s latest announcement: the coming of it’s Linux-based, web-oriented operating system for netbooks: Google Chrome OS. You’ve probably already read a lot of the other posts about the Chrome OS and know something about how it works. It’s an operating system at the core, but more importantly its a platform tuned to running web apps. It’s a clear signal from Google to pretty much every other operating system maker out there, including Microsoft and Apple, but also the Linux distribution providers like Red Hat and Canonical. The message is simple and clear; move over OS makers, the browser is the new application platform.

Google Chrome as the operating system for web apps

Though the reactions from around the web are mostly positive, there are some articles that are raising real issues. ZDNet Australia criticizes Chrome OS on the grounds that it will further fragment the Linux community (who will be contributing the kernel of the new OS) and a better solution would have been to join with Ubuntu which already has pushed Linux to new heights. A prediction from The Next Web makes the claim that Chrome OS will be “the beginning of the end for Ubuntu & co” and the real battle will be between Apple and Google, leaving everyone else in the dust. There’s also concern about the fact that Google already has a operating system for the web: Android, even though it’s only for mobile devices (though it has been ported to x86). Personally, I feel that these criticisms and fears surrounding Chrome contain the more interesting food for thought.

Microsoft isn't beat yet (image from Engadget)

Chrome OS is undoubtedly going to be interesting, both in terms of technology and in terms of the market forces that it will affect. Also certain is that Google is more clearly than ever taking a swipe at Microsoft. Even though Google may have become the most powerful player in  the web sphere, the desktop operating system stronghold was undoubtedly held by Microsoft. Even many of Google’s own applications (including Chrome) target Windows as the primary platform. Microsoft is still a force to be reckoned with. Windows 7 is shaping up well and they have a few tricks up their sleeve, including a new browser project: Gazelle and even a cloud-centered operating system called Midori in the works. They also have a powerful research wing which does some really interesting work and a very big budget (which is enough for them to sit things out for a few years while they make a better product). Whether or not they will actually do so is still questionable, but lets not write them off just yet.

And there is Apple. The last few years have seen Apple’s gradual re-rise to stardom starting with the beautiful new OS X and continuing today with it’s dominance of the online music store arena and the strength of the iPhone platform. Not many people seem interested in pitting Google against Apple, especially since Apple has stayed out of the mainstream operating system and netbook markets. However, when it comes to the internet, Apple has a considerable stake. The iPhone is as much a portable internet device as it is a phone. And though it has carefully stayed away from the low-cost netbook market, it’s unlikely that they’ll sit by while Google plays its hand in the portable computer market.

Apple may the best suited to withstand Google (from The Next Web)

However Apple’s strength in the current situation probably stems directly from the closed, proprietary nature of it’s technology. Apple has a reputation for both creating and support great desktop apps. Good design has always been a hallmark of software running on a Mac and most web apps are still far for matching the polish that Apple has  to offer. The user experience offered by the complete OS X operating system by virtue of the way it can tie together information across different apps is still something that web applications (even suites like Google Apps) have not matched to a large extent. I agree with the Next Web post that Apple probably has the most chance of retaining its user base as Google begins it’s foray into the operating system arena. With the iPhone they’ve shown that they’re still capable of market-shaking innovation and that will probably help them survive the coming OS wars.

One more important player in this market is Linux. Thanks in no small measure to Canonical, desktop Linux has gained some ground in the last few years. However, it’s still holding a very small piece of the desktop market. It’s a valid concern that Google’s entry into the market might eat into the Linux market share. Though it’s certainly possible, I’m not quite sure if this will come to pass. A lot will depend on how easy it is to get things working on Chrome OS besides the browser and web apps. What new webapps have to offer will also be influential. I personally have never been very hopeful of Linux’s position on the consumer desktop. It’s great for hacker-types like me, but I’m still not fully convinced if I would recommend it for everybody. In my opinion, most Linux desktop apps still lack some amount of external polish. That being said, I wouldn’t recommend Vista either. I do think that OS X is the best OS for most users. I don’t see Chrome as contributing to the ‘fragmentation’ of the Linux distribution scene because I expect it to be very different from traditional distros, but in this case, only time will tell.

So what can we expect in the months to come before and after Chrome OS hits the markets? Undoubtedly Google’s announcement will cause the other big players in the field to sit up and take notice. I think this move might consider other companies, especially Microsoft to push out web-centric products sooner than they otherwise would have. Google is clearly looking to shake things up in the near future and it would be folly not to plan to do something about it. However, it’s also worth keeping in mind that Chrome OS is still some time away and there is a lot of work to be done — Chromium works on Linux, but only just.

Before we make and declarations about drastic change in the OS market, it would be prudent to wait and watch and see what Chrome OS actually looks like when it releases. There is also the fact that Google will have to get people to actually use it and that may be easier said than done (considering the fact that most netbooks run Windows XP). Of course, as the iPhone has shown, there is room in the market for a sleek new product if it is made right. I will be interested in seeing how Chrome OS turns out, but I certainly won’t be giving up my Linux laptop or my Mac Mini anytime soon. I wish Chrome OS luck and hope to see some good ideas being implemented. As Yoda would say, ‘Begun the Chrome Wars have’, I’m not ready to pick sides just yet.

Social networks. We all use them (to some extent) and there are a lot of them. I appreciate the services they provide and enjoy using them, but it can be hard to not spend too much time and energy micromanaging. For people who use just one social utility there isn’t much of a problem: everything that they do goes into that one environment. However, chances are you use more than one. It’s perfectly fine to keep them completely separated from each other and I’m sure that works for a lot of users. But I personally would like to provide a coherent image of myself across the sites that I do use. As I found out over the weekend, that’s not as simple as it sounds.

The Facebook Factor

Before I talk about how I actually went about trying my social networks, there are a  few things about the networks themselves that need to be said. First off, Facebook is the unstoppable juggernaut when it comes to social networks. I didn’t really think about this until I started out on my quest, but Facebook does (in a limited way) a lot of things which the other social networks individually focus on. Like most other people, you can use Facebook to keep in touch with friends and make new ones. You can post photos to Facebook albums. I haven’t heard any word on limits except being limited to 60 photos per album. There’s also the status line where you can broadcast what you’re doing to everyone who wants to listen. Conversely the newsfeed lets you keep track of everyone else. You can write notes and use them as a lightweight blog. Interesting links and videos you find around the web can also go on Facebook. And everything you do has a chance to become the basis of a longwinded, interactive discussion between your friends.

Facebook’s purpose as a social network is to essentially surround all the other social networking utilities. Posting a video on YouTube or saved an interesting link on Delicious? Put a link to it on Facebook. Wrote a blog post? Post a link to it or have it be automatically mirrored in your notes (learned this trick from a new friend of mine). Want to share photos of your hike with everyone who went with you? Put in a Facebook album, tag your friends and they’ll be automatically notified. No need to decide whether to use Flickr, Picasa, Zooomr or worry about how you’r going to let others know that their photos are up. If you need a place to discuss something, you can start a Facebook group and invite your friends rather than having to set up a mailing list or forum. Though Facebook does all these things decently well, it doesn’t do them quite as well as the other sites that are out there for specifically those purposes. And Facebook is definitely an ‘inside-out’ community: to use it well you need to have a network of friends first. On the other hand, if your goal is to reach out and find new people to connect with, you’re better off using another social site.

Facebook was my first problem. I tend to think of Facebook as a place for connecting with people I already know fairly well. It’s a good way to keep my friends updated on what I’m doing, especially if they aren’t using Twitter or Delicious or following my blog regularly. I really want Facebook to be part of my social network, but since it won’t be directly help me connect with new people, I want to use it passively as much as possible. To be fair, I suppose Facebook could probably be used for ‘reaching-out’ purposes, but I personally don’t want to do that since in my opinion there are better tools for that. Luckily for me, Facebook seems to understand it’s role as an aggregator of sorts: it makes it simple to integrate your blogs as notes and post your Delicious links and shared Google Reader feeds to the newsfeed. So even though most of online activities take place elsewhere, almost all of it is seamlessly mirrored on Facebook for all my friends to see and discuss.

A Network of Networks

The second part of the problem involves the many services that I use. The services that I use most frequently are Twitter, Delicious, this blog, Last.fm and Facebook (not necessarily in that order). I do have a Picasa account, but I mainly use that for sharing pictures with my parents and close family and so it’s not really part of my social net. For a while I really wanted to integrate all this together. The Ping.fm service let me do just that. I could post something to Ping.fm and it would be sent to Twitter and Facebook and any links would be saved to Delicious. Using Twitterfeed I could have blog updates routed to Ping.fm as well. This setup was worked pretty well and there was nothing wrong with the implementation itself. However, I found some flaws in working this way that involved the very concept of trying to pull everything together.

All the services I’ve named above are all very different. That’s probably part of the reason why Facebook only offers generic clones of them (for now at least). Twitter is good at communicating small snippets of information while my blog is for far more detailed writings (my posts are routinely over a thousand words). Using Delicious with Ping.fm allows me to quickly save anything from the web, but I also don’t fully utilize their tagging functionality which helps greatly when trying to find something later. By using Ping.fm to quickly post to everything, I was losing out on the focused functionality that each of them offered.

I found myself faced with a classic dilemma: I wanted people who ‘followed’ me to able to see everything that I did socially, but I also wanted to be able to use the individual social services to their full potential. I found the answer (to some extent) in the form of Friendfeed. It’s an aggregator for all your social content, not entirely unlike the role that Facebook fills. Friendfeed gathers all your activity from as many as 58 service and provides a single unified feed that people can follow and interact with. My Friendfeed collects information from all the services I listed above as well as Google Reader’s shared feeds. In an ideal world, everyone would see my Friendfeed and interact mainly via comments on that (or on the original sources).

However, it’s not an ideal world. In particular, most people I know (and who know me) don’t actually use Friendfeed (unfortunately). To get around the fact that not everyone is going to see the collected feed that I present, the most reasonable solution is to maintain a careful amount of redundancy. I say careful because if I were to simply link everything to everything, not only would people be seeing the same thing many times over, I could also potentially set up infinite loops with messages being propagated from service to service and back ad infinitum. My first step in this careful redundancy setup is to keep Facebook separate. I use its own native integration features to pull information from the other services, but nothing pulls from Facebook. I generate very little original content on Facebook and so it’s mainly just for collection and discussion.

The second step is to take into account the distinctions between the services and use them accordingly. For example, Twitter is used for status updates so it gets used the most. Delicious is used for saving stuff from the web. Previously I had everything that I saved in Delicious appear on Twitter as well (via Ping.fm), but I’ve come to realize that I save a fair of stuff for my research work which I don’t think everyone wants to know. By separating the services, I can only post things which I think people will find interesting to Twitter. I can also use Twitter to elicit responses and feedback from followers while I use Delicious more for classification. I’m also considering moving to Diigo, but that’s a subject for another post. My blog also gets mirrored to Twitter via Twitterfeed because I think that’s an important part of what I want people to know about. Of course, if you are following me on Friendfeed, you will see all my bookmarks and some duplicates. That’s something I’m still working to solve, I’m not comfortable with having my followers see the same things multiple times.

The third and final piece of the puzzle involves Twitter. Though I use it mainly to send out updates to my followers, it also becomes a medium for discussion with people retweeting and replying to what I’ve said. Unfortunately the native Twitter interface isn’t the best for managing a multi-way communication stream and I’ve found that I can miss out on a lot if I don’t pay careful attention. Luckily there are a number of Twitter clients out there that do a good job of managing your Twitter traffic for you. The one I currently use is TweetDeck which provides a nice multi-panel layout for seeing general tweets, replies, mentions of your name and direct messages side by side.

In conclusion, the problem of managing your social networks is still a tricky one to solve and requires some careful thought to get right. Even then, it’s still not a perfect solution by any means (especially if your friends aren’t on the same networks). As time progresses, it’ll be interesting to see how social networks evolve. Facebook in particular is looking to place itself as the center of the social web. Personally I’m not a big fan of having Facebook be the center of everything, but it could have it’s benefits. But for the time being, Facebook does have serious competitors each with their own strengths and there are some things (like this blog) that I would prefer to happen outside any single social network.

I’m spending the better part of my summer working on a software engineering research project at Virginia Tech’s computer science department. Coming from a small liberal arts college with a small computer science department, it’s quite an interesting experience for me to learn about all the cool things that the various groups, faculty members and grad students are doing. The computer science has a well established Research Experience for Undergraduates program for students in the area of human-computer interaction (HCI). Though my work isn’t really HCI, I tag along with the other students in the program and as a result get to attend interesing presentations and discussions by faculty members working on interesting HCI related problems.

Today we got to listen to a presentation by Dr. Nicholas Polys, the Director of the Visual Computing Group. His presentation was mainly about 3D interfaces, how they were being implemented (on open W3C standards) and how people were using them to solve real problems. I thought that the projects that he showed us were interesting, but I had one major problem with what we were shown: even though these 3D environments were really well thought out, user’s interactions would still be via flat 2D monitors and devices like mice and keyboards, which weren’t the best for navigating a complex 3D environment. The way I’d like to see a 3D environment work would be like the Iron Man movie, where Tony Stark manipulates a 3D projection of his suit design using simple, intuitive and direct hand movements.

Dr. Polys answered my question by suggesting that the 3D interfaces helped primarily by helping put more information on a 3D surface and that a fully immersive “move-your-hands” interaction system might be too cumbersome for day to day use. But he acknowledged that I brought up valid questions worth looking into. The presentation was fun, but was even better was the demonstration we got a later of the Virginia Tech CAVE. The CAVE is an immersive virtual reality environment. But instead of being used for entertainment or simulation, it’s used for visualization of scientific data.

The system is actually quite simple in concept. Projectors are used to project images onto large screens on three sides and below the user. The images are special in that they take advantage of the fact our two eyes see slightly different images. Combined with special glasses that synchronize with the projectors to block corresponding images for each eye, they give the illusion of being in fully three dimensional environment. There can be any number of people wearing these glasses and being in the CAVE, but there is one ‘pilot’ who wears a set of glasses that have sensors allowing for head-movement tracking: the images change to adjust for how the user is looking at it. Movement is via a simple pointing device that allows for 6 degrees of movement. The CAVE information page has more details and videos as well.

After interacting with various different environments and simulations, there are a number of things I observed (which most of my friends agreed with). The most realistic simulations aren’t the fully immersive environments, but rather projections of smaller discrete objects. Our first demonstration was projections of various insects. These seemed very real and it was quite easy to believe that they weren’t projected onto the walls, but actually occupying the space in between them. I think this was due to the fact that they were small, discrete objects and that they were rendered with a very high level of detail as compared to some of the later demonstrations.

The most interesting demo we saw was a 3D model of the myoglobin molecule. This was representative of the scientific visualization work that the CAVE is primarily used for. It was interesting to fly around the molecule and see how the different atoms and ions connected together. The remaining simulations were all of real-world environments. One was a model of a solar powered house that Virginia Tech students had designed and another was a model of a city that was used for early detection of Alzheimer’s by testing subjects’ abilities to navigate streets and complete day-to-day tasks. They were both very interesting, but seemed less real than the bug or molecule simulations. For one thing the graphics were of much lower quality in order to be rendered fast enough to be responsive, and because they took up the entire view field, it was easier to notice the edges where the walls met and that hindered the illusion considerably.

We all had a really good and I’m sure it’s an experience many of us will remember for a long time. In the end, I have mixed feelings about 3D interfaces. The bug simulation has convinced me that it’s very useful for looking at small objects or designs where you’d like to be able move the object around and look at it from all angles. It’s also good for larger models (such as molecules) as long as you’re not looking for photo-realism. I think it’s definitely worth using when a simple 2D image or even a 3D model on a small screen just doesn’t cut it. However, the technology isn’t quite at the stage where a full-blown immersive simulation like a city can be made to look real enough to be truly satisfying (especially if your standards are the 3D environments you see in modern computer games). Immersive virtual reality technologies like the CAVE are definitely important and I’m sure more and more scientists will be using them for modelling and visualization work in the near future.

3D environments on the desktop are a somewhat different matter. Using 3D models for things like design are of course very helpful, but as a general paradigm for interaction, I think 3D on the desktop isn’t a very good idea, at least not with the current interface tools that we have. Controlling a 3D setup with a mouse can be very tiresome at times and I don’t like the idea of having to ‘walk’ through a virtual space to find something when I could find it much faster if it was laid out as simple menu or set of buttons. Things like BumpTop and Shock Desktop 3D look really cool, but I wonder how easy they would be to use in day-to-day work. Then again, I’m pretty much a confirmed minimalist so I’m probably not unbiased. I think that the way modern desktops work in 2.5D with having 3D-like effects (like piling windows on top of each other, transparency, gradients) but still having a 2D interface is good way for people to work while they’re looking at 2D screens.  Of course automating those features and making them easy to use is wonderful. Apple’s Expose is a good example.

What I’d really like to see is some sort of projection technology allowing people to interact with 3d representations in a simple way. That being said, I don’t think that 3D interfaces are going to take over any time soon. The truth is that the simple 2D format is deeply entrenched and works well enough for most intents and purposes. Also the keyboard and plain text is a very efficient way for communication with a computer that’s not going away anytime soon. Though I would really like to see large touchscreens become cheap. 3D interfaces are an interesting technology and I would love to see them evolve. I have my doubts as to whether or not they are ready for the mainstream, but they’re certainly worth looking at, especially if you do data visualization of any kind.

If you’re at all interested in Open Source or Linux you’ve probably heard about the debate surrounding the inclusion of Mono and Mono-based apps in the default Ubuntu distribution.  Here is the original post that seemed to trigger it all. Here is the reply that was posted by Jo Shields, a member of the Debian Mono group. Richard Stallman also weighed in on the matter, in favor of removing Mono. Now I think having ideals and standing up for them is great and that people should always stand up for what they believe in. However, it’s one thing to stand up for your ideals and do what you have to, but it’s quite another to use your ideals as an excuse for making people use something that is of lower quality and that they don’t want to.

I think that people who advocate open source often forget that a lot of people (including a lot of people working with computer technology) care less about the licensing of their software and more about whether or not it works well. The reason open source succeeded isn’t so much due to a religious zeal to use only Free products but rather due to the fact that it let people open up the innards of their software and make changes so that things worked better. The reason I use Linux over Windows is that I can start up in 30 seconds as opposed to 5 minutes and have things organized just the way I like. And I like OS X better for managing my media and doing graphic related stuff.

The second bone I have to pick regards all the talk of “alternatives”. The original post says “There are alternatives to every Mono application that for the most part are better” and quite conveniently fails to mention any of them. Stallman says that the ‘probem’ is with applications like Tomboy which depend on Mono and also fails to mention any alternatives that would not depend on Mono. It’s this sort of condescending attitude towards developers who are doing real work that really irritates me. Stallman is a very inspiring figure who has done a lot of work for open source, but let’s not forget that a lot of ‘pure’ free software GNU projects have been stalling for years while alternatives leap ahead. Free software owes its position today in no small part to the Linux kernel, while the GNU’s own Hurd kernel which started development before Linux is still unusable for most intents and purposes. The Guile scripting language which was supposed to become standard across all GNU tools is also mostly a pipe dream. GNU’s Flash replacement is mostly inadequate. GNU Emacs is a wonderful piece of technology, but it’s losing out to newer IDEs. I personally believe these IDEs are inferior to Emacs in many ways, but are much more in-tune with modern developer needs while Emacs is in many ways is stuck in the past. “Show me the code” was a popular slogan when Microsoft threatened to use it’s patent portfolio against linux, I think it’s time Open Source enthusiasts followed their own slogans.

Finally, this whole business of “Microsoft is evil” is getting tiresome. Yes Microsoft did bad things. Yes, they’re out to get Linux and free software. Maybe Novell is a sell out. So? I don’t like them either. Don’t use Windows and close down your Hotmail account. If you read Jo Shields’reply, you’ll see that Microsoft had no hand in the development of Mono and the agreements with Novell do not cover it. Tomboy and F-Spot are free applications running on a free runtime system. It is reimplementation of a Microsoft standard, containing no Microsoft code. If you still think that Mono is ‘tainted’ because of the association, then that’s your choice and you can take Mono off your system and vow never to use Mono-reliant apps. As a user of free software that is your choice. But it is not your right nor your duty to force other people to do the same. Choice is at the core of free software and if you are trying to tell people that they can’t use or distribute some piece of software, then you are no better than Microsoft or other commercial software makers pulling users into vendor lock in.

At this point, I’m going to change track and talk about a post that appeared as a blip on Reddit a few days ago. It’s called Thumbs Down on Clojure and it basically denounces Clojure as a false Lisp because it interfaces tightly with Java and so has to abandon a lot of the ‘pure’ principles behind real Lisp. Now I have nothing at all against the author’s actual post and I actually like it that he is speaking his mind. The author drew a lot of flak from Reddit users because he placed purity above practicality and more importantly because he had nothing but vaporware to offer as an alternative.

To tell the truth, I have very mixed opinions about what this guy is doing (and talking about). Having seen the power of Lisp, I understand the attraction of purity. Having struggled to find a good cross-platform UI toolkit that doesn’t take a CS major to install makes me believe that when it comes to software, practicality trumps purity every single time (though pure and practical can coexist). And one bird in the hand is worth two in the bush. Though I use Linux and OS X, I have issues with modern operating systems in general. I would love to have an operating system with a fast, lean core but with a uniform, powerful, managed runtime on top in which all user applications are built and which achieves close-to-the-metal performance. I have some ideas on how it could be done, but I don’t have the knowledge or the resources to implement something like that. And till I’m in a position to make real contributions to the world around me, I don’t think it’s fair for me to deride the works that others are doing to make the world a little better for all of us.

The core thread behind both the above issues is that free software really isn’t free at all. There is a significant time and energy investment behind every single line of open source code that you’re using, which includes the software that runs this blog and probably the browser you’re using to read it. Ideals are well and good, but there are real .people who are behind all the free software we use today. Ultimately, the software and ideals don’t really matter, it’s the people that do. The people who make the software, the people who believe and stand up for the ideals and the people who can benefit from both the software and ideals without consciously taking part in either process. It makes me laugh when people like Stallman talk about software being ‘ethical or not’ when the only things that are really ethical are people and their actions. It almost makes me cry when people laugh and deride the work of other people but are incapable of doing anything about it. This isn’t the first time that I’ve heard a Lisp admirer say that they’re appalled at the state of modern Lisps. It’s an understandable sentiment, but what’s equally disturbing to me is that all these smart people (and some of them are really smart) can’t seem to be able to pull together and create something that can blow the socks of all the supposedely substandard solutions that people are using to get real work done right now.

So where does this leave all of us? The same place as where we started. Neither the Mono controversy nor the Lispers’ pining for the good old days is really anything new. Writing good software is hard and always has been, even with the best of tools. And it doesn’t help anyone to criticize productive projects if you don’t have an equally well working alternative. Opposing ideas on a purely ideological basis is often not the best idea (though there are notable exceptions which must not be forgotten). I think Linus Torvalds probably sums things up the best into two quotes. The first one, as I already mentioned, is “Talk is cheap. Show me the code.”. The other one which is equally valid given the circumstances and the rest of this article is “Anybody who tells me I can’t use a program because it’s not open source, go suck on rms. I’m not interested. 99% of that I run tends to be open source, but that’s my choice, dammit.”

So what are you all waiting for? Start showing some code.

It’s been an eventful week with Michael Jackson and Farrah Fawcett passing away and all news of the Iran election situation being knocked off Twitter as a result. But life must go on and it’s sunday again.

Reading

Music Mind and Meaning This article from one of the legends of AI, Marvin Minsky takes a look at music and how it could affect the way we think about the mind (and vice versa). No directly about AI, but certainly worth reading if you have an hour’s time on your hands

Media

Merlin Mann on time and attention Too much to do and too little time? Can’t get yourself to focus on the important things in life? If that’s the case, then this video doesn’t have all the answers, but it can point you in the right direction

Software

Neuroarena I’m not much of a gamer, but that doesn’t mean that I can’t appreciate them (or the cool technology behind them). This is a fast paced action strategy games that should give you strategy fans something interesting to do on hot lazy weekend afternoons. What’s more, the game backends are written in a combination of Common Lisp and Erlang. Not bad at all.

My summer research job involves a fair amount of reading. It also involves searching for papers on databases such as the ACM’s Digital Library. The only problem is that once you get to more than a few papers, it becomes really hard to figure out which paper is which. The papers often have long titles, so you really don’t want the filenames to be the paper titles. There’s also a lot of metadata associated with each paper (title, authors, where and when it was published and so on). It quickly became apparent to me that I really needed a better way to organize all the papers I was downloading and reading. The problem became even worse when I decided to start making detailed outlines and notes for the more important papers, because now I had to have someway to connect each paper to its notes.

I looked at a number of really good tools out there before making a choice. The very first program I remember looking at was Papers for the Mac. Papers is a really slick application. Not only can you collect and organize your papers, you can also read them right in Papers, take notes and send a copy to fellow researchers. It also lets you search popular databases and download papers without needing to step out into a browser. If I was a full time Mac user, then I would almost certainly be using Papers. But since most of my more scholarly is done on my Linux laptop, it’s not really an option. It’s priced at a very reasonable $42 with a 40% discount for undergraduate students. Not a bad deal at all.

The Papers interface (from the website)

I happen to do most of my scholarly work on my linux laptop and so I needed something that worked well on Linux. There is a program called gPapers for Linux which is similar to Papers. I didn’t actually check this out myself. From the screenshots it seems like a good tool, but development on it seems to have stopped for a while and I wasn’t sure if it was complete enough for daily use. My next choice was the Firefox plugin Zotero. Since it lives right in your browser, it makes it very easy to collect papers that you read on the Internet. If you get a Zotero account you can even sync your papers and notes between multiple machines. Once again, Zotero is a great tool. Not only can you add any web page to it’s library, you can also attach notes to any item in your library. The most awesome feature is that for PDFs it will automatically retrieve bibliographic information from Google Scholar. You can also export bibliographic information in a variety of formats. Zotero makes a lot of things very easy. To be honest, I haven’t entirely ruled it out yet. Perhaps the only real reason that I’m not using it right now is that the interface seems a bit cramped on my not-too-big laptop screen.

Zotero in Firefox (from the documentation)

After trying all these out I took a look at one online reference management tool: CiteUlike. I was very interested in this as it would make sense for whenever I worked from library computers. It’s a really good resource and works very well with popular databases in terms of automatically gathering metadata. The only problem I have is that it doesn’t work if I point it directly to the PDF of the article. There is an option to store the PDF online, but this requires an upload from the local computer and not from an URL. I can understand why this is needed because many of the journals require subscriptions, but it is a bit of hassle. I’m not using it now because I really don’t need an online service at the moment. I like keeping all my papers organized locally. However, if I find myself moving about a lot once school starts again, I might seriously consider using CiteUlike more regularly. There are similar online services such as Connotea, but I haven’t tried them out myself.

So what do I use? There were a number of factors that affected my choice. Firstly, I wanted to have copies of the PDFs on my disk. I needed to know where they were and they had to be named according to some common scheme so that if I had to switch tools I could do so easily. Secondly, I had to be able to easily extract reference information as a BibTex file. I use Latex to write my own papers, so there was no compromising on that. Thirdly, there had to be a good way to edit and view notes for each PDF. Zotero was certainly the tool that came closest to meeting my needs (except perhaps for Papers). In fact, in the process of writing this review I’ve been sorely tempted to actually use it full time. And I would too, if weren’t for one almost unrelated piece of software: Org-mode for Emacs.

Org-mode is a package for Emacs that turns it into a powerful note-taking and organizational tool. It offers some really good features including very easy-to-use (and smart) show/hide allowing you to concentrate on parts of your notes. It also allows to attach tags to parts of your org files and then search and re-organize based on them. You can insert links to URLs or other files (which can be opened in Emacs or in other programs). Over the past few days I invested a few hours to make a homegrown solution based around Org-mode and Emacs.

I combine some low-tech organization with some simple scripting to make my system work. Here’s how: each project for which I need to do research has a papers and a notes directory. The name of any file that goes into either directory is of the format <publication><year>-<first author> with the appropriate extension. I also have a BibTex file that contains bibliographic information. The key for each entry is in the same format as the filename. I then have a quick Python script that matches each PDF with it’s notes file and bibliographic data and combines it into an easy to read org file with appropriate links. Since I have Emacs open all the time, whenever I need to look up a paper all I need to do is open up the org file and I get organized links to both the paper and its notes. I also attach tags to the papers’ names to make locating them easier. Here’s a screenshot of what an example org file looks like after I’ve added some tags:

Listing of papers with metadata and links in Org-mode

I’m currently using this system on a project with about 15 papers and growing. It works well so far and I’m really comfortable in it. The only part of it that I don’t like is having to manually get the BibTex data. Since gathering metadata is a feature that Zotero has, I’m considering using Zotero as the ‘front-end’ of my system i.e. I use to Zotero to download and store the PDFs and their data, but I then use my Emacs-based system as the actual interface. This is something I still have to explore, but I think it could work. I should also note that I use Delicious to quickly bookmark interesting articles that pop up in searches, before going back and doing a preliminary quick scan prior to actually downloading the PDF. As usual I would love to hear any comments or suggestions you might have.

For most of last week, I’ve been evaluating various options for starting a public, personal wiki. I’ve looked at a number of solutions, both large-scale by wiki providers and homegrown using open source tools. I still haven’t made my decision and to tell the truth I am getting a little frustrated at this point. However, if there’s anything that I learned, it’s that there are way too many formats out there.

I’m not talking about large-scale industry standard formats like HD-DVD vs Blu-Ray or something on the level of paper vs digital. I’m talking about much simpler things like the variety of publishing formats and ad-hoc text formats floating around in cyberspace. The lingua franca of the internet is still HTML, at least if you want a simple website as opposed to something running on a content management system. However, if you’ve done any sort of web development, then you’ll know that building websites from scratch using HTML is not fun. It’s XML after all, and no one should have to write XML by hand.

Even without straight HTML, there’s still a ton of formats to choose form. If you want others to read something you’ve written, what do you choose? You could use a word processor format like .doc or the newer OOXML (.docx) or if you’re more of an open source fan, OpenDocument might be your thing. But it’s a bit harder to spread an office file like that. You can’t just drop it onto a webpage unless you convert it to HTML first. You could email to people, but that doesn’t scale and most people might not care enough to actually open it. Same arguments go for PDFs.

If you really  want a lot of people to read what you’ve written, you want to provide in a form that’s accessible via a plain browser, hopefully without plugins. You could use something like Scribd’s iPaper, but I think that’s more useful if you have a complicated PDF that you want to show without people having to actually download and open the PDF. It’s a bit overkill for normal text. But we alread decided that writing HTML is bad and so like the hackers we are, we’re going to find a way around it.

You could go the CMS-route and use something like WordPress (for blogs) or MediaWiki (for wikis) or Drupal (for something more versatile). These have the advantage of having a pretty WYSIWYG interface and a whole list of administrative features. But they require you to have your own server or web space or find a free host like WordPress.com. But if you’re someone like me, you want a simpler solution that’s more under your direct control and that you can add on to later as your needs increase. The good news and bad news is that there are a bunch of simpler, human-readable (and writeable) markup formats that can be translated to HTML with fairly good results.

The good news is that these formats are simple and the HTML conversion tools are mostly open source and of good quality. The bad news is that there are a multitude of such markups, all of which are mutually incompatible. The ones that seem most popular seem to be Markdown, Textile and reStructured Text. All of them have their strengths and weaknesses and I don’t really like any of them. If you can pick one and use it right, you can have a good experience. But you can just as easily push against the boundaries of what they offer and end up being frustrated by their limitations.

The above are all markups that are meant to be translated to HTML at some point, but can be read by people directly. Though they do allow some form of structuring (in the form of HTML headings) and allow inline HTML too, they’re not really all that good for highly structured text, like when you’re trying to make outlines for scholarly papers. There are other text-based tools to do that and my personal favorite is Org-mode for Emacs. It’s a package for Emacs that turns it into a powerful outlining, note-taking and organization tool. It lets you create an outline as a series of headings and subheadings (nested many levels deep) along with plain text and normal lists. The different levels can then be hidden arbitrarily letting you take a bird’s eye view or just focus on one part. Many people use it for GTD or some other productivity system. I prefer Google Docs and Tasks for that, but Org-mode is a note taking tool unparalleled in it’s simplicity and ease of use.

Org-mode uses a simple, custom text format to actually store any notes you make. It’s also human readable, so you can easily copy/paste it into an email and share with others. But without actually using Org-mode, it’s hard to exploit the format to it’s full potential. It becomes even less ideal when it’s exported to some other format like HTML. The org-mode concept of headers don’t neatly map onto HTML headings. Org-mode encourages header nesting which looks terrible in HTML unless you carefully lay out a CSS stylesheet for it. You can use the headers more judiciously preferring to use plain lists, but that defeats the purpose of Org-mode to some extent. It seems to me that you can’t have the bost of best worlds.

At this point any self-respecting hacker will point out that I could just stop whining and start writing converters from one format to another. After all, they all convert to HTML, it can’t be that hard to convert between them, right? It’s probably not and the Pandoc system does it to some extent, but the problem isn’t with the formats themselves as much as with the tools that work on them. HTML is good for publishing normal documents, but if you have something with many levels of nesting such as an Org-mode document, you really need something that doesn’t show all the information at once. Unfortunately there isn’t an easy way to do this without resorting to JavaScript or something similar. HTML in it’s raw form is still a very static format, presentation wise and it doesn’t always scale well to complex sets of information.

I’m going to take a break for a moment and think about what an ideal system would be like. It would be based on a simple text format that was explicit about what things meant, so you didn’t need a reference. One of my complaints about Markdown is that anything indenting 4 spaces gets treated as a block of unformatted HTML placed withing <pre><code> tags. There’s no way you would guess this by looking at the plain text and it also makes list nesting and using indentation to present your text very awkward. But I digress. While you could write this text in a plain editor, the preferred way would be in an editor that supported folding and searching and all the other editing niceties. The editor would actually double as the presentation so that there would be any export or rendering step. And it will probably be on the web. The editor will be in a browser and the actual data will be on a remote server. However, unlike many web services today, export and import to a local form will be a core strength. This way you can pull the bare text out of the editor and send to others who don’t use the same tools as you without having to go through some silly registration/sign-up step.

I started this post thinking it would be about the markup, but now at the end its turned out to be about the tools as well. HTML is a great medium because it’s so easy to render and produce. However, our data needs are going beyond what a simple document-oriented format can easily supported. I don’t think Org-mode or the like will ever become the de facto standard. But the fact that a lot of very smart people choose it over mode ‘modern’ Web 2.0 stuff is a testament to the power of simple, easily editable formats. I hope we could standardize around a dual markup system that had a simple human readable form for quick writing and a more snazzy display form. I doubt that’s going to happen any time soon, so till then I’m looking for the perfect set of text tools to store my data and ultimately show it to the world. All this ties in directly to my efforts to having a wiki that’s easy to create, easy to back up and good to look at. More on that later.

It’s a warm Sunday afternoon in Southern Virginia and I need to do laundry and other such mundane stuff. But before that it’s time for another installment of Sunday Selection

Reading

The Benefits of a Classical Education Nothing to do with computers but certainly worth reading for anyone who does work that involves the intellect in any way (and I do hope that your work with the computer involves your intellect)

Media

CouchDB and me Once again not entirely about computers. It’s more about what to do with your life if you really love computer technology and want to work on your terms rather than slave away on what someone else thinks is important.

Software

Opera 10 beta This includes the much hyped and talked about Unite web server. As I’ve said before, I have my doubts as to whether or not it will have any noticeable impact on the web, but you might think differently. If you might some cool, interesting way to use it, do let me know.

Today Opera announced the release of their new ‘Unite’ product. The basic concept behind Unite is something that has been around ever since the beginning of the internet: users aren’t just consumers, but producers as well. Unite will turn your browser into a mini server allowing you to connect to other people and share things directly from your computer. It sounds like a good idea, but the implementation is not something that I find very comfortable with.

The idea

Don’t get me wrong, I think the idea is a great one. Being able to share your own material without having to depend on a third party and risk them stealing your stuff (or just locking it up) is a great boon. It would be wonderful if we all just had our own private servers, keyed to our personal identities in some uniquely identifiable way and exert total control over what we put online. However, the truth is that the implementation details of doing something like are very complicated,

For example, if we all started directly publishing our own content, we’d all need massive bandwidth connections and have to pay for them. We’d need to install hardware and software and keep it all up-to-date. We’d need to deal with all the potential security issues related to allowing other people to access our computers. It would also be difficult to maintain any sense of uniformity across the web. Sure, we could agree to some common protocol, but that protocol would have to be set in stone because it’s going to be very hard to get millions of people around the world to all update to a new protocol. The idea is a very good and powerful one, but it’s useless without proper implementation.

The implementation

That being said, I think Opera has done a lot to alleviate some of these problems. In particular, Unite is easy enough for just about anyone to use. They’ve taken a large part of the maintenance headaches out of the equation, at least for the software component. They also seem to have found a way around the issue of keeping everyone on the same page and playing by the same rules: producers use Opera’s custom system, but consumers can use a plain web browser. But while this strategy means that it’s easy for users to start becoming producers, it also means that people will be locked into using Opera’s product and account system. It’s this part of the bargain that I find somewhat uncomfortable with.

Unite requires an Opera account

It seems to me that Opera may have solved one problem by replacing it with another one. It’s now easy for anyone to distribute their content from their own computers, as long as they buy into Opera’s system. I don’t use buy in the monetary sense of the term, but in the ‘free as in freedom’ sense. Opera claims that unite will allow “sharing data and services without the need for any third-party Web sites/applications to be involved at all”. Problem is, Opera is the third party. Sure my content is still physically on my own computer, but Opera is the gatekeeper. I feel that’s even less of a deal than uploading my data to Facebook or YouTube. Not only do I now have to pay for all the bandwidth and space I use, I also have to play on Opera’s terms. I don’t see much of a bargain in that. Perhaps I would if I was really more concerned about people ’stealing my content’, but I honestly think that you shouldn’t put stuff on the Internet if you don’t want people to share it and spread it around.

Unite isn’t for me, is it for you?

Opera Unite is really quite an interesting piece of technology. It’s one of those ideas that no one really thinks of, but once you hear about it either seems ridiculous or very obvious. It’s a great idea to let users directly share their own content, but I’m confused as to who Opera is targeting here. Let’s start with the fact that Opera’s market share is really quite tiny. Using Unite means that people have to go and download yet another browser. Secondly, how many people will really want to pay for the bandwidth prices that they need to in order to really share their own media? Third, even if you do start using it, you’ll need to have your computer on all the time and connected, something that’s not an option for people on the move with laptops or netbooks. Finally, the market of people who will actually use this seems rather small to me. If you’re really interested in becoming an internet content producer, you’re going to want your domain name, be always on and outsource the technical details to people with more reliable services. If you’re the average internet user who just wants to share your photos with your friends, chances are you’re already on Facebook or MySpace and it works good enough. And if you’re savvy enough to be worried about people stealing your content, you know your way around the web and probably have your own server in the basement already.

I feel that Unite is one of those things that unfortunately just missed the proper timing window. Had Opera released this before social networks and YouTube made media sharing easy, they might have had a fighting chance to make something out of it. But with Facebook and the likes deeply entrenched and sharing tools like Google Wave promising a more open model for those who care, Opera seems to be outmatched and outgunned.

Feel free to use the comments to disagree with me.

Dear readers, yesterday I noticed a sharp drop in the number of RSS feed subscribers. I apologize if all this jumping hosts has been confusing, but The Bytebaker feed remains the same. The feed is currently distributed via Feedburner and I will continue to use that as the main feed. I use Feedburner precisely so that nothing changes if I move across hosts. If you happened to switch to the default WordPress.com feed, I’d like to ask you to move back to the Feedburner one. Once again, I’m really sorry for the confusion. The RSS button on the right links to the Feedburner feed (I’m aware that the button is distorted in Google Chrome).

It’s possible that this is just a glitch in Feedburner’s tracking system (this wouldn’t be the first time), but I’d rather not take that chance with my readers. Thanks again for all your help and consideration.