Spam and user responsibility

Twitter, like any communication medium (physical or electronic) has a spam problem. They’ve been able to mostly keep it under control, but it surfaces every now and then. I was talking to a friend about this recently and he noted that certain kinds of spam are easier to detect than others. For example, a tweet full of suspicious URLs may very well be spam. But is an account that never tweets but follows lots of people a spam account? Part of this is an perception issue. I personally don’t care about being followed by (possibly) spam accounts. I get more annoyed if I get a tweet from one of them. However, I do know people who are really careful about who follows them. For Twitter, the question is how to tell spam tweets and spammers apart from legitimate traffic and users.

Getting users to report spammers is perhaps the first line of defense. If a large enough group of people report an account as a spammer, there’s a high probability that it actually is a spammer. But on the users side, what is my incentive to report an account as spam? Unfortunately, I feel that Twitter’s nature plays against it in this regard. In particular, a spam tweet is much less annoying than a spam email. Tweets and the Twitter stream are, by design, ephemeral and temporary. You see something and a few hours later it’s gone. You’re not expected to read each and every tweet and you’re certainly not expected to reply to everything and everyone (unless you’re  a self-styled social media guru, but that’s another matter altogether). Reporting a spammer is a pretty high overhead activity, especially with the current web interface. As far as I can tell, you can’t directly report a tweet, you have to click on the user image, click on the profile button and only then do you see the option to report spam. If I see a spam tweet, the easiest thing for me to do is shrug my shoulders and move on — it’ll be out of sight and out of mind in a while anyway.

Contrast this with email: I don’t reply to each and every email, but I do read most of it and I reply to a lot. Reading and answering emails (especially if they’re technical in nature) are fairly high load activities and my inbox definitely isn’t as ephemeral as my Twitter stream. Consequently, getting spam in my email inbox is correspondingly more annoying, especially if it’s more sophisticated spam (like a phishing attempt). It’s in my own best interests to report spam so that I have less to deal with in the future. Furthermore Gmail (and I would hope, most email interfaces) make it easy to report spam — generally just one click (two if you need to open a menu first, sometimes there’s a confirmation request).

(Aside: Note that Twitter associates “spam-ness” with the user, while email conventions associate it with the email itself. Objectively the two are isomorphic: the sender of spam message is very likely a spammer and a spammer account very likely sends out mostly spam. I find the difference is interface and convention interesting from an HCI point of view.)

An argument could be made that reporting spam is the users’ best interest, irrespective of quality of interface and immediate overhead. But I wonder if even this argument doesn’t quite work in Twitter’s case. Twitter is, and has always been, a largely public medium. Private accounts exist and you can block followers, but by and large you use Twitter to share everything with everyone. Email by contrast has always been more personal and private. But this means that reporting spam on Twitter is a lot like picking up litter off the street — it is bothersome, and you know that at some level you should do it, but is it really your problem? What’s really in it for you, and if you don’t do it someone else surely will, right? By contrast, spam email is pretty much exactly  like getting junk mail delivered right to your home.

While Twitter’s image as a public space has probably been key to its success, I do believe that it’s hard to get people interested in taking care of something they don’t have any ownership over (as far as I can tell the only way to pay for Twitter is to buy ads and it’s becoming increasingly harder to hook up non-official clients). To its credit, I haven’t found spam to be much of a problem — most days I don’t see (or at least don’t notice) any spam at all. Whatever they’re doing, it seems to be working. I don’t have any suggestions, just the above ruminations. But if you’re in the process of creating (or interacting with) any sort of user platform, it’s worth spending some time thinking about how to deal with spam.

Starting with Inbox Zero

As part of joining Cornell I have had a shiny new Cornell email address for a few weeks now. Cornell uses Gmail for its email service (along with Calendar, Docs, Sites, Groups and a few more). I’m not sure if I entirely approve with Universities and large organizations using Gmail for their mail, I do love Gmail as a mail client and interface.

Previously, Lafayette used Zimbra and I would forward all my Lafayette email to my personal Gmail accont so that I could use the Gmail interface. While it meant that all my email was in one place, it also meant that it was a bit of a mess. I’d regularly end up sending relatives email from my school account and professors email from my personal account even though I wanted to keep them separate.

Since my Cornell account gets sa Gmail interface, I’m going to keep it separate from my personal email completely. And since it’s a brand new, empty inbox I’m going to try to keep it to Inbox Zero. The basic of idea of Inbox Zero is that you use a combination of automated filtering and quick, decisive action to stay on top of email. By having all email get sorted automatically and manually processing only the things that need human intervention you can end each day with a clean inbox (a state of Inbox Zero).

I don’t really have a problem with email because I barely get a dozen emails a day (even on the busiest of days). However I do have problems replying to emails. Sometimes I’ll let things sit for days on end, lose them in the pile of things sitting (even though everything else has been read) and eventually follow up days later much to the annoyance of the sender.

The tenets of my Inbox Zero mission for my new inbox are simple. For starters, my email address is precious. It only goes out for reasons that are directly related to me being a graduate student at Cornell. As much as I can, I’m going to avoid using it to sign up for services. I’m not going to provide it as an alternate for any of my existing email accounts. How much will this work? I don’t know, but I’m hoping it’ll keep out at least some of the useless not-quite-but-almost spam that I get for my personal account.

Next step is automated processing. Whatever can be automatically filed away using Gmail’s filters and labels gets filed away, within limits. I already have labels for 2 project mailing lists and one for automated mail from the University. What I keep in mind while I set up these filters is that I don’t want to splinter my inbox into a dozen different places. Whatever doesn’t get into my inbox I can deal with later. Later here is defined as: I’ll look at it at some point in the next 24 hours and respond to it if necessary. The email that does make it to my inbox is stuff that requires my attention quickly or needs to spawn a filter so that similar stuff doesn’t come back to my inbox. Once I have read and responded to (or filtered) all incoming email, everything gets archived. Since everything is labeled I can use the labels to come back to something later (or just use the powerful search tools).

The final piece of the puzzle uses Gmail’s support for multiple inboxes. Besides the normal inbox I have two more. One is for high priority mail that hasn’t been filed away (using Gmail’s automated Important tag which seems to work pretty well for me). The other is for email that I have to respond to, marked with a blue star. This leaves the normal inbox to contain only unread email that is not high-priority. This way I can see important things that I should look at right now, things that I need to respond to as well as unread, probably unimportant things that might need to be filtered.

This is the same organizational system I’ve been using with my personal inbox which I wrote about a few months ago. However, the fatal flaw was that I didn’t actually clear out my inbox, I just made sure I read and marked everthing with some label. Unfortunately a “clean” inbox doesn’t have the same psychological effect as Inbox Zero. It’s easy to lose stuff in a sea of “read”. So this time around no compromises: Zero or bust.

The real test of my resolve (and system) won’t come until things start gearing up again in late August. The one tweak that I will need to make is when I check my mail. Right now I leave the tab open all day, but I think that’ll be far too distracting if I’m going to get email every few minutes. Ill probably adopt some kind of “check every few hours” policy, but what hours remain to be seen.

As I’ve been writing this I’ve been thinking that I should go back and revisit the system I use for my personal mail. I need to clear the inbox all the way down to zero for it to work, but that pile of 11000+ mostly answered email doesn’t look too appealing. Maybe someday in the not too distant future. But I’ve learned some lessons (I think) and it’s good to be able to make a clean start.

My Gmail power user workflow

My email has started to consume me. The amount I get has slowly risen over the last few years and a lot of what I get s not spam, but not important either. This means that I’ve been devoting increasing amounts of time to processing my email or letting things pile up and hence mixing the important and the unimportant It makes it harder to see at a glance and later find what the actually useful things. Psychologically I also worry about the number of unread emails in my inbox, even though most are relatively unimportant. I’ve also become rather attached to my email — leaving it open all the time and compulsively checking it whenever anything comes up.

A few days ago I decided that enough was enough and I was going to do something about the emotional and intellectual energy and time sink that my email was becoming. Looking around the internet I came across both Inbox Zero my Merlin Mann as well as an article by Christine Moran about her Gmail power user workflow. I’ve modeled my workflow on Christine’s with a few additions and tweaks.

The Concepts

I’ve come up with a few guiding concepts that help shape how my system is implemented and maintained.

  1. I want to focus on “important” email — things that I need to respond to or that contain vital information.
  2. Anything that is not “important” by the above definition must get filtered away and sorted automatically for possible future reference.
  3. At any moment I should be able to easily lookup email that I need to respond to without always performing searches.
  4. I should be able to check email only a few times a day, ideally once in the morning, afternoon, evening and night. An exception is made for my iPod which I can use to keep tabs on things that are really important but not for anything else.

The Workflow

Gmail is my mail client of choice, mainly because I change computers a lot and would like to get to my email from any machine connected to the Web. I make use of Gmail’s filters, labels and multiple inboxes to automatically sort email and present only the most important ones.

Filters and labels make up the “backend” of my system. Filters attach subject-specific labels to most of my incoming email. I can generally sort by sender, though sometimes I need to filter by content. I call these my importance filters. The labels are fairly fine-tuned and with one level of nesting (for now). As I get involved in more things I add more labels and the old ones get hidden and stay out of sight. I also use Gmail’s “Important” tag for the priority inbox, though not by itself.

The filters are designed to make sure that the only email that actually makes it to my inbox is stuff I need to read or respond to. Anything that gets past the filters but is not important enough generates a new one for that and similar cases. Everything else is labeled and sent to the Archive. Some of them I will just ignore completely (Netflix sent/received notifications) and some I go through on a weekly basis to see if there is anything interesting (ACM and IEEE newsletters).

While the labeling is mostly automatic, I use stars to manually mark emails that I need to respond to. This is done once I’ve actually read the email. I don’t have any plans for making this automatic anytime soon. This could be done with labels, but the stars are easier to see in the Gmail interface.

The frontend of the system makes use of Gmail’s multiple inboxes. Besides the default inbox I have two more. The first one shows me unread email that has successfully gotten through my importance filters. The second inbox is for starred mail that I need to respond to. The default inbox for now captures stuff that isn’t of high importance but isn’t getting filtered out automatically. As my filtering and labeling improves, it should become an “already read” section. This is different from Merlin Mann’s Inbox Zero in that the inbox is a generic holding area for read items. The actual “inbox” for all intents and purposes is the filtered unread-important list.

Ending thoughts

I’ve only been using the system for a few days, but it’s already making a good dent in the number of things that pile up. The greatest success in removing the not-spam but not-important stuff that I get a lot of every day. The final piece of the puzzle is just checking less often and not leaving Gmail open all the time. That’s something I’ll be working over the next few weeks. That’s more habitual but the fact that I’m getting fewer pings for inessential incoming stuff should help.

Though I’ve used Gmail it should be possible to implement a similar system on any client that supports filtering, tagging and custom views. I’d love to hear what measures you take to manage your influx of mail.

Google Wave has already spoiled me

Google Wave is ostensibly supposed to be what email would look like if it were invented today. Personally, I’m finding out that to be quite the case. Case in point: A few days ago I got an email from another student wanting some information on one of my previous summer projects. I promptly forgot all about it until last evening when my professor (who had supervised the project) emailed me and one of my collaborators and politely told us to reply to the original email. I went back to the original email, replied and CC’d my professor. While I was doing this I was thinking about how much simpler it would be if we were all on Wave.

Let’s look at all the different communications that took place as this happened:

  1. The new student emailed me and my collaborator separately asking if we would get him up to speed. He may also have emailed my professor separately. That’s 2, maybe 3 emails that went out.
  2. Both me and my collaborator neglected to reply, causing my professor to send out one more email (the same one to both of us).
  3. I replied to the original email and CC’d it to my professor. He now has one more email.

In the end, we each have between 2 and 4 emails in our inboxes, in at least two separate threads. Gmail with its conversation view makes things  a bit cleaner to handle, but Wave would make it easier still. Plus there is the human element. I didn’t CC my collaborator on the reply to the original email and probably won’t know what arrangements he makes with the new student. The student might end up meeting separately with both of us when it would make sense to meet all together.

In Wave, the new student would create a new Wave and add himself, my professor, my collaborator and me to it. We would then forget about it until my professor replied on the Wave which floated it back up to the top of the Inbox. I would post my reply and so would my collaborator. We would all have an idea of how we were going to meet and get each other up to speed. There would be no problem of “forgetting to CC”.

I’m not meaning to say that Wave is the solution to all problems. And a lot of the times the problem with new technologies like this is social, not technological. In the previous example, we wouldn’t really have needed Wave if the student had sent out a single email to all of us and my professor had just replied to that to remind us. In that case, Gmail with its conversation view (or any other client with threading) would have been sufficient. On the other hand, Wave would also have failed if there had been multiple waves made or if my professor wasn’t included on the original Wave. But even there Wave would have had a slight advantage because of how easy it is to add another person to an existing Wave – much easier than CC-ing someone on an entire email conversation.

I should say that a lot of the functionality I mentioned above could be handled by any modern wiki software. Instead of using a Wave for the discussion, use a wiki and send people the URL for it. Admittedly, there are some administrative questions (how does hosting and permissions of the wiki work) but the essence is the same: instead of sending packages around, have a wall where everyone posts.

But the similarity in strengths between Wave and modern wikis also highlights what I consider the biggest challenge to Wave’s popularity. When we used wikis for my Software Engineering class, we would have keep all our team information in the wiki (and fairly up to date when got used to it) but whenever anyone made a change, we would send out an email to everyone else asking them to look it over. Wave will be useless if I have to send out an email to people telling them to check their Waves. It will become just one more thing that I have to devote mental bandwidth to instead of focusing on my job. Need I reiterate the fact that this is again a social, not technological problem? Facebook has managed to beat this (at least among my fellow college students). We check Facebook out of habit often enough that we don’t need email reminders telling us to go check Facebook. Twitter also seems to have beaten the social inertia, but I suspect that the brevity of it’s messages changes the game somewhat.

It’s still too early to tell whether Wave will succeed in its professed goal of being what email would be if it were invented today. I like the way it’s going but still not sure if I’m ready to give up email for it any time soon. There are still some organixational issues that need to be solved. In particular long conversations in one Wave can be hard to comprehend especially if there are multiple discussions taking place in the same Wave. It’ll also be interesting to see if other solutions like email or wikis start to learn some lessons from Wave and start incorporating its features. Google Wave has a lot of potential and it’ll be intersting to see how it changes the web and the way we communicate on i.

Would you give up email?

A little over a week ago personal development blogger Leo Babauta of Zen Habits announced that he was basically giving out email. While he certainly had his reasons and alternative communication tools to use, it seems to me that giving up email is one of the more drastic ways to cut down on information overload. It’s not a decision to be taken lightly in todays world and I think that it’s a choice that could be hard to stick by.

Personally I think it would be almost impossible for me to give up email. I would much rather give up other communication tools like Facebook or Twitter before even considering ditching email. The main reason why I would be really reluctant to give up on email is that its such a flexible medium and I’m a sucker for flexible tools. Email works just as well for short, quick “how are you” messages to longer detailed conversations (especially if you have a good client like Gmail). Sending attachments may not be the best way to collaborate online, but it’s possible and for simply sharing a file (without editing) it’s generally much easier to add an attachment than it is to upload to FTP or something similar.

The second most wonderful thing about email is that it has become a completely open medium. There are dozens of free email services and if need be you can even run your own email server on you own domain and be completely independent. This is far from the case with popular email replacements like Twitter or Facebook. However, there is an open source microblogging platform called Laconi.ca which you can run on your own server. It powers the free indenti.ca service.  I feel really uncomfortable about tying my major communication arteries to closed proprietary services which could go down (with or without the intent of the developers). Incidentally, as I write this, I can’t get to twitter or connect via Twitterfox.

Edit: Shortly after I wrote the above it turned out that Twitter had been down for several hours due to a DDoS attack. While it’s true that this could happen to any service (including webmail), I think this shows that Twitter at least isn’t quite ready to be the primary communications channel

The more I think about it, the more it seems that the reasons behind using or ditching email are less technological and more psychological and social. A lot depends on your own communication preferences. An important question is: do you want a small volume of in-depth conversations or do you want a high volume of short-length communications? For someone like Leo Babauta who has thousands of leaders and needs to keep in touch with a large community it makes sense to choose something that will encourage lots of short, to-the-point comments (ie. 140 characters on Twitter). For longer communications he says he wants to use either IM or Skype. This seems to mean that he wants to focus on realtime, direct communication. Again that’s fine for him.

My communication preferences are different. I don’t need to keep up a multi-thousand-way conversation and I get less than a dozen emails a day. It’s more important for me to have in-depth detailed communications with a few people whose ideas are important to me at this time. I also like the asynchronous nature of email since I like to be able to think things through before giving a reaction. I dislike real-time communication like IM and especially phones unless I’ve consciously signed up for a fast paced, brainstorming style discussion with a friend. I also value the ability to be explain my ideas in depth if I need to. Twitter’s 140 character limit is good for posting updates and trading short replies, but I feel it’s really limiting for any sort of detailed discussion (and splitting across multiple tweets just makes things ugly). Leo claims that he checks his Twitter inbox fewer times a day and spends less time on it. He doesn’t keep it always on. For me, I keep Twitter always on and glance at it about once an hour just to see if anything interesting has come. It takes up little of my mental RAM whereas my email is something I consciously spend more time and energy on.

Now this isn’t to say that email doesn’t have its share of problems. Spam is certainly a problem, but I think you’d get that on any communication medium. Email certainly does not work if you want your communications to scale to more than a few people. Sending out an email to lots of people about something they aren’t specifically interested in is often a good way of making sure they don’t read it. If you need to write something that gets read by a lot of people, then a blog, wiki or some other more open platform is definitely the best way to go. Or maybe twitter if what you have to say is short enough.

If you’re currently having problems handing your email, what you need to understand most is that no single tool is good for all people or all tasks. The question of whether or not you should ditch email depends almost entirely on how you as a person work. It depends on whether you need to have public conversations with many or private conversations with not so many. It depends on whether you want to have your replies done as soon as you can or do you prefer to mull over things before putting finger to key. It’s important to keep in mind that no matter what tool or system you use, you’re going to have an inbox and a message queue that you need to manage and respond to. Email is currently the best way for me to maintain a persistent, workable message queue, but it may very well be different for you. To each his or her own.