why I use gmail (or, the list of daily worries of a self-hoster)

Posted on by Luis Villa

In class Thursday, during a discussion of privacy and security, Prof. Moglen asked me how I do email; I told him gmail. I was going to write a long post explaining why (which will probably form part of an essay in the near future) but Jesse nails a fair number of them in one sentence:

Now, I no longer have to think about keeping spam stuff up to date, no longer have to worry about that next security vuln …, no longer have to worry about having a decent interface for getting mail from mobile devices, etc…

I’d add no longer have to worry about storage space (at least not for future emails); not have to worry about data backup; not have to worry about hardware failure and reliability; not have to worry that I can’t leave (since I mostly ‘hide’ @gmail behind @tieguy.org).

Jesse and I are not alone in this- gmail is the most popular user-agent at gmane.org, and an lkml admin tells me that over half of current lkml subscribers are @gmail.com. (This bleg was unsuccessful in getting Debian data, but I imagine their numbers are similar.)

Prof. Moglen is right to worry about privacy and security, but for the vast, vast majority of us those are very irregular problems. If they have non-trivial impact, that impact is once or twice in a lifetime. The problems I’ve listed here are all daily problems with self-hosted email. (You can take steps to reduce some of the worry, but you still have to use your precious time to recover when things go wrong, and you have to do it on the hardware or network’s schedule, not yours.) Solving daily problems at the expense of once-in-a-lifetime problems is a tradeoff most people will happily make. So gmail and the like are winning, and will win for the foreseeable future, even amongst those like Jesse who are skilled in the fine arts of software maintenance.

This is principled software’s biggest challenge- not how to stay relevant in the face of google’s vast server farms (which are important but not insurmountable for many classes of service), but how to stay relevant in the face of how convenient centrally-hosted web software is for both users and developers.

[It doesn’t hurt that gmail is very nice software. The keyboard nav is very good, search is powerful, conversation view is the first real innovation in email in ages, archiving of IMs as emails is so blindingly obvious that I’m still shocked no other mail/IM pairs that I know of do it, and the intense scriptability (which is now officially supported) means I have more plugins for this than I used to have for evo. None of these were the things that pulled me away from evo, though- it was all about not having to have the responsibility of running my own server.]

Published by Luis Villa

33 thoughts on “why I use gmail (or, the list of daily worries of a self-hoster)”

  1. Luis Villa: why I use gmail (or, the list of daily worries of a self-hoster) – Nov 7, 2007 Jef Spaleta: Bracing for Impact….  – Nov 7, 2007 Fabian Affolter: *BSD Statistiken  – Nov 7, 2007 Christopher Blizzard: awesome, dennis!  – Nov 7, 2007

  3. Linkblogwhy I use gmail (or, the list of daily worries of a self-hoster)- Over half of LKML subscribers use gmail, wowProprietary Software Does Not Scale – Wow, good point. ‘The whole point about cloud computing is that it has to be effectively infinite – the more people want, the more they get. You can’t do that with

  4. Hey Luis,

    Well said. I switched from Evo (on Linux) and Thunderbird (on OS X) to Gmail back in April and I will never, ever go back to fat client email. I was hoping Hula could have been something, but you know.

    If open source advocates want to compete with things like Gmail, they need to create software of comparable functionality and usability (sorry SquirrelMail) and somehow fund the server farms necessary to provide them to users. I’m not sure how “open source” does this, but certainly there are corporate backers who could.

    Joe

  5. I never found any web mail interface comparable to a “real” client and I just can’t make myself trust all my mail to a free and beta service. I can kinda see how coming from POP, Gmail would be fantastic, but a good mail client and IMAP(yes and even SquirrelMail for the occasional webmail usage) is really hard to beat.

  6. I came from IMAP, for the record. Several gigs of it. POP sucks.

    I just can’t make myself trust all my mail to a free service.

    I’m not happy with that, but I can’t make myself trust all my mail to a server I administer either. I’m in the vast and (if you trust lkml/gmane stats) growing majority, I think.

  8. I’d guess that many are in the same boat I’m in. Had gmail offered IMAP when I started using gmail, I would probably have stuck with evo and used gmail as a ‘mere’ IMAP server. I definitely wasn’t going to use POP. But now that I’m used to gmail’s interface, I don’t see any reason to go back to any local mail, be it IMAP, POP, or any other local delivery mechanism. What is the benefit?

  9. I’m currently self-hosting a Zimbra installation and it is working quite well for me. I have access to IMAP, to the Zimbra calendaring software through their extension for Evolution, and a pretty good web-based email client.

    In their next iteration (5.0), they are adding a XMPP-based IM component to their software (http://www.igniterealtime.org/projects/openfire/) among other improvements.

    Zimbra is released under the Yahoo Public License.

  10. I loved Thunderbird, but switched to gmail 2.5 years ago to escape my crappy mail service provider (who still had the gall to charge me for 20 megs with POP but not IMAP). At first I just used gmail as a POP server for Thunderbird, but pretty quickly I found myself using gmail’s UI more and more.

    My wife switched because her IMAP service was really buggy and would crash Thunderbird. After that, we just stopped using Thunderbird. At first I would fire it up occasionally as a means to back up stuff from Google’s servers (for hypothetical offline access), but I gave up after never using it.

    Recently I decided to give gmail+IMAP+Thunderbird2 a try. TB2 is really great, but I just can’t see any benefit to using it over gmail. I generally have good network access, I don’t fly much, and I don’t really feel like setting up TB on each OS installation.

    Maybe one day I’ll set up my home server to quietly download my gmail messages “just in case”, but there’s just not a lot of urgency to it.

    Like Joe, I’m not sure what this all means for free software. Will a free service definition be enough for most people? Will free software developers be able to afford to deploy web apps competitively? Competitors with more money will be able to provide the same software with a better overall user experience due to better server infrastructure, so where is the incentive to open the source to your web app? Will Online Desktop help the cause by providing a free alternative, or hinder it by increasing our dependence on proprietary web services? And why did Hula fail?

  11. Somehow in that very long comment I forgot to mention something else that bothers me: won’t it be harder for Joe Hobbyist to download your free web software and tweak it to meet his needs? Running it on his local Apache instance or whatever, will he be able to access the data on your servers?

    Maybe this is just a technical hurdle.

  12. Emil: but you’re still self-hosting. How long did it take you to set up? What are you doing for redundant data backup? How often does it go down?

    Sandy: some very good questions.
    Competitors with more money will be able to provide the same software with a better overall user experience due to better server infrastructure, so where is the incentive to open the source to your web app?
    I’d guess the answer here is the same reason RMS started writing GNU- because you want to do the right thing. But I agree that it seems like commercial involvement would be difficult.
    why did Hula fail? If I had to guess, Novell politics, but I left before Hula was even announced publicly.
    will he be able to access the data on your servers? Stay tuned. :) I think data is as critical if not more so than source in the New Age.

  13. Luis, a few weeks ago weren’t you telling me that if people had symmetric broadband connections that a lot of them would want to run servers from their homes?

    Personally, I’m still using my alma mater’s email client. They let me keep it in the hopes I’ll give them money, and I have some hopes they’ll be moderately more resistant to doing evil things with it than a commercial company would. I’ve been using a fat client for a decade now, so it would take a major adjustment to get used to a web-based interface.

  14. Luis, a few weeks ago weren’t you telling me that if people had symmetric broadband connections that a lot of them would want to run servers from their homes?

    I was. I think at least some people would, and I think you could make an interesting business out of supplying those servers if the bandwidth story were more favorable. With the current bandwidth story we’ll never know.

    More relevantly to this discussion, I’d like to be wrong about this- I’d like there to be businesses which push back on these assumptions, and make servers (and redundant remote backup) as almost trivially easy as gmail, so that there is no worry in self-hosting. But, again, that will never happen if the bandwidth story sucks.

    [I’m also dropping some points here, because I’m trying to make this and the next post fit into a thousand or so words for reuse elsewhere.]

  15. I used to run Postfix / Dovecot on an OpenBSD box over dynamic DNS. I grew very tired of doing IT at home, especially when my wife would call me at work saying the mail is down and when can I fix it! Needless to say, we switched to GMail and later Gmail’s apps for your domain service (free). We haven’t looked back.

    Sean

  16. Luis:

    I took me at most two to three hours to install and setup using their guides. It works out of the box for most of my needs and if that doesn’t suffice I can make changes through their web-based administration GUI.

    Upgrades are handled through RPM and/or DEB packages that are provided by Zimbra for your specific distro and architecture. I have not had any problems with this process yet.

    My backups are handled on a hourly, daily, and weekly rsync process that takes the data, copies it to a backup server, and on a weekly basis writes it to a DVD for archiving purposes.

    The entire solution has been mostly problem-free for the last year except for situations that I caused myself. I would say that the Zimbra mail server has had a problem once and it took less than 30 minutes to fix — it’s availability has been more than outstanding.

  18. Luis. It seems like your main concern here is server administration and gmail provided that for free and you haven’t found another entity that also does free. Does that mean that if another entity provider IMAP server with loads of storage and full administration along with redundancy etc etc then you would consider it ‘equal’ to gmail. Perhaps using a fat client for access and a web interface for the time you do not have fat client access?

  19. Damien: mostly, yes. I’d even be willing to pay for it, probably. I’m not averse to paying for Freedom, but I’m in a very, very distinct minority there, and one that is getting smaller, I think, since no one is doing a very good job of articulating why Freedom matters in a service-oriented context.

  20. Saying “I don’t care about my privacy, I’ve got nothing to hide” is the first step toward dropping any kind of privacy.

    I like GMail, and I use it for all the mailing-lists I’m subscribed to, but I’ll never use it for a personal/professional use. In France, the CNRS (main research agency) prohibits the use of GMail in a professional context.

  21. In France, the CNRS (main research agency) prohibits the use of GMail in a professional context.
    Every sane business prohibits the use of gmail in a professional context.

    Saying “I don’t care about my privacy, I’ve got nothing to hide” is the first step toward dropping any kind of privacy.
    Sure, but unless you gpg encrypt all your email, you shouldn’t expect that your email is private anyway.

  22. “Sure, but unless you gpg encrypt all your email, you shouldn’t expect that your email is private anyway.”

    Well, in France you need the agreement of a judge to get an access to private emails. I wouldn’t bet it’s really difficult in the US. I’ve heard in August about a US law that makes that a lot easier…

  23. What does “every sane business prohibits the use of gmail in a professional context” mean?

    Is it because it is visibly a third party email address (which can be got around with Apps for your Domain), because it’s a a privacy risk (as is all email, as you say), because it’s a third party hosting your archive (a reasonably popular option for businesses, particularly at the smaller scale) or something else?

    If I was in charge of the vast majority of small to medium businesses, non-profits, charities etc. then I’d be transitioning my users to Google apps as a far for secure, performant and featureful alternative to the status quo (whether MSFT or OSS based, self-hosted or not).

  24. Hi Luis, interesting article but i think you make a mistake by comparing gmail with a “do it all yourself”-approach.

    There is also something in between. There are other mail sevices than google and if you have a homepage/domain you can host your emails there.

    For example i have a own domain at a (small german) service provider which uses 100% Free Software, take privacy and security really serious and even releases own software as Free Software.

    I trust this provider definitely more than Google.

    Let me try to give you an answer to this question:

    > I don’t see any reason to go back to any local mail, be it
    > IMAP, POP, or any other local delivery mechanism. What is
    > the benefit?

    If you use only a web interface you always have to be online to work on your mails. With a local client i can fetch all my mail (e.g. if i travel) than read the mails offline and write responses and the next time i have an internet connection i can send out my mails and fetch again new mails. For me that’s much more convenience than an online service.

    Also if you use a high adaptable mail client like mutt or gnus i’m sure that you can make with it much more than with any web mailer and you can really adapt it to your needs.

    So you gain flexibility both in handling your mails and in regard to how your mail client works if you use a local mail client instead of a web interface.

  25. “I’m not averse to paying for Freedom, but I’m in a very, very distinct minority there, and one that is getting smaller, I think, since no one is doing a very good job of articulating why Freedom matters in a service-oriented context.” – Luis

    I don’t think it’s necessarily a problem of articulation, so much as the economic problem implied in your remarks here. Free and OS Software has the inherent economic advantage over its proprietary rival that it is usually cheaper and, in some cases, much cheaper, libre and gratuit. Open Services, on the other hand, don’t enjoy that advantage. At best they are going to cost the user as much as their proprietary rivals, at worst more.

    Perhaps the point about your use of gmail is not that you are compromising your freedom, but that you are exchanging some freedoms in return for the benefits of free hosting etc. It’s a strange kind of social contract….

  26. One of the reasons I’ve switched to Gmail is, I can get out of it any time I want, taking away all my data out. Recently added, Gmail supports IMAP as well.

  27. Luis, you’ve taken a plunge I’m not ready to yet. I kind of backed myself into a corner, as well, because I said a few weeks before Gmail’s IMAP access was publicly announced that if they had it, I would switch. Now I’m in a state of cognitive dissonance – :-p I’m trying to find other reasons not to use Gmail. Being that I am notoriously paranoid and feel the need to self-host everything, I am still sticking to my own hosted solution. However, all your points are very valid, and I think you made the right decision for yourself, perhaps. My weights for the different values, though, are different. As it stands, my hosted IMAP solution is “Good Enough”, and I also am confident that, God-willing, in time, there will be FOSS solutions that will make Gmail less outstandingly spectacular, though not necessarily inferior, compared to a self-hosted FOSS solution.

  28. […] Luis Villa: “This is principled software�s biggest challenge- not how to stay relevant in the face of google�s vast server farms (which are important but not insurmountable for many classes of service), but how to stay relevant in the face of how convenient centrally-hosted web software is for both users and developers.” Tim O’Reilly: OpenSocial is pointless. A little slow on the uptake, but I’m glad to see reality setting in. […]

  29. “Competitors with more money will be able to provide the same software with a better overall user experience due to better server infrastructure, so where is the incentive to open the source to your web app?”

    The danger is actually that a competitor will be able to take advantage of all your hard work, add some proprietary features, and *then* deploy on superior infrastructure.

    This is known as the ‘web app loophole’. It can be closed by using a license such as the Affero GPL, at which point commercial (but nonproprietary) forks are at the usual disadvantage vis-a-vis the original reciprocally-licensed project: no community.

    When AGPL3 is finalized, I think we’ll see a lot of new web app (and web app infrastructure) projects start in this space.

  31. Sandy, on reason to use gmail+IMAP+Thunderbird2 over gmail-through-the-web is it let’s you ditch google’s cookie from following your habits across other google web service (blogger, youtube).

Comments are closed.