Hello!
Just wanted to ask what setup you use or recommend for large scale sites (~10k users, some hundreds of imap(s), smtp, webmail connections, about 5-10 mails incoming and outgoing per second):
- what smtpd? why? - what mail retrieval solution for which protocol? - if you offer webmail, what do you use? - it should be *nix software, preferable FOSS (free and open source software) - Which Antispam / Antivirus "protection" do you use? And what is the success rate?
I'm personally deep into qmail+patches+patches+patches of patches plus courier and dovecot.
The problem with qmail is that you need either a big patchset or a once patched setup and reuse that. Plus qmail really has some not-so-nice bugs.
On the other hand, there seems to be no MTA out there, that can be configured so easy like qmail (ever created a mail addr with "cp .qmail-template .qmail-domain-user" with another MTA?).
Oh, btw, "froehliche Weihnachten"
Nico
Nico -telmich- Schottelius wrote:
Hello!
Just wanted to ask what setup you use or recommend for large scale sites (~10k users, some hundreds of imap(s), smtp, webmail connections, about 5-10 mails incoming and outgoing per second):
- what smtpd? why?
- what mail retrieval solution for which protocol?
- if you offer webmail, what do you use?
- it should be *nix software, preferable FOSS (free and open source software)
- Which Antispam / Antivirus "protection" do you use? And what is the success rate?
I'm personally deep into qmail+patches+patches+patches of patches plus courier and dovecot.
The problem with qmail is that you need either a big patchset or a once patched setup and reuse that. Plus qmail really has some not-so-nice bugs.
On the other hand, there seems to be no MTA out there, that can be configured so easy like qmail (ever created a mail addr with "cp .qmail-template .qmail-domain-user" with another MTA?).
That all depends on what you are used to.
The mini-isp setup: - Debian unstable - nsd or pdns as authoritive dns - pdns as a recursing dns - postfix for SMTP + optional STARTTLS + Submission - amavis + clamav & Spamassassin using milter inline in postfix - http://saupdates.openprotect.com/ to update SA. - dovecot for IMAP-SSL - squirrelmail or Horde IMP as a webmail frontend or when living on the alpha side: hula (hula-project.org)
To make it all configureable one can go for a MySQL backend or write scripts which can generate the configs. MySQL (or other SQL backend) is prolly the way to go though.
Yet again, it all depends on what you need and what you are used to...
Greets, Jeroen
Jeroen Massar [Sun, Dec 24, 2006 at 02:18:18PM +0000]:
That all depends on what you are used to.
The mini-isp setup:
- Debian unstable
unstable? for production system?
- nsd or pdns as authoritive dns
- pdns as a recursing dns
dns was not really questioned, but I would prefer djbdns (+patches, again) or bind.
- postfix for SMTP + optional STARTTLS + Submission
Submission does what task?
- amavis + clamav & Spamassassin using milter inline in postfix
Seem both to be just 'the standard antivir and antispam' solution
- http://saupdates.openprotect.com/ to update SA.
nice, did not know that.
- dovecot for IMAP-SSL
yep, sounds for me as the best solution. I remember having some problems with courier and performance with Maildirs
= 100k Mails (my inbox).
- squirrelmail or Horde IMP as a webmail frontend or when living on the alpha side: hula (hula-project.org)
Will have a look at hula; squirrel and horde are also both quite 'standard' in their area.
To make it all configureable one can go for a MySQL backend or write scripts which can generate the configs. MySQL (or other SQL backend) is prolly the way to go though.
Don't know whether I really like the idea being dependent of MySQL as backend, but it is at least one possibility to do central administration.
Yet again, it all depends on what you need and what you are used to...
Well, it does not really depend on what I am used to for me (anymore).
I am currently really thinking of giving postfix a try, but it somehow "feels" like stepping backwards.
I am really interested in some experiences of postfix admins and how they handle virtual domains and users.
Nico
I am currently really thinking of giving postfix a try, but it somehow "feels" like stepping backwards.
I am really interested in some experiences of postfix admins and how they handle virtual domains and users.
Postfix is great, efficient and easy to use. Also there are lots of plugins developed for it. For 10k mailboxes it will run smoothly. Back in 1998, we had a postfix server running more than 50k accounts on an old Sun Ultra server, and it was not causing much trouble. Well, the users were of course mostly dialup ones, and spammers weren't that active that time...
Comparing to qmail, I'd advice to go for postfix.
Nico -telmich- Schottelius wrote:
Jeroen Massar [Sun, Dec 24, 2006 at 02:18:18PM +0000]:
That all depends on what you are used to.
The mini-isp setup:
- Debian unstable
unstable? for production system?
Yes, why not? Unstable == newest of the newest. The only thing that can break is when you upgrade packages, thus only upgrade when you need to. Aka install debsecan and only upgrade for security issues. I only had 1 minor problem with it this year and that was an libattr upgrade which broke ls ;)
- nsd or pdns as authoritive dns
- pdns as a recursing dns
dns was not really questioned, but I would prefer djbdns (+patches, again) or bind.
Patches, patches, patches. Bind9 is fine (and actually what I usually use) but pdns&nds are simply faster, thus for scalability I would go for those, then again it depends on ones needs.
- postfix for SMTP + optional STARTTLS + Submission
Submission does what task?
Port 587. SMTP Message Submission - http://www.ietf.org/rfc/rfc2476.txt
The way that your users send mail to the rest of the world. Just in case, they are not supposed to use port 25, that is for server<->server message transfers. This also allows you to easily redirect port 25 outbound to a greylister/ratelimit if you want as they should never be using it. Of course that should be an option that one can easily turn of, some people love to run their own SMTP too ;)
- amavis + clamav & Spamassassin using milter inline in postfix
Seem both to be just 'the standard antivir and antispam' solution
There is afaik nothing better, especially in combo with:
- http://saupdates.openprotect.com/ to update SA.
nice, did not know that.
That thing ;) Which is indeed pretty sweet.
- dovecot for IMAP-SSL
yep, sounds for me as the best solution. I remember having some problems with courier and performance with Maildirs
= 100k Mails (my inbox).
Well cras (ehm Timo Sirainen ;) is simply an extremely talented and good coder thus I definitely trust his stuff.
To make it all configureable one can go for a MySQL backend or write scripts which can generate the configs. MySQL (or other SQL backend) is prolly the way to go though.
Don't know whether I really like the idea being dependent of MySQL as backend, but it is at least one possibility to do central administration.
Otherwise go for postgresql if you want a technically better DB. LDAP is always also an option but it doesn't provide the flexibility that SQL gives, especially for storing all kinds of unrelated data. Having everything (including accounting etc) in the same database with one single data model makes management a lot easier and one can also do weird queries and do statistics making $management happy.
Yet again, it all depends on what you need and what you are used to...
Well, it does not really depend on what I am used to for me (anymore).
I am currently really thinking of giving postfix a try, but it somehow "feels" like stepping backwards.
postfix is IMHO at the moment the best MTA there is.
I am really interested in some experiences of postfix admins and how they handle virtual domains and users.
"vi /etc/postfix/virtual/" but you can stick *all* of that in SQL.
There are loads of HOWTO's on how to do it: http://wiki.dovecot.org/DovecotLDAPostfixAdminMySQL http://www.howtoforge.com/virtual_postfix_mysql_quota_courier
Cool thing about Dovecot&Postfix btw: http://www.postfix.org/SASL_README.html#server_dovecot Aka you can let dovecot authenticate SMTP AUTH for submission
Or if you really want the truly virtual and complete doc: http://postfix.pentachron.net/ :)
Happy playing over X-mas with all the new toys everybody is getting :)
Greets, Jeroen
Jeroen Massar wrote:
dns was not really questioned, but I would prefer djbdns (+patches, again) or bind.
Patches, patches, patches. Bind9 is fine (and actually what I usually use) but pdns&nds are simply faster, thus for scalability I would go for those, then again it depends on ones needs.
Given the potentially high DNS traffic (all those xBL lookups), a dedicated caching DNS resolver may make sense. Additionally, you should consider running a local rbldnsd for mirrored zones (proxying from the resolver to rbldnsd).
- amavis + clamav & Spamassassin using milter inline in postfix
Seem both to be just 'the standard antivir and antispam' solution
There is afaik nothing better, especially in combo with:
Detection rates of ClamAV are pretty low. If you want to advertise "virus protection" as a feature, you may want to integrate at least one additional scanner.
-- Matthias
On Sunday 24 December 2006 15:01, Nico -telmich- Schottelius wrote:
I'm personally deep into qmail+patches+patches+patches of patches plus courier and dovecot.
The problem with qmail is that you need either a big patchset or a once patched setup and reuse that. Plus qmail really has some not-so-nice bugs.
We use a qmail-ldap [1] setup (qmail + single patchset) for years with a similar workload and it works quite fine. However as you already said, there are some annoyances with qmail which are not that nice... one of the biggest drawbacks is that it's not actively maintained anymore and you have to either use alternative components if you need a new specific feature (like smtp daemon with support for filtering, greylisting or whatever) or manually adjust patches because usually the different patches are not interoperable. However, many annoyances are already addressed in the qmail-ldap patchset and because qmail is modular it's not a big deal to replace parts which you don't like.
What you have to consider when using qmail is that the disk i/o performance is far more critical than for other MTA's - fast disks are a must, at least for the queue partition.
Regards, Matthias
we've been operating ISPMan+openldap+postfix+Cyrus+horde, but then moved our services to Plesk. Having looked back, I'd still go again with Plesk. It saves a lot of time, and normally a sysadmin's working time is more expensive than the software license.
It has its limitations and unconveniences, but in general it's a great product.
A strong advice is to put a greylisting server in front of it on a separate machine (might be a virtual machine).
Besides, I studied together in the same University with Sw-Soft's core team ;-)
regards, stan
--- Nico -telmich- Schottelius nico-swinog@schottelius.org wrote:
Hello!
Just wanted to ask what setup you use or recommend for large scale sites (~10k users, some hundreds of imap(s), smtp, webmail connections, about 5-10 mails incoming and outgoing per second):
Nico,
For domain toaster of that size, qmail-ldap is a nice solution, but software seems to be orphaned now, as soon as some major SMTP enhancements come in I am afraid qmail will slowly die out.
As some have already mentioned, the Maildir format used by Qmail is suboptimal for handling large boxes, expecially with IMAP, I would strongly advise to use IMAP daemon which features caching mechanisms, such as Cyrus IMAP. Cyrus is great plus you get Sieve server side filtering but it is a challenge to get it working with Qmail, especially with virtual boxes.
If you go with some non-caching IMAP daemon like courier, then it might make sence to setup IMAP Proxy too which would keep alife the IMAP connections for let`s say 2 min and reuse it for same user, so new IMAP processes do not have to be spawned and mailboxes rescanned every time web mail refreshes. Of course this hogs memory but is useful sometimes, especially if users have thousands of mails.
Frontend: Squirrelmail. By far most powerful web based MUA out there IMHO. Also very easy to ehnance, we wrote a LDAP control panel to it for example, so users can setup aliases, forwarders and away messages themselves. (Contact me offlist if you want it, I can send it to you).
Cheers
Kirill
On Mon, Dec 25, 2006 at 06:33:45AM +0100, Kirill Ponazdyr wrote:
Nico,
For domain toaster of that size, qmail-ldap is a nice solution, but software seems to be orphaned now, as soon as some major SMTP enhancements come in I am afraid qmail will slowly die out.
qmail-ldap is not orphaned, I just don't have enough useful stuff together to make a new release. Btw. I don't see any major SMTP enhancements on the horizon.
As some have already mentioned, the Maildir format used by Qmail is suboptimal for handling large boxes, expecially with IMAP, I would strongly advise to use IMAP daemon which features caching mechanisms, such as Cyrus IMAP. Cyrus is great plus you get Sieve server side filtering but it is a challenge to get it working with Qmail, especially with virtual boxes.
Any format has pro and cons. Maildir is extremly robust. I have some large mailboxes and IMO it is not very slow. You need to make sure that the IMAP client smart enough and does not open every file in the Maildir. Caching is nice (mutt-hcache rocks) but if you access your Maildirs via different protocols you normaly end up with a total mess.
If you go with some non-caching IMAP daemon like courier, then it might make sence to setup IMAP Proxy too which would keep alife the IMAP connections for let`s say 2 min and reuse it for same user, so new IMAP processes do not have to be spawned and mailboxes rescanned every time web mail refreshes. Of course this hogs memory but is useful sometimes, especially if users have thousands of mails.
Frontend: Squirrelmail. By far most powerful web based MUA out there IMHO. Also very easy to ehnance, we wrote a LDAP control panel to it for example, so users can setup aliases, forwarders and away messages themselves. (Contact me offlist if you want it, I can send it to you).
I miss a powerful search option in most webfrontends. That's the only good thing of gmail.
* on the Sun, Dec 24, 2006 at 03:01:04PM +0100, Nico -telmich- Schottelius wrote:
Just wanted to ask what setup you use or recommend for large scale sites (~10k users, some hundreds of imap(s), smtp, webmail connections, about 5-10 mails incoming and outgoing per second):
- what smtpd? why?
postfix, exim or qmail -- in that order of preference. I would very much advise against anything else, even for small or medium mailservers (and for goddess sake, don't use any mickey-mouse solution, unless you like the pain, like of being repeatedly kicked into the crotch).
And I'd use Debian "Etch", which is right now in testing, but will soon be stable. But there's loads of software available which which is lacking or outdated in Debian "Sarge".
Cheers Seegras
-
Claudio Jeker -
Jeroen Massar -
Kirill Ponazdyr -
Matthias Blaser -
Matthias Leisi -
Nico -telmich- Schottelius -
Peter Keel -
Stanislav Sinyagin