Webserver howto

From Wiki@comjat.net

Jump to: navigation, search

This article describes some actions done in order to set up a new web server. Tasks like this will not be done too often by you, though every switch from one server to another might force you to do so.

This article will get more complete the next time(s) I move to another web server and got to reinstall / move my sites and mailboxes on to that new server.

First of all I followed the instructions found in the article Debian Mailserver though adjustments were necessary to get my configuration running. This article describes the configuration and changes which finally worked for me. Upon next installion I will remove reference to other articles and replace the missing parts with my proven (previous) configuration.

Contents

[edit] Preconditions

  • Linux Box
  • Debian 4.0 (Etch)

[edit] Result

  • Apache 2
  • PHP 5
  • Cyrus Mail

[edit] Apt basics

Apt manages your software packages. We will come acress apt quite a few times in this article. Therefore here's a cheat sheet for some handy commands keeping your Linux box up to date.

[edit] apt-get

  • Useful switches for 'apt-get
    • -s: No action; perform a simulation of events that would occur but do not actually change the system. ... Simulate prints out a series of lines each one representing a dpkg operation, Configure (Conf), Remove (Remv), Unpack (Inst). Square brackets indicate broken packages with and empty set of square brackets meaning breaks that are of no consequence (rare).
    • -y: Assume Yes to all queries and do not prompt
    • -u: Show a list of upgraded packages as well
    • update: Retrieve new lists of packages
    • dist-upgrade: Distribution upgrade

[edit] Package update

In order for apt to know of the packages you can install, you got to get the most cirrent list of packages from the Debian repositories: 

>apt-get update

[edit] Distribution update

To update your Debian distribution you type this: 

>apt-get -u dist-upgrade

[edit] Create users

First you should create some users who need shell access to your Linux box: 

>useradd -m -g users <username>
  • Useful switches for 'useradd:
    • -g initial group: Force use initial group for the new user account. Group name or number of the user's primary group.
    • -G grp1,grp2,...: A list of additional groups, which the user will be made a member of.
    • -d homedir: The user's home directory. Defaults to "/home/${username}".
    • -m: This option will cause the user's home directory to be created if it does not already exist. Any files and directories which exist in /etc/skel will also be copied into the user's home directory.

[edit] Locales

Users are assigned to locales, software on your server relies on locales. To avoid problems we now set up the locales for your Linux box.

Install locale support: 

>apt-get install locales language-env

Configire your locales: 

>dpkg-reconfigure locales

Each user who needs an own Locales runns this command: 

>set-language-env

[edit] Web server

To get the apache 2 web server running, some packages got to be fetched and installed: 

>apt-get -y install apache2

Now for php 5 this is the right command: 

>apt-get -y install libapache2-mod-php5 php5-cli php5-common php5-cgi

[edit] Database

To install MySQL: 

>apt-get install mysql-common mysql-server mysql-server-5.0


Secure your MySQL installation: 

>mysqladmin -u root -p YourNewPassword

Manage MySQL with PhpMyAdmin: 

>apt-get install -y phpmyadmin

[edit] Cyrus

To install Cyrus do it like this: 

>apt-get install -y cyrus-imapd-2.2

I followed the instructions as found in the chapters on Cyrus with SASL and Cyrus of the article on setting up a Debian Mailserver.

[edit] imapd.conf

Edit the following file got in order to enable dots in mail addresses such as john.mueller@domain.tld: 

/etc/imapd.conf

Set the following to yes in order to enable dots in email addresses: 

...
unixhierarchysep: yes
...

[edit] Web-Cyradm

To get the Web-Cyradm web frontend, get the newest archive such as: 

>wget http://www.web-cyradm.org/web-cyradm-svn-0.5.5.tar.gz

...and install it in a folder accessible by your web server. Don't make a symlink, as this seems to cause some infinte loops for the web server (didn't investigate it any further).

To get the web frontend working properly I had to use the newest SVN Snapshot instead of the Latest release. Adding new accounts else might cause problems.

[edit] conf.php

In addition to the changes you made to the file imapd.conf the following changes got to be applied to enable dots in mail addresses such as john.mueller@domain.tld.

Copy the file: 

/var/www/web-cyradm/config/conf.php.dist

... to: 

/var/www/web-cyradm/config/conf.php

... and configure it as described in the file itself. In addition to enable dots in email addresses, set the following to 1: 

...
DOMAIN_AS_PREFIX = 1;
...

[edit] Spamassassin

SpamAssassin is used to shield your mailboxes from to much spam mail. I followed the instructions as found in the chapter on SpamAssassin of the article on setting up a Debian Mailserver applying the below mentioned changes to get it run with my configuration. First install it: 

>apt-get install spamassassin

[edit] spamassassin

As SpamAssassin is invoked by postfix, it tries to run in nobody's home folder. This will cause an error (see [1] for mor information). To cause SpamAssassin to run as user nobody edit the following file: 

/etc/default/spamassassin

Adjust the section with the options: 

OPTIONS="--create-prefs --max-children 5 --helper-home-dir -D -x -u nobody --virtual-config-dir=/srv/spamassassin"

Create the folder /srv/spamassassin with user nobody and group nogroup: 

>mkdir /srv/spamassassin
>chmod nobody:nogroup /srv/spamassassin

[edit] ClamAV

To enable virus scanning by ClamAV, install and configure it: 

apt-get install clamav clamsmtp

To configure CalmSMTP see the article clamsmtp-postfix-installation-unter-debian. Though there seems to be trouble with the ports used there with my configuration. Below find the ports pointed out which I found working with my configuration.

Take care on the port numbers you use for your configuration, see below for details.

[edit] clamsmtpd.conf

To configure the ClamAV SMTP daemon edit the following file: 

/etc/clamsmtpd.conf

Adjust the following entries in there: 

...
OutAddress: 10025
...
Listen: 127.0.0.1:10026
...

The ClamAV SMTP daemon uses port 10025 to connect to the postfix daemon (configured in the master.cf seen below). It listens to port 10026 for any mail scanning request from the postfix daemon (as configured in the main.cf seen below).

[edit] clamd.conf

For configuration of the ClamAV daemon itself, edit the following file: 

/etc/clamav/clamd.conf

Add or adjust these line accordingly: 

...
LocalSocket /var/run/clamav/clamd.ctl
# TCPSocket 3310
...

The directive TCPSocket 3310 must be deactivated and replaced by LocalSocket /var/run/clamav/clamd.ctl in order to ensure that ClamAV SMTP daemon, postfix and ClamAV daemon don't conflict each others with any same ports (sometimes I saw it configured to port 10025 which conflicts with our postfix daemon as configured in the master.cf) and in order for communication between the ClamAV daemon and the ClamAV SMTP daemon.

[edit] Postfix

Postfix configuration is kind of tricky. First of all I flowed the steps for postfix configuration as described in the chapter Postfix of the Debian Mailserver article, mainly creating the several scripts. Install it first: 

>apt-get install openssl postfix postfix-mysql postfix-pcre

Configure it as mentioned in the Postfix chapter. The below adjustments and settings turned out to work for my configuration:

[edit] master.cf

Edit the file: 

 /etc/postfix/master.cf

... to look similar to this one: 

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master").
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       n       -       -       smtpd
  -o content_filter=spamassassin
#submission inet n       -       -       -       -       smtpd
#  -o smtpd_enforce_tls=yes
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
smtps     inet  n       -       n       -       -       smtpd
  -o content_filter=spamassassin
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
#628      inet  n       -       -       -       -       qmqpd
pickup    fifo  n       -       -       60      1       pickup
cleanup   unix  n       -       -       -       0       cleanup
qmgr      fifo  n       -       n       300     1       qmgr
#qmgr     fifo  n       -       -       300     1       oqmgr
tlsmgr    unix  -       -       -       1000?   1       tlsmgr
rewrite   unix  -       -       -       -       -       trivial-rewrite
bounce    unix  -       -       -       -       0       bounce
defer     unix  -       -       -       -       0       bounce
trace     unix  -       -       -       -       0       bounce
verify    unix  -       -       -       -       1       verify
flush     unix  n       -       -       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
smtp      unix  -       -       -       -       -       smtp
# When relaying mail as backup MX, disable fallback_relay to avoid MX loops
relay     unix  -       -       -       -       -       smtp
  -o fallback_relay=
# -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq     unix  n       -       -       -       -       showq
error     unix  -       -       -       -       -       error
discard   unix  -       -       -       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       -       -       -       lmtp
anvil     unix  -       -       -       -       1       anvil
scache	  unix	-	-	-	-	1	scache
#
# ====================================================================
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
#
# Many of the following services use the Postfix pipe(8) delivery
# agent.  See the pipe(8) man page for information about ${recipient}
# and other message envelope options.
# ====================================================================
#
# maildrop. See the Postfix MAILDROP_README file for details.
# Also specify in main.cf: maildrop_destination_recipient_limit=1
#
maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
#
# See the Postfix UUCP_README file for configuration details.
#
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
#
# Other external delivery methods.
#
ifmail    unix  -       n       n       -       -       pipe
  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix	-	n	n	-	2	pipe
  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman   unix  -       n       n       -       -       pipe
  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
  ${nexthop} ${user}
cyrus     unix -        n       n       -       -       pipe
 flags= user=cyrus argv=/usr/sbin/cyrdeliver -r ${sender} -m ${extension} ${user}

# ============================================================================
# Antivirus: AV scan filter (used by content_filter)
# ============================================================================
scan      unix  -       -       n       -       16      smtp
  -o smtp_send_xforward_command=yes
# For injecting mail back into postfix from the filter
127.0.0.1:10025	inet	n 	-       n       -       16      smtpd
  -o content_filter=
  -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
  -o smtpd_helo_restrictions=
  -o smtpd_client_restrictions=
  -o smtpd_sender_restrictions=
  -o smtpd_recipient_restrictions=permit_mynetworks,reject
  -o mynetworks_style=host
  -o smtpd_authorized_xforward_hosts=127.0.0.0/8 
# ============================================================================
# Spamassassin:
# ============================================================================
spamassassin unix -     n       n       -       -       pipe
 user=nobody argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Note the port assignment to 10025: This is the OutAddress configured above in the clamsmtpd.conf file, which the ClamAV SMTP daemon uses to connect to your postfix daemon.

[edit] main.cf

Edit the file: 

 /etc/postfix/main.cf

... to look, again, similar to this one: 

# See /usr/share/postfix/main.cf.dist for a commented, more complete version

# ============================================================================
# Common settings:
# ============================================================================

# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

myhostname = subdomain.dmoan.tld
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = subdomain.dmoan.tld, localhost.domain.tld, localhost, mysql:/etc/postfix/mysql-mydestination.cf
relayhost =
mynetworks = 127.0.0.0/8
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
message_size_limit = 0
mailbox_transport = cyrus

virtual_alias_maps = mysql:/etc/postfix/mysql-virtual.cf
sender_canonical_maps = mysql:/etc/postfix/mysql-canonical.cf

# ============================================================================
# TLS parameters (Transport Layer Security):
# ============================================================================
smtpd_use_tls=yes
# smtpd_tls_auth_only = yes
# smtpd_tls_cert_file=/etc/postfix/mailserver.cert
# smtpd_tls_key_file=/etc/postfix/mailserver.key
# smtpd_tls_CAfile = /etc/postfix/mailserver.cert
smtpd_tls_cert_file = /etc/ssl/certs/smtps.pem
smtpd_tls_key_file = /etc/ssl/certs/smtps.pem
smtpd_tls_CAfile = /etc/ssl/certs/smtps.pem

smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

# ============================================================================
# SASL (Simple Authentication and Security Layer): smtpd mit SASL
# ============================================================================
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain =
broken_sasl_auth_clients = yes

# ============================================================================
# Greylisting aktivieren: "check_policy_service inet:127.0.0.1:60000"
# ============================================================================
smtpd_recipient_restrictions = 
  permit_sasl_authenticated,
  permit_mynetworks,
  reject_unauth_destination,
  check_policy_service inet:127.0.0.1:60000

# ============================================================================
# ClamAV: Antivirus
# ============================================================================
content_filter = scan:127.0.0.1:10026
receive_override_options = no_address_mappings

Note the port 10026 for the virus scanner: Here we got a a hook to the ClamAV SMTP daemon which listens to exactly this port, as configured in the clamsmtpd.conf.

[edit] Cyrus mail migration

Migrating your mail accounts from one server to another turned out to be some time consuming investigation and online search until I figured out how to do it.

[edit] Crate Cyrus mail accounts

First of all create your mail accounts. Do it with the cyradm tool, not with the web frontend web-cyradm cause cyradm creates the needed directory structure and initial files for the new account. Web-cyradm seemd to forget to do that:

Log on to cyradm: 

cyradm --user cyrus --auth login localhost

Then create the mail accounts for your users. 

cm user/somebody

Important: prefix your mail-account with user/ instead of user. as this will result in your account to be created in the wrong folder below /var/spool/cyrus.

Verify thr creation of your mailbox (still inside cyradm) by listing the mailboxes: 

lm

Finally use the web-frontend web-cyradm to finish setting up the mailbox and password, now your mailboxes are ready for use.

[edit] Shut down the daemons

Stop the postfix and the cyrus daemon: 

/etc/init.d/cyrus2.2 stop
/etc/init.d/postfix stop

[edit] Copy mailboxes

On a Debian based machine, the mailboxes are found in the following folder: 

/var/spool/cyrus/mail/...

Here you got directories from a to z: 

a b c d e f g h i j k l m n o p q r s t u v w x y z

On the source host you copy the mailboxes' folder (and sub folders) to be migrated (found below the folder with the according letter witch which your mailboxes starts, i.e. if the mailboxes' name is somebody then the mailboxes is found below s) preserving all the attributes (you might want to use tar) to the according folder on the destination host.

Due to naming conventions dots in the mailboxes' name are represented by a ^ character.

For example you might migrate the mailbox for somebody. On the source host you do the following: 

tar --preserve -c -v -z -f somebody.tar.gz /var/spool/cyrus/mail/s/user/somebody

On the destination host you do the same in reverse order. You copy the tar-ball from source host to destination host and untar it there accordingly: 

cd /var/spool/cyrus/mail/s/user/somebody
tar --preserve -x -v -z -f somebody.tar.gz

[edit] Copy mail mailboxes' meta information

You also got to copy the folder's meta information such as the mail mailboxes' folder structure and the information if a mail has already been read. This is found here: 

/var/lib/cyrus/users

Here you got directories from a to z: 

a b c d e f g h i j k l m n o p q r s t u v w x y z

Depending on the mailbox's name you have to locate the following files: 

 somebody.sub
 somebody.seen

One file contains information on the mail mailboxes' folder structure (*.sub), the other whether a mail has been read or not (*.seen). Same procedure as above: Copy the files from the source host to the according location on the destination host. For mailbox somebody the location of the files would be: 

/var/lib/cyrus/user/s/somebody.sub
/var/lib/cyrus/user/s/somebody.seen

[edit] Reconstruct your mail mailboxes

As the user cyrus on the shell you got to issue the reconfigure command for your mailbox: 

>login cyrus
>/usr/sbin/cyrreconstruct -r -f user/somebody

[edit] Start up the daemons

Startup the postfix and the cyrus daemon: 

/etc/init.d/cyrus2.2 start
/etc/init.d/postfix start

You should now be able to access your mailboxes on the destination host.

[edit] Links

This cheat sheet has pulled together information form the following resources:

Retrieved from "http://wiki.comjat.net/index.php/Webserver_howto"
Personal tools