You are here: Fladi.at>Main Web>XenLvmCman (16 Jan 2009, FladischerMichael)EditAttach

XenLvmCman

Setting up a cluster with at least three nodes with GFS2 volumes, LVM2 and various services. All nodes are running inside Xen-DomU instances.

Goals

  • DONE Have an easy way to create new nodes for the cluster by cloning existing DomU systems.
  • DONE Share one or more resizeable volumes between all nodes in the cluster and provide parallel access (R/W) to these volumes.
  • DONE Run Dovecot and Postfix on each node utilizing the shared volume.
  • TODO Provide some ways of controlling each service on the nodes.
  • DONE Manage configuration on all nodes equally with a centralized system.

Requirements

Xen

As of the time this document was written, the Debian package xen-linux-system-2.6.26-1-xen-686 provide the latest available Xen system. It depends on the binary Linux kernel and the Xen hypervisor. Once the Xen system is installed, the machine has to be rebooted and in Grub there should be a new entry for booting the Xen hypervisor. Select this entry and see if it works. Now we have a running Dom0 which will serve as the host system for all our future DomUs.

LVM

If the Dom0 is not already running one or more LVG VGs it would now be the time to create one. I will assume that there is a spare block device on /dev/sdx which will serve as a physical volume for our new LVM VG named domu:
Language 'sh' not supported 


#> pvcreate /dev/sdx
#> vgcreate -s 32M domu /dev/sdx

Packages on cluster nodes

  • cman
  • clvm
  • rgmanager
  • gfs2-tools
Install them with all their dependencies:
Language 'sh' not supported 


#> aptitude install cman clvm rgmanager gfs2-tools

Configuration

Enable clustered LVM

To use LVM in a clustered environment it has to uses it's built-in cluster locking. This can be enabled inside /etc/lvm/lvm.conf inside the global section: 
locking_type = 3
It seems that the Debian package of clvm does not create the Sys-V init symlinks automatically so I had to set them up by hand:
Language 'sh' not supported 


#> update-rc.d clvm defaults

Create cluster.conf

Here is a sample cluster.conf which is used to power my experimental cluster for dovecot and postfix: 
<?xml version="1.0"?>
<cluster name="test1" config_version="11">
    <clusternodes>
        <clusternode name="cluster1" nodeid="1">
            <fence>
                <method name="xvm">
                    <device name="xvm" nodename="cluster1" domain="cluster1"/>
                </method>
            </fence>
        </clusternode>
        <clusternode name="cluster2" nodeid="2">
            <fence>
                <method name="xvm">
                    <device name="xvm" nodename="cluster2" domain="cluster2"/>
                </method>
            </fence>
        </clusternode>
        <clusternode name="cluster3" nodeid="3">
            <fence>
                <method name="xvm">
                    <device name="xvm" nodename="cluster3" domain="cluster3"/>
                </method>
            </fence>
        </clusternode>
    </clusternodes>
        <fencedevices>
          <fencedevice name="last_resort" agent="fence_manual"/>
          <fencedevice name="xvm" agent="fence_xvm"/>
        </fencedevices>
    <rm>
        <failoverdomains>
            <failoverdomain name="cluster1" restricted="1">
                <failoverdomainnode name="cluster1"/>
            </failoverdomain>
            <failoverdomain name="cluster2" restricted="1">
                <failoverdomainnode name="cluster2"/>
            </failoverdomain>
            <failoverdomain name="cluster3" restricted="1">
                <failoverdomainnode name="cluster3"/>
            </failoverdomain>
        </failoverdomains>
        <resources>
            <clusterfs device="/dev/mapper/cluster-mail" force_unmount="0" fstype="gfs2" mountpoint="/home" name="mail" options="acl"/>
            <script name="Dovecot" file="/etc/init.d/dovecot"/>
            <script name="Postfix" file="/etc/init.d/postfix"/>
            <script name="Spamassassin" file="/etc/init.d/spamassassin"/>
        </resources>
        <service name="cluster1-mail" autostart="1" domain="cluster1">
            <script ref="Spamassassin"/>
            <script ref="Dovecot"/>
            <script ref="Postfix"/>
            <clusterfs ref="mail"/>
        </service>
        <service name="cluster2-mail" autostart="1" domain="cluster2">
            <script ref="Spamassassin"/>
            <script ref="Dovecot"/>
            <script ref="Postfix"/>
            <clusterfs ref="mail"/>
        </service>
        <service name="cluster3-mail" autostart="1" domain="cluster3">
            <script ref="Spamassassin"/>
            <script ref="Dovecot"/>
            <script ref="Postfix"/>
            <clusterfs ref="mail"/>
        </service>
    </rm>
</cluster>
Edit | Attach | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | More topic actions
Topic revision: r4 - 16 Jan 2009 - 08:14:54 - FladischerMichael
 
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Fladi.at? Send feedback