[Rear-users] New feature: savelayout and checklayout

Schlomo Schapiro schlomo at schapiro.org
Thu Dec 16 13:10:54 CET 2010


Hi,

we got a new feature in ReaR:

$ sudo rear checklayout
[rear] Locking: removing stale lock of nonexistent PID 27460
[rear] restarting myself
[rear] Locking: success, installed signal handlers
Relax & Recover Version 0.0.0 / 2010-12-01
Saving disk partitions.
Saving LVM layout.
Saving Filesystem layout.
Disk layout is identical.
Finished in 1 seconds.

With the savelayout one can save the layout information. Please note
that this is totally independent of the disaster recovery code in ReaR
and uses different commands and a different on-disk storage for the
information. Therefore it is not guaranteed that the ReaR disaster
recovery code has the same opinion as the save|checklayout code. Maybe
in the future we can merge these two areas so that the checklayout
relies on the data in the recovery area, thus making extra savelayout
runs unnecessary.

The main use case is probably in a cron job to quickly find out if it is
required to to a new "rear mkrescue" run to update the boot media.

Please give it a try and thanks a lot to Jeroen.

Kind Regards,
Schlomo

-------- Original-Nachricht --------
Betreff: SF.net SVN: rear:[183] trunk/usr/share/rear
Datum: Thu, 16 Dec 2010 09:36:00 +0000
Von: gdha at users.sourceforge.net
An: rear at schlomo.schapiro.org

Revision: 183
          http://rear.svn.sourceforge.net/rear/?rev=183&view=rev
Author:   gdha
Date:     2010-12-16 09:36:00 +0000 (Thu, 16 Dec 2010)

Log Message:
-----------
- added patch of Jeroen to check/save-layout workflows
- aligned the mkdir and mkvendorrpm

Modified Paths:
--------------
    trunk/usr/share/rear/CHANGES
    trunk/usr/share/rear/TODO
    trunk/usr/share/rear/contrib/mkvendorrpm
    trunk/usr/share/rear/lib/mkdist-workflow.sh

Added Paths:
-----------
    trunk/usr/share/rear/layout/
    trunk/usr/share/rear/layout/cleanup/
    trunk/usr/share/rear/layout/cleanup/default/
    trunk/usr/share/rear/layout/cleanup/default/90_remove_temp.sh
    trunk/usr/share/rear/layout/compare/
    trunk/usr/share/rear/layout/compare/default/
    trunk/usr/share/rear/layout/compare/default/50_compare_layout.sh
    trunk/usr/share/rear/layout/precompare/
    trunk/usr/share/rear/layout/precompare/default/
    trunk/usr/share/rear/layout/precompare/default/11_check_layout_file.sh
    trunk/usr/share/rear/layout/save/
    trunk/usr/share/rear/layout/save/GNU/
    trunk/usr/share/rear/layout/save/GNU/Linux/
    trunk/usr/share/rear/layout/save/GNU/Linux/10_create_layout_file.sh
    trunk/usr/share/rear/layout/save/GNU/Linux/20_partition_layout.sh
    trunk/usr/share/rear/layout/save/GNU/Linux/21_raid_layout.sh
    trunk/usr/share/rear/layout/save/GNU/Linux/22_lvm_layout.sh
    trunk/usr/share/rear/layout/save/GNU/Linux/23_filesystem_layout.sh
    trunk/usr/share/rear/lib/checklayout-workflow.sh
    trunk/usr/share/rear/lib/savelayout-workflow.sh

Modified: trunk/usr/share/rear/CHANGES
===================================================================
--- trunk/usr/share/rear/CHANGES	2010-12-16 08:28:37 UTC (rev 182)
+++ trunk/usr/share/rear/CHANGES	2010-12-16 09:36:00 UTC (rev 183)
@@ -1,7 +1,10 @@
 Version 1.9.0  - not released
 ---------------------------
+2010-12-16	GD	* aligned mkdist-workflow.sh and mkvendorrpm
+			* added patch of Jeroen Hoeckx to check- and savelayout workflows
+			  (SF #3124343, sponsored by Belgian Federal Police)
+			* doc/configuration-examples.txt added for the beginners.
 2010-12-15	GD	* add DRBD support patch of Dag/Jeroen (sponsored by
Belgian Federal Police #3124341)
-			* doc/configuration-examples.txt added for the beginners.
 2010-12-14	GD	* lib/network-functions.sh: added my_ipcalc replacement
for ipcalc (only standard in RHEL/Fedora)
 			* minor fixes in skel/default/bin/dhclient-script
 			* skel/default/etc/scripts/system-setup.d/58-start-dhclient.sh loop
over valid interfaces

Modified: trunk/usr/share/rear/TODO
===================================================================
--- trunk/usr/share/rear/TODO	2010-12-16 08:28:37 UTC (rev 182)
+++ trunk/usr/share/rear/TODO	2010-12-16 09:36:00 UTC (rev 183)
@@ -1,13 +1,19 @@
 * finalize/Fedora/i386/17_rebuild_initramfs.sh
-- needs to be written, seems like RH builds rich initramfs with all
modules included
-- be aware, recent Fedora uses dracut instead of mkinitrd
+	- needs to be written, seems like RH builds rich initramfs with all
modules included
+	- be aware, recent Fedora uses dracut instead of mkinitrd

 * finalize/Gentoo/i386/17_rebuild_initramfs.sh
-- needs to be written, I have no idea what should be done here (since I
don't have Gentoo)
+	- needs to be written, I have no idea what should be done here (since
I don't have Gentoo)

 * write network migration code for all other distributions
+	- some work has been done in the meantime, but not sure if it is
sufficient enough to cover all
+	- waiting on customer feedback

 * support dhcp for network migration
+	- has been added in 1.9.0
+	- waiting on customer feedback

 * prep/Linux-ia64/20_getty_or_agetty.sh might be obsolete after the
USE_SERIAL_CONSOLE piece has been added and tested successfully
+	- ia64 architecture will not support Linux anymore in future? Dropped
off the list in RHEL6.

+* Can we drop the mkvendorrpm workflow?

Modified: trunk/usr/share/rear/contrib/mkvendorrpm
===================================================================
--- trunk/usr/share/rear/contrib/mkvendorrpm	2010-12-16 08:28:37 UTC
(rev 182)
+++ trunk/usr/share/rear/contrib/mkvendorrpm	2010-12-16 09:36:00 UTC
(rev 183)
@@ -130,11 +130,14 @@
 # Create ReaR rescue media as ISO image
 OUTPUT=ISO

-# optionally set your backup software
+# optionally define (non-default) backup software, e.g. TSM, NBU, DP,
BACULA
 # BACKUP=TSM

 # the following is required on older VMware VMs
 MODULES_LOAD=( vmxnet )
+
+# to see boot messages on the serial console (uncomment next line)
+# KERNEL_CMDLINE="console=tty0 console=ttyS1"
 EOF
 	
 tee
./$CONFIG_DIR/templates/{PXE_pxelinux.cfg,ISO_isolinux.cfg,USB_syslinux.cfg}
>/dev/null <<EOF

Added: trunk/usr/share/rear/layout/cleanup/default/90_remove_temp.sh
===================================================================
--- trunk/usr/share/rear/layout/cleanup/default/90_remove_temp.sh	
                  (rev 0)
+++ trunk/usr/share/rear/layout/cleanup/default/90_remove_temp.sh
2010-12-16 09:36:00 UTC (rev 183)
@@ -0,0 +1,4 @@
+# Remove temporary files
+
+Log "Removing temporary files."
+rm $TEMP_LAYOUT

Added: trunk/usr/share/rear/layout/compare/default/50_compare_layout.sh
===================================================================
--- trunk/usr/share/rear/layout/compare/default/50_compare_layout.sh	
                     (rev 0)
+++ trunk/usr/share/rear/layout/compare/default/50_compare_layout.sh
2010-12-16 09:36:00 UTC (rev 183)
@@ -0,0 +1,9 @@
+# Test if ORIG_LAYOUT and TEMP_LAYOUT are the same
+
+diff $ORIG_LAYOUT $TEMP_LAYOUT > /dev/null
+
+if [ $? -eq 0 ] ; then
+    LogPrint "Disk layout is identical."
+else
+    LogPrint "Disk layout has changed."
+fi

Added:
trunk/usr/share/rear/layout/precompare/default/11_check_layout_file.sh
===================================================================
---
trunk/usr/share/rear/layout/precompare/default/11_check_layout_file.sh	
                       (rev 0)
+++
trunk/usr/share/rear/layout/precompare/default/11_check_layout_file.sh
2010-12-16 09:36:00 UTC (rev 183)
@@ -0,0 +1,5 @@
+# Check if the disk layout file exists.
+
+if [ ! -e "$ORIG_LAYOUT" ] ; then
+    Error "Please run \"# rear savelayout\" first."
+fi

Added: trunk/usr/share/rear/layout/save/GNU/Linux/10_create_layout_file.sh
===================================================================
--- trunk/usr/share/rear/layout/save/GNU/Linux/10_create_layout_file.sh
                        (rev 0)
+++ trunk/usr/share/rear/layout/save/GNU/Linux/10_create_layout_file.sh
2010-12-16 09:36:00 UTC (rev 183)
@@ -0,0 +1,9 @@
+# Create the layout file
+
+Log "Preparing layout directory."
+mkdir -p $VAR_DIR/layout
+
+if [ -e "$DISKLAYOUT_FILE" ] ; then
+    Log "Removing old layout file."
+fi
+: > $DISKLAYOUT_FILE

Added: trunk/usr/share/rear/layout/save/GNU/Linux/20_partition_layout.sh
===================================================================
--- trunk/usr/share/rear/layout/save/GNU/Linux/20_partition_layout.sh	
                      (rev 0)
+++ trunk/usr/share/rear/layout/save/GNU/Linux/20_partition_layout.sh
2010-12-16 09:36:00 UTC (rev 183)
@@ -0,0 +1,43 @@
+# Save the partition layout
+
+LogPrint "Saving disk partitions."
+
+(
+    # Disk sizes
+    # format: disk <disk> <sectors>
+    devices=()
+    for disk in /sys/block/* ; do
+        case $(basename $disk) in
+            hd*|sd*|cciss*)
+                if [ "$(cat $disk/removable)" = "1" ] ; then
+                    continue
+                fi
+
+                # fix cciss
+                devname=$(basename $disk | tr '!' '/')
+                devsize=$(cat $disk/size)
+                echo "disk $devname $devsize"
+
+                devices=( "${devices[@]}" "$devname" )
+                ;;
+        esac
+    done
+
+    # Partitions
+    # format : part <partition size(sectors)> <partition id>
+    for device in "${devices[@]}" ; do
+        if [ -e /dev/$device ] ; then
+            while read line ; do
+                if [ -z "$(echo $line | grep start)" ] ; then
+                    continue
+                fi
+
+                psize=$(echo $line | cut -d "," -f 2 | cut -d "=" -f 2
| tr -d " ")
+                pid=$(echo $line | cut -d "," -f 3 | cut -d "=" -f 2 |
tr -d " ")
+
+                echo "part $device $psize $pid"
+            done < <(sfdisk -uS -d /dev/$device)
+        fi
+    done
+
+) >> $DISKLAYOUT_FILE

Added: trunk/usr/share/rear/layout/save/GNU/Linux/21_raid_layout.sh
===================================================================
--- trunk/usr/share/rear/layout/save/GNU/Linux/21_raid_layout.sh	
                 (rev 0)
+++ trunk/usr/share/rear/layout/save/GNU/Linux/21_raid_layout.sh
2010-12-16 09:36:00 UTC (rev 183)
@@ -0,0 +1,21 @@
+# Save the Software RAID layout
+
+if [ -e /proc/mdstat ] &&  grep -q blocks /proc/mdstat ; then
+    LogPrint "Saving Software RAID configuration."
+    (
+        while read array device level ndevices metadata uuid junk ; do
+            if [ "$array" != "ARRAY" ] ; then
+                continue
+            fi
+
+            device=$(basename $device)
+            ndevices=$(echo "$ndevices" | cut -d "=" -f 2)
+            uuid=$(echo "$uuid" | cut -d "=" -f 2)
+
+            read jname junk jstate level devices < <(grep $device
/proc/mdstat)
+            devices=$(echo "$devices" | sed 's/\[[[:digit:]]\]//g')
+
+            echo "raid $device $level $ndevices $uuid $devices"
+        done < <(mdadm --detail --scan --config=partitions)
+    ) >> $DISKLAYOUT_FILE
+fi

Added: trunk/usr/share/rear/layout/save/GNU/Linux/22_lvm_layout.sh
===================================================================
--- trunk/usr/share/rear/layout/save/GNU/Linux/22_lvm_layout.sh	
                (rev 0)
+++ trunk/usr/share/rear/layout/save/GNU/Linux/22_lvm_layout.sh
2010-12-16 09:36:00 UTC (rev 183)
@@ -0,0 +1,38 @@
+# Save LVM layout
+
+LogPrint "Saving LVM layout."
+
+(
+    ## Get physical_device configuration
+    # format: lvmdev <volume_group> <device> [<uuid>] [<size(bytes)>]
+    lvm pvdisplay -c | while read line ; do
+        pdev=$(echo $line | cut -d ":" -f "1")
+        vgrp=$(echo $line | cut -d ":" -f "2")
+        size=$(echo $line | cut -d ":" -f "3")
+        uuid=$(echo $line | cut -d ":" -f "12")
+
+        echo "lvmdev $vgrp $pdev $uuid $size"
+    done
+
+    ## Get the volume group configuration
+    # format: lvmgrp <volume_group> <extentsize> [<size(extents)>]
[<size(bytes)>]
+    lvm vgdisplay -c | while read line ; do
+        vgrp=$(echo $line | cut -d ":" -f "1")
+        size=$(echo $line | cut -d ":" -f "12")
+        extentsize=$(echo $line | cut -d ":" -f "13")
+        nrextents=$(echo $line | cut -d ":" -f "14")
+
+        echo "lvmgrp $vgrp $extentsize $nrextents $size"
+    done
+
+    ## Get all logical volumes
+    # format: lvmvol <volume_group> <name> <size(extents)> [<size(bytes)>]
+    lvm lvdisplay -c | while read line ; do
+        lvol=$(echo $line | cut -d ":" -f "1" | cut -d "/" -f "4")
+        vgrp=$(echo $line | cut -d ":" -f "2")
+        size=$(echo $line | cut -d ":" -f "7")
+        extents=$(echo $line | cut -d ":" -f "8")
+
+        echo "lvmvol $vgrp $lvol $extents $size "
+    done
+) >> $DISKLAYOUT_FILE

Added: trunk/usr/share/rear/layout/save/GNU/Linux/23_filesystem_layout.sh
===================================================================
--- trunk/usr/share/rear/layout/save/GNU/Linux/23_filesystem_layout.sh	
                       (rev 0)
+++ trunk/usr/share/rear/layout/save/GNU/Linux/23_filesystem_layout.sh
2010-12-16 09:36:00 UTC (rev 183)
@@ -0,0 +1,35 @@
+# Save Filesystem layout
+
+LogPrint "Saving Filesystem layout."
+
+(
+    # Filesystems
+    # format: fs <device> <mountpoint> <filesystem> [uuid=<uuid>]
[label=<label>]
+    while read line ; do
+        if [ "${line#/}" = "$line" ] ; then
+            continue
+        fi
+
+        device=$(echo $line | cut -d " " -f 1)
+        mountpoint=$(echo $line | cut -d " " -f 3)
+        fstype=$(echo $line | cut -d " " -f 5)
+
+        echo -n "fs $device $mountpoint $fstype "
+        case "$fstype" in
+            ext*)
+                uuid=$(tune2fs -l $device | grep UUID | cut -d ":" -f 2
| tr -d " ")
+                label=$(e2label $device)
+                echo -n "uuid=$uuid label=$label "
+                ;;
+            xfs)
+                uuid=$(xfs_admin -u $device | cut -d'=' -f 2 | tr -d " ")
+                label=$(xfs_admin -l $device | cut -d'"' -f 2)
+                echo -n "uuid=$uuid label=$label "
+                ;;
+        esac
+
+        # TODO: filesystem attributes
+
+        echo
+    done < <(mount)
+) >> $DISKLAYOUT_FILE

Added: trunk/usr/share/rear/lib/checklayout-workflow.sh
===================================================================
--- trunk/usr/share/rear/lib/checklayout-workflow.sh	
     (rev 0)
+++ trunk/usr/share/rear/lib/checklayout-workflow.sh	2010-12-16 09:36:00
UTC (rev 183)
@@ -0,0 +1,35 @@
+# checklayout-workflow.sh
+#
+# checklayout workflow for Relax & Recover
+#
+#    Relax & Recover is free software; you can redistribute it and/or
modify
+#    it under the terms of the GNU General Public License as published by
+#    the Free Software Foundation; either version 2 of the License, or
+#    (at your option) any later version.
+
+#    Relax & Recover is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+
+#    You should have received a copy of the GNU General Public License
+#    along with Relax & Recover; if not, write to the Free Software
+#    Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
02110-1301  USA
+#
+#
+
+WORKFLOW_checklayout_DESCRIPTION="Check if the disk layout has changed
since the last run of savelayout."
+WORKFLOWS=( ${WORKFLOWS[@]} checklayout )
+WORKFLOW_checklayout () {
+    ORIG_LAYOUT=$VAR_DIR/layout/disklayout.conf
+    TEMP_LAYOUT=$VAR_DIR/layout/disklayout.conf.new
+
+    SourceStage "layout/precompare"
+
+    DISKLAYOUT_FILE=$TEMP_LAYOUT
+    SourceStage "layout/save"
+
+    SourceStage "layout/compare"
+
+    SourceStage "layout/cleanup"
+}

Modified: trunk/usr/share/rear/lib/mkdist-workflow.sh
===================================================================
--- trunk/usr/share/rear/lib/mkdist-workflow.sh	2010-12-16 08:28:37 UTC
(rev 182)
+++ trunk/usr/share/rear/lib/mkdist-workflow.sh	2010-12-16 09:36:00 UTC
(rev 183)
@@ -26,8 +26,8 @@
 	# reverted back to symlinking because we put more MegaByte into doc
and should not package it twice
 	ln -s .$SHARE_DIR/{doc,contrib}  .  1>&8
 	# to prevent RPMs from installing symlinks into the doc area we
actually copy the text files
-	cp -r .$SHARE_DIR/{COPYING,CHANGES,README}  .  1>&8
-	ProgressStopIfError $? "Could not copy
.$SHARE_DIR/{COPYING,CHANGES,README,doc,contrib}"
+	cp -r .$SHARE_DIR/{COPYING,CHANGES,README,AUTHORS,TODO}  .  1>&8
+	ProgressStopIfError $? "Could not copy
.$SHARE_DIR/{COPYING,CHANGES,README,AUTHORS,TODO,doc,contrib}"
 	

 # I want the generic SPEC file to be always shipped 2009-11-16 Schlomo
@@ -55,7 +55,7 @@
 # Create ReaR rescue media as ISO image
 OUTPUT=ISO

-# optionally define (non-default) backup software, e.g. TSM, NBU, DP
+# optionally define (non-default) backup software, e.g. TSM, NBU, DP,
BACULA
 # BACKUP=TSM

 # the following is required on older VMware VMs

Added: trunk/usr/share/rear/lib/savelayout-workflow.sh
===================================================================
--- trunk/usr/share/rear/lib/savelayout-workflow.sh	
    (rev 0)
+++ trunk/usr/share/rear/lib/savelayout-workflow.sh	2010-12-16 09:36:00
UTC (rev 183)
@@ -0,0 +1,26 @@
+# savelayout-workflow.sh
+#
+# savelayout workflow for Relax & Recover
+#
+#    Relax & Recover is free software; you can redistribute it and/or
modify
+#    it under the terms of the GNU General Public License as published by
+#    the Free Software Foundation; either version 2 of the License, or
+#    (at your option) any later version.
+
+#    Relax & Recover is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+
+#    You should have received a copy of the GNU General Public License
+#    along with Relax & Recover; if not, write to the Free Software
+#    Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
02110-1301  USA
+#
+#
+
+WORKFLOW_savelayout_DESCRIPTION="Save the disk layout of the system."
+WORKFLOWS=( ${WORKFLOWS[@]} savelayout )
+WORKFLOW_savelayout () {
+    DISKLAYOUT_FILE=$VAR_DIR/layout/disklayout.conf
+    SourceStage "layout/save"
+}


This was sent by the SourceForge.net collaborative development platform,
the world's largest Open Source development site.




More information about the rear-users mailing list