[rear-devel] [rear/rear] 619a3b: Revert "Fix for removing directory in case that th...

Johannes Meixner noreply at github.com
Fri Jun 18 15:02:46 CEST 2021


  Branch: refs/heads/master
  Home:   https://github.com/rear/rear
  Commit: 619a3b3ff76780e327269f73407fc25467c7801b
      https://github.com/rear/rear/commit/619a3b3ff76780e327269f73407fc25467c7801b
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/framework-functions.sh
    M usr/share/rear/output/default/100_mount_output_path.sh
    M usr/share/rear/output/default/980_umount_output_dir.sh

  Log Message:
  -----------
  Revert "Fix for removing directory in case that they are not empty"

The "fix" actually has not fixed anything (#781 was a different issue)
and reintroduces bug #465 (re-reported as #2611). The issue is that if
the directories are not empty, it is because they contain potentially
valuable data (old backups and backups from other machines), so removing
them in order to silence some error is the worst thing to do.

This reverts commit 4e9c2a1b05f87762fb06355cf959b24eacc21f62.


  Commit: fdea31457cf558b64dad87bbf31c75ab907e2674
      https://github.com/rear/rear/commit/fdea31457cf558b64dad87bbf31c75ab907e2674
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    R usr/share/rear/build/YUM/default/600_create_python_symlink.sh
    A usr/share/rear/build/YUM/default/600_create_python_symlink.sh
    R usr/share/rear/prep/YUM/default/070_set_backup_archive.sh
    A usr/share/rear/prep/YUM/default/070_set_backup_archive.sh
    R usr/share/rear/restore/YUM/default/100_mount_YUM_path.sh
    A usr/share/rear/restore/YUM/default/100_mount_YUM_path.sh
    R usr/share/rear/restore/YUM/default/980_umount_YUM_dir.sh
    A usr/share/rear/restore/YUM/default/980_umount_YUM_dir.sh
    R usr/share/rear/verify/YUM/default/050_check_YUM_requirements.sh
    A usr/share/rear/verify/YUM/default/050_check_YUM_requirements.sh
    R usr/share/rear/verify/YUM/default/060_mount_YUM_path.sh
    A usr/share/rear/verify/YUM/default/060_mount_YUM_path.sh
    R usr/share/rear/verify/YUM/default/070_set_backup_archive.sh
    A usr/share/rear/verify/YUM/default/070_set_backup_archive.sh
    R usr/share/rear/verify/YUM/default/980_umount_YUM_dir.sh
    A usr/share/rear/verify/YUM/default/980_umount_YUM_dir.sh

  Log Message:
  -----------
  Partially revert "Convert YUM files which were symlinks to other, non-YUM files to copies of those files"

Files that did not differ from their originals and have not gained
significant updates afterwards are better kept as symlinks to reduce
duplication and maintenance effort.

This reverts commit 3d9cc245799317102ec5311d67c104494da1cbaf
except for usr/share/rear/restore/YUM/default/970_set_root_password.sh


  Commit: eeed37d5091f6b740a8820145e17c0d1fa2c1ca2
      https://github.com/rear/rear/commit/eeed37d5091f6b740a8820145e17c0d1fa2c1ca2
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/verify/YUM/default/050_check_YUM_requirements.sh
    M usr/share/rear/verify/YUM/default/060_mount_YUM_path.sh
    M usr/share/rear/verify/YUM/default/070_set_backup_archive.sh
    M usr/share/rear/verify/YUM/default/980_umount_YUM_dir.sh

  Log Message:
  -----------
  Change YUM symlinks to more appropriate targets

No functional change.


  Commit: ea130127a925204449a0e58a6293814e8efe6b08
      https://github.com/rear/rear/commit/ea130127a925204449a0e58a6293814e8efe6b08
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    R usr/share/rear/backup/YUM/default/400_create_include_exclude_files.sh
    A usr/share/rear/backup/YUM/default/400_create_include_exclude_files.sh

  Log Message:
  -----------
  Convert a YUM script into a symlink

Introduced as a copy in e68326f7ef602d2746b0308bebf6db7e4db8f495.

No functional change.


  Commit: 696a4cb86a09674b059ab928d055bd65460ce26d
      https://github.com/rear/rear/commit/696a4cb86a09674b059ab928d055bd65460ce26d
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    R usr/share/rear/restore/DUPLICITY/default/100_mount_duplicity_path.sh
    A usr/share/rear/restore/DUPLICITY/default/100_mount_duplicity_path.sh
    R usr/share/rear/restore/DUPLICITY/default/980_unmount_duplicity_path.sh
    A usr/share/rear/restore/DUPLICITY/default/980_unmount_duplicity_path.sh
    R usr/share/rear/verify/DUPLICITY/default/100_mount_duplicity_path.sh
    A usr/share/rear/verify/DUPLICITY/default/100_mount_duplicity_path.sh
    R usr/share/rear/verify/DUPLICITY/default/980_unmount_duplicity_path.sh
    A usr/share/rear/verify/DUPLICITY/default/980_unmount_duplicity_path.sh

  Log Message:
  -----------
  Reduce duplication in DUPLICITY

Replace identical script copies by symlinks.


  Commit: cdfe8d981fef23f36de46e8d5e5ebc8c86bf3c91
      https://github.com/rear/rear/commit/cdfe8d981fef23f36de46e8d5e5ebc8c86bf3c91
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/980_umount_output_dir.sh

  Log Message:
  -----------
  Remove the removal of outputfs/$RSYNC_PREFIX

$BUILD_DIR/outputfs/$RSYNC_PREFIX has never been used by rsync. Removing
it is even dangerous, because usually $RSYNC_PREFIX == $NETFS_PREFIX ==
$OUTPUT_PREFIX, so this would lead to a loss of data if the unmount
fails. It could even remove all the backups if $RSYNC_PREFIX is empty.


  Commit: 4b39c5c8b3ea5db70ef37bc0a2159fd4ef39b990
      https://github.com/rear/rear/commit/4b39c5c8b3ea5db70ef37bc0a2159fd4ef39b990
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/980_umount_output_dir.sh

  Log Message:
  -----------
  Replace NETFS_PREFIX by OUTPUT_PREFIX in output

NETFS_PREFIX has not been used in the output stage since
a8fdc445d0d6f7f9184dc6633817928f96aae9bc. This was the last instance.


  Commit: b9364e8dca546970fa0eb327ec4b5ad403dabce1
      https://github.com/rear/rear/commit/b9364e8dca546970fa0eb327ec4b5ad403dabce1
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Apply Coding-Style to function declarations

Use the function keyword to declare functions.

https://github.com/rear/rear/wiki/Coding-Style#functions


  Commit: 00d2d3e41a79915ebd7ee0a5593792c41ca53bd3
      https://github.com/rear/rear/commit/00d2d3e41a79915ebd7ee0a5593792c41ca53bd3
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Add the scheme_supports_filesystem helper function

Checks whether a given URL scheme supports direct filesystem access when
mounted.


  Commit: d1687284bf521fb7c2e9e6631ca143fd8370ae10
      https://github.com/rear/rear/commit/d1687284bf521fb7c2e9e6631ca143fd8370ae10
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Add obdr as output scheme w/o filesystem access

obdr is similar to tape, so it should be in the same code branch.

At this time mostly academic since OBDR support was most likely broken
by isofs changes in 4ef0f30 (ppc64le), 3658b79 (ppc64) and 41efc97
(i386).


  Commit: a6613e5932c16a25594735f69ed3dc8352fee8e6
      https://github.com/rear/rear/commit/a6613e5932c16a25594735f69ed3dc8352fee8e6
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/backup/NETFS/default/150_save_copy_of_prefix_dir.sh
    M usr/share/rear/backup/NETFS/default/200_make_prefix_dir.sh
    M usr/share/rear/backup/NETFS/default/250_create_lock.sh
    M usr/share/rear/backup/NETFS/default/970_remove_lock.sh
    M usr/share/rear/output/default/150_save_copy_of_prefix_dir.sh
    M usr/share/rear/output/default/250_create_lock.sh

  Log Message:
  -----------
  Apply Coding-Style to command substitutions

Space-separate commands from the enclosing parens.


  Commit: f99851d5f111d344f7fce5552a35c966b3fd3105
      https://github.com/rear/rear/commit/f99851d5f111d344f7fce5552a35c966b3fd3105
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/200_make_prefix_dir.sh

  Log Message:
  -----------
  Fix typo in comment


  Commit: 458c2b570fa730c5eee4a330e688133276283caf
      https://github.com/rear/rear/commit/458c2b570fa730c5eee4a330e688133276283caf
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/backup/NETFS/default/150_save_copy_of_prefix_dir.sh
    M usr/share/rear/backup/NETFS/default/200_make_prefix_dir.sh
    M usr/share/rear/backup/NETFS/default/250_create_lock.sh
    M usr/share/rear/output/default/150_save_copy_of_prefix_dir.sh
    M usr/share/rear/output/default/250_create_lock.sh

  Log Message:
  -----------
  Show last error messages using Error

Use Error instead of StopIfError which is problematic for several
reasons (see the comments near its definition).

Error also shows last error messages from the failed commands.

Analogous to changes in PR #1877.


  Commit: 5185eee0ac7801ee9811fefb129b6053343f4a98
      https://github.com/rear/rear/commit/5185eee0ac7801ee9811fefb129b6053343f4a98
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/backup/NETFS/default/150_save_copy_of_prefix_dir.sh
    M usr/share/rear/backup/NETFS/default/200_make_prefix_dir.sh
    M usr/share/rear/backup/NETFS/default/250_create_lock.sh
    M usr/share/rear/backup/NETFS/default/970_remove_lock.sh
    M usr/share/rear/output/default/150_save_copy_of_prefix_dir.sh
    M usr/share/rear/output/default/200_make_prefix_dir.sh
    M usr/share/rear/output/default/250_create_lock.sh

  Log Message:
  -----------
  Remove code that depended on the current stage

Since a8fdc445d0d6f7f9184dc6633817928f96aae9bc the handling of
BACKUP_URL and OUTPUT_URL is separate and the code is not shared between
backup and output stages. Also, the code that handles the lockfile is
different anyway. This allows some simplification by removing the
indirection based on the current stage that originally allowed sharing
the code.

Making this code generic and shared between the stages might be a good
idea, but would have to be done a bit differently.


  Commit: fa6a1f96ff58284d000315bc9d6055f8b45e04d7
      https://github.com/rear/rear/commit/fa6a1f96ff58284d000315bc9d6055f8b45e04d7
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/PXE/default/810_create_pxelinux_cfg.sh

  Log Message:
  -----------
  Update file name in comment


  Commit: afccd1c2aee7d0a1cb3506455705ce77aa2b9e63
      https://github.com/rear/rear/commit/afccd1c2aee7d0a1cb3506455705ce77aa2b9e63
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/PXE/default/800_copy_to_tftp.sh
    M usr/share/rear/output/PXE/default/810_create_pxelinux_cfg.sh

  Log Message:
  -----------
  Remove superfluous declaration of unused variable


  Commit: 6ce72b25de2ea2d1e1dee6e1cfd45f38dd03a764
      https://github.com/rear/rear/commit/6ce72b25de2ea2d1e1dee6e1cfd45f38dd03a764
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/PXE/default/800_copy_to_tftp.sh
    M usr/share/rear/output/PXE/default/810_create_pxelinux_cfg.sh

  Log Message:
  -----------
  Protect against unsupported schemes in PXE output

The PXE output (except for the legacy support) needs direct filesystem
access to OUTPUT_URL. Therefore complain if OUTPUT_URL can not be
mounted.


  Commit: 06849737a53d76bd0c4c328dd8c34bd10fb2a130
      https://github.com/rear/rear/commit/06849737a53d76bd0c4c328dd8c34bd10fb2a130
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/150_save_copy_of_prefix_dir.sh
    M usr/share/rear/output/default/200_make_prefix_dir.sh
    M usr/share/rear/output/default/250_create_lock.sh
    M usr/share/rear/output/default/970_remove_lock.sh

  Log Message:
  -----------
  Check if OUTPUT_URL supports filesystem access

Instead of testing whether the path returned by output_path() is empty,
use the new dedicated scheme_supports_filesystem() function for that.
This is less error prone, because it is easy to ignore the empty string
returned from output_path() and accidentally create files in the root
directory.

Functional change: output schemes that do not support mounting (anything
"ftp-like" or "tape-like") will stop creating an empty directory
$BUILD_DIR/outputfs.


  Commit: 07ece1f2bb9f52c489e72a6f24a7b52d78b00b52
      https://github.com/rear/rear/commit/07ece1f2bb9f52c489e72a6f24a7b52d78b00b52
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/950_copy_result_files.sh

  Log Message:
  -----------
  Proper test for empty RESULT_FILES array


  Commit: 9ea1e3e52259143fe077807923f32724ecdcdefa
      https://github.com/rear/rear/commit/9ea1e3e52259143fe077807923f32724ecdcdefa
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Introduce function scheme_accepts_files()

Determines whether one can upload files to a URL.

True for most schemes except for null and tape-based schemes (tape and
obdr).

Does not tell how exactly one is supposed to upload files to the URL:
it could be via a local mountpoint (mounted by mount_url) or by using a
file transfer utility for URLs that can not be mounted (usually lftp or
rsync). Use scheme_supports_filesystem() to distinguish between the two
cases.


  Commit: 57f0605722054a1a5b8f9a33a926c5b19e27480a
      https://github.com/rear/rear/commit/57f0605722054a1a5b8f9a33a926c5b19e27480a
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/950_copy_result_files.sh

  Log Message:
  -----------
  Use scheme_accepts_files in 950_copy_result_files

Replaces a hardcoded list of schemes that do not support uploading files
and can be skipped. Also stop relying on output_path() returning an
empty string for paths that do not accept files and move the
output_path() call down to the place where we actually know that we will
use the result.

Functional change: if the OUTPUT method has produced files but we don't
know how to upload them, abort. This prevents the output from being
silently lost in case the selected scheme does not accept them. This
will affect the tape scheme. The only exception is null, because it
indicates that the user does not expect the files to be uploaded
anywhere.


  Commit: 93aadafb43b0e0df1e47a5b358df348beb2c8fd2
      https://github.com/rear/rear/commit/93aadafb43b0e0df1e47a5b358df348beb2c8fd2
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/950_copy_result_files.sh

  Log Message:
  -----------
  Remove a FIXME for ${array[*]} expansion

${array[*]} instead of "${array[@]}" was indeed intended. Those uses are
inside bigger quoted strings and ${array[@]} would split them into
multiple strings.

Use ${array[*]} in other places where it was not originally intended,
but should have been.


  Commit: d4782dfd8551e74987c6a63873d1ad65b4ca3fdf
      https://github.com/rear/rear/commit/d4782dfd8551e74987c6a63873d1ad65b4ca3fdf
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Abort if output_path gets called with wrong scheme

"Wrong" here means schemes that do not support filesystem access. Since
this function returns a filesystem path that can be used to access the
URL destination directly, it is crucial that this is actually supported.

Returning an empty string for them is not satisfactory: it could lead to
caller putting its files under / instead of the intended location if the
result is not checked for emptiness.

Returning ${BUILD_DIR}/outputfs/${OUTPUT_PREFIX} even for unmountable
URLs (as done before) is also not satisfactory: caller could put its
files there expecting them to be safely at their destination, but if the
directory is not a mountpoint, they would get silently lost.

All callers have been modified to avoid calling this function if it
would abort.


  Commit: b5da19075ddeeffa1b34cc7be70c091e295a2a39
      https://github.com/rear/rear/commit/b5da19075ddeeffa1b34cc7be70c091e295a2a39
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/980_umount_output_dir.sh

  Log Message:
  -----------
  Remove removal of outputfs/$OUTPUT_PREFIX

After umount_url, this directory should not be there. Removing it is
even dangerous, because it could lead to a loss of data if the unmount
fails. It could even remove all the outputs if $OUTPUT_PREFIX happens to
be empty.


  Commit: 0fc7dcc37d60a897a5efad00a05e0d022872a296
      https://github.com/rear/rear/commit/0fc7dcc37d60a897a5efad00a05e0d022872a296
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Replace one instance of StopIfError

This function has multiple problems, see comments in code - better to
use explicit tests and Error.


  Commit: 15892253e3895065ce8b9ac4cd27088d23ee76d0
      https://github.com/rear/rear/commit/15892253e3895065ce8b9ac4cd27088d23ee76d0
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Untabify umount_url and fix misleading indent


  Commit: 1efcf7045794851567cacf068f4d210224d8ce4a
      https://github.com/rear/rear/commit/1efcf7045794851567cacf068f4d210224d8ce4a
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Introduce function remove_temporary_mountpoint()

There are many places where a temporary mount point is being removed, so
introduce a shared function for it, even if it is a tiny one-liner.


  Commit: 538e8c0cae2d7e9150c852a2f9bc529663b7a24f
      https://github.com/rear/rear/commit/538e8c0cae2d7e9150c852a2f9bc529663b7a24f
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/backup/DUPLICITY/default/100_mount_duplicity_path.sh
    M usr/share/rear/backup/DUPLICITY/default/980_unmount_duplicity_path.sh
    M usr/share/rear/backup/NETFS/default/100_mount_NETFS_path.sh
    M usr/share/rear/backup/NETFS/default/980_umount_NETFS_dir.sh
    M usr/share/rear/lib/global-functions.sh
    M usr/share/rear/output/default/100_mount_output_path.sh
    M usr/share/rear/output/default/980_umount_output_dir.sh
    M usr/share/rear/prep/BORG/default/250_mount_usb.sh

  Log Message:
  -----------
  Move mountpoint removal/creation into u/mount_url

All callers of mount_url() create a temporary mount point for it, so we
can move the mount point creation there, together with all the exit task
logic, and the corresponding removal into umount_url().

Functional change: mount point does not get created and removed for URLs
that do not need to be mounted. This mount point would be unused and if
not, it would be a bug (nobody would save files that appear there, so
they would get lost).

Bug fix: call the filesystem-specific umount handling for davfs and
sshfs (it used to be ignored).


  Commit: 1ea884cfa378328ac2f7d5e1ab507dded78fd3db
      https://github.com/rear/rear/commit/1ea884cfa378328ac2f7d5e1ab507dded78fd3db
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-04 (Fri, 04 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Detect unmountable URL with scheme_supports_filesystem

If scheme_supports_filesystem returns false, the URL is definitely not
mountable. Use this instead of keepeing a long list of unmountable URL
schemes in mount_url() and umount_url().

There are some URLs that allow filesystem accces even if we don't
mount and umount them (file: and sometimes iso: ). They need to be
handled as special case. Still, this reduces the amount of hardcoded
schemes quite a lot.


  Commit: 74243f94c1cb8765381a2de41bce2b72ea765aba
      https://github.com/rear/rear/commit/74243f94c1cb8765381a2de41bce2b72ea765aba
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Optionally perform lazy unmount

Add an additional parameter to perform_umount_url() and
umount_mountpoint() that, if present and nonempty, instructs them to try
lazy unmount if regular unmount fails. To be used in exit tasks in order
to allow cleanup of the temporary mount points.

Inspired by the lazy unmount in cleanup_build_area_and_end_program().

The davfs handling is now too coplicated so move it to its own helper
function.


  Commit: b8f094dadd7e562c17e86f97112cbfc2a5acba08
      https://github.com/rear/rear/commit/b8f094dadd7e562c17e86f97112cbfc2a5acba08
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Use perform_umount_url in exit tasks

This ensures that a filesystem-specific unmount command is invoked
instead of the generic `umount`, if necessary. It also makes
improvements like lazy unmount easy to consistently use in all places
that benefit from them.

Does not work for the var:// scheme, but this has been broken anyway
(by the "fix" for #170, see the discussion for commit
20359a987662cc0c3fcfa52d62d1feac7cd55850, and custom umount command has
never been used in exit tasks.)


  Commit: 0af0d4f4315f8912c3aee16b4f8ba4a5216b22a4
      https://github.com/rear/rear/commit/0af0d4f4315f8912c3aee16b4f8ba4a5216b22a4
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Add a comment about ftpfs

Does it need fusermount -u for umount? Seems similar to the sshfs case,
but does not get the same special handling.


  Commit: bcb95ba9248c57532df70a95534e8d7be34a1fd7
      https://github.com/rear/rear/commit/bcb95ba9248c57532df70a95534e8d7be34a1fd7
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/output/PXE/default/800_copy_to_tftp.sh
    M usr/share/rear/output/PXE/default/810_create_pxelinux_cfg.sh

  Log Message:
  -----------
  Remove mountpoint handling from PXE

Temporary mountpoint for output is now handled internally in
mount_url()/umount_url().

Fixes a possible data loss: the exit task was using rm -Rf, which would
remove the output files for other machines, if the same tftpboot
directory is shared among multiple machines (and ReaR encourages this,
since it uses a unique prefix to identify backups from different
machines) and unmounting it fails. The shared implementation of exit
tasks now uses rmdir instead of rm -r.


  Commit: 42ad4ddc9761ef0eea25623e52d785b7d91e1ad1
      https://github.com/rear/rear/commit/42ad4ddc9761ef0eea25623e52d785b7d91e1ad1
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/framework-functions.sh

  Log Message:
  -----------
  Use remove_temporary_mountpoint at the end

Instead of special code in cleanup_build_area_and_end_program(), use the
shared helper to remove $BUILD_DIR/outputfs. This translates to using
rmdir always instead of using rm -Rf in case the filesystem is not
mounted. This is simpler and safer (disaster could arise if the
mountpoint command fails for any unexpected reason, for example).

It intentionally reverts PR #578 (commit
606d8ce380c130b5fc7412d856ba2fedd194a505). It is not clear why was
.lockfile present in the unmounted outputfs (this was the reason for the
change). According to my understanding of the code, this should not
happen. Even if this happens, it means that some part of the code is
using outputfs incorrectly (it should be used only if mounted, otherwise
data put there are lost) and it is better to be warned about the
problem rather than ignoring it by blindly removing the whole directory.


  Commit: 3e8bb2600b1ce6f2d7ee1d3813f48ac0087e2bc4
      https://github.com/rear/rear/commit/3e8bb2600b1ce6f2d7ee1d3813f48ac0087e2bc4
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/framework-functions.sh

  Log Message:
  -----------
  Suggest a safe way to remove build area

When the build area is left for debugging purposes, we tell the user
that they should remove it later. Since the output filesystem with
valuable old backups or backups from other machines may be still mounted
at $BUILD_DIR/outputfs, tell the user to use rm -Rf --one-file-system
instead of just rm -Rf. This prevents data loss if the user follows the
intruction literally.

All supported distributions now support the --one-file-system option, so
there should be no problem with this.


  Commit: f7939e3b94e538a681ad38bfda4e1c4d04695054
      https://github.com/rear/rear/commit/f7939e3b94e538a681ad38bfda4e1c4d04695054
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/framework-functions.sh

  Log Message:
  -----------
  Use the shared helper to umount output at the end

umount_mountpoint_lazy() was introduced for this purpose earlier, so use
it also in cleanup_build_area_and_end_program().

Note: the cleanup in cleanup_build_area_and_end_program() duplicates the
exit tasks used by mount_url()/umount_url(). This has been the case for
a long time and should not be a real problem, althought it would be
probably safe to remove this cleanup from
cleanup_build_area_and_end_program() and leave only those in
mount_url()/umount_url().


  Commit: 4c8c0ba76476f19bf0e0c43b3c662d2516024569
      https://github.com/rear/rear/commit/4c8c0ba76476f19bf0e0c43b3c662d2516024569
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/950_copy_result_files.sh

  Log Message:
  -----------
  Use scheme_supports_filesystem when copying output

Instead of hardcoding a list of output URL schemes that can be accessed
as a filesystem when copying result files to their destination, use the
scheme_supports_filesystem() function that tells us exactly this
information. This extends supported schemes to anything that can be
mounted by mount_url().


  Commit: 1e71249f01881101c90ed4cf9745f523ae3c5440
      https://github.com/rear/rear/commit/1e71249f01881101c90ed4cf9745f523ae3c5440
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Remove temporary mountpoint only if it exists

If the temporary mountpoint does not exist, we should be happy, since
there is nothing to do. Unfortunately rmdir (unlike rm -Rf) reports an
error in this case, so test for the directory existence before.


  Commit: 84cf6d008afb44c2bcd82607354bef49bc2c2418
      https://github.com/rear/rear/commit/84cf6d008afb44c2bcd82607354bef49bc2c2418
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/output/default/950_copy_result_files.sh

  Log Message:
  -----------
  Treat empty OUTPUT_URL as "null"

This used to be the case before. This case is encountered without any
local configuration, because by default BACKUP_URL is empty and
OUTPUT_URL is equal to BACKUP_URL. It might be better to use
OUTPUT_URL=null as the default value instead of empty, but for now
preserve the previous behavior.


  Commit: 3f750480d5e27a0320bd00a4c1d5e08147405cf8
      https://github.com/rear/rear/commit/3f750480d5e27a0320bd00a4c1d5e08147405cf8
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Don't do anything special with stdout

Apparently it is not needed since #2416.


  Commit: b57dfdc6291abd44062df5bac7cf11b5087630bf
      https://github.com/rear/rear/commit/b57dfdc6291abd44062df5bac7cf11b5087630bf
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/lib/global-functions.sh

  Log Message:
  -----------
  Change API for lazy umount of umount helper funcs

Functions that optionally perform lazy umount now require that the
optional argument is either empty or equal to "lazy". An arbitrary
nonempty string is not enough and is now considered a bug. This improves
readability. Suggestion by @jsmeix.


  Commit: 14a9a61d62b02a27f55495320f2d0e1090a72c30
      https://github.com/rear/rear/commit/14a9a61d62b02a27f55495320f2d0e1090a72c30
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    R usr/share/rear/output/PXE/default/820_copy_to_net.sh

  Log Message:
  -----------
  Revert "add script from Andreas Lemke to xfer to PXE server"

This reverts commit d850c4094238a03c9b926b88d7e1582ecd28af52.

The script probably made sense when it was submitted, but in the
meantime generic handling of transfer of output files got implemented
( a8fdc44 ), which made it redundant when it was committed.


  Commit: 2b56f383ee329501031121d422d85d93bc4d2efc
      https://github.com/rear/rear/commit/2b56f383ee329501031121d422d85d93bc4d2efc
  Author: Pavel Cahyna <pcahyna at redhat.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh

  Log Message:
  -----------
  Remove executable bit from a script

Fixes mistaken commit 27de4cd95dabe86f531f132108744459b2dbcfd0 in PR #2608


  Commit: 98a22402594b23f6aad78cd448c73cb525a69ad7
      https://github.com/rear/rear/commit/98a22402594b23f6aad78cd448c73cb525a69ad7
  Author: Johannes Meixner <jsmeix at suse.com>
  Date:   2021-06-18 (Fri, 18 Jun 2021)

  Changed paths:
    M usr/share/rear/backup/DUPLICITY/default/100_mount_duplicity_path.sh
    M usr/share/rear/backup/DUPLICITY/default/980_unmount_duplicity_path.sh
    M usr/share/rear/backup/NETFS/default/100_mount_NETFS_path.sh
    M usr/share/rear/backup/NETFS/default/150_save_copy_of_prefix_dir.sh
    M usr/share/rear/backup/NETFS/default/200_make_prefix_dir.sh
    M usr/share/rear/backup/NETFS/default/250_create_lock.sh
    M usr/share/rear/backup/NETFS/default/970_remove_lock.sh
    M usr/share/rear/backup/NETFS/default/980_umount_NETFS_dir.sh
    R usr/share/rear/backup/YUM/default/400_create_include_exclude_files.sh
    A usr/share/rear/backup/YUM/default/400_create_include_exclude_files.sh
    R usr/share/rear/build/YUM/default/600_create_python_symlink.sh
    A usr/share/rear/build/YUM/default/600_create_python_symlink.sh
    M usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh
    M usr/share/rear/lib/framework-functions.sh
    M usr/share/rear/lib/global-functions.sh
    M usr/share/rear/output/PXE/default/800_copy_to_tftp.sh
    M usr/share/rear/output/PXE/default/810_create_pxelinux_cfg.sh
    R usr/share/rear/output/PXE/default/820_copy_to_net.sh
    M usr/share/rear/output/default/100_mount_output_path.sh
    M usr/share/rear/output/default/150_save_copy_of_prefix_dir.sh
    M usr/share/rear/output/default/200_make_prefix_dir.sh
    M usr/share/rear/output/default/250_create_lock.sh
    M usr/share/rear/output/default/950_copy_result_files.sh
    M usr/share/rear/output/default/970_remove_lock.sh
    M usr/share/rear/output/default/980_umount_output_dir.sh
    M usr/share/rear/prep/BORG/default/250_mount_usb.sh
    R usr/share/rear/prep/YUM/default/070_set_backup_archive.sh
    A usr/share/rear/prep/YUM/default/070_set_backup_archive.sh
    R usr/share/rear/restore/DUPLICITY/default/100_mount_duplicity_path.sh
    A usr/share/rear/restore/DUPLICITY/default/100_mount_duplicity_path.sh
    R usr/share/rear/restore/DUPLICITY/default/980_unmount_duplicity_path.sh
    A usr/share/rear/restore/DUPLICITY/default/980_unmount_duplicity_path.sh
    R usr/share/rear/restore/YUM/default/100_mount_YUM_path.sh
    A usr/share/rear/restore/YUM/default/100_mount_YUM_path.sh
    R usr/share/rear/restore/YUM/default/980_umount_YUM_dir.sh
    A usr/share/rear/restore/YUM/default/980_umount_YUM_dir.sh
    R usr/share/rear/verify/DUPLICITY/default/100_mount_duplicity_path.sh
    A usr/share/rear/verify/DUPLICITY/default/100_mount_duplicity_path.sh
    R usr/share/rear/verify/DUPLICITY/default/980_unmount_duplicity_path.sh
    A usr/share/rear/verify/DUPLICITY/default/980_unmount_duplicity_path.sh
    R usr/share/rear/verify/YUM/default/050_check_YUM_requirements.sh
    A usr/share/rear/verify/YUM/default/050_check_YUM_requirements.sh
    R usr/share/rear/verify/YUM/default/060_mount_YUM_path.sh
    A usr/share/rear/verify/YUM/default/060_mount_YUM_path.sh
    R usr/share/rear/verify/YUM/default/070_set_backup_archive.sh
    A usr/share/rear/verify/YUM/default/070_set_backup_archive.sh
    R usr/share/rear/verify/YUM/default/980_umount_YUM_dir.sh
    A usr/share/rear/verify/YUM/default/980_umount_YUM_dir.sh

  Log Message:
  -----------
  Merge pull request #2625 from pcahyna/fix-backup-removal-in-exit-task-updated

Fix backup removal in exit task and cleanup handling of URL mountpoints:
Cleanup of temporary mount point handling, particularly for output.
Unification of mount point umount and cleanup move
to the mount_url() and umount_url() functions.
Replaced the various "rm -rf" of the mountpoint by "rmdir" which fixes
https://github.com/rear/rear/issues/2611
Added lazy umount in case normal umount does not succeed.
If build dir is kept (cf. KEEP_BUILD_DIR), propose a safe way to remove it
to the user via "rm -Rf --one-file-system" instead of just "rm -Rf" where
the user risks to remove everything below that mountpoint if still mounted.
Fixes also some other bugs noted in the process:
Filesystem-specific umount command not called
https://github.com/rear/rear/commit/20359a987662cc0c3fcfa52d62d1feac7cd55850#r51319634
Unknown schemes considered invalid, see the discussion under
https://github.com/rear/rear/pull/932
Identical scripts under DUPLICITY and YUM replaced by symlinks.
Reverted
https://github.com/rear/rear/pull/782
that had reintroduced
https://github.com/rear/rear/issues/465
which got re-reported as
https://github.com/rear/rear/issues/2611
Reverted
https://github.com/rear/rear/pull/578
because it is not clear how .lockfile can exist in the
unmounted filesystem, and if it does, it is a bug.
Reverted 
https://github.com/rear/rear/commit/d850c4094238a03c9b926b88d7e1582ecd28af52
because it became meanwhile obsoleted by
https://github.com/rear/rear/commit/a8fdc445d0d6f7f9184dc6633817928f96aae9bc


Compare: https://github.com/rear/rear/compare/2433c72f614a...98a22402594b


More information about the rear-devel mailing list