[Rear-users] Profiling ReaR

Schlomo Schapiro schlomo at schapiro.org
Thu Jun 2 11:59:42 CEST 2011


On 02/06/11 01:34, Dag Wieers wrote:
> Using 'gzip --fast', instead of 'gzip --best', the pack stage is reduced 
> from 34 seconds to just 4 seconds. We may want to look at the trade-offs 
> we want to make here.
>      http://stephane.lesimple.fr/wiki/blog/lzop_vs_compress_vs_gzip_vs_bzip2_vs_lzma_vs_lzma2-xz_benchmark_reloaded
> This article seems to indicate that the default 'gzip -6' is a better 
> trade-off. When using the default, the pack stage takes 9 seconds with 
> minimal size differences (400k on a file of 51MB).
> So I commited this change.

This is really a good improvement and a few MB more or less indeed does not matter.

I am actually thinking about ReaR optimization each time I wait for a test run :-)

That was also the reason why you find timing infos in the log file, I wanted to
see what takes so long.

I would like to point out another aspect before you start optimizing the
mkrescue part: The backup will take much more time than the mkrescue part.
Especially using rsync which requires a full filesystem scan on source and
destination I would be surprised if the mkrescue part of ReaR is still so
relevant. With tar or other backups this is even worse.

Or do you operators really only do a mkrescue? I thought you do a full backup
onto the USB device as well...

If you want to go and start optimizing things maybe this could be a way to do it
(please tell us your thoughts about how to optimize):

* Put entire scriptlets into background if they are totally self contained.
Running cfg2html would be a prime example, it has no connection to ReaR at all.
* Use wait at the end of a stage to collect all background processes (I would
like us to keep the stages independant from each other as much as is possible).
Unfortunately you must use one wait for each subprocess to collect the return
code or use files as temporary storage.
* Create some log collection mechanism that collects all log output from a
background task and puts it as one piece into the ReaR log so that we don't get
log messages from different background tasks interleaved.

Sadly this would add a lot of complexity to ReaR so I would hope that it can be
turned off for debugging purposes.

Kind Regards,

More information about the rear-users mailing list