Compression

[NewsArchive]

I'm using SetupBuilder to replace a simplistic self-extracting RAR
approach. SB definitely brings many benefits to the table, but I
happened to notice that the size of the distributable is 250MB,
compared with only 150MB when using RAR.

I've already set Compression to "Maximum" Generator Settings. Is there
anything else I can do?

Mike Hanson
www.boxsoft.net

[NewsArchive]

Mike,

> I'm using SetupBuilder to replace a simplistic self-extracting RAR
> approach. SB definitely brings many benefits to the table, but I
> happened to notice that the size of the distributable is 250MB,
> compared with only 150MB when using RAR.
>
> I've already set Compression to "Maximum" Generator Settings. Is
> there anything else I can do?

Interesting question, I haven't seen it for a while. In short, compression
ratio (compression power) isn't the only valid factor. The compression
method used in SetupBuilder is still the perfect combination of compression
ratio, compression speed and reconstruction speed across a wide range of
different file types.

RAR makes use of the "solid compression" method for data compression of
multiple files. All the uncompressed files are concatenated and treated as
a single data block. Let us assume, you have 5000 files in your archive and
you only need the last file (and skip file 1 - 4999). Then you have to
unpack all the 5000 files in RAR to get your hand on the last file.
Compression ratio is very good in this case, but speed and performance wise
it is an absolute nightmare. In SetupBuilder, you can skip the 4999 files
within seconds and just decompress (install "on-the-fly") the last file.
This is impossible to do with RAR.

When you execute a SetupBuilder generated setup.exe, it starts immediately.
Most installation systems (including .MSI) have to unpack the compressed
archives before the real installation process can start. In SetupBuilder,
there is no "Please wait..." dialog because the setup.exe is a native
Windows executable with no "solid archive" attached.

There cannot be a perfect compression algorithm. It's proven that any
algorithm that can reduce the size of a certain set of data must also
increase the size of another set of data. Some algorithms are more
interested in getting smaller compression ratios while gaining on
decompression speed, while others will pick algorithms for maximizing
compression ratios even if compression and decompression take a long time.
For example, 7-Zip gets far better compression rates than standard Zip, even
better than RAR. But performance wise it's not the preferred algorithm
(especially for installation systems).

Hope this makes sense.

--
Friedrich Linder
Lindersoft | SetupBuilder | www.lindersoft.com
954.252.3910 (within US) | +1.954.252.3910 (outside US)

--SetupBuilder "point. click. ship"
--Helping You Build Better Installations
--Create Windows 10 ready installations in minutes
--Official COMODO Code Signing and SSL Certificate Partner

[NewsArchive]

Thanks for the extra info.

Our self-extracting RAR (EXE) loads pretty quickly, even though it's
150+ MB. It's definitely not unpacking everything first. Once the
user choses the desired decompression location, that's when it takes
time churning through the files.

The larger file is primarily an issue for data transfer speed. Since
it's almost always "all or nothing", we could also pack the files as a
RAR, then install and execute that during installation, as controlled
by SB.

Mike Hanson
www.boxsoft.net

[NewsArchive]

Hi Mike,

Yes, when I said 'unpacking everything first' I was referring to a real
installer, not to a simple self-extracting archive.

Let us build an installer for one of our typical Consulting projects.
16,239 files (a wide range of different data and file types, including
already highly-compressed Microsoft redistributables) in 1,195 folders.
Uncompressed data size: 1,915,610,890 bytes. Compressed data size with
WinZip: 1,313,939,159 bytes.

We create 1.) a SetupBuilder 10 installer including uninstaller, 2.) a
"non-solid" self-extracting RAR, 3.) a "solid" self-extracting RAR and 4.)
an installer with a known product (let's name it Install*****; Price:
$3,999). The test environment is running on a super-high speed NVMe SSD
(M.2 Standard).

1. SB10 installer and uninstaller (fully scripted deployment logic)

Compression : LSPack (backward compatible to Win95)
Generation Time : 2:15 minutes (135 seconds)
Original Data Size : 1,915,610,890 bytes
..EXE Size : 1,305,516,271 bytes
Decompression Time : 0:37 minutes (37 seconds)
Compression Ratio : 31.85 %

2. Self-extracting RAR (no deployment logic)

Compression : RAR
Generation Time : 5:38 minutes (338 seconds)
Original Data Size : 1,915,610,890 bytes
..EXE Size : 1,250,190,748 bytes
Decompression Time : 1:20 minutes (80 seconds)
Compression Ratio : 34.74 %

3. Self-extracting "solid" RAR (no deployment logic)

Compression : RAR
Generation Time : 6:49 minutes (409 seconds)
Original Data Size : 1,915,610,890 bytes
..EXE Size : 938,196,668 bytes
Decompression Time : 1:05 minutes (65 seconds)
Compression Ratio : 51.02 %

4. Install***** installer and uninstaller (scripted deployment logic)

Compression : LZMA (top speed only ~26 MB/s)
Generation Time : 21:19 minutes (1,279 seconds)
Original Data Size : 1,915,610,890 bytes
..EXE Size : 921,881,111 bytes
Decompression Time : 1:43 minutes (103 seconds)
Compression Ratio : 51.88 %

As you can see, there is a direct relation to compression ratio, performance
(and efficiency). If compression ratio is more important than compression
and decompression speed, then LZMA wins. The generation of the LZMA and RAR
files always takes much longer and requires more memory (e.g. LZMA takes
1,279 seconds compared to 135 seconds with LSPack). The self-extracting RAR
executables always decompresses all files. The SB10 installer has full
deployment and uninstall logic for all 16,239 files. In other words, in 37
seconds SetupBuilder does not only decompresses the 16,239 files but it also
executes the deployment logic for each and every file (including uninstall
generation).

For installation development systems, there has to be a good balance between
compression speed (affects the developer), decompression speed (affects the
customer) and compression ratio (affects developer and customer). But who
knows, maybe there will be a new lossless compression algorithm in the
future which can provide both great compression ratio and speed :-)

Friedrich

[NewsArchive]

Thanks for all the additional information. We were considering
creating a RAR, then building that with SB. I'll do some speed tests,
and if it turns out the SB approach is faster at install time, then
that might tip the scales for them.

Mike