MESSAGE
| DATE | 2016-12-15 |
| FROM | Ruben Safir
|
| SUBJECT | Re: [Learn] zlib demo with vector::resize
|
From learn-bounces-at-nylxs.com Thu Dec 15 20:23:31 2016
Return-Path:
X-Original-To: archive-at-mrbrklyn.com
Delivered-To: archive-at-mrbrklyn.com
Received: from www.mrbrklyn.com (www.mrbrklyn.com [96.57.23.82])
by mrbrklyn.com (Postfix) with ESMTP id 07014161312;
Thu, 15 Dec 2016 20:23:30 -0500 (EST)
X-Original-To: learn-at-nylxs.com
Delivered-To: learn-at-nylxs.com
Received: from [10.0.0.62] (flatbush.mrbrklyn.com [10.0.0.62])
by mrbrklyn.com (Postfix) with ESMTP id 22749160E77
for ; Thu, 15 Dec 2016 20:23:28 -0500 (EST)
To: learn-at-nylxs.com
References: <87r35898t0.fsf-at-contrapunctus.net>
From: Ruben Safir
Message-ID: <41da0fff-8c6f-e1fa-65a7-adc3560e2a7c-at-mrbrklyn.com>
Date: Thu, 15 Dec 2016 20:23:28 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.5.1
MIME-Version: 1.0
In-Reply-To: <87r35898t0.fsf-at-contrapunctus.net>
Subject: Re: [Learn] zlib demo with vector::resize
X-BeenThere: learn-at-nylxs.com
X-Mailman-Version: 2.1.17
Precedence: list
List-Id:
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: learn-bounces-at-nylxs.com
Sender: "Learn"
On 12/15/2016 04:33 PM, Christopher League wrote:
>
> Here's a cool little demo that generates some pseudo-random chars in a
> buffer (represented by `vector`), compresses them using zlib,
> decompresses, and checks that the final result matches the input.
>
> Thanks Ruben, for the afternoon hack. :/
>
> CL
>
> ~~~~ {.cpp}
> // zdemo
> // % make zdemo CXXFLAGS="-g -Wall" LDFLAGS=-lz
> #include
> #include
> #include
> #include
> #include
> #include "zlib.h"
> using namespace std;
>
> // This many bytes per kilobyte (kibibyte)
> const unsigned KIBI = 1024;
>
> // Since the deflating and inflating procedures have a lot in common, this is
> // an attempt to capture the commonality. Yes I wrote a class, try not to
> // faint.
> class Flater {
> public:
> const int LEVEL = Z_DEFAULT_COMPRESSION;
> const bool TRACE_RESIZES = true;
> const unsigned MINIMAL_CHUNK = 16 * KIBI; // Start point for output size
> Flater();
> virtual ~Flater() { };
> void run(vector& in, vector& out);
> protected:
> virtual int flate() = 0;
> z_stream zs;
> };
>
> Flater::Flater()
> {
> zs.zalloc = Z_NULL;
> zs.zfree = Z_NULL;
> zs.opaque = Z_NULL;
> }
>
> // Here's the generic algorithm. The input buffer is fully available. The
> // output buffer doubles its size as needed. It can start out empty, and we'll
> // allocate MINIMAL_CHUNK bytes up front.
> void Flater::run(vector& in, vector& out)
> {
> zs.avail_in = in.size(); // #bytes available to be read
> zs.next_in = in.data(); // ptr to bytes available to be read
>
> unsigned int bytes_written = 0;
> do {
> out.resize(max(MINIMAL_CHUNK, 2*(unsigned)out.size()));
> if(TRACE_RESIZES) {
> cout << " Resize " << out.size() << '\n';
> }
>
> zs.avail_out = out.size() - bytes_written;
> zs.next_out = out.data() + bytes_written;
>
> int r = flate();
> assert(Z_STREAM_ERROR != r);
>
> bytes_written = out.size() - zs.avail_out;
>
> } while (zs.avail_out == 0);
> assert(zs.avail_in == 0); // All input will be used
>
> out.resize(bytes_written); // Reduce size
> if(TRACE_RESIZES) {
> cout << " Finish " << out.size() << '\n';
> }
> }
>
> ////////////////////////////////////////////////////////////////
>
> // Here's the customizations for deflating (compression).
>
> class Deflater : public Flater {
> public:
> Deflater();
> ~Deflater();
> int flate();
> };
>
> Deflater::Deflater()
> {
> int r = deflateInit(&zs, LEVEL);
> assert(Z_OK == r);
> }
>
> int Deflater::flate()
> {
> return deflate(&zs, Z_FINISH);
> }
>
> Deflater::~Deflater()
> {
> deflateEnd(&zs);
> }
>
> ////////////////////////////////////////////////////////////////
>
> // Here's the customizations for inflating (decompression).
>
> class Inflater : public Flater {
> public:
> Inflater();
> ~Inflater();
> int flate();
> };
>
>
> Inflater::Inflater()
> {
> int r = inflateInit(&zs);
> assert(Z_OK == r);
> }
>
> int Inflater::flate()
> {
> return inflate(&zs, Z_FINISH);
> }
>
> Inflater::~Inflater()
> {
> inflateEnd(&zs);
> }
>
> ////////////////////////////////////////////////////////////////
>
> // For testing purposes, fill buffer with a mixture of lowercase letters and
> // random bytes. If the data is completely uniform-random in the range 0-255,
> // then we wouldn't be able to compress. Having a significant portion of
> // lower-case letters (maybe) makes it more compressible.
> void generate_data(vector& buffer)
> {
> assert(buffer.size() > 0);
> const unsigned RANDOM_RATIO = 10;
> srand(time(0));
> for(unsigned i = 0; i < buffer.size(); i++) {
> buffer[i] =
> (rand()%RANDOM_RATIO == 0) ?
> rand() :
> 'a' + (rand()%26);
> }
> }
>
>
> // Print sizes, and first several bytes in hex and ASCII, similar to a hex
> // dump.
> void output_sample(const vector& buffer)
> {
> cout << " "
> << buffer.size() << " bytes (~"
> << buffer.size()/KIBI << "K, ~"
> << buffer.size()/KIBI/KIBI << "M)\n";
>
> const unsigned SEVERAL = min(16, (int)buffer.size());
> cout.fill('0');
> cout << hex << " ";
> for(unsigned i = 0; i < SEVERAL; i++) {
> cout << setw(2) << (int)buffer[i] << ' ';
> }
> for(unsigned i = 0; i < SEVERAL; i++) {
> cout << (char)(isprint(buffer[i])? buffer[i] : '.');
> }
> cout << dec << '\n';
> }
>
> ////////////////////////////////////////////////////////////////
>
> int main()
> {
> const float INPUT_SIZE_MB = 25.7;
> const unsigned INPUT_SIZE_BYTES = INPUT_SIZE_MB * KIBI * KIBI;
>
> cout << "Generating some data...\n";
> vector original(INPUT_SIZE_BYTES);
> generate_data(original);
> output_sample(original);
>
> cout << "Compressing...\n";
> vector compressed;
> Deflater().run(original, compressed);
> output_sample(compressed);
>
> double ratio = (double)compressed.size() / original.size();
> cout << "Compression ratio: " << fixed << setprecision(3) << ratio << '\n';
>
> vector copy;
> cout << "Decompressing...\n";
> Inflater().run(compressed, copy);
> output_sample(copy);
>
> assert(copy.size() == INPUT_SIZE_BYTES);
> assert(copy == original);
> cout << "Exact match!\n";
>
> return 0;
> }
> ~~~~
>
>
>
>
> _______________________________________________
> Learn mailing list
> Learn-at-nylxs.com
> http://lists.mrbrklyn.com/mailman/listinfo/learn
>
wow
--
So many immigrant groups have swept through our town
that Brooklyn, like Atlantis, reaches mythological
proportions in the mind of the world - RI Safir 1998
http://www.mrbrklyn.com
DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002
http://www.nylxs.com - Leadership Development in Free Software
http://www2.mrbrklyn.com/resources - Unpublished Archive
http://www.coinhangout.com - coins!
http://www.brooklyn-living.com
Being so tracked is for FARM ANIMALS and and extermination camps,
but incompatible with living as a free human being. -RI Safir 2013
_______________________________________________
Learn mailing list
Learn-at-nylxs.com
http://lists.mrbrklyn.com/mailman/listinfo/learn
|
|
