MESSAGE
| DATE | 2016-12-15 |
| FROM | John Bowler
|
| SUBJECT | Re: [Learn] [png-mng-implement] 4 byte length storage
|
From learn-bounces-at-nylxs.com Thu Dec 15 15:13:26 2016
Return-Path:
X-Original-To: archive-at-mrbrklyn.com
Delivered-To: archive-at-mrbrklyn.com
Received: from www.mrbrklyn.com (www.mrbrklyn.com [96.57.23.82])
by mrbrklyn.com (Postfix) with ESMTP id 983DA161312;
Thu, 15 Dec 2016 15:13:26 -0500 (EST)
X-Original-To: learn-at-nylxs.com
Delivered-To: learn-at-nylxs.com
Received: by mrbrklyn.com (Postfix, from userid 1000)
id 625D9161311; Thu, 15 Dec 2016 15:13:22 -0500 (EST)
Resent-From: Ruben Safir
Resent-Date: Thu, 15 Dec 2016 15:13:22 -0500
Resent-Message-ID: <20161215201322.GA31872-at-www.mrbrklyn.com>
Resent-To: learn-at-nylxs.com
X-Original-To: ruben-at-mrbrklyn.com
Delivered-To: ruben-at-mrbrklyn.com
Received: from lists.sourceforge.net (lists.sourceforge.net [216.34.181.88])
by mrbrklyn.com (Postfix) with ESMTP id 7EC58161311
for ; Thu, 15 Dec 2016 10:20:17 -0500 (EST)
Received: from localhost ([127.0.0.1] helo=sfs-ml-2.v29.ch3.sourceforge.com)
by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from )
id 1cHXor-0002ZF-DE; Thu, 15 Dec 2016 15:19:41 +0000
Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
helo=mx.sourceforge.net)
by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from )
id 1cHXoq-0002Yg-2e for png-mng-implement-at-lists.sourceforge.net;
Thu, 15 Dec 2016 15:19:40 +0000
Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.213.177 as permitted sender)
client-ip=209.85.213.177;
envelope-from=john.cunningham.bowler-at-gmail.com;
helo=mail-yb0-f177.google.com;
Received: from mail-yb0-f177.google.com ([209.85.213.177])
by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:AES128-SHA:128)
(Exim 4.76) id 1cHXon-0004gn-MQ
for png-mng-implement-at-lists.sourceforge.net;
Thu, 15 Dec 2016 15:19:39 +0000
Received: by mail-yb0-f177.google.com with SMTP id d59so30533633ybi.1
for ;
Thu, 15 Dec 2016 07:19:37 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:in-reply-to:references:from:date
:message-id:subject:to;
bh=G1Ne1tWZSVyJRPhRbiDKMW1QT6sRrAiybR5JxGW6R7Y=;
b=mOP9vn2g2j2Xaluo3Ln9eFm8j7l4dWeMMyj5N1RxGra6DsbTXPOK0APEFZ981pj2L2
jHixNWsCjtXlnC04Q1zpeqonWQFhGV6YfB4E8SoF7jax+9wp79CFT5xp9vj1wTx0agGV
iAeq6zfcRseinGDfFe1+Bk9CY7FN4ccRPc19HxYXdePddhD9kkkSUfHh5Uuhg41284di
mjgNWY+4z9B85l3Xi5gmfTGr0upY+24yuD7Qyc+YVePhvvEGz2tbFMMYAd1KtgLZo8Yr
dZuXmzk5czDuKsT0MrG1aIsxE0Ug6Q1HVvauKbqFO0dxDHS0qDLxXGEOaIVQj7NbQLpQ
YUqA==
X-Gm-Message-State: AIkVDXJ2b3ePWif6f20Oc6Lx3v6Q8f+sgNMEdsxPLE7n8ZZ8i/TrjSfDjb3aXa0z4ba62vcStQHTECKTr7RtFQ==
X-Received: by 10.37.76.2 with SMTP id z2mr1403629yba.68.1481815171780; Thu,
15 Dec 2016 07:19:31 -0800 (PST)
MIME-Version: 1.0
Received: by 10.129.46.66 with HTTP; Thu, 15 Dec 2016 07:19:28 -0800 (PST)
In-Reply-To: <100bb932-d8c5-a32a-3f14-3e4a18f0cc21-at-mrbrklyn.com>
References: <17705460-c78b-6310-807f-417193dc4f6b-at-mrbrklyn.com>
<100bb932-d8c5-a32a-3f14-3e4a18f0cc21-at-mrbrklyn.com>
From: John Bowler
Date: Thu, 15 Dec 2016 10:19:28 -0500
Message-ID:
To: "PNG/MNG implementation discussion list"
X-Spam-Score: -1.1 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(john.cunningham.bowler[at]gmail.com)
0.5 RCVD_IN_SORBS_SPAM RBL: SORBS: sender is a spam source
[209.85.213.177 listed in dnsbl.sorbs.net]
-0.0 SPF_PASS SPF: sender matches SPF record
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1cHXon-0004gn-MQ
X-BeenThere: png-mng-implement-at-lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
Subject: Re: [Learn] [png-mng-implement] 4 byte length storage
X-BeenThere: learn-at-nylxs.com
Reply-To: PNG/MNG implementation discussion list
List-Id:
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: learn-bounces-at-nylxs.com
Sender: "Learn"
On Mon, Dec 12, 2016 at 5:45 PM, Ruben Safir wrote:
> So now I have a question about the data in the IDAT chuncks.
>
> do they likiwise need to be reversed in the byte order before sending
> them to be decompressed with zlib?
The zlib specification defines the compressed data as a *byte* stream.
This is then split into sections (arbitrarily, except that each
section must be less than 2^31 bytes long) and sections are stored in
a sequence of IDAT chunks (no intervening chunks permitted).
The only 8/32 bit issue here is that each *section* is checksumed
using CRC32 (the cyclic redundancy code defined for Ethernet packets)
and, while this also checksums a *byte* stream, the result is a 32-bit
number which is encoded at the end of each chunk as a big-endian
sequence of four bytes. This is exactly the same as the check sum on
every other chunk.
Internally the zlib stream is defined to end with an Adler32 checksum.
This is another 32-bit checksum of a sequence of bytes but now over
the whole data stream *before* compression. This is, in turn, encoded
as a sequence of 4 bytes at the end of the *compressed* stream but
that definition comes from the zlib spec. IRC it is big-endian (i.e.
the same as the CRC32 that immediately follows it.)
Anyway, the internals of zlib are not part of the PNG spec and they
have to deal with the simple fact that the zlib stream (well, the
deflate sub-part) is actually a *bit* stream that has to be mapped to
*bytes* to make a byte stream.
Roll-your-own zlib decompression is, in fact, very straight-forward
however nowhere near as simple as decoding a PNG stream using an
existing decompression implementation.
The relevant references are, high level to low:
PNG specification: http://www.libpng.org/pub/png/spec/iso/
(Glenn gave an equivalent link to the W3C web site; I prefer the
ISO spec because it is the normative version, it also has lots of nice
explanation and pretty pictures.)
Zlib specification: https://www.ietf.org/rfc/rfc1950.txt
Defines a big-endian multi-byte format
Deflate specification: https://www.ietf.org/rfc/rfc1951.txt
Defines a little-endian multi-byte format
Notice that the zlib and deflate specifications use different ordering
for multi-byte numbers. It is important to read the specification and
to read the *correct* specification when decoding data! Indeed, as
the deflate specification points out, different numbers within the
different parts of the spec are encoded in different ways; in some
cases numbers are written as Huffman codes and, in that case, the most
significant parts of the Huffman code are written first; the code is
effectively big-endian (although that description has its own set of
assumptions.)
John Bowler
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
png-mng-implement mailing list
png-mng-implement-at-lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/png-mng-implement
_______________________________________________
Learn mailing list
Learn-at-nylxs.com
http://lists.mrbrklyn.com/mailman/listinfo/learn
|
|
