MESSAGE
| DATE | 2016-11-21 |
| FROM | ruben safir
|
| SUBJECT | Subject: [Learn] Fwd: PNG coding
|
From learn-bounces-at-nylxs.com Mon Nov 21 09:41:31 2016
Return-Path:
X-Original-To: archive-at-mrbrklyn.com
Delivered-To: archive-at-mrbrklyn.com
Received: from www.mrbrklyn.com (www.mrbrklyn.com [96.57.23.82])
by mrbrklyn.com (Postfix) with ESMTP id 685D6161314;
Mon, 21 Nov 2016 09:41:31 -0500 (EST)
X-Original-To: learn-at-nylxs.com
Delivered-To: learn-at-nylxs.com
Received: from [10.0.0.62] (flatbush.mrbrklyn.com [10.0.0.62])
by mrbrklyn.com (Postfix) with ESMTP id 219CC161312
for ; Mon, 21 Nov 2016 02:12:22 -0500 (EST)
References:
<31145d76-97ae-0d80-363f-b0b2ebbaffc5-at-verizon.net>
<529b73c9-0c8c-4f30-8291-56e4d31f61e4-at-googlegroups.com>
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
<20161115234903.5851f221-at-maxa-pc>
<20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
<20161115234214.3c2bf7e4-at-maxa-pc>
<582eb801.1344453-at-news.xs4all.nl>
<5685c80f-f8d3-4c8e-9f8d-27dc1890c8e4-at-googlegroups.com>
<1e3e797f-c498-4939-b683-f6b09d1c506b-at-googlegroups.com>
To: learn-at-nylxs.com
From: ruben safir
X-Forwarded-Message-Id:
<0926c75e-8ccf-4e74-9887-82396932235d-at-googlegroups.com>
<31145d76-97ae-0d80-363f-b0b2ebbaffc5-at-verizon.net>
<529b73c9-0c8c-4f30-8291-56e4d31f61e4-at-googlegroups.com>
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
<20161115234903.5851f221-at-maxa-pc>
<20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
<20161115234214.3c2bf7e4-at-maxa-pc>
<582eb801.1344453-at-news.xs4all.nl>
<5685c80f-f8d3-4c8e-9f8d-27dc1890c8e4-at-googlegroups.com>
<1e3e797f-c498-4939-b683-f6b09d1c506b-at-googlegroups.com>
Message-ID: <8fd6b681-2941-7311-9902-179ef12b9e55-at-mrbrklyn.com>
Date: Mon, 21 Nov 2016 02:12:22 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
MIME-Version: 1.0
In-Reply-To:
Content-Type: multipart/mixed; boundary="------------A8BF854E67DD7FF514138687"
X-Mailman-Approved-At: Mon, 21 Nov 2016 09:41:27 -0500
Subject: [Learn] Fwd: PNG coding
X-BeenThere: learn-at-nylxs.com
X-Mailman-Version: 2.1.17
Precedence: list
List-Id:
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Errors-To: learn-bounces-at-nylxs.com
Sender: "Learn"
This is a multi-part message in MIME format.
--------------A8BF854E67DD7FF514138687
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
this is the full thread worth reading
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="PNG coding.eml"
Path: reader2.panix.com!panix!not-for-mail
From: ruben safir
Newsgroups: comp.lang.c
Subject: PNG coding
Date: Tue, 15 Nov 2016 14:40:04 -0500
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID:
NNTP-Posting-Host: www.mrbrklyn.com
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Trace: reader2.panix.com 1479238806 4510 96.57.23.82 (15 Nov 2016 19:40:06 GMT)
X-Complaints-To: abuse-at-panix.com
NNTP-Posting-Date: Tue, 15 Nov 2016 19:40:06 +0000 (UTC)
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
Xref: panix comp.lang.c:1094867
Does anyone know of any sample code for reading PNG files that doesn't
include using libpng.... just straight direct access.
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!not-for-mail
From: ruben safir
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Tue, 15 Nov 2016 15:20:21 -0500
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID:
References:
NNTP-Posting-Host: www.mrbrklyn.com
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Trace: reader2.panix.com 1479241221 8935 96.57.23.82 (15 Nov 2016 20:20:21 GMT)
X-Complaints-To: abuse-at-panix.com
NNTP-Posting-Date: Tue, 15 Nov 2016 20:20:21 +0000 (UTC)
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Xref: panix comp.lang.c:1094871
On 11/15/2016 03:02 PM, Rick C. Hodgin wrote:
> On Windows and with a C++ compiler, you can use the built-in
> Gdiplus::Bitmap object and load the png and then write it out
> as a standard .bmp format, which is much easier to access
> line-by-line
I can't use MS, and I'm not looking for a solution, just source code to
learn by, read and adapt.
FWIW, Rick, I wouldn't touch a slaveware MS OS if it was the last
platform on earth. I suggest strongly for you to try something else.
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
X-Received: by 10.157.46.106 with SMTP id c39mr8674695otd.106.1479241558217;
Tue, 15 Nov 2016 12:25:58 -0800 (PST)
X-Received: by 10.157.4.119 with SMTP id 110mr1831563otc.11.1479241558180;
Tue, 15 Nov 2016 12:25:58 -0800 (PST)
Path: reader2.panix.com!panix!bloom-beacon.mit.edu!bloom-beacon.mit.edu!168.235.88.217.MISMATCH!feeder.erje.net!2.us.feeder.erje.net!news.ripco.com!news.glorb.com!w132no121263ita.0!news-out.google.com!c26ni415itd.0!nntp.google.com!w132no121260ita.0!postnews.google.com!glegroupsg2000goo.googlegroups.com!not-for-mail
Newsgroups: comp.lang.c
Date: Tue, 15 Nov 2016 12:25:57 -0800 (PST)
In-Reply-To:
Complaints-To: groups-abuse-at-google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=68.44.229.196;
posting-account=BcpLkAoAAACbVwkzAAKP0XXOd-MDREpp
NNTP-Posting-Host: 68.44.229.196
References:
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <0926c75e-8ccf-4e74-9887-82396932235d-at-googlegroups.com>
Subject: Re: PNG coding
From: "Rick C. Hodgin"
Injection-Date: Tue, 15 Nov 2016 20:25:58 +0000
Content-Type: text/plain; charset=UTF-8
Xref: panix comp.lang.c:1094873
On Tuesday, November 15, 2016 at 3:20:29 PM UTC-5, ruben safir wrote:
> On 11/15/2016 03:02 PM, Rick C. Hodgin wrote:
> > On Windows and with a C++ compiler, you can use the built-in
> > Gdiplus::Bitmap object and load the png and then write it out
> > as a standard .bmp format, which is much easier to access
> > line-by-line
>
> I can't use MS, and I'm not looking for a solution, just source code to
> learn by, read and adapt.
>
> FWIW, Rick, I wouldn't touch a slaveware MS OS if it was the last
> platform on earth. I suggest strongly for you to try something else.
Couldn't agree more, though I also extend the definition to include
Mac OS and Linux among others:
http://www.libsf.org/wiki/index.php/Main_Page
https://github.com/RickCHodgin/libsf/tree/master/exodus
I named it Exodus in the 1990s, about six years before I became a
Christian. It was given that name because I wanted it to be "a mass
departure from evil" which was Microsoft.
Today I am pursuing Exodus an OS/2 compatible kernel with some more
modern extensions, called ES/2.
Best regards,
Rick C. Hodgin
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin3!goblin1!goblin.stu.neva.ru!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: "James R. Kuyper"
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Tue, 15 Nov 2016 15:34:19 -0500
Organization: A noiseless patient Spider
Message-ID:
References:
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Info: mx02.eternal-september.org; posting-host="de2b9cf661b11c67a4716a5ec8a7fe30";
logging-data="27585"; mail-complaints-to="abuse-at-eternal-september.org"; posting-account="U2FsdGVkX1+CQfV3knADvJVrZrYeGLRbBWC7K/HJyWk="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Cancel-Lock: sha1:N/Th+6PQMye+kEl8N1DlwBAnDlg=
Xref: panix comp.lang.c:1094875
On 11/15/2016 02:40 PM, ruben safir wrote:
> Does anyone know of any sample code for reading PNG files that doesn't
> include using libpng.... just straight direct access.
How about looking at libpng itself? It is the official reference
implementation for PNG, and the source code is available under a
permissive free software license. I'd think you'd learn at least as much
by looking at the code for the official reference implementation as
you'd learn by looking at any other code.
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!not-for-mail
From: Popping mad
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Tue, 15 Nov 2016 20:57:07 +0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID:
References:
NNTP-Posting-Host: www.mrbrklyn.com
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Trace: reader2.panix.com 1479243427 20608 96.57.23.82 (15 Nov 2016 20:57:07 GMT)
X-Complaints-To: abuse-at-panix.com
NNTP-Posting-Date: Tue, 15 Nov 2016 20:57:07 +0000 (UTC)
User-Agent: Pan/0.140 (Chocolate Salty Balls; GIT b8fc14e
git.gnome.org/git/pan2)
Xref: panix comp.lang.c:1094877
On Tue, 15 Nov 2016 15:34:19 -0500, James R. Kuyper wrote:
> How about looking at libpng itself?
because after 15 years they suddenly broke half my images with their bug
update and I specifically need to work around it to fix my images, not
through it.
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!bloom-beacon.mit.edu!bloom-beacon.mit.edu!168.235.88.217.MISMATCH!2.us.feeder.erje.net!feeder.erje.net!1.eu.feeder.erje.net!news.unit0.net!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: "James R. Kuyper"
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Tue, 15 Nov 2016 16:12:15 -0500
Organization: A noiseless patient Spider
Message-ID: <31145d76-97ae-0d80-363f-b0b2ebbaffc5-at-verizon.net>
References:
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Info: mx02.eternal-september.org; posting-host="de2b9cf661b11c67a4716a5ec8a7fe30";
logging-data="3992"; mail-complaints-to="abuse-at-eternal-september.org"; posting-account="U2FsdGVkX181gQkjqI79eHiifEUqaoTDh8WDbXHWv04="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Cancel-Lock: sha1:6+wMfsNtwUOd7wG43Eb9qdtdtxs=
Xref: panix comp.lang.c:1094879
On 11/15/2016 03:57 PM, Popping mad wrote:
> On Tue, 15 Nov 2016 15:34:19 -0500, James R. Kuyper wrote:
>
>> How about looking at libpng itself?
>
> because after 15 years they suddenly broke half my images with their bug
> update and I specifically need to work around it to fix my images, not
> through it.
Reading the source code might be the easiest way to figure out how to
work around it.
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
X-Received: by 10.99.155.9 with SMTP id r9mr1239677pgd.127.1479247255664;
Tue, 15 Nov 2016 14:00:55 -0800 (PST)
X-Received: by 10.157.56.132 with SMTP id p4mr1893848otc.20.1479247255593;
Tue, 15 Nov 2016 14:00:55 -0800 (PST)
Path: reader2.panix.com!panix!bloom-beacon.mit.edu!bloom-beacon.mit.edu!168.235.88.217.MISMATCH!feeder.erje.net!2.us.feeder.erje.net!news.ripco.com!news.glorb.com!o1no3018ito.0!news-out.google.com!c26ni20itd.0!nntp.google.com!o1no3009ito.0!postnews.google.com!glegroupsg2000goo.googlegroups.com!not-for-mail
Newsgroups: comp.lang.c
Date: Tue, 15 Nov 2016 14:00:55 -0800 (PST)
In-Reply-To:
Complaints-To: groups-abuse-at-google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=2001:480:240:0:0:0:0:2;
posting-account=rH_E3woAAADR_9M5Q9bWAyDPjAMcf7LJ
NNTP-Posting-Host: 2001:480:240:0:0:0:0:2
References:
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <529b73c9-0c8c-4f30-8291-56e4d31f61e4-at-googlegroups.com>
Subject: Re: PNG coding
From: jadill33-at-gmail.com
Injection-Date: Tue, 15 Nov 2016 22:00:55 +0000
Content-Type: text/plain; charset=UTF-8
Xref: panix comp.lang.c:1094892
On Tuesday, November 15, 2016 at 3:57:17 PM UTC-5, Popping mad wrote:
> On Tue, 15 Nov 2016 15:34:19 -0500, James R. Kuyper wrote:
>
> > How about looking at libpng itself?
>
> because after 15 years they suddenly broke half my images with their bug
> update and I specifically need to work around it to fix my images, not
> through it.
One option is to use a tool like Beyond Compare to compare the source trees
of the two versions of libpng to try to isolate the file change(s) that
caused your problem. If the versions are not too far apart (so that there
are not many large file differences), you may be able to merge the new file
versions from the new libpng into the old libpng and try to rebuild
incrementally to eliminate file changes that do not affect your images.
An alternative is that you can do the same in Subversion/git or other source
control program to visualize the differences. Commit the newest version of
libpng that works with your images to a repo, then copy over files from the
new version of libpng one at a time until you hack a version that works for
you.
You can also try contacting a libpng developer and if you have a couple
sample images that demonstrate the problem that you're able to share, they
may be considerate enough to do this analysis for you.
I've experienced something similar myself when I need to migrate Wireshark
plugin code to new major versions (e.g. 1.10->1.12->2.0->2.2) as the API
changes can be somewhat significant.
Best regards,
John D.
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin3!goblin.stu.neva.ru!news-1.dfn.de!news.dfn.de!news.informatik.hu-berlin.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: Jorgen Grahn
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: 20 Nov 2016 11:14:54 GMT
Message-ID:
References:
<529b73c9-0c8c-4f30-8291-56e4d31f61e4-at-googlegroups.com>
X-Trace: individual.net UjE1bN8X45SfFD18DCVgPwLM5jK7uQijZV/KMXt+S8J55MUhv+
Cancel-Lock: sha1:yrTZtCuxwclTzRqwo3WvmUAUtsg=
User-Agent: slrn/pre1.0.0-18 (Linux)
Xref: panix comp.lang.c:1095288
On Tue, 2016-11-15, jadill33-at-gmail.com wrote:
> On Tuesday, November 15, 2016 at 3:57:17 PM UTC-5, Popping mad wrote:
>> On Tue, 15 Nov 2016 15:34:19 -0500, James R. Kuyper wrote:
>>
>> > How about looking at libpng itself?
>>
>> because after 15 years they suddenly broke half my images with their bug
>> update and I specifically need to work around it to fix my images, not
>> through it.
>
> One option is to use a tool like Beyond Compare to compare the source trees
> of the two versions of libpng to try to isolate the file change(s) that
> caused your problem. If the versions are not too far apart (so that there
> are not many large file differences), you may be able to merge the new file
> versions from the new libpng into the old libpng and try to rebuild
> incrementally to eliminate file changes that do not affect your images.
>
> An alternative is that you can do the same in Subversion/git or other source
> control program to visualize the differences. Commit the newest version of
> libpng that works with your images to a repo, then copy over files from the
> new version of libpng one at a time until you hack a version that works for
> you.
I'd just clone the libpng git repo (listed at http://libpng.sourceforge.net/)
and do a 'git bisect' on it. Or do the binary search for the oldest
bad version manually.
> You can also try contacting a libpng developer and if you have a couple
> sample images that demonstrate the problem that you're able to share, they
> may be considerate enough to do this analysis for you.
Yes, they may find that interesting. Especially if there's software
out there generating such files.
/Jorgen
--
// Jorgen Grahn \X/ snipabacken.se> O o .
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin2!goblin1!goblin.stu.neva.ru!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Richard Heathfield
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Tue, 15 Nov 2016 21:45:21 +0000
Organization: Fix this later
Message-ID:
References:
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Tue, 15 Nov 2016 21:44:50 -0000 (UTC)
Injection-Info: mx02.eternal-september.org; posting-host="5aa817f98dde6e19527f193bb7e85bbb";
logging-data="12567"; mail-complaints-to="abuse-at-eternal-september.org"; posting-account="U2FsdGVkX186CDciL9uz0wnxOnOK6AASd9y2b8PdEzlpiH/aHBotuw=="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Cancel-Lock: sha1:vCTFmHElt84jkAbxuGg3K+ApIXc=
Xref: panix comp.lang.c:1094886
On 15/11/16 20:34, James R. Kuyper wrote:
> On 11/15/2016 02:40 PM, ruben safir wrote:
>> Does anyone know of any sample code for reading PNG files that doesn't
>> include using libpng.... just straight direct access.
>
> How about looking at libpng itself? It is the official reference
> implementation for PNG, and the source code is available under a
> permissive free software license. I'd think you'd learn at least as much
> by looking at the code for the official reference implementation as
> you'd learn by looking at any other code.
Just to encourage people to take James up on his suggestion, here are
some statistics from the libpng code.
Firstly, there'd better be LOTS of it. We want to learn a lot, don't we?
C files: 17
H files: 6
That looks promising, but not too impressive. Maybe we'd better count
lines instead:
Lines of C: 33593
Lines of H: 6791
Ah, that's better! But how much does it all *weigh*?
Kilobytes of C: 1116
Kilobytes of H: 304
Ah, that's very reassuring. Over a megabyte of C code. Let's remind
ourselves what we're doing: reading a PNG into memory, or writing it
out. Okay, a megabyte (of *source*) seems perfectly reasonable for that.
Now, we all know that there's nothing like the pre-processor for making
code readable, so let's hope they use the pre-processor a *lot*:
Number of lines of H starting #: 1667
Number of lines of C starting #: 2223
Yep, excellent.
Disappointingly, they supply a file, example.c, that shows you how to
use the library. You'd have thought you'd learn more from reading the
whole of the library code line by line in order to understand how to
call it.
But they redeem themselves in a most unexpected manner --- example.c
/does not compile/! And it's not supposed to. There's a comment to that
effect:
"This file does not currently compile, because it is missing certain
parts, like allocating memory to hold an image. You will have to supply
these parts to get it to compile."
They refer the interested reader, instead, to a 2000+ line test program,
which I presume was originally developed as an IOCCC entry. The main()
function calls a function named test_one_file(), which sounds quite a
promising place to start. Let's take a look at it:
/* Test one file */
static int
test_one_file(PNG_CONST char *inname, PNG_CONST char *outname)
{
static png_FILE_p fpin;
static png_FILE_p fpout; /* "static" prevents setjmp corruption */
pngtest_error_parameters error_parameters;
png_structp read_ptr;
png_infop read_info_ptr, end_info_ptr;
#ifdef PNG_WRITE_SUPPORTED
png_structp write_ptr;
png_infop write_info_ptr;
png_infop write_end_info_ptr;
#ifdef PNG_WRITE_FILTER_SUPPORTED
int interlace_preserved = 1;
#endif /* WRITE_FILTER */
#else /* !WRITE */
png_structp write_ptr = NULL;
Actually, since it's about 900 lines long, I don't think I'll show any
more of it. It's obviously all quite self-explanatory, and you won't
need any help at all in understanding it completely. Five minutes should
do it. Maybe ten if you're having a bad day. For the whole library?
Perhaps an hour. After all, it's only 40,000 lines.
--
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin1!goblin.stu.neva.ru!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: "James R. Kuyper"
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Tue, 15 Nov 2016 17:15:55 -0500
Organization: A noiseless patient Spider
Message-ID: <58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
References:
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Info: mx02.eternal-september.org; posting-host="de2b9cf661b11c67a4716a5ec8a7fe30";
logging-data="19856"; mail-complaints-to="abuse-at-eternal-september.org"; posting-account="U2FsdGVkX19Tv2XiPMojeSBLI0WT9SAIq3NRihtZk7Y="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Cancel-Lock: sha1:2o959Bce0c+ZbPzLIaNC1idXxQ4=
Xref: panix comp.lang.c:1094895
On 11/15/2016 04:45 PM, Richard Heathfield wrote:
> On 15/11/16 20:34, James R. Kuyper wrote:
>> On 11/15/2016 02:40 PM, ruben safir wrote:
>>> Does anyone know of any sample code for reading PNG files that doesn't
>>> include using libpng.... just straight direct access.
>>
>> How about looking at libpng itself? It is the official reference
>> implementation for PNG, and the source code is available under a
>> permissive free software license. I'd think you'd learn at least as much
>> by looking at the code for the official reference implementation as
>> you'd learn by looking at any other code.
>
> Just to encourage people to take James up on his suggestion, here are
> some statistics from the libpng code.
...
> Ah, that's very reassuring. Over a megabyte of C code. Let's remind
> ourselves what we're doing: reading a PNG into memory, or writing it
> out. Okay, a megabyte (of *source*) seems perfectly reasonable for that.
With you, it's sometimes hard to tell, but I'm going to assume you're
being sarcastic.
Keep in mind that this is the official reference implementation of PNG.
Any fully functional alternative to libpng is likely to be comparably
complicated - unless the implementors of libpng are exceptionally
incompetent. I know nothing about that personally, but it seems unlikely
that PNG would be as popular as it is if that were the case.
Would a less-than-fully-functional alternative be acceptable? In that
case, such an alternative could be a lot simpler - but that depends upon
which features of PNG ruben needs to handle correctly.
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin3!goblin1!goblin.stu.neva.ru!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Richard Heathfield
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Tue, 15 Nov 2016 22:41:23 +0000
Organization: Fix this later
Message-ID:
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Tue, 15 Nov 2016 22:40:52 -0000 (UTC)
Injection-Info: mx02.eternal-september.org; posting-host="5aa817f98dde6e19527f193bb7e85bbb";
logging-data="25540"; mail-complaints-to="abuse-at-eternal-september.org"; posting-account="U2FsdGVkX1+VWrJmpLJljsPfPUBHYgaTcdnU2x/8SGF083E5NSNotw=="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To: <58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
Cancel-Lock: sha1:uBx7ipwAF2ofWMjcMOqtuJveb+A=
Xref: panix comp.lang.c:1094896
On 15/11/16 22:15, James R. Kuyper wrote:
> On 11/15/2016 04:45 PM, Richard Heathfield wrote:
>> On 15/11/16 20:34, James R. Kuyper wrote:
>>> On 11/15/2016 02:40 PM, ruben safir wrote:
>>>> Does anyone know of any sample code for reading PNG files that doesn't
>>>> include using libpng.... just straight direct access.
>>>
>>> How about looking at libpng itself? It is the official reference
>>> implementation for PNG, and the source code is available under a
>>> permissive free software license. I'd think you'd learn at least as much
>>> by looking at the code for the official reference implementation as
>>> you'd learn by looking at any other code.
>>
>> Just to encourage people to take James up on his suggestion, here are
>> some statistics from the libpng code.
> ...
>> Ah, that's very reassuring. Over a megabyte of C code. Let's remind
>> ourselves what we're doing: reading a PNG into memory, or writing it
>> out. Okay, a megabyte (of *source*) seems perfectly reasonable for that.
>
> With you, it's sometimes hard to tell,
Good. :-)
> but I'm going to assume you're being sarcastic.
On this occasion, you have correctly divined my intent.
> Keep in mind that this is the official reference implementation of PNG.
> Any fully functional alternative to libpng is likely to be comparably
> complicated - unless the implementors of libpng are exceptionally
> incompetent. I know nothing about that personally, but it seems unlikely
> that PNG would be as popular as it is if that were the case.
I don't think for a moment that they're incompetent. But readable code
clearly is not their priority.
>
> Would a less-than-fully-functional alternative be acceptable?
Hell yes!
> In that
> case, such an alternative could be a lot simpler - but that depends upon
> which features of PNG ruben needs to handle correctly.
I can't speak for Ruben's needs. But if anyone is planning to write a
simple PNG library, I would recommend the following interface:
/* returns width*height unsigned longs, encoding RGBA 00-FF */
unsigned long *png_load(const char *filename,
size_t *width,
size_t *height);
int png_save(const char filename,
unsigned long *frame,
size_t width,
size_t height);
And *nothing else*!
--
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!bloom-beacon.mit.edu!bloom-beacon.mit.edu!168.235.88.217.MISMATCH!2.us.feeder.erje.net!feeder.erje.net!1.eu.feeder.erje.net!news.albasani.net!.POSTED!not-for-mail
From: Melzzzzz
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Tue, 15 Nov 2016 23:49:03 +0100
Organization: albasani.net
Message-ID: <20161115234903.5851f221-at-maxa-pc>
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Trace: news.albasani.net A5EQeyuQ/4lyLSflSXcR/Vo8RvndWcKLGv2xy0FDVgm7DAeK75CGqLcSw397tCOP4lP75/bJE34O6TSkhFGToQ==
NNTP-Posting-Date: Tue, 15 Nov 2016 22:49:03 +0000 (UTC)
Injection-Info: news.albasani.net; logging-data="KnZMwYx+R2Js3XWERbh+PuAdxZUPOLwxKL+oWH//r98XEJ9HwMIzlWGWsRWhpbErjGxLnlDQzrCuulQEnGOlWguGLBidoJEUjIpbM0sfpkYhk+rNgeF7fr12GSxSplNb"; mail-complaints-to="abuse-at-albasani.net"
X-Newsreader: Claws Mail 3.14.1 (GTK+ 2.24.31; x86_64-unknown-linux-gnu)
Cancel-Lock: sha1:SfoWz+6H8L84tX5CWtzLA1qzjJI=
Xref: panix comp.lang.c:1094898
On Tue, 15 Nov 2016 22:41:23 +0000
Richard Heathfield wrote:
>
> I can't speak for Ruben's needs. But if anyone is planning to write a
> simple PNG library, I would recommend the following interface:
>
> /* returns width*height unsigned longs, encoding RGBA 00-FF */
> unsigned long *png_load(const char *filename,
> size_t *width,
> size_t *height);
>
> int png_save(const char filename,
> unsigned long *frame,
> size_t width,
> size_t height);
>
> And *nothing else*!
>
Well, one can write wrapper around libpng, possibly C++ ;)
--
press any key to continue or any other to quit
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin2!goblin1!goblin.stu.neva.ru!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Anton Shepelev
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Wed, 16 Nov 2016 15:23:43 +0300
Organization: A noiseless patient Spider
Message-ID: <20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Injection-Info: mx02.eternal-september.org; posting-host="d86994faa7e08fe05bc91e89243eb468";
logging-data="1552"; mail-complaints-to="abuse-at-eternal-september.org"; posting-account="U2FsdGVkX18C67BUBd78iyB76zqtKj7wBNjzfNqumoI="
X-Newsreader: Sylpheed 3.5.0 (GTK+ 2.24.23; i686-pc-mingw32)
Cancel-Lock: sha1:rR/ewsQ4X42cV08uROlWCLXNZj0=
Xref: panix comp.lang.c:1094917
Richard Heathfield:
>I can't speak for Ruben's needs. But if anyone is
>planning to write a simple PNG library, I would
>recommend the following interface:
>
>/* returns width*height unsigned longs, encoding RGBA 00-FF */
>unsigned long *png_load(const char *filename,
> size_t *width,
> size_t *height);
>
>int png_save(const char filename,
> unsigned long *frame,
> size_t width,
> size_t height);
>
>And *nothing else*!
Why do you return a potentially sparce matrix?
Whereas many APIs accept a raw companct pixel array,
it might be good to return that, i.e. void* or
char*, where each four bytes represent an RGBA pix-
el.
--
() ascii ribbon campaign - against html e-mail
/\ http://preview.tinyurl.com/qcy6mjc [archived]
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!bloom-beacon.mit.edu!bloom-beacon.mit.edu!168.235.88.217.MISMATCH!2.us.feeder.erje.net!feeder.erje.net!1.eu.feeder.erje.net!feeder3.usenet.farm!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Richard Heathfield
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Wed, 16 Nov 2016 13:57:10 +0000
Organization: Fix this later
Message-ID:
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
<20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Wed, 16 Nov 2016 13:56:38 -0000 (UTC)
Injection-Info: mx02.eternal-september.org; posting-host="e03646e6fad0bb563bff67abd59053c1";
logging-data="18282"; mail-complaints-to="abuse-at-eternal-september.org"; posting-account="U2FsdGVkX185O0LLqP78pSCq+ULEJ7FDmJikEuNpgxEzFTeObYhXeA=="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To: <20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
Cancel-Lock: sha1:mAJUnEkYKX/elrE+HjAqvbP0BS8=
Xref: panix comp.lang.c:1094919
On 16/11/16 12:23, Anton Shepelev wrote:
> Richard Heathfield:
>
>> I can't speak for Ruben's needs. But if anyone is
>> planning to write a simple PNG library, I would
>> recommend the following interface:
>>
>> /* returns width*height unsigned longs, encoding RGBA 00-FF */
>> unsigned long *png_load(const char *filename,
>> size_t *width,
>> size_t *height);
>>
>> int png_save(const char filename,
>> unsigned long *frame,
>> size_t width,
>> size_t height);
>>
>> And *nothing else*!
>
> Why do you return a potentially sparce matrix?
Well, I don't return anything, because that's a suggested interface, not
a report of existing code. But why /would/ I? Because it's easy to
understand and easy to incorporate into one's own code, that's why. Them
as wants complicated can use libpng.
--
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin1!goblin.stu.neva.ru!news.albasani.net!.POSTED!not-for-mail
From: BGB
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Fri, 18 Nov 2016 01:24:23 -0600
Organization: albasani.net
Message-ID:
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
<20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Trace: news.albasani.net 8H/9LrRfU3rFB9mVjdWW1C5PJ3P9eILlASm/jVf9P2+OROs7SaDnx0VZi5fFAfsKfvz4OcPeRz5fuJ5HyTKjYiyJlntY/GUgyIFWtQ3tuW+GLF/m3EUf9vUHlJP12jSb
NNTP-Posting-Date: Fri, 18 Nov 2016 07:24:04 +0000 (UTC)
Injection-Info: news.albasani.net; logging-data="P9vHHEvtRqZhsa+gkf57LZUYvd1JzzIfSBgdmpYZGMRo5cPgSzB8R+zpPMd/L19ZJWGqhYr6wrhDtYPHyI1sAonNK0GRG7i+8fkW6pKjqjB6KpXdPCZyNY4j4jbNudvlq0AY23HgTvMq0m+Gg8jhh0F4BBs30atM8FR+lmkdhdE="; mail-complaints-to="abuse-at-albasani.net"
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Cancel-Lock: sha1:j0uFYp6KUNyYl3sCW5Nr3/Sfwgg=
Xref: panix comp.lang.c:1095014
On 11/16/2016 7:57 AM, Richard Heathfield wrote:
> On 16/11/16 12:23, Anton Shepelev wrote:
>> Richard Heathfield:
>>
>>> I can't speak for Ruben's needs. But if anyone is
>>> planning to write a simple PNG library, I would
>>> recommend the following interface:
>>>
>>> /* returns width*height unsigned longs, encoding RGBA 00-FF */
>>> unsigned long *png_load(const char *filename,
>>> size_t *width,
>>> size_t *height);
>>>
>>> int png_save(const char filename,
>>> unsigned long *frame,
>>> size_t width,
>>> size_t height);
>>>
>>> And *nothing else*!
>>
>> Why do you return a potentially sparce matrix?
>
> Well, I don't return anything, because that's a suggested interface, not
> a report of existing code. But why /would/ I? Because it's easy to
> understand and easy to incorporate into one's own code, that's why. Them
> as wants complicated can use libpng.
>
FWIW, the reason mine used buffers, rather than files or file-names, was
because buffers are generally more versatile (pretty much everything can
be done via a buffer on modern systems).
a lot of my other formats have a more complicated interface and
implementation, but mostly for other reasons of speed. PNG remained
simple, partly as earlier on I have basically given up on making it
fast. some aspects of PNG's design basically make fast decoding (much
past around 40-50 megapixels/second) essentially a road-block.
I had tried before, but just couldn't really manage to get it much
faster than this.
some of my other codecs can generally be a lot faster (faster ones
getting 300-400 Mpix/sec per thread for decoding, *), but at the cost of
a lot of hair, and frequently lackluster quality/bitrate.
*: on a 3GHz K10, peak is ~ 1.4 Gpix/sec for 4 threads. have gotten
around 3 Gpix/sec on a dual-socket Xeon E5410 (2.33 GHz) with 8 threads.
though, this is with inherently lossy VQ based designs.
could work on a fast lossless codec (probably based around blocky
Haar-like filter and Rice coding or similar, 1), but lack a strong
use-case ATM.
1: ex, possible design:
colorspace=GDbDrA (G, B-G, R-G, A)
blocks use 8x8 blocky filter.
at a basic level, pairs of points may be transformed:
A,B -> A+B,A-((A+B)/2)
a1=a+b; b1=a-(a1>>1);
a2=(a1>>1)+b1; b2=a1-a2;
these could be stacked up into a 2D block transform.
coefficients would use RLE and Rice coding.
count coded as a signed Rice symbol
0=EOB
<0=number of zeroes (followed by single coeff)
>0=number of non-zero coeffs.
each coeff is a signed Rice-coded value
could probably use an adaptive length-limited variant.
...
I had promising results with a similar design in the past, but didn't
really pursue it, as, for lossly cases, speeds fell short of what was
possible with VQ, but if doing lossless it could make sense, given VQ is
basically incapable of doing lossless.
though, unclear is if it would offer any sufficient advantages (in terms
of speed) vs PNG to make it worthwhile.
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin2!goblin1!goblin.stu.neva.ru!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: BartC
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Fri, 18 Nov 2016 11:56:01 +0000
Organization: A noiseless patient Spider
Message-ID:
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
<20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Fri, 18 Nov 2016 11:55:30 -0000 (UTC)
Injection-Info: mx02.eternal-september.org; posting-host="0ee68c94a1cd0e432e7f11f2d521d1fd";
logging-data="6380"; mail-complaints-to="abuse-at-eternal-september.org"; posting-account="U2FsdGVkX1+WFsv+gi387J/sb0QjdzAi"
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Cancel-Lock: sha1:vSOGS4fVX9kDv86FMBVTxXFSPGE=
Xref: panix comp.lang.c:1095030
On 18/11/2016 07:24, BGB wrote:
> a lot of my other formats have a more complicated interface and
> implementation, but mostly for other reasons of speed. PNG remained
> simple, partly as earlier on I have basically given up on making it
> fast. some aspects of PNG's design basically make fast decoding (much
> past around 40-50 megapixels/second) essentially a road-block.
That doesn't sound too bad. The fastest C jpeg decoder I've tried took 6
seconds for an 80 Mpixel image (IIRC), so some 13Mpix/second. (Although
that image had full chroma resolution so more info than normal.)
> I had tried before, but just couldn't really manage to get it much
> faster than this.
>
> some of my other codecs can generally be a lot faster (faster ones
> getting 300-400 Mpix/sec per thread for decoding, *), but at the cost of
> a lot of hair, and frequently lackluster quality/bitrate.
Again that's very good unless you're talking about something like
uncompressed BMP where it depends on how fast you can load a file into
memory. (But BMP is lossless so probably not.)
Doing a test now, of simply reading an 80MB file and expanding each byte
to three, to end up with 240MB in memory (and doing nothing more with
the result), took 0.27 secs so an upper limit of just under
900MB/second, using gcc-O3, and even that takes advantage of the file
already being cached.
With your figures, what size of input file does 300-400Mpix represent?
(I guess this will be video rather than still images.)
--
Bartc
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin3!goblin.stu.neva.ru!news.roellig-ltd.de!open-news-network.org!news.albasani.net!.POSTED!not-for-mail
From: BGB
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Fri, 18 Nov 2016 12:48:02 -0600
Organization: albasani.net
Message-ID:
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
<20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Trace: news.albasani.net seTvLkmKbViaVtbVbRFed+F8pYrggQVXiISJtH23Ryh4kURwUwn+XxpEvjloxv6X+cZ/iBo6W7mvJt0IuN9XkimkFKAkaabcGFRwdhiontrWywad+vnQMnZXug1rLLEm
NNTP-Posting-Date: Fri, 18 Nov 2016 18:47:41 +0000 (UTC)
Injection-Info: news.albasani.net; logging-data="cqiUxx32fD9suoZjMyCdSC18m4+T8rIqTZsd3iOM+tWD1HospXwID70snv88dfAc2DykMLqqvvkHhnoWnREj3T6wKOZXe2GvznB16qVbC2I3+Bz9smINEMaloliuCvuBcieQ/gtuNpyTAMauv8FUJ4Vdv5Am1Q8Axw9egyNnECo="; mail-complaints-to="abuse-at-albasani.net"
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Cancel-Lock: sha1:CRyJswMyTvmSGnPKRcr7tS2OSUI=
Xref: panix comp.lang.c:1095105
On 11/18/2016 5:56 AM, BartC wrote:
> On 18/11/2016 07:24, BGB wrote:
>
>> a lot of my other formats have a more complicated interface and
>> implementation, but mostly for other reasons of speed. PNG remained
>> simple, partly as earlier on I have basically given up on making it
>> fast. some aspects of PNG's design basically make fast decoding (much
>> past around 40-50 megapixels/second) essentially a road-block.
>
> That doesn't sound too bad. The fastest C jpeg decoder I've tried took 6
> seconds for an 80 Mpixel image (IIRC), so some 13Mpix/second. (Although
> that image had full chroma resolution so more info than normal.)
>
yeah. mine are generally a little faster than this.
IME, can get JPEG decoding up to around 70-90 Mpix/sec, but typically
with 4:2:0 subsampling.
>
>> I had tried before, but just couldn't really manage to get it much
>> faster than this.
>>
>> some of my other codecs can generally be a lot faster (faster ones
>> getting 300-400 Mpix/sec per thread for decoding, *), but at the cost of
>> a lot of hair, and frequently lackluster quality/bitrate.
>
> Again that's very good unless you're talking about something like
> uncompressed BMP where it depends on how fast you can load a file into
> memory. (But BMP is lossless so probably not.)
>
for uncompressed BMP, one is going to run head-on with being IO bound
(well, at least if using normal HDDs, which is my case).
but, no, it is not raw BMPs.
though, I typically still use the BMP header for still images, for other
reasons.
FWIW: for loading, I use a VFS which can help optimize IO some by
reading in groups of files all in a shared buffer (as reading lots of
small files individually from the HDD tends to be fairly slow).
generally files are clustered by type, and if Deflate is used, it is
applied over the group of files (vs per-file, as it would be in ZIP).
> Doing a test now, of simply reading an 80MB file and expanding each byte
> to three, to end up with 240MB in memory (and doing nothing more with
> the result), took 0.27 secs so an upper limit of just under
> 900MB/second, using gcc-O3, and even that takes advantage of the file
> already being cached.
>
> With your figures, what size of input file does 300-400Mpix represent?
> (I guess this will be video rather than still images.)
>
my test images had generally been around 1 to 16 Mpix (ex: 1024x1024 to
4096x4096), but the figure is mostly how quickly they can be decoded in
a loop (note that images this size are too big to fit in cache for the
BGRA decode path).
bitrates for still images are typically 0.8 to 1.6 bits/pixel.
quality/bitrate (Q/bpp) is typically still worse than JPEG, but not
drastically (and it has an advantage here at lower quality levels).
in the use-case, I was mostly trying for something "competitive" with
JPEGs, where it mostly needed to be able to maximize decode speed, not
be overly large, have passable image quality, ...
I have codecs for both still images and video (video is easier to make
fast, and because of delta compression it can break the 400 Mpix/sec
limit by not needing to update the whole framebuffer).
here is a basic format spec for one of my faster designs (BTIC4B):
https://github.com/cr88192/bgbtech_misc/blob/master/tool_btic4b/docs/2016-06-25_BTIC4B2.txt
but, basic overview is this design:
it encodes 8x8 pixel blocks as a color-vector, and a block of bits is
used to select interpolated colors from this vector. typically, the
vector is unpacked into a LUT, and for pixels, the values from this LUT
are copied to the output pixels.
for blocks which use lower resolutions, the same pixel values are copied
to multiple output pixels (and a flat 8x8 block is basically a
flood-fill operation).
there are blocks with higher levels of chroma subsampling (4:1:0, 4:2:0,
4:2:2, or 4:4:4), which can give better image quality, but the encoder
uses them sparingly (as they are fairly expensive).
typically pixels are worked with as 32-bit words, and in some cases,
SIMD/SSE2 operations may be used to optimize memory-fill operations
(speed advantages from SSE2 are modest but still significant).
an older, albeit more widely used slightly slower and with worse Q/bpp
design is this (BTIC1H):
https://github.com/cr88192/bgbtech_misc/wiki/BTIC1H
like the above, it is also reasonably fast.
BTIC1H was a successor to BTIC1C, which was a Deflate-compressed
heavily-extended variant of the "Apple QuickTime RPZA" video format
(from the early 90s), where 1H had notably improved over 1C's Q/bpp.
BTIC4B was derived from BTIC1H, with the most significant differences being:
BTIC4B uses 8x8 blocks rather than 4x4, which reduces per-block overheads;
it also uses an LSB-first bitstream rather than MSB first, where LSB
first has an advantage in terms of reducing the cost of bitstream
operations.
though, the 8x8 blocks result in a fair bit more complexity vs 4x4.
for 4B I had also developed a slightly better MTF scheme for doing
symbol entropy coding:
1H had used a scheme (SMTF) where symbols were either swapped towards
the front (index 1..31), or the symbol space was rotated and a symbol
was swapped into the resulting hole (index 32..255).
4B uses a scheme I had called STF2, which simply swaps symbols at index
I with ((I*7)/8), which seems overall both more effective, and is also
faster (given no "if()" branches are needed, where things like "if()"
branches and "for()" loops have a fairly high risk of adverse effects on
performance).
interestingly, there is a company known as Binomial which is working on
commercializing vaguely similar technology.
though, they seem to be going at things in a different direction.
and, my stuff here is basically MIT licensed.
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin1!goblin.stu.neva.ru!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: BartC
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Sat, 19 Nov 2016 00:21:43 +0000
Organization: A noiseless patient Spider
Message-ID:
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
<20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Sat, 19 Nov 2016 00:21:12 -0000 (UTC)
Injection-Info: mx02.eternal-september.org; posting-host="a574928bc16e2af6d10272c4ff1ffd00";
logging-data="8011"; mail-complaints-to="abuse-at-eternal-september.org"; posting-account="U2FsdGVkX1/2ymwjbx2oqMhcSVqEIjCe"
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Cancel-Lock: sha1:QtOF+dtNqOmY7bFrA16Y/rDJi7I=
Xref: panix comp.lang.c:1095145
On 18/11/2016 18:48, BGB wrote:
> On 11/18/2016 5:56 AM, BartC wrote:
>> On 18/11/2016 07:24, BGB wrote:
>>
>>> a lot of my other formats have a more complicated interface and
>>> implementation, but mostly for other reasons of speed. PNG remained
>>> simple, partly as earlier on I have basically given up on making it
>>> fast. some aspects of PNG's design basically make fast decoding (much
>>> past around 40-50 megapixels/second) essentially a road-block.
>>
>> That doesn't sound too bad. The fastest C jpeg decoder I've tried took 6
>> seconds for an 80 Mpixel image (IIRC), so some 13Mpix/second. (Although
>> that image had full chroma resolution so more info than normal.)
>>
>
> yeah. mine are generally a little faster than this.
> IME, can get JPEG decoding up to around 70-90 Mpix/sec, but typically
> with 4:2:0 subsampling.
OK, a bit of a challenge them! On a 2Mpix test image ('2x2' subsampling
if that means anything; I think chroma resolution is half luminance in X
and Y), I get 1.2Mpix/second on interpreted code, 12Mpix using my
compiler and 15Mpix using C and gcc-O3. (Pixel rates tend to reduce on
bigger images.)
However that's using a decoding algorithm I don't understand so it would
be interesting to try my own.
> bitrates for still images are typically 0.8 to 1.6 bits/pixel.
(Colour? I first played with this stuff in 1981 with 4-bit greyscale
images from a frame-grabber. If I wanted it efficient, I just used the
top bit to end up with 1.0 bits/pixels. But it /was/ just 1 bit per
pixel! Although high noise level, especially on video, made it seem a
little better.
An interesting experiment however a few years later, when I was
digitising live composite video using 6 bits (trying to get around some
scrambling then sending it out again via a DAC). I could disconnect
several of the lower bits and still get a reasonable colour picture with
just 3 or 4 bits! 1 bit was problematical though; not surprising as I
had to encode hoz and vert syncs in that one bit too.)
> here is a basic format spec for one of my faster designs (BTIC4B):
> https://github.com/cr88192/bgbtech_misc/blob/master/tool_btic4b/docs/2016-06-25_BTIC4B2.txt
That looks pretty comprehensive to me...
TBH when I work on this jpeg stuff I have to do it blindly without
understanding, and just see it gives something that looks about right as
the output.
--
Bartc
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin3!goblin.stu.neva.ru!news.roellig-ltd.de!open-news-network.org!news.albasani.net!.POSTED!not-for-mail
From: BGB
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Sat, 19 Nov 2016 01:50:36 -0600
Organization: albasani.net
Message-ID:
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
<20161116152343.8f327ba657c914b38a9a8492-at-g{oogle}mail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Trace: news.albasani.net GTDW4xmSGHxcDNbVxpIuyobrt45kQS9t2b8V1Iw18LrQ8kXxy7R5+ZkQA8nH54N1ewt7s0NhNLce1upe2wniaw==
NNTP-Posting-Date: Sat, 19 Nov 2016 07:50:13 +0000 (UTC)
Injection-Info: news.albasani.net; logging-data="7ZZxMjJpkw0frZtQSwvhOhYLZxgiFtpRjJ45BRl7aEjlYmDyFD1p0gDTzOSYWBNN6wOlu/G3b+DolnCnCW5XYSb2iCv66W/1gGNIOiIOlfyoxFpO9+aU2iSTr0igVXhC"; mail-complaints-to="abuse-at-albasani.net"
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
Thunderbird/45.4.0
In-Reply-To:
Cancel-Lock: sha1:8zclN0f0Q5Yi1h/EDrRx41QYQ4I=
Xref: panix comp.lang.c:1095171
On 11/18/2016 6:21 PM, BartC wrote:
> On 18/11/2016 18:48, BGB wrote:
>> On 11/18/2016 5:56 AM, BartC wrote:
>>> On 18/11/2016 07:24, BGB wrote:
>>>
>>>> a lot of my other formats have a more complicated interface and
>>>> implementation, but mostly for other reasons of speed. PNG remained
>>>> simple, partly as earlier on I have basically given up on making it
>>>> fast. some aspects of PNG's design basically make fast decoding (much
>>>> past around 40-50 megapixels/second) essentially a road-block.
>>>
>>> That doesn't sound too bad. The fastest C jpeg decoder I've tried took 6
>>> seconds for an 80 Mpixel image (IIRC), so some 13Mpix/second. (Although
>>> that image had full chroma resolution so more info than normal.)
>>>
>>
>> yeah. mine are generally a little faster than this.
>> IME, can get JPEG decoding up to around 70-90 Mpix/sec, but typically
>> with 4:2:0 subsampling.
>
> OK, a bit of a challenge them! On a 2Mpix test image ('2x2' subsampling
> if that means anything; I think chroma resolution is half luminance in X
> and Y), I get 1.2Mpix/second on interpreted code, 12Mpix using my
> compiler and 15Mpix using C and gcc-O3. (Pixel rates tend to reduce on
> bigger images.)
>
> However that's using a decoding algorithm I don't understand so it would
> be interesting to try my own.
>
speed is fairly dependent on optimizations, and I had optimized mine a bit.
one strategy is basically to make the decoder go through everything an
MCU/macroblock at a time, rather than using multiple stages (this helps
to keep things from going out of cache).
similar goes for doing a lot of stuff with unrolled loops, ...
>> bitrates for still images are typically 0.8 to 1.6 bits/pixel.
>
> (Colour? I first played with this stuff in 1981 with 4-bit greyscale
> images from a frame-grabber. If I wanted it efficient, I just used the
> top bit to end up with 1.0 bits/pixels. But it /was/ just 1 bit per
> pixel! Although high noise level, especially on video, made it seem a
> little better.
>
it is an average, but yes.
the reason it can go so low is not every pixel needs bits.
say, one has an 8x8 block, but the encoder decided to use 4x4x2 bits.
so, we have 64 pixels in 32 bits, or 0.5 bpp for the pixel data.
if it used a 2x2x2 block, it is 0.125 bpp, and no bits are used for a
flat color. in effect, for a 2x2 block, the 8x8 output block is filled
with 4 large pixels.
granted, one still needs bits for the color vector.
the encoder tries to select blocks such that cheaper blocks are used
where it wont matter as much, and more expensive blocks are used where
it is likely to be more noticeable (using a lot decision trees and
heuristics).
in places where there is more detail, more expensive blocks are used,
which can express more complex patterns, but frequently in large parts
of the image not a whole lot is going on, so the blocks can be
downsampled or expressed as a flat color.
generally the vector is (Y,U,V,Dy,Du,Dv), which may be interpreted
either as a vector between a pair of endpoints colors, or as a
bounding-box in YUV space (where YUV gives the box center, and Dyuv
gives the size).
in the most common blocks, 2 bits are used, which interpolate along the
Y axis.
Ym=Y-(Dy>>1); Yn=Ym+Dy
Um=U-(Du>>1); Un=Um+Du
Vm=V-(Dv>>1); Vn=Vm+Dv
say, for pixel bits:
00=Ym
01=2/3*Ym+1/3*Yn
10=1/3*Ym+2/3*Yn
11=Yn
in some cases, the same bits also select the U/V values:
00=Ym,Um,Vm
...
11=Yn,Un,Vn
so, Du and Dv may be negative (if negatively correlated with Y).
some other (more expensive) blocks separately interpolate Y,U,V values.
for unpacking block pixels, one can first generate a 4-element table
with all 4 possible colors in it, then the 2 bits will select which of
the table elements goes into the output pixel.
some other block formats use larger lookup tables.
for each block, the color vector is stored as a difference from a
predicted vector, generally using a modified Paeth variant (vaguely like
in PNG). the vector deltas are Rice-coded, along with block-commands,
but pixel bits are dumped into the bitstream as-is (the pixel-bits data
isn't particularly compressible)
> An interesting experiment however a few years later, when I was
> digitising live composite video using 6 bits (trying to get around some
> scrambling then sending it out again via a DAC). I could disconnect
> several of the lower bits and still get a reasonable colour picture with
> just 3 or 4 bits! 1 bit was problematical though; not surprising as I
> had to encode hoz and vert syncs in that one bit too.)
>
typically the logical colorspace has a full 8-bit depth.
because the pixel bits mostly select the colors, rather than define
them, there are fewer adverse effects to using fewer bits per-pixel.
>> here is a basic format spec for one of my faster designs (BTIC4B):
>> https://github.com/cr88192/bgbtech_misc/blob/master/tool_btic4b/docs/2016-06-25_BTIC4B2.txt
>>
>
> That looks pretty comprehensive to me...
>
> TBH when I work on this jpeg stuff I have to do it blindly without
> understanding, and just see it gives something that looks about right as
> the output.
>
BTIC4B is not JPEG, basically completely different technology (mostly
based on designs that were popular in the early/mid 90s but pretty much
died when JPEG and MPEG came along).
though, it is worth noting that at the time, storage and bandwidth were
the dominant concerns, and CPU speeds were going up rapidly. however,
CPU speeds have since effectively hit a plateau, but storage space and
bandwidth continue to increase, likewise for resolutions.
for example, a simple case is my PC, vs a 13 year old laptop.
if I want to do full-screen capture, I actually get higher framerates on
the laptop than on my PC.
reason is because the budget of clock-cycles-per-pixel is lower on a
3GHz processor dealing with 1920x1080, than on a 1.7GHz processor
dealing with 1024x768.
between them is a laptop with a 2.1 GHz processor and a 1440x900 display.
so, CPU clock speed went up 77%, but screen resolution went up by 164%.
difference in RAM is 3200%, and difference in available HDD space is
6600%. likewise, I have 18200% as much internet bandwidth as well.
...
--------------A8BF854E67DD7FF514138687
Content-Type: message/rfc822;
name="Re: PNG coding.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Re: PNG coding.eml"
Path: reader2.panix.com!panix!goblin3!goblin1!goblin.stu.neva.ru!news.albasani.net!.POSTED!not-for-mail
From: Melzzzzz
Newsgroups: comp.lang.c
Subject: Re: PNG coding
Date: Tue, 15 Nov 2016 23:42:14 +0100
Organization: albasani.net
Message-ID: <20161115234214.3c2bf7e4-at-maxa-pc>
References:
<58044c2b-c4d4-cfd7-7142-2b5ca73a6c28-at-verizon.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.albasani.net VlgqHu+wXj+N/gKf5tsGFbgUWt0o9qUN81wJ+DbUAXvTm0Hij0ROCzQRJCeuVE79/kKvE5a1D84KLnkFzOQu8Q==
NNTP-Posting-Date: Tue, 15 Nov 2016 22:42:14 +0000 (UTC)
Injection-Info: news.albasani.net; logg |
|
