t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Us=
ing Valgrind-3.15.0 and LibVEX; rerun with -h for copyright infov>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Co=
mmand: apache2 -k start -X
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3Dpan>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D In=
valid read of size 8
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D at=
0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0)an>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so)>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so)>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so)div>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so)an>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x1663D3: ap_run_open_logs (in /usr/sbin/apache2)
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x14043F: main (in /usr/sbin/apache2)
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Ad=
dress 0x5a44000 is not stack'd, malloc'd or (recently) free'd
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3Dpan>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3Dpan>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Pr=
ocess terminating with default action of signal 11 (SIGSEGV)
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Ac=
cess not within mapped region at address 0x5A44000
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D at=
0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0)an>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so)>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so)>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so)div>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so)an>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x1663D3: ap_run_open_logs (in /usr/sbin/apache2)
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x14043F: main (in /usr/sbin/apache2)
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
When using debug symbols, gdb indicated that it was erroring in very e=
arly in perl's runtime before it had got to any perl code - the exact line =
it was failing on was perl.c:2365
scriptname =3D argv[=
0];. It wasn't possible to reason beyond that as stepping through op=
timised code even with debug symbols is next to impossible to make any sens=
e of.
I did find that building perl without optimisations made the error go =
away.
I found the following closed issue: https://github.com/Perl/perl5/issu=
es/15806 which describes the same issue I was having.
Looking at the source for mod_perl, I found that the argv array passed to p=
erl_parse() is not NULL terminated as is required by perl - ( documentation=
: https://perldoc.perl.org/perlembed#Adding-a-Perl-interpreter-to-your-C-pr=
ogram )
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
As for why this works in 99.99% of cases, I have no idea. With the set up I=
have (Ubuntu 20.04 LXD container) it is very reproducible initially, but a=
fter I fiddle a bit (uninstall & reinstall packages, install debugging =
packages, etc etc) it stops being reproducible.
Such is undefined behaviour and invalid memory accesses, I suppose, but qu=
ite irritating.
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
I've attached my suggested patch.
t; color:rgb(0,0,0)">
eadonly_1">
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
I don't believe it's strictly relevant, but package versions are:
t; color:rgb(0,0,0)">
apache2: 2.4.41-4ubuntu3.1
t; color:rgb(0,0,0)">
libapache2-mod-perl2: 2.0.11-2
t; color:rgb(0,0,0)">
perl: 5.30.0-9ubuntu0.2
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
Thanks,
t; color:rgb(0,0,0)">
Charles
--
aining/DO178C/?utm_source=3Demail_footer">Register for our online DO-178C &=
amp; Multicore training with ConsuNova: 8-12th March 2021.--_000_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_--
--_004_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_
Content-Type: application/octet-stream;
name="mod_perl-argv-null-terminator.diff"
Content-Description: mod_perl-argv-null-terminator.diff
Content-Disposition: attachment;
filename="mod_perl-argv-null-terminator.diff"; size=756;
creation-date="Thu, 18 Feb 2021 15:54:38 GMT";
modification-date="Thu, 18 Feb 2021 16:14:35 GMT"
Content-Transfer-Encoding: base64
LS0tIGEvc3JjL21vZHVsZXMvcGVybC9tb2RwZXJsX2NvbmZpZy5jCTIwMTktMTAtMDUgMTI6MDQ6
NDEuMDAwMDAwMDAwICswMTAwCisrKyBiL3NyYy9tb2R1bGVzL3BlcmwvbW9kcGVybF9jb25maWcu
YwkyMDIxLTAyLTE3IDE5OjA3OjIzLjY0NjIwNDM2NCArMDAwMApAQCAtMTYzLDcgKzE2Myw4IEBA
CiAgICAgc2NmZy0+UGVybFBvc3RDb25maWdSZXF1aXJlID0KICAgICAgICAgYXByX2FycmF5X21h
a2UocCwgMSwgc2l6ZW9mKG1vZHBlcmxfcmVxdWlyZV9maWxlX3QgKikpOwogCi0gICAgc2NmZy0+
YXJndiA9IGFwcl9hcnJheV9tYWtlKHAsIDIsIHNpemVvZihjaGFyICopKTsKKyAgICAvKiAyIGFy
Z3VtZW50cyArIE5VTEwgdGVybWluYXRvciAqLworICAgIHNjZmctPmFyZ3YgPSBhcHJfYXJyYXlf
bWFrZShwLCAzLCBzaXplb2YoY2hhciAqKSk7CiAKICAgICBzY2ZnLT5zZXR2YXJzID0gYXByX3Rh
YmxlX21ha2UocCwgMik7CiAgICAgc2NmZy0+Y29uZmlndmFycyA9IGFwcl90YWJsZV9tYWtlKHAs
IDIpOwpAQCAtMjE5LDYgKzIyMCw5IEBACiAKICAgICAqYXJnYyA9IHNjZmctPmFyZ3YtPm5lbHRz
OwogCisgICAgLyogcGVybF9wYXJzZSgpIGV4cGVjdHMgYSBOVUxMIHRlcm1pbmF0ZWQgYXJndiBh
cnJheSAqLworICAgIG1vZHBlcmxfY29uZmlnX3Nydl9hcmd2X3B1c2goTlVMTCk7CisKICAgICBN
UF9UUkFDRV9nX2RvKGR1bXBfYXJndihzY2ZnKSk7CiAKICAgICByZXR1cm4gKGNoYXIgKiopc2Nm
Zy0+YXJndi0+ZWx0czsK
--_004_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Hangout mailing list
Hangout-at-nylxs.com
http://lists.mrbrklyn.com/mailman/listinfo/hangout
--_004_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_--
--_004_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_
Content-Type: multipart/alternative;
boundary="_000_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_"
--_000_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hi,
TL;DR: mod_perl is using perl_parse() incorrectly and not NULL-terminating =
the argv array passed to it.
While setting up a perl web application with mod_perl & apache, apache kept=
segfaulting.
Broke out gdb, and found that it was segfaulting within perl itself:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5=
.30
(gdb) bt
#0 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.s=
o.5.30
#1 0x00007ffff764cd0c in modperl_startup () from /usr/lib/apache2/modules/m=
od_perl.so
#2 0x00007ffff764cc97 in modperl_startup () from /usr/lib/apache2/modules/m=
od_perl.so
#3 0x00007ffff764d0fa in modperl_init () from /usr/lib/apache2/modules/mod_=
perl.so
#4 0x00007ffff764d27b in modperl_hook_init () from /usr/lib/apache2/modules=
/mod_perl.so
#5 0x00005555555b23d4 in ap_run_open_logs ()
#6 0x000055555558c440 in main ()
# valgrind apache2 -k start -X
=3D=3D22529=3D=3D Memcheck, a memory error detector
=3D=3D22529=3D=3D Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward =
et al.
=3D=3D22529=3D=3D Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyr=
ight info
=3D=3D22529=3D=3D Command: apache2 -k start -X
=3D=3D22529=3D=3D
=3D=3D22529=3D=3D Invalid read of size 8
=3D=3D22529=3D=3D at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/li=
bperl.so.5.30.0)
=3D=3D22529=3D=3D by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/module=
s/mod_perl.so)
=3D=3D22529=3D=3D by 0x55A8C96: modperl_startup (in /usr/lib/apache2/module=
s/mod_perl.so)
=3D=3D22529=3D=3D by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/m=
od_perl.so)
=3D=3D22529=3D=3D by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modu=
les/mod_perl.so)
=3D=3D22529=3D=3D by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2)
=3D=3D22529=3D=3D by 0x14043F: main (in /usr/sbin/apache2)
=3D=3D22529=3D=3D Address 0x5a44000 is not stack'd, malloc'd or (recently) =
free'd
=3D=3D22529=3D=3D
=3D=3D22529=3D=3D
=3D=3D22529=3D=3D Process terminating with default action of signal 11 (SIG=
SEGV)
=3D=3D22529=3D=3D Access not within mapped region at address 0x5A44000
=3D=3D22529=3D=3D at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/li=
bperl.so.5.30.0)
=3D=3D22529=3D=3D by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/module=
s/mod_perl.so)
=3D=3D22529=3D=3D by 0x55A8C96: modperl_startup (in /usr/lib/apache2/module=
s/mod_perl.so)
=3D=3D22529=3D=3D by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/m=
od_perl.so)
=3D=3D22529=3D=3D by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modu=
les/mod_perl.so)
=3D=3D22529=3D=3D by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2)
=3D=3D22529=3D=3D by 0x14043F: main (in /usr/sbin/apache2)
When using debug symbols, gdb indicated that it was erroring in very early =
in perl's runtime before it had got to any perl code - the exact line it wa=
s failing on was perl.c:2365 scriptname =3D argv[0];. It wasn't possible to=
reason beyond that as stepping through optimised code even with debug symb=
ols is next to impossible to make any sense of.
I did find that building perl without optimisations made the error go away.
I found the following closed issue: https://github.com/Perl/perl5/issues/15=
806 which describes the same issue I was having.
Looking at the source for mod_perl, I found that the argv array passed to p=
erl_parse() is not NULL terminated as is required by perl - ( documentation=
: https://perldoc.perl.org/perlembed#Adding-a-Perl-interpreter-to-your-C-pr=
ogram )
As for why this works in 99.99% of cases, I have no idea. With the set up I=
have (Ubuntu 20.04 LXD container) it is very reproducible initially, but a=
fter I fiddle a bit (uninstall & reinstall packages, install debugging pack=
ages, etc etc) it stops being reproducible. Such is undefined behaviour and=
invalid memory accesses, I suppose, but quite irritating.
I've attached my suggested patch.
See also Ubuntu bug report - https://bugs.launchpad.net/ubuntu/+source/liba=
pache2-mod-perl2/+bug/1915959
I don't believe it's strictly relevant, but package versions are:
apache2: 2.4.41-4ubuntu3.1
libapache2-mod-perl2: 2.0.11-2
perl: 5.30.0-9ubuntu0.2
Thanks,
Charles
--
Register for our online DO-178C & Multicore training with ConsuNova: 8-12th=
March 2021
il_footer>.
--_000_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
1">
t; color:rgb(0,0,0)">
Hi,
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
TL;DR: mod_perl is using perl_parse() incorrectly and not NULL-terminating =
the argv array passed to it.
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
While setting up a perl web application with mod_perl & apache, apache =
kept segfaulting.
Broke out gdb, and found that it was segfaulting within perl itself:div>
Program receive=
d signal SIGSEGV, Segmentation fault.
0x00007ffff7358=
ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30>
(gdb) bt=
#0 0x00007ffff7=
358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30div>
#1 0x00007ffff7=
64cd0c in modperl_startup () from /usr/lib/apache2/modules/mod_perl.son>
#2 0x00007ffff7=
64cc97 in modperl_startup () from /usr/lib/apache2/modules/mod_perl.son>
#3 0x00007ffff7=
64d0fa in modperl_init () from /usr/lib/apache2/modules/mod_perl.so<=
/div>
#4 0x00007ffff7=
64d27b in modperl_hook_init () from /usr/lib/apache2/modules/mod_perl.sopan>
#5 0x0000555555=
5b23d4 in ap_run_open_logs ()
#6 0x0000555555=
58c440 in main ()
# valgrind apa=
che2 -k start -X t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Me=
mcheck, a memory error detector
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Co=
pyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Us=
ing Valgrind-3.15.0 and LibVEX; rerun with -h for copyright infov>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Co=
mmand: apache2 -k start -X
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3Dpan>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D In=
valid read of size 8
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D at=
0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0)an>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so)>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so)>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so)div>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so)an>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x1663D3: ap_run_open_logs (in /usr/sbin/apache2)
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x14043F: main (in /usr/sbin/apache2)
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Ad=
dress 0x5a44000 is not stack'd, malloc'd or (recently) free'd
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3Dpan>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3Dpan>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Pr=
ocess terminating with default action of signal 11 (SIGSEGV)
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D Ac=
cess not within mapped region at address 0x5A44000
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D at=
0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0)an>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so)>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so)>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so)div>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so)an>
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x1663D3: ap_run_open_logs (in /usr/sbin/apache2)
t; color:rgb(0,0,0)">
=3D=3D22529=3D=3D by=
0x14043F: main (in /usr/sbin/apache2)
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
When using debug symbols, gdb indicated that it was erroring in very e=
arly in perl's runtime before it had got to any perl code - the exact line =
it was failing on was perl.c:2365
scriptname =3D argv[=
0];. It wasn't possible to reason beyond that as stepping through op=
timised code even with debug symbols is next to impossible to make any sens=
e of.
I did find that building perl without optimisations made the error go =
away.
I found the following closed issue: https://github.com/Perl/perl5/issu=
es/15806 which describes the same issue I was having.
Looking at the source for mod_perl, I found that the argv array passed to p=
erl_parse() is not NULL terminated as is required by perl - ( documentation=
: https://perldoc.perl.org/perlembed#Adding-a-Perl-interpreter-to-your-C-pr=
ogram )
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
As for why this works in 99.99% of cases, I have no idea. With the set up I=
have (Ubuntu 20.04 LXD container) it is very reproducible initially, but a=
fter I fiddle a bit (uninstall & reinstall packages, install debugging =
packages, etc etc) it stops being reproducible.
Such is undefined behaviour and invalid memory accesses, I suppose, but qu=
ite irritating.
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
I've attached my suggested patch.
t; color:rgb(0,0,0)">
eadonly_1">
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
I don't believe it's strictly relevant, but package versions are:
t; color:rgb(0,0,0)">
apache2: 2.4.41-4ubuntu3.1
t; color:rgb(0,0,0)">
libapache2-mod-perl2: 2.0.11-2
t; color:rgb(0,0,0)">
perl: 5.30.0-9ubuntu0.2
t; color:rgb(0,0,0)">
t; color:rgb(0,0,0)">
Thanks,
t; color:rgb(0,0,0)">
Charles
--
aining/DO178C/?utm_source=3Demail_footer">Register for our online DO-178C &=
amp; Multicore training with ConsuNova: 8-12th March 2021.--_000_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_--
--_004_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_
Content-Type: application/octet-stream;
name="mod_perl-argv-null-terminator.diff"
Content-Description: mod_perl-argv-null-terminator.diff
Content-Disposition: attachment;
filename="mod_perl-argv-null-terminator.diff"; size=756;
creation-date="Thu, 18 Feb 2021 15:54:38 GMT";
modification-date="Thu, 18 Feb 2021 16:14:35 GMT"
Content-Transfer-Encoding: base64
LS0tIGEvc3JjL21vZHVsZXMvcGVybC9tb2RwZXJsX2NvbmZpZy5jCTIwMTktMTAtMDUgMTI6MDQ6
NDEuMDAwMDAwMDAwICswMTAwCisrKyBiL3NyYy9tb2R1bGVzL3BlcmwvbW9kcGVybF9jb25maWcu
YwkyMDIxLTAyLTE3IDE5OjA3OjIzLjY0NjIwNDM2NCArMDAwMApAQCAtMTYzLDcgKzE2Myw4IEBA
CiAgICAgc2NmZy0+UGVybFBvc3RDb25maWdSZXF1aXJlID0KICAgICAgICAgYXByX2FycmF5X21h
a2UocCwgMSwgc2l6ZW9mKG1vZHBlcmxfcmVxdWlyZV9maWxlX3QgKikpOwogCi0gICAgc2NmZy0+
YXJndiA9IGFwcl9hcnJheV9tYWtlKHAsIDIsIHNpemVvZihjaGFyICopKTsKKyAgICAvKiAyIGFy
Z3VtZW50cyArIE5VTEwgdGVybWluYXRvciAqLworICAgIHNjZmctPmFyZ3YgPSBhcHJfYXJyYXlf
bWFrZShwLCAzLCBzaXplb2YoY2hhciAqKSk7CiAKICAgICBzY2ZnLT5zZXR2YXJzID0gYXByX3Rh
YmxlX21ha2UocCwgMik7CiAgICAgc2NmZy0+Y29uZmlndmFycyA9IGFwcl90YWJsZV9tYWtlKHAs
IDIpOwpAQCAtMjE5LDYgKzIyMCw5IEBACiAKICAgICAqYXJnYyA9IHNjZmctPmFyZ3YtPm5lbHRz
OwogCisgICAgLyogcGVybF9wYXJzZSgpIGV4cGVjdHMgYSBOVUxMIHRlcm1pbmF0ZWQgYXJndiBh
cnJheSAqLworICAgIG1vZHBlcmxfY29uZmlnX3Nydl9hcmd2X3B1c2goTlVMTCk7CisKICAgICBN
UF9UUkFDRV9nX2RvKGR1bXBfYXJndihzY2ZnKSk7CiAKICAgICByZXR1cm4gKGNoYXIgKiopc2Nm
Zy0+YXJndi0+ZWx0czsK
--_004_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Hangout mailing list
Hangout-at-nylxs.com
http://lists.mrbrklyn.com/mailman/listinfo/hangout
--_004_CWXP123MB3336E482A11D7866264653A6AE849CWXP123MB3336GBRP_--
