MESSAGE
| DATE | 2021-02-07 |
| FROM | James Smith
|
| SUBJECT | Re: [Hangout - NYLXS] Moving ExecCGI to mod_perl - performance and
|
Agree in this - you need to always think that a mod_perl app is running in a loop where each loop is an iteration, so if you don't initialise something at the start of the script - it can have the value at the end of it's last call.... Use Perl critic it is a good one to find your gotchas...
Try
my $admin = 1 if $user_is_logged_in and $user eq 'admin';
may seem innocent in as script and compiles even under use strict... but now put this in a loop
1st request user is logged in and is admin - this gets set to true.
2nd request user isn't logged in - but this doesn't get reset to "false"....
-----Original Message-----
From: Chris
Sent: 07 February 2021 21:18
To: modperl-at-perl.apache.org
Subject: Re: Moving ExecCGI to mod_perl - performance and custom'modules'[EXT]
On Sun, Feb 07, 2021 at 09:21:41PM +0800, Wesley Peng wrote:
> If you can take time to rewrite all codes with modPerl handlers, that will improve performance a lot.
I've never used Template, but I just eventually wrote all handlers.
I moved from Registry to all handlers, bit by bit.
You can mix Registry and handlers together without any problems.
Just remember to always write clean code that resets variables after doing tasks. The same code runs multiple times. Variables might retain old values. It's a good habit to keep for all perl code, even outside of mod_perl.
I considered changing all of my mod_perl code to something newer, but I decided to just keep it. No regrets.
Chris
>
> On Sun, Feb 7, 2021, at 9:14 PM, Steven Haigh wrote:
> > In fact, I just realised that 'ab' test is rather restrictive.... So here's a bit more of an extended test:
> >
> > # ab -k -n 1000 -c 32
> >
> > Apache + ExecCGI:
> > Requests per second: 14.26 [#/sec] (mean)
> > Time per request: 2244.181 [ms] (mean)
> > Time per request: 70.131 [ms] (mean, across all concurrent requests)
> >
> > Apache + mod_perl (ModPerl::PerlRegistry):
> > Requests per second: 132.14 [#/sec] (mean)
> > Time per request: 242.175 [ms] (mean)
> > Time per request: 7.568 [ms] (mean, across all concurrent requests)
> >
> > Interestingly, without Keepalives, the story is much the same:
> >
> > # ab -n 1000 -c 32
> >
> > Apache + ExecCGI:
> > Requests per second: 14.15 [#/sec] (mean)
> > Time per request: 2260.875 [ms] (mean)
> > Time per request: 70.652 [ms] (mean, across all concurrent requests)
> >
> > Apache + mod_perl (ModPerl::PerlRegistry):
> > Requests per second: 154.48 [#/sec] (mean)
> > Time per request: 207.140 [ms] (mean)
> > Time per request: 6.473 [ms] (mean, across all concurrent requests)
> >
> > Running some benchmarks across various parts of my site made me realise I also had some RewriteRules in the apache config that still had H=cgi-script - changed those to H=perl-script and saw similar improvements:
> >
> > ExecCGI - Requests per second: 11.84 [#/sec] (mean)
> > mod_perl - Requests per second: 130.97 [#/sec] (mean)
> >
> > That's quite some gains for a days work.
> >
> > --
> > Steven Haigh 📧 netwiz-at-crc.id.au 💻
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.crc.id.au&d
> > =DwIDaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj
> > 4oDX0XM7vQ&m=jUH_UYtcqlwO076IYfxVYrdow4cIntukmuDhs07wzzE&s=EheaEdaj-
> > 0DWuhH62PQjVTVfPw5WRAazyGkJFaYUC8E&e=
> >
> > On Sun, Feb 7, 2021 at 23:58, Steven Haigh wrote:
> >> Interestingly, I did get things working with ModPerl::PerlRegistry.
> >>
> >> What I couldn't find *anywhere* is that the data I was loading in Template Toolkit was included in the file in the __DATA__ area - which causes mod_perl to fall over!
> >>
> >> The only way I managed to find this was the following error in the *system* /var/log/httpd/error_log (didn't show up in the vhost error_log!):
> >> readline() on unopened filehandle DATA at /usr/lib64/perl5/vendor_perl/Template/Provider.pm line 638.
> >>
> >> Took me a LONG time to find a vague post that reading in lines from kills mod_perl. Not sure why - but I stripped all the templates out and put them in a file instead and re-wrote that bit of code, and things started working.
> >>
> >> I had to fix a few lib path issues, but after getting my head around that, most things seem to work as before - however I don't notice much of an improvement in execution times, I do see this improvement using 'ab -n 100 -c32':
> >>
> >> Apache + ExecCGI: Requests per second: 13.50 [#/sec] (mean)
> >> Apache + mod_perl: Requests per second: 59.81 [#/sec] (mean)
> >>
> >> This is obviously a good thing.
> >>
> >> I haven't gotten into the preload or DBI sharing yet - as that'll
> >> end up needing a bit of a rewrite of code to take advantage of. I'd
> >> be open to suggestions here from those who have done it in the past
> >> to save me going down some dead ends :D
> >>
> >> --
> >> Steven Haigh 📧 netwiz-at-crc.id.au 💻
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.crc.id.au&
> >> d=DwIDaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1e
> >> cj4oDX0XM7vQ&m=jUH_UYtcqlwO076IYfxVYrdow4cIntukmuDhs07wzzE&s=EheaEd
> >> aj-0DWuhH62PQjVTVfPw5WRAazyGkJFaYUC8E&e=
> >>
> >> On Sun, Feb 7, 2021 at 12:49, James Smith wrote:
> >>> As welsey said – try Registry, that was the standard way of using
> >>> mod_perl to cache perl in the server – but your problem might be
> >>> due to the note in PerlRun…
> >>>
> >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__perl.apache.o
> >>> rg_docs_2.0_api_ModPerl_PerlRun.html-23Description&d=DwIDaQ&c=D7By
> >>> GjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m
> >>> =jUH_UYtcqlwO076IYfxVYrdow4cIntukmuDhs07wzzE&s=X28W3EsAPwLXjwa4Im6
> >>> 8Ak7wzU4ss-EYYegNaEoii18&e=
> >>> META: document that for now we don't chdir() into the script's dir, because it affects the whole process under threads. `ModPerl::PerlRunPrefork ` should be used by those who run only under prefork MPM.
> >>> {tbh most people don’t use mod perl under threads anyway as there
> >>> isn’t really a gain from using them}
> >>>
> >>> It suggests you use ModPerl/PerlRunPrefork – as this does an additional step to cd to the script directory – which might be your issue….
>
> >>>
>
> >>> *From:* Steven Haigh
> >>> *Sent:* 07 February 2021 01:00
> >>> *To:* modperl-at-perl.apache.org
> >>> *Subject:* Moving ExecCGI to mod_perl - performance and custom
> >>> 'modules' [EXT]
>
> >>>
>
> >>> Hi all,
>
> >>>
>
> >>> So for many years I've been slack and writing perl scripts to do various things - but never needed more than the normal apache +ExecCGI and Template Toolkit.
>
> >>>
>
> >>> One of my sites has become a bit more popular, so I'd like to spend a bit of time on performance. Currently, I'm seeing ~300-400ms of what I believe to be execution time of the script loading, running, and then blatting its output to STDOUT and the browser can go do its thing.
>
> >>>
>
> >>> I believe most of the delay would be to do with loading perl, its
> >>> modules etc etc
>
> >>>
>
> >>> I know that the current trend would be to re-write the entire site
> >>> in a more modern, daemon based solution - and I started down the
> >>> Mojolicious path - but the amount of re-writing to save 1/3rd of a
> >>> second seems to be excessive
>
> >>>
>
> >>> Would I be correct in thinking that mod_perl would help in this case?
>
> >>>
>
> >>> I did try a basic test, but I have a 'use functions' in all my scripts that loads a .pm with some global vars and a lot of common subs - and for whatever reason (can't find anything on Google as to why), none of the subs are recognised in the main script when loaded via ModPerl::PerlRun.
>
> >>>
>
> >>> So throwing it out to the list - am I on the right track? wasting my time? or just a simple mistake?
>
> >>>
>
> >>> --
>
> >>> Steven Haigh 📧 netwiz-at-crc.id.au 💻
> >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.crc.id.au
> >>> &d=DwIDaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge
> >>> 1ecj4oDX0XM7vQ&m=jUH_UYtcqlwO076IYfxVYrdow4cIntukmuDhs07wzzE&s=Ehe
> >>> aEdaj-0DWuhH62PQjVTVfPw5WRAazyGkJFaYUC8E&e= [crc.id.au]
> >>> > >>> u_&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0
> >>> ge1ecj4oDX0XM7vQ&m=bosoTbkecbnrPukObNK-5Duc1p3JTllIM7_FHhBYKW4&s=v
> >>> QDi0ezyEZDOz86GVraerPdT76UjN2in3UdPh8fglRM&e=>
>
> >>> -- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.
--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
_______________________________________________
Hangout mailing list
Hangout-at-nylxs.com
http://lists.mrbrklyn.com/mailman/listinfo/hangout
|
|
