MESSAGE
| DATE | 2021-01-21 |
| FROM | Zack Weinberg
|
| SUBJECT | Re: [Hangout - NYLXS] Future plans for Autotools
|
On Wed, Jan 20, 2021 at 5:15 PM Zack Weinberg wrote:
> Now we've all had a while to recover from the long-awaited Autoconf
> 2.70 release, I'd like to start a conversation about where the
> Autotools in general might be going in the future.
> Now we've all had a while to recover from the long-awaited Autoconf
> 2.70 release, I'd like to start a conversation about where the
> Autotools in general might be going in the future.
Thanks for all the great thoughtful responses that came in overnight!
I see many people expressing similar sentiments, so rather than
respond to each of you individually I'm going to summarize and reply
to each strand in this one message.
> continuous integration for Autotools
This is one of my highest priorities for what volunteer time I have in
the next few months. The biggest difficulty with CI, as I see it, is
that Autotools needs to be tested on a very wide variety of operating
systems, and ideally *versions* of operating systems, and unusual
compilers, and so on, in order to confirm that it's still fulfilling
its contract. Testing just on the "big three" operating systems
(Linux, Windows, MacOS) typically supported by commercial CI providers
... would be better than nothing, but would not have helped all that
much with the 2.70 release, if I'm honest. GNU's Hydra instance
(https://www.gnu.org/software/devel.en.html#Hydra -- I didn't
previously know about this, thank you Dan for pointing it out) is a
strong contender just because it has FreeBSD and Solaris as well.
> implementation languages - can't we do better than shell+perl+m4?
I deeply sympathize with frustration with M4 and portable shell as
implementation languages. Perl isn't so bad IMO, but being restricted
to perl 5.6.0 with no CPAN modules is also a headache.
Having said that, switching to *anything else* would be a gigantic
task -- multiple full-time person-years of effort just for the core --
and would mean either porting or losing all of the third-party macro
libraries. I don't think it makes sense to explore this possibility
any further unless both a lot of funding and a lot of developers
materialize.
A lesser possibility we could think about, would be dusting off GNU M4
2.0 and thinking about what we could change about the M4 language to
make it friendlier for Autoconf's purposes. Again I would want
funding and developers lined up, but we'd not need quite so much of
either.
And we probably could bump up the minimum version requirement for Perl
quite some ways, since it's only needed at autoreconf time. I'm not
sure how much Autoconf would gain from this, though, since it's only
incrementally changed since 5.6 and Autoconf doesn't do *that* much
with it. Automake uses it much more heavily, though, perhaps people
from that project would like to comment?
[Nate Bargmann]
> One strength of the Autotools that I think stands above the rest is
> the fact that a user of a distributed package does not need to
> install any of the Autotools packages.
I definitely agree with this, and I'm putting it right next to the
"implementation languages" note because this is a compelling reason to
stick to shell as the language for generated configure scripts. The
only alternative that would come anywhere near as close in terms of
"just unpack and run" portability is Perl. Which is a tempting
possibility -- it would wipe out dozens of internal headaches due to
shell's limitations and portability quirks -- but again, multiple
full-time person-years of effort, losing the third-party macro
libraries, and we *would* be making life harder for people starting
from scratch on OSes that don't bundle Perl.
> Autotools are too difficult to work on
Yes indeed. On the assumption that we are stuck with shell, M4, and
Perl, however, what can we do about it? I have a few ideas, maybe you
have more:
- A linter for configure.ac and third-party macro libraries.
This wouldn't be any fun to write, because it would have to parse
both M4 and shell, *accurately*, but it would be valuable to every
user of autoconf.
- The parser for the linter could also be used as the basis for a
reimplementation of autoupdate that *doesn't* rely on running
configure.ac through M4 with most of the macros disabled. That
implementation choice is responsible for most of the bugs in
autoupdate, and for most of its limitations (e.g. not being able to
fix quotation errors).
- Audit existing configure.ac's looking for pain points: where are
people having trouble doing the checks they need, where are people
not realizing that they don't need to work so hard, where are people
not realizing that they don't need to check for that at all...
- Audit the core for workarounds that are no longer necessary now we
can rely on shell functions (I'm looking at you, AC_LANG_*).
- Better documentation and better debugging tools. Speaking of which,
here's an interesting bit from Russell:
[Russell Shaw]
> The biggest problem i had that wasted the most time is that none of
> the documentation gave a good overview of the sequence of steps the
> main perl script did to process the files and produce the output,
> and intermediate files involved.
The "Autom4te" layer really is quite poorly documented, and it gives
the impression that it's more important than it actually is. It
exists primarily for the sake of programs like autoheader and automake
that want to extract information from a configure.ac *instead of*
generating configure; when you're running autoconf proper, the main
things it does are select the appropriate .m4f file and postprocess
diagnostics. And it's a hindrance to debugging problems with the M4
layer. I have lost count of the number of times, while fixing bugs
in the run-up to 2.70, that I had to run autoconf --verbose, extract
the actual M4 invocation command line, revise it heavily in an editor
window, and then run it separately.
> Using a language like C++ where a step-by-step debugger like gdb can
> be used would have been good.
I'm a printf-debugging person myself, but that is *also* too hard with
autoconf. GNU M4's trace functionality could be improved, and a way
to single-step through macro expansion could be added, and an
interactive REPL would be nice. This is all in the land of things
we'd probably need to pay people to work on, though.
[Bob Friesenhahn]
> There is a fork of Automake which was re-done to be based on GNU
> Make. This assumes that makefiles written in GNU make can noticeably
> improve things. I like your idea of supporting other underlying
> build tools besides 'make'. Make's dependence on matching rules and
> file time stamps is problematic and it does not scale.
I haven't ever followed automake development closely. I know that
at least one person has tried to write a set of GNU Make library files
intended to replace it altogether, but I've never seen anyone *finish*
that project. I'd very much like to see someone give that another go.
The alternative to the Make family of tools, that I'm most familiar
with, is Ninja, and it also depends on matching rules and file time
stamps. What do you have in mind as a replacement for these things?
Is there already an alternative tool that you like?
Part of why I want to see Automake be able to generate configurations
for different build drivers, is that if Automake actually took
advantage of build-driver capabilities beyond the lowest common
denominator, that would give the people working on build drivers more
of a reason to improve those capabilities. Hard to be motivated to
hack on something when no one uses it, after all.
[Nate Bargmann]
> At the moment the project(s) that seem to be working toward replacing
> all things GNU do allow the later installation of GNU software
> ...
> What negative impact would relying solely on GNU make have on third
> party projects intending to build on platforms where the
> installation of GNU make may become discouraged?
Speaking personally, I am not interested in making any effort to cater
to organizations and projects that are averse to the GPL. I think
copyleft is a vital tool for preventing corporations from building on
my hard work without compensating me in any way whatsoever. The
existing Autoconf license exception -- you can ship a generated
configure script with your non-GPL program without being required to
conform with the GPL because of it -- is already enough of an indulgence,
IMNSHO.
As a concrete example of a situation where this *already* causes
friction with redistributors: OpenBSD has heavily patched the copy of
Autoconf 2.69 they ship with their base system so that it can use
their implementation of M4 and they don't have to ship GNU M4. I
don't think we should worry about that *at all* when considering
improvements to GNU M4 for Autoconf's sake.
I have a lot more sympathy with people who are annoyed with the GNU
*project* and/or the FSF. I have any number of my own grievances with
them. However, I would prefer not to get into that in this
conversation, because it's liable to drag it off-topic and/or turn
into a flame war. Let's declare the known obstacles imposed on
Autotools by the FSF -- concretely, the requirement for copyright
assignment paperwork and the requirement not to use proprietary CI and
forge services -- immobile and out-of-scope for this discussion.
zw
_______________________________________________
Hangout mailing list
Hangout-at-nylxs.com
http://lists.mrbrklyn.com/mailman/listinfo/hangout
|
|
