Discussion:
[Axiom-developer] building wh-sandbox on Windows
Bill Page
2007-04-26 12:16:15 UTC
Permalink
Waldek,

I have finally succeeded in building wh-sandbox on Windows.
I have attached the patch set with diffs against revision 515.
My approach was to introduce a "BASE" variable as I discussed
in a previous email to allow the Windows applications to have
a different absolute path than the MSYS tools used during the
build. This is in contrast to Gaby's approach of making all paths
relative. In the end this seemed like the shortest path to getting
wh-sandbox to compile under Windows.

Please let me know what you think about this approach.

I will continue to test this patchset to make sure that it does
not break the linux build.

Regards,
Bill Page.
Martin Rubey
2007-04-26 13:24:14 UTC
Permalink
Post by Bill Page
Waldek,
I have finally succeeded in building wh-sandbox on Windows. I have attached
the patch set with diffs against revision 515. My approach was to introduce a
"BASE" variable as I discussed in a previous email to allow the Windows
applications to have a different absolute path than the MSYS tools used
during the build. This is in contrast to Gaby's approach of making all paths
relative. In the end this seemed like the shortest path to getting wh-sandbox
to compile under Windows. Please let me know what you think about this
approach. I will continue to test this patchset to make sure that it does not
break the linux build. Regards, Bill Page.
WOW! It remains to get the graphics working, I guess. I had a small
presentation today, but people where dismayed by the fact that there is no
help, no graphics under MS windows.

And of course, they really want a better expression domain, i.e., provisos.

I guess, both is out of reach for the moment?



Martin
Bill Page
2007-04-26 20:13:26 UTC
Permalink
Post by Bill Page
...
I will continue to test this patchset to make sure that it does
not break the linux build.
My test build of wh-sandbox (rev. 515) plus the patches for
Windows just completed normally and runs simple confidence
tests. with errors on the axiom-developer.org Linux server
(RedHat 9+).

I have also checked that this patch set is independent of the
earlier changes for portability and they both can be applied
simultaneously.

Waldek, I have modified the portability patch set, as you
suggested in your previous email, to remove the regression
of the cfuns-c socket code and drop the patch to the obsolete
'construc' function. Would you like me to commit either or
both of these patch sets to wh-sandbox? Note: One of the
revisions that I pulled from build-improvements triggers the
requirement for autoconf >= 2.60.

Gaby, the portability changes are also applicable to the
build-mprovements branch. Would you like me to make an
initial commit? There will be more (less critical) changes
like this to follow.

Regards,
Bill Page
Bill Page
2007-04-26 20:16:04 UTC
Permalink
Post by Bill Page
... I will continue to test this patchset to make sure that it does
not break the linux build.
My test build of wh-sandbox (rev. 515) plus the patches for
Windows just completed normally and runs simple confidence
tests. with errors on the axiom-developer.org Linux server
(RedHat 9+).
...
Of course I meant to write "without errors" ...
Gabriel Dos Reis
2007-04-27 16:06:25 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

| Gaby, the portability changes are also applicable to the
| build-mprovements branch. Would you like me to make an
| initial commit? There will be more (less critical) changes
| like this to follow. Regards,

Which parts of your patch is the "protability changes"?

-- Gaby
Waldek Hebisch
2007-04-26 21:17:58 UTC
Permalink
Post by Bill Page
Waldek,
I have finally succeeded in building wh-sandbox on Windows.
I have attached the patch set with diffs against revision 515.
My approach was to introduce a "BASE" variable as I discussed
in a previous email to allow the Windows applications to have
a different absolute path than the MSYS tools used during the
build. This is in contrast to Gaby's approach of making all paths
relative. In the end this seemed like the shortest path to getting
wh-sandbox to compile under Windows.
Please let me know what you think about this approach.
Congratulations. Few quick remarks about the diff:

1) It does not apply cleanly to 515 (I had problem with
README.build-improvements, configure.ac.pamphlet and
src/interp/sys-pkg.lisp.pamphlet)
2) Some parts seem to be whitespace-only diff (differnces
in whitespace probably are responsible for failures in 1)
3) Use of '(in-package \"boottran\")' in document.in is wrong,
because in Ansi Lisp the package name here is case sensitive,
so we want '(in-package \"BOOTTRAN\")'
4) Do you really need subshells in src/algebra/Makefile.pamphlet?
5) In src/interp/patches.lisp.pamphlet the first hunk inroduces
conditionals for |$saturn| etc. I prefer to set those
variables unconditionally, because ATM there is _no_ platform
on which choosing Saturn works.
6) We can avoid using pathnames during interpreter build by
eliminationg int/interp subdirectory -- one reason for
keeping this directory is that we want sane pathname handling
at user level, so have to solve the underlying problem. The
same solution should work for build. But if we have to use
gross hacks to get around build problems then I prefer to
move files to single directory.
7) In config/var-def.mk I ommited a few unused variables that Gaby
added -- I feel that we should not slavishy copy the bloat
created by autotools. More preciesly, I do not like bloat
in machine generated files but as long as thing are handled
automatically such bloat is acceptable. But we are
maintaning var-def.mk by hand and bloat is a problem. So
I would like to have only usefull stuff in config/var-def.mk
-- it looks that your patch copies a bunch of unused variables
from build-improvements.
8) Concerning BASE variable: it does not look nice, but I do not
think that other solutions would look better.
--
Waldek Hebisch
***@math.uni.wroc.pl
Bill Page
2007-04-26 22:56:42 UTC
Permalink
Post by Waldek Hebisch
1) It does not apply cleanly to 515 (I had problem with
README.build-improvements, configure.ac.pamphlet and
src/interp/sys-pkg.lisp.pamphlet)
2) Some parts seem to be whitespace-only diff (differnces
in whitespace probably are responsible for failures in 1)
Perhaps one cause of the apparent whitespace differences
has to do with the annoyance of different line endings between
Linux and Windows. SVN for Windows automatically terminates
lines with CR+LF, but the MSYS tools assume normal Unix
convention of just newline (LF).

I would prefer if we set the EOL attributes on the SourceForge
SVN repository so that SVN does not mangle line endings
in this way. But this is just an inconvenient a fact-of-life for
Windows developers. Most Windows and MSYS tools are
tolerant for these differences but some are not. When I
produced the patch file I sent you, I filtered it by tr -d '\r' to
convert back to Unix line endings.

A significant thing to remember about MSYS is that it is really
a cross-compilation/porting environment intended to produce
native Windows applications from Unix-oriented sources. It is
not quite a native development environment for Windows (like
Visual C, for example).
Post by Waldek Hebisch
3) Use of '(in-package \"boottran\")' in document.in is wrong,
because in Ansi Lisp the package name here is case sensitive,
so we want '(in-package \"BOOTTRAN\")'
Ok. BTW, the build I did of wh-sandbox on Windows was using
the ANSI mode of GCL. I have compiled the same sources under
Linux using CLtL1. Thank you very much for resolving the
differences that make it possible to use GCL in ANSI mode.
I did not realize that it would also be possible to use a dialect
of GCL that was compatible with *both*. This is a nice feature to
try to retain at least during the transition to full ANSI compliance.
Post by Waldek Hebisch
4) Do you really need subshells in
src/algebra/Makefile.pamphlet?
Subshells make it easier to make local changes to the current
directory to avoid some path issues. But, no they are probably
not needed in a more careful design.
Post by Waldek Hebisch
5) In src/interp/patches.lisp.pamphlet the first hunk inroduces
conditionals for |$saturn| etc. I prefer to set those
variables unconditionally, because ATM there is _no_ platform
on which choosing Saturn works.
Ok. I am in favour of eliminating as much of this cruft as
possible (before we start adding more of our own ;).
Post by Waldek Hebisch
6) We can avoid using pathnames during interpreter build
by eliminationg int/interp subdirectory -- one reason for
keeping this directory is that we want sane pathname
handling at user level, so have to solve the underlying
problem.
I am not sure if I understand what you call the "underlying
problem". Do you mean the fact that Axiom's pamphlet
files group code into "chunks" that may not correspond to
files at the lowest level? I suppose the model that we want
to present to the user is: "Axiom's source code as a set
of literate documents...". How these documents are woven
from text and code chunks is (mostly) our problem. But
what the build machinery needs is the chunks of various
pamphlets organized into files. I worry that the tools we
have for doing this right now (e.g. noweb, make stanzas,
and even awk scripts) are a little too primitive.
Post by Waldek Hebisch
The same solution should work for build.
Again, I am not sure exactly sure to what "solution" you
are referring, but if you mean continuing the use of an
intermediate file system-oriented view of the source code,
then I suppose that right now we have no other option.
Post by Waldek Hebisch
But if we have to use gross hacks to get around build
problems then I prefer to move files to single directory.
I don't view the introduction of a "BASE" variable as a gross
hack. But this is different than what I did in two cases marked
"hack #1 billpage in the Makefile.pamphlet source. In that
case I made use of the intermediate directory as a temporary
staging area for both the input and output files. A 'cd' to change
the current directory eliminates one absolute path and the use
of a 'mv' command moves the other path out of the native
environment (Windows) and back into the development
environment (MSYS). But as you say this could have been
handled by the use of $(BASE) instead. So can I assume
that you would be in favor of the latter option?
Post by Waldek Hebisch
7) In config/var-def.mk I ommited a few unused variables
that Gaby added -- I feel that we should not slavishy
copy the bloat created by autotools. More preciesly,
I do not like bloat in machine generated files but as
long as thing are handled automatically such bloat is
acceptable. But we are maintaning var-def.mk by
hand and bloat is a problem.
I understand your point. What I tried to do was to use
'svn merge' just to pull as much as possible an entire
revision from build-improvements into wh-sandbox. My
thinking was that this might prevent problems in the future
in the case of an eventual merge of both build-improvements
and wh-sandbox back into the "Axiom Silver" trunk. But
where we are going with this is not clear to me yet.
Post by Waldek Hebisch
So I would like to have only usefull stuff in config/var-def.mk
-- it looks that your patch copies a bunch of unused
variables from build-improvements.
That is correct. I think there are advantages and disadvantages
of either choice. In favor of retaining the unused (but "standard")
autoconf variables is in the eventual move to a complete
automated make environment. (I think that was Gaby's original
intent.) In which case, as you say, this complexity would be
hidden by the machinery. The question is then: how much of
this can we tolerate now? Maybe the answer is different
between build-improvements and wh-sandbox... For now I will
try to eliminate the unused variables in my patch wh-sandbox,
but doing so requires a little more analysis.
Post by Waldek Hebisch
8) Concerning BASE variable: it does not look nice, but
I do not think that other solutions would look better.
I agree. If we can afford the divergence of the code between
wh-sandbox and build-improvements then it makes sense
to experiment with two rather opposite solutions to the same
problem.

Regards,
Bill Page.
Waldek Hebisch
2007-04-27 02:51:09 UTC
Permalink
Post by Bill Page
Post by Waldek Hebisch
1) It does not apply cleanly to 515 (I had problem with
README.build-improvements, configure.ac.pamphlet and
src/interp/sys-pkg.lisp.pamphlet)
2) Some parts seem to be whitespace-only diff (differnces
in whitespace probably are responsible for failures in 1)
Perhaps one cause of the apparent whitespace differences
has to do with the annoyance of different line endings between
Linux and Windows. SVN for Windows automatically terminates
lines with CR+LF, but the MSYS tools assume normal Unix
convention of just newline (LF).
No, AFAICS differences are due to trailing spaces -- actually it
seems that we have hundreds of lines with trailing whitespece
while IMHO such lines should be extremally rare (that is all
useless trailing whitespece should be stripped).
Post by Bill Page
I would prefer if we set the EOL attributes on the SourceForge
SVN repository so that SVN does not mangle line endings
in this way. But this is just an inconvenient a fact-of-life for
Windows developers. Most Windows and MSYS tools are
tolerant for these differences but some are not. When I
produced the patch file I sent you, I filtered it by tr -d '\r' to
convert back to Unix line endings.
This setting is intended as a convenience for Windows developers...
And it seems to work -- at least it seems that the repository
is free from unwanted carriage returns.
Post by Bill Page
Post by Waldek Hebisch
4) Do you really need subshells in
src/algebra/Makefile.pamphlet?
Subshells make it easier to make local changes to the current
directory to avoid some path issues. But, no they are probably
not needed in a more careful design.
I appreciate advantages of subshells. But on some platforms
(including Windows) subshells are rather slow to create, so
we probably want to avoid them in performance critical parts.

The shell code in question is _not_ performance critical, but
since the only change to src/algebra/Makefile.pamphlet I see
(or did I overlook something?) is introduction of subshells
I would like to know if there is a special problem here that
braces have but subshells avoid.
Post by Bill Page
Post by Waldek Hebisch
6) We can avoid using pathnames during interpreter build
by eliminationg int/interp subdirectory -- one reason for
keeping this directory is that we want sane pathname
handling at user level, so have to solve the underlying
problem.
I am not sure if I understand what you call the "underlying
problem". Do you mean the fact that Axiom's pamphlet
files group code into "chunks" that may not correspond to
files at the lowest level? I suppose the model that we want
No. Extraction from pamphlets is a different problem.
Post by Bill Page
Post by Waldek Hebisch
The same solution should work for build.
Again, I am not sure exactly sure to what "solution" you
are referring, but if you mean continuing the use of an
intermediate file system-oriented view of the source code,
then I suppose that right now we have no other option.
I am specifically referring to:

$(addsuffix .$(FASLEXT), $(IN_from_MID)): \
%.$(FASLEXT) : $(MID)/%.clisp
- @ echo 10 making $@ from $<
- @ ( B=`pwd`;\
+ @ echo 10a making $@ from $< # hack 1 by Bill Page
+ # avoid paths in file names for Windows compatibility

(and the hack 2). The `pwd` command I put there is already a hack
to work around fact that meaning of relative :output parameter to
compile-file is implementation dependent. In principle using

:output (merge-pathnames "$@")

should help but I still had problems after doing that, so
I used `pwd` as temporary hack before I get :output working
in consistent way.

But if using separate directory here causes extra problems,
I think that we can just put all .clisp in single directory.
Post by Bill Page
Post by Waldek Hebisch
But if we have to use gross hacks to get around build
problems then I prefer to move files to single directory.
I don't view the introduction of a "BASE" variable as a gross
hack. But this is different than what I did in two cases marked
"hack #1 billpage in the Makefile.pamphlet source. In that
case I made use of the intermediate directory as a temporary
staging area for both the input and output files. A 'cd' to change
the current directory eliminates one absolute path and the use
of a 'mv' command moves the other path out of the native
environment (Windows) and back into the development
environment (MSYS). But as you say this could have been
handled by the use of $(BASE) instead. So can I assume
that you would be in favor of the latter option?
Above I used wrong words. Given problem $(BASE) is a logical
solution. Your 'cd' + 'notdir' + 'mv' combination is also a
logical solution, if we _have to_ keep current files loctions.
But it is getting heavy, and we may put .clisp files in current
directory which should solve problem in 'IN_from_MID' rule
and simplify solution to 'AUTO_from_MID' rule
Post by Bill Page
Post by Waldek Hebisch
7) In config/var-def.mk I ommited a few unused variables
that Gaby added -- I feel that we should not slavishy
copy the bloat created by autotools. More preciesly,
I do not like bloat in machine generated files but as
long as thing are handled automatically such bloat is
acceptable. But we are maintaning var-def.mk by
hand and bloat is a problem.
I understand your point. What I tried to do was to use
'svn merge' just to pull as much as possible an entire
revision from build-improvements into wh-sandbox. My
thinking was that this might prevent problems in the future
in the case of an eventual merge of both build-improvements
and wh-sandbox back into the "Axiom Silver" trunk. But
where we are going with this is not clear to me yet.
I am trying to avoid unnecessary differences. But IMHO global
variables (even unused ones) significantly increase complexity and
consequently we should limit use of them. In particular
I think that we should decide at merge time if we really
want the variables in question. Of course the problem may
solve itself earlier: the variables may turn out to be
usefull ones. But ATM the difference is to remind us about
the problem.
Post by Bill Page
Post by Waldek Hebisch
So I would like to have only usefull stuff in config/var-def.mk
-- it looks that your patch copies a bunch of unused
variables from build-improvements.
That is correct. I think there are advantages and disadvantages
of either choice. In favor of retaining the unused (but "standard")
autoconf variables is in the eventual move to a complete
automated make environment. (I think that was Gaby's original
intent.) In which case, as you say, this complexity would be
hidden by the machinery. The question is then: how much of
this can we tolerate now? Maybe the answer is different
between build-improvements and wh-sandbox... For now I will
try to eliminate the unused variables in my patch wh-sandbox,
but doing so requires a little more analysis.
Please do. I would probably first add no new variables and
then look which new variables are used by the rest of the patch and
which exisiting values changed.
--
Waldek Hebisch
***@math.uni.wroc.pl
Waldek Hebisch
2007-04-27 03:06:08 UTC
Permalink
Post by Bill Page
My test build of wh-sandbox (rev. 515) plus the patches for
Windows just completed normally and runs simple confidence
tests. with errors on the axiom-developer.org Linux server
(RedHat 9+).
I have also checked that this patch set is independent of the
earlier changes for portability and they both can be applied
simultaneously.
Waldek, I have modified the portability patch set, as you
suggested in your previous email, to remove the regression
of the cfuns-c socket code and drop the patch to the obsolete
'construc' function. Would you like me to commit either or
both of these patch sets to wh-sandbox? Note: One of the
revisions that I pulled from build-improvements triggers the
requirement for autoconf >= 2.60.
ATM I use autoconf 2.59 and I would prefer to stay with this
version. AFAICS the problem is that autoconf 2.59 does not
set value of 'top_builddir'. So the following changes to
variable settings are problematic:

-build_libdir = $(abs_top_builddir)/src/lib
+build_libdir = $(top_builddir)/src/lib

+axiom_top_builddir = $(top_builddir)/build
+axiom_builddir = $(axiom_top_builddir)/$(build)
-axiom_configdir = $(abs_top_builddir)/config
+axiom_configdir = $(top_builddir)/config
--
Waldek Hebisch
***@math.uni.wroc.pl
Waldek Hebisch
2007-04-27 14:00:49 UTC
Permalink
Post by Bill Page
I have also checked that this patch set is independent of the
earlier changes for portability and they both can be applied
simultaneously.
Waldek, I have modified the portability patch set, as you
suggested in your previous email, to remove the regression
of the cfuns-c socket code and drop the patch to the obsolete
'construc' function. Would you like me to commit either or
both of these patch sets to wh-sandbox? Note: One of the
revisions that I pulled from build-improvements triggers the
requirement for autoconf >= 2.60.
Please go on with portability patch.

Concerning Windows build, attached you will find modified version
of your patch (diff with respect to revision 517). I eliminated extra
variable settings from config/var-def.mk -- this is likely to break
Windows build, but fixed autoconf 2.59 problem. Also, 517 changes
one of the variable setting differently than your original patch.
So there is probably some work to find out which settings in
config/var-def.mk are needed for Windows.

I eliminated use of MID directory from src/interp/Makefile.pamhlet
-- this should make path problems easier.

One extra comment: change to src/lisp/Makefile.pamhlet loads
initial-env.lisp which in turn creates BOOTTRAN package -- that
is why change to src/interp/sys-pkg.lisp.pamphlet is needed.

Could you merge the attached patch with your original one (and if
possible consider also comments from other mails) so that it works
both of Windows and Linux? Assuming new version works OK please
go on and commit.

BTW. I will be leaving out town for few days, so I will probably be
unable to immediatly look at your final version.
--
Waldek Hebisch
***@math.uni.wroc.pl
Waldek Hebisch
2007-04-27 14:09:35 UTC
Permalink
Post by Waldek Hebisch
Concerning Windows build, attached you will find modified version
of your patch (diff with respect to revision 517).
I forget to attach the patch, so I attach it to this mail.
--
Waldek Hebisch
***@math.uni.wroc.pl
Waldek Hebisch
2007-04-27 16:25:07 UTC
Permalink
Post by Gabriel Dos Reis
| Gaby, the portability changes are also applicable to the
| build-mprovements branch. Would you like me to make an
| initial commit? There will be more (less critical) changes
| like this to follow. Regards,
Which parts of your patch is the "protability changes"?
-- Gaby
IIUC Bill writes about patch attached to:

http://lists.nongnu.org/archive/html/axiom-developer/2007-04/msg00329.html
--
Waldek Hebisch
***@math.uni.wroc.pl
Waldek Hebisch
2007-04-27 17:57:10 UTC
Permalink
Post by Waldek Hebisch
Post by Waldek Hebisch
Concerning Windows build, attached you will find modified version
of your patch (diff with respect to revision 517).
I forget to attach the patch, so I attach it to this mail.
--
Waldek Hebisch
Axiom build with this patch fails numerous tests, for example
calculus2.input. AFAICS runtime compilation is broken: the comiler
can not read back intermediate files it wrote. ATM I do not see what
is the reason, but the only Lisp related change seems to be to
src/lisp/Makefile.pamphlet, my first guess is that initial-env.lisp
is causing the problem.
--
Waldek Hebisch
***@math.uni.wroc.pl
Waldek Hebisch
2007-04-27 18:19:31 UTC
Permalink
Post by Waldek Hebisch
Axiom build with this patch fails numerous tests, for example
calculus2.input. AFAICS runtime compilation is broken: the comiler
can not read back intermediate files it wrote. ATM I do not see what
is the reason, but the only Lisp related change seems to be to
src/lisp/Makefile.pamphlet, my first guess is that initial-env.lisp
is causing the problem.
It is probably false alarm: I just noticed that I made a mistake
and build Axiom based on gcl-2.6.6 -- this is likely the reason
for test failures.
--
Waldek Hebisch
***@math.uni.wroc.pl
Bill Page
2007-04-27 22:24:40 UTC
Permalink
Post by Gabriel Dos Reis
| Gaby, the portability changes are also applicable to the
| build-mprovements branch. Would you like me to make an
| initial commit? There will be more (less critical) changes
| like this to follow. Regards,
Which parts of your patch is the "protability changes"?
Those originally done by Mike Thomas. That is that patch
that I sent you earlier. My point is just that those
changes and the patches I made more recently to wh-sandbox
to build on Windows are orthogonal.
g***@cs.tamu.edu
2007-04-27 23:33:12 UTC
Permalink
Post by Bill Page
Post by Gabriel Dos Reis
| Gaby, the portability changes are also applicable to the
| build-mprovements branch. Would you like me to make an
| initial commit? There will be more (less critical) changes
| like this to follow. Regards,
Which parts of your patch is the "protability changes"?
Those originally done by Mike Thomas. That is that patch
that I sent you earlier. My point is just that those
changes and the patches I made more recently to wh-sandbox
to build on Windows are orthogonal.
Aha, OK; thanks for the clarification.

-- Gaby
Bill Page
2007-04-28 05:46:33 UTC
Permalink
Post by Waldek Hebisch
...
Post by Bill Page
I would prefer if we set the EOL attributes on the SourceForge
SVN repository so that SVN does not mangle line endings
in this way. But this is just an inconvenient a fact-of-life
for Windows developers. Most Windows and MSYS tools are
tolerant for these differences but some are not. When I
produced the patch file I sent you, I filtered it by
tr -d '\r' to convert back to Unix line endings.
This setting is intended as a convenience for Windows developers...
And it seems to work -- at least it seems that the repository
is free from unwanted carriage returns.
Do you mean that it is intended to protect the repository from
commits by Windows developers of source code containing Windows
line endings? I suppose. I never thought of it that way before.

But as a "Windows developer" I only consider it a pain that SVN
corrupts the linux sources in this manner. If I check-out code
from a "linux" project, I expect it to be valid linux source.
And the "corruption" can be subtle, for example the format of
the Axiom database files look like text files but contains byte
offsets encoded as text. These offsets are wrong if the line
endings are changed by SVN.

Windows porting tools like MSYS and the cygwin linux emulation
environment assume that source files will have unix line endings.
Even most native Windows text editors support unix line endings
as an option.
Post by Waldek Hebisch
Post by Bill Page
Post by Waldek Hebisch
4) Do you really need subshells in
src/algebra/Makefile.pamphlet?
Subshells make it easier to make local changes to the current
directory to avoid some path issues. But, no they are probably
not needed in a more careful design.
I appreciate advantages of subshells. But on some platforms
(including Windows) subshells are rather slow to create, so
we probably want to avoid them in performance critical parts.
The shell code in question is _not_ performance critical, but
since the only change to src/algebra/Makefile.pamphlet I see
(or did I overlook something?) is introduction of subshells
I would like to know if there is a special problem here that
braces have but subshells avoid.
Braces like { } in the Makefile caused the make to fail.
Replacing these with ( ) worked. I have not encountered this
problem before.

I have

$ make --version
GNU Make version 3.79.1, by Richard Stallman and Roland McGrath.

Perhaps I need to upgrade it?
Post by Waldek Hebisch
...
$(addsuffix .$(FASLEXT), $(IN_from_MID)): \
%.$(FASLEXT) : $(MID)/%.clisp
+ # avoid paths in file names for Windows compatibility
(and the hack 2). The `pwd` command I put there is already a
hack to work around fact that meaning of relative :output
parameter to compile-file is implementation dependent. In
principle using
should help but I still had problems after doing that, so
I used `pwd` as temporary hack before I get :output working
in consistent way.
After reading the Common Lisp documentation here:

http://www.cs.queensu.ca/software_docs/gnudev/gcl-ansi/gcl_1130.html

It is not clear to me how I could specify something like

C:/msys/1.0

to be added to a pathname by this method. How can I expect GCL
to know this BASE component of native Windows paths when called
from which an MSYS build environment?
Post by Waldek Hebisch
...
Above I used wrong words. Given problem $(BASE) is a logical
solution. Your 'cd' + 'notdir' + 'mv' combination is also a
logical solution, if we _have to_ keep current files loctions.
But it is getting heavy, and we may put .clisp files in current
directory which should solve problem in 'IN_from_MID' rule
and simplify solution to 'AUTO_from_MID' rule
Ok, I will try.
Post by Waldek Hebisch
...
Regards,
Bill Page.
Gabriel Dos Reis
2007-04-28 12:09:05 UTC
Permalink
Post by Bill Page
But as a "Windows developer" I only consider it a pain that SVN
corrupts the linux sources in this manner. If I check-out code
from a "linux" project, I expect it to be valid linux source.
I agree.

-- Gaby
C Y
2007-04-28 16:51:12 UTC
Permalink
Perhaps I am missing something, but shouldn't it be possible for the
person doing a checkout to use the svn:eol-style property flag to get
the desired behavior on a per-client basis? In this case, perhaps that
means unix style eol for all platforms?

http://svnbook.red-bean.com/en/1.1/ch07s02.html

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Waldek Hebisch
2007-04-28 17:57:37 UTC
Permalink
Post by Bill Page
Post by Waldek Hebisch
...
Post by Bill Page
I would prefer if we set the EOL attributes on the SourceForge
SVN repository so that SVN does not mangle line endings
in this way. But this is just an inconvenient a fact-of-life
for Windows developers. Most Windows and MSYS tools are
tolerant for these differences but some are not. When I
produced the patch file I sent you, I filtered it by
tr -d '\r' to convert back to Unix line endings.
This setting is intended as a convenience for Windows developers...
And it seems to work -- at least it seems that the repository
is free from unwanted carriage returns.
Do you mean that it is intended to protect the repository from
commits by Windows developers of source code containing Windows
line endings? I suppose. I never thought of it that way before.
My point was that the the current setting is intended to work
for Windows developers -- if it causes problem we can change it.
We could try setting svn:eol-style to LF or remove this property.
But we should check first if this gives desired consequences.
BTW: Same files have no svn:eol-style property on them, are
they transferred as is?
Post by Bill Page
But as a "Windows developer" I only consider it a pain that SVN
corrupts the linux sources in this manner. If I check-out code
from a "linux" project, I expect it to be valid linux source.
And the "corruption" can be subtle, for example the format of
the Axiom database files look like text files but contains byte
offsets encoded as text. These offsets are wrong if the line
endings are changed by SVN.
AFAICS there are no properties on database files -- were they
corrupted?
Post by Bill Page
Windows porting tools like MSYS and the cygwin linux emulation
environment assume that source files will have unix line endings.
Even most native Windows text editors support unix line endings
as an option.
svn should allow user control of line convertion but the documentation
leaves impression that the settings are hardwired. I consider this
as a design (or maybe documentation) bug. MSYS and cygwin svn
are runninig in Unix environment so should use Unix line conventions.
Post by Bill Page
Post by Waldek Hebisch
Post by Bill Page
Post by Waldek Hebisch
4) Do you really need subshells in
src/algebra/Makefile.pamphlet?
Subshells make it easier to make local changes to the current
directory to avoid some path issues. But, no they are probably
not needed in a more careful design.
I appreciate advantages of subshells. But on some platforms
(including Windows) subshells are rather slow to create, so
we probably want to avoid them in performance critical parts.
The shell code in question is _not_ performance critical, but
since the only change to src/algebra/Makefile.pamphlet I see
(or did I overlook something?) is introduction of subshells
I would like to know if there is a special problem here that
braces have but subshells avoid.
Braces like { } in the Makefile caused the make to fail.
Replacing these with ( ) worked. I have not encountered this
problem before.
I have
$ make --version
GNU Make version 3.79.1, by Richard Stallman and Roland McGrath.
Perhaps I need to upgrade it?
Could you say what is the error message and post corresponding fragment
of the build log. The log should contain (rather longish) shell
command. If this command is garbled, then we have make problem.
Othewise it may be some shell weirdness.
Post by Bill Page
Post by Waldek Hebisch
...
$(addsuffix .$(FASLEXT), $(IN_from_MID)): \
%.$(FASLEXT) : $(MID)/%.clisp
+ # avoid paths in file names for Windows compatibility
(and the hack 2). The `pwd` command I put there is already a
hack to work around fact that meaning of relative :output
parameter to compile-file is implementation dependent. In
principle using
should help but I still had problems after doing that, so
I used `pwd` as temporary hack before I get :output working
in consistent way.
http://www.cs.queensu.ca/software_docs/gnudev/gcl-ansi/gcl_1130.html
It is not clear to me how I could specify something like
C:/msys/1.0
to be added to a pathname by this method. How can I expect GCL
to know this BASE component of native Windows paths when called
from which an MSYS build environment?
Ansi Lisp has no concept of current directory, so is of no help here
(and almost anyting about files is implementation dependent anyway).
But I think that I have reasonable solution, just the solution
depends on implementation. For clisp, cmucl, gcl and sbcl the following
works:

(merge-pathnames "$@" (truename ""))

But Poplog clisp needs:

(merge-pathnames "$@" (truename "."))

which works incorrectly in gcl, and gives error in clisp and other
implementations may need something entirely different.

So, the full solution to relative name problem will be too large
for Makefile -- we should put it in separate portability library.
--
Waldek Hebisch
***@math.uni.wroc.pl
Gabriel Dos Reis
2007-04-28 18:22:09 UTC
Permalink
Post by Waldek Hebisch
svn should allow user control of line convertion but the documentation
leaves impression that the settings are hardwired. I consider this
as a design (or maybe documentation) bug. MSYS and cygwin svn
are runninig in Unix environment so should use Unix line conventions.
They let you set it to whatever you want.

[...]
Post by Waldek Hebisch
Post by Bill Page
http://www.cs.queensu.ca/software_docs/gnudev/gcl-ansi/gcl_1130.html
It is not clear to me how I could specify something like
C:/msys/1.0
to be added to a pathname by this method. How can I expect GCL
to know this BASE component of native Windows paths when called
from which an MSYS build environment?
Ansi Lisp has no concept of current directory,
check default-pathname-defaults.

-- Gaby
Ralf Hemmecke
2007-04-28 18:42:14 UTC
Permalink
Post by Waldek Hebisch
AFAICS there are no properties on database files -- were they
corrupted?
No. I think I removed a lot of properties in particular from the
databases (revisions 137 and 136). But that happened in trunk and not in
build-improvements.

Ralf
Waldek Hebisch
2007-04-28 18:46:54 UTC
Permalink
Post by Gabriel Dos Reis
Post by Waldek Hebisch
Ansi Lisp has no concept of current directory,
check default-pathname-defaults.
I know about it:

1) content of *default-pathname-defaults* is implementation dependent
2) on some implementations *default-pathname-defaults* is a relative
path
3) on Poplog clisp:

== *default-pathname-defaults*
#P"temp.lsp"

which if you use merge-patnames nicely defaults name to "temp" and
type to "lsp".
--
Waldek Hebisch
***@math.uni.wroc.pl
Gabriel Dos Reis
2007-04-28 20:25:17 UTC
Permalink
Post by Waldek Hebisch
Post by Gabriel Dos Reis
Post by Waldek Hebisch
Ansi Lisp has no concept of current directory,
check default-pathname-defaults.
1) content of *default-pathname-defaults* is implementation dependent
So what?
Post by Waldek Hebisch
2) on some implementations *default-pathname-defaults* is a relative
path
only if set to relative path.
But, again I don't see why that is a problem.

Having read this thread many times, it appears to me that the
whole thread is confused.

First, I think people should state clearly and unambigously what
is the problem they are trying to solve.

Second, people should state very clearly what is the problem they
are trying to avoid with relative paths.
Post by Waldek Hebisch
== *default-pathname-defaults*
#P"temp.lsp"
which if you use merge-patnames nicely defaults name to "temp" and
type to "lsp".
Yes. As expected. What is the real problem?

-- Gaby
Waldek Hebisch
2007-04-28 21:10:51 UTC
Permalink
Post by Gabriel Dos Reis
Having read this thread many times, it appears to me that the
whole thread is confused.
First, I think people should state clearly and unambigously what
is the problem they are trying to solve.
Second, people should state very clearly what is the problem they
are trying to avoid with relative paths.
If you missed it we are trying to solve the following exercise:

Given make variables $< and $@ write a make rule which compiles
Lisp file from $< and puts the results in $@.

You can easily check that

echo '(compile-file "$<" :output-file "$@") (quit)' | ${DEPSYS}

does not work with relative paths (it works fine with absolute
paths).

Related problem is handling '--output' parameter to 'document'
script when compiling.

When I am writing about underlying problem I mean that we call
'compile-file' in Axiom and we want output in specific place
(ATM what we get is "implementation dependent").

Note that compilation is part of user interface. We can
choose crippled interface (say only names in current directory),
accept inconsincy between Lisps or we resolve this problem in
general (for arbitrary paths). I feel that we should resolve
the problem.
--
Waldek Hebisch
***@math.uni.wroc.pl
Martin Rubey
2007-04-28 21:21:04 UTC
Permalink
Waldek Hebisch <***@math.uni.wroc.pl> writes:

[...]
Post by Waldek Hebisch
You can easily check that
does not work with relative paths (it works fine with absolute
paths).
Don't you think you should forward that to comp.lang.lisp?

Martin
Gabriel Dos Reis
2007-04-28 21:24:24 UTC
Permalink
Post by Martin Rubey
[...]
Post by Waldek Hebisch
You can easily check that
does not work with relative paths (it works fine with absolute
paths).
Don't you think you should forward that to comp.lang.lisp?
I just received his mail. He could of course send the message
to comp.lang.lisp, but I also believe that he is confusing the
medium with the message.

More on that later.

-- Gaby
Waldek Hebisch
2007-04-28 21:53:11 UTC
Permalink
Post by Martin Rubey
[...]
Post by Waldek Hebisch
You can easily check that
does not work with relative paths (it works fine with absolute
paths).
Don't you think you should forward that to comp.lang.lisp?
I do not think it make any sense:

1) 'compile-file' works as specified -- the "only" problem is some
aspect of specified behaviour are implementation dependent and
impementation as doing something else compared to what we need
(and different implementation and doing different something else).

2) We could complain about this implementation dependent behaviour
but the problem is well-known and such message would be treated
as an attempt to start a flame war.

3) We could ask how to archive what we want. Again, there is a
library (cl-fad), which solves bunch of similar exercises.
cl-fad does not work on gcl, but it is IMHO unreasonable to
ask a newsgroup to do a port for us.
--
Waldek Hebisch
***@math.uni.wroc.pl
Gabriel Dos Reis
2007-04-28 22:15:02 UTC
Permalink
Post by Waldek Hebisch
3) We could ask how to archive what we want. Again, there is a
library (cl-fad), which solves bunch of similar exercises.
cl-fad does not work on gcl, but it is IMHO unreasonable to
ask a newsgroup to do a port for us.
Indeed, if you ask the newsgroup to do a port for you, it would not
make much sense. But, again, I understood something different
from Martin's suggestion.

-- Gaby
Waldek Hebisch
2007-04-28 22:06:23 UTC
Permalink
Post by Waldek Hebisch
Post by Gabriel Dos Reis
Having read this thread many times, it appears to me that the
whole thread is confused.
First, I think people should state clearly and unambigously what
is the problem they are trying to solve.
Second, people should state very clearly what is the problem they
are trying to avoid with relative paths.
You can easily check that
does not work with relative paths (it works fine with absolute
paths).
In case you did not notice: I wrote earlier that we want to solve
inderlying problem, but in the Makefile we can arrange "$<" to
be in current directory, which solves the immediate problem.
--
Waldek Hebisch
***@math.uni.wroc.pl
Gabriel Dos Reis
2007-04-28 22:16:27 UTC
Permalink
Post by Waldek Hebisch
Post by Waldek Hebisch
Post by Gabriel Dos Reis
Having read this thread many times, it appears to me that the
whole thread is confused.
First, I think people should state clearly and unambigously what
is the problem they are trying to solve.
Second, people should state very clearly what is the problem they
are trying to avoid with relative paths.
You can easily check that
does not work with relative paths (it works fine with absolute
paths).
In case you did not notice: I wrote earlier that we want to solve
inderlying problem, but in the Makefile we can arrange "$<" to
be in current directory, which solves the immediate problem.
And many of us have been using that and building with relative paths.

-- Gaby
Bill Page
2007-04-28 22:32:10 UTC
Permalink
Post by Waldek Hebisch
Post by Gabriel Dos Reis
Having read this thread many times, it appears to me
that the whole thread is confused.
I agree.
Post by Waldek Hebisch
Post by Gabriel Dos Reis
First, I think people should state clearly and
unambigously what is the problem they are trying to
solve.
Second, people should state very clearly what is the
problem they are trying to avoid with relative paths.
If you missed it we are trying to solve the following
Gabriel Dos Reis
2007-04-28 22:53:54 UTC
Permalink
On Sat, 28 Apr 2007, Bill Page wrote:

[...]
Post by Waldek Hebisch
You can easily check that
does not work with relative paths (it works fine with
absolute paths).
Waldek claims that the various Lisps do not implement a
uniform concept of "current directory", so therefore
relative paths behave differently in different Lisps
because it is not clear what is relative to what. This
does however work as expected (i.e. in the sense of
"current directory" implemented by both Linux and Windows)
when building Axiom from build-improvements using GCL on
both Linux and Windows.
In MSYS on Windows absolute paths do to work as you might
expect, but this has anything to do with Lisp. The problem
is that MSYS is a porting environment and so there are
really two *different* concepts of absolute path: one that
applies exclusively to the MSYS environment and associated
tools, and another one that applies in the native Windows
applications. During the Axiom build we need to use both
of these because we are building Axiom and its intermediate
phases as a native apps. If we choose to use absolute paths,
as is (mostly) done in wh-sandbox, then we must be careful
when we specify paths that must be interpreted by these
native windows programs.
Alternatively when compile Axiom on MSYS using GCL, as is
done in build-improvements, we can code things carefully so
that all paths are relative and this works because the
concept of current path is the same between MSYS and native
Windows.
You have adequately summarized my findings while working last
december on Axiom builds on Windows. An important point is
to understand what MSYS/MinGW are.
Gabriel Dos Reis
2007-04-28 23:09:19 UTC
Permalink
On Sat, 28 Apr 2007, Bill Page wrote:

[...]
Waldek Hebisch
2007-04-28 23:57:21 UTC
Permalink
[...]
Gabriel Dos Reis
2007-04-29 01:20:38 UTC
Permalink
[...]
Waldek Hebisch
2007-04-29 04:17:08 UTC
Permalink
Post by Bill Page
Post by Waldek Hebisch
...
$(addsuffix .$(FASLEXT), $(IN_from_MID)): \
%.$(FASLEXT) : $(MID)/%.clisp
+ # avoid paths in file names for Windows compatibility
Above I used wrong words. Given problem $(BASE) is a logical
solution. Your 'cd' + 'notdir' + 'mv' combination is also a
logical solution, if we _have to_ keep current files loctions.
But it is getting heavy, and we may put .clisp files in current
directory which should solve problem in 'IN_from_MID' rule
and simplify solution to 'AUTO_from_MID' rule
Ok, I will try.
I now commited a change that keeps all lisp files in src/interp directory.
--
Waldek Hebisch
***@math.uni.wroc.pl
Waldek Hebisch
2007-04-29 18:14:41 UTC
Permalink
Post by Waldek Hebisch
Note that compilation is part of user interface. We
can choose crippled interface (say only names in current
directory), accept inconsincy between Lisps or we resolve
this problem in general (for arbitrary paths). I feel
that we should resolve the problem.
I do not think that resolving this problem for multiple
Lisp targets needs to be much of a priority for Axiom
right now.
Continue reading on narkive:
Loading...