Discussion:
build-improvements on cygwin
(too old to reply)
Gabriel Dos Reis
2006-11-26 21:14:27 UTC
Permalink
Hi,

I would like to test build-improvements on cygwin, but gcl-2.6.8pre
(and gcl-2.6.7) would not compile there. If you know of the status of
GCL on cygwin, please tell me; otherwise, I fear Axiom on cygwin will
be delayed till I have time to work on GCL for cygwin (which will be
none this year).

-- Gaby
Bill Page
2006-11-26 21:24:08 UTC
Permalink
Post by Gabriel Dos Reis
I would like to test build-improvements on cygwin, but gcl-2.6.8pre
(and gcl-2.6.7) would not compile there. If you know of the status of
GCL on cygwin, please tell me; otherwise, I fear Axiom on cygwin will
be delayed till I have time to work on GCL for cygwin (which will be
none this year).
gcl-2.6.8pre compiles natively under Windows using MSYS/MinGW.
cygwin is not necessary.

Regards,
Bill Page.
Gabriel Dos Reis
2006-11-26 23:39:44 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

| On November 26, 2006 4:14 PM Gabriel Dos Reis wrote:
| >
| > I would like to test build-improvements on cygwin, but gcl-2.6.8pre
| > (and gcl-2.6.7) would not compile there. If you know of the status of
| > GCL on cygwin, please tell me; otherwise, I fear Axiom on cygwin will
| > be delayed till I have time to work on GCL for cygwin (which will be
| > none this year).
| >
|
| gcl-2.6.8pre compiles natively under Windows using MSYS/MinGW.
| cygwin is not necessary.

Thanks, I'll give it a shot.
I tried cygwin because it is was I'm used when I'm forced to use windows.

-- Gaby
Gabriel Dos Reis
2006-11-27 05:30:53 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

| On November 26, 2006 4:14 PM Gabriel Dos Reis wrote:
| >
| > I would like to test build-improvements on cygwin, but gcl-2.6.8pre
| > (and gcl-2.6.7) would not compile there. If you know of the status of
| > GCL on cygwin, please tell me; otherwise, I fear Axiom on cygwin will
| > be delayed till I have time to work on GCL for cygwin (which will be
| > none this year).
| >
|
| gcl-2.6.8pre compiles natively under Windows using MSYS/MinGW.
| cygwin is not necessary.

Well, I installed all I would find on mingw website. The configure for
GCL-2.6.8pre failed because it could not find some of the basic
standard headers. Meanwhile, I learnt what -mno-cygwin can trick GCC
to do. I'm falling back to that. It's late, I'm going to bed.

-- Gaby
Bill Page
2006-11-27 15:56:45 UTC
Permalink
...
| >
| > Well, I installed all I would find on mingw website. The
| > configure for GCL-2.6.8pre failed because it could not find
| > some of the basic standard headers.
|
| I built GCL and AXIOMsys on Windows recently (last month). It
| should still work. How to setup MSYS/MinGW to compile GCL and
|
| http://wiki.axiom-developer.org/BuildAxiom
As I said, installed all I could find on mingw webite labelled as
current.
I have no idea if installing the packages marked current on the
mingw site will work, however I did test the following recipe on
the BuildAxiom page:

[Install] In the following order:

1. MinGW-5.0.3.exe

This package will install mingw-runtime, w32api, GCC, binutils
and mingw32-make. See the instructions for Getting Started

2. MSYS-1.0.11-2004.04.30-1.exe

This installation file contains basic Unix-style commands plus
a commandline shell. See How to install MSYS

3. msysDTK-1.0.1.exe

This package provides common packages that are needed by
developers.
Configuring Axiom fails from the outset becaure there is no
<regex.h> under mingw (there is one with cygwin).
If you are trying to build Axiom from build-improvements I am
sure that will fail because there are patches specfic to Windows
in the axiom--windows--1 branch that were never incorporated
into the axiom--main--1 branch from which build-improvements was
derived.

And of course under native windows you will not be able to build
all of the non-lisp components because some depend on X11 and
some require pty support neither of which is available on native
Windows.

But you can probably get at least some of the missing libraries
required by build-improvements from

http://sourceforge.net/projects/mingwrep

In particular regex is here:

http://sourceforge.net/project/showfiles.php?group_id=7382&package_id=12650
Attemping to configure GCL only also fails because some fundamental
headrs are missing. I can't copy-n-patse the error because: (1) the
windows machine is separate from the one I'm using, (2) the windows
machine is at home and there is no access to it from the outside.
I built gcl-2.6.8pre and Axiom (from axiom--windows--1) in September
as described here:

http://lists.nongnu.org/archive/html/axiom-developer/2006-09/msg00625.html
http://lists.nongnu.org/archive/html/axiom-developer/2006-09/msg00617.html

If you are serious about building Axiom on Windows, I would be glad to
repeat this to verify that it still works. I would really like to work
with someone to create a new release of Axiom for Windows. The Windows
version is extremely popular if you believe the download statistics but
there seem to be very few people willing to work on Axiom development
on Windows. :-(
I've a long day ahead of me, so I'll not return to Axiom matters
before tonight.
| Once built, GCL call be called from cygwin if you wish. This
yes, I know.
| would be necessary for example to build the parts of Axiom that
| depend on pty (like sman) since there is no pty under native
| windows.
|
| > Meanwhile, I learnt what -mno-cygwin can trick GCC to do.
| > I'm falling back to that.
|
| Yes, I think it should be possible to cross-compile from cygwin
| but that seems like a lot of trouble. If you get this to work
| then please let me know.
It is not much of cross-compilation. Recent versions of cygwin
include package for mingw.
If you can make this work, great! Please keep me informed.

Regards,
Bill Page.
Gabriel Dos Reis
2006-11-28 10:16:31 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

[...]

| I built gcl-2.6.8pre and Axiom (from axiom--windows--1) in September

I installed mingw on yet another machine, restarted everything from
scratch, and now I can report that I was able to build GCL-2.6.8pre
on mingw and preliminary tests I made were working.

However, I was unable to build a working gcl on cygwin using the mingw
package. The generated executable does not use cygwin dll, but
raw_pre_gcl generated a stack overflow (my mingw/cygwin-fu is not
adequate to tackle a debug yet). Consequently, the link for saved_gcl
with for unresolved symbols (which are present in libgcl.a!).

Then I decided to try build-improvements on mingw to get a feeling.
That led to to rethink some configure test. Pristine noweb cannot be
compiled on mingw because the archive contains symlinks. So untarred
noweb using cygwin, modified the makefile using cygwin, then invoked
make [all install] under mingw. I had a functional noweb installed.
Next, the modified build-improvement's configure completed (under
mingw). Make stopped on bsdsignal.c. Which prompted me to look into,
and get out horrified: Axiom has a tendency of checking for platforms,
when in fact it is interested in functionalities most of the time.
I changed bsdsignal.c to test for availability of sigaction(),
SA_RESTART, SA_INTERRUPT, etc. The build proceeded till func_key.c
which has some calls to fork() and wait(). At that point, I decided
that I had enough experience for tonight and tackle daytime job
things.

I'll install a cross-compiling environment linux-x-mingw and use that
for further investigation. And I'll tackle the mingw/cygwin issue
much later when time permits.


-- Gaby
Bill Page
2006-11-27 14:01:37 UTC
Permalink
| >
| > I would like to test build-improvements on cygwin, but
| > gcl-2.6.8pre (and gcl-2.6.7) would not compile there. If
| > you know of the status of GCL on cygwin, please tell me;
| > otherwise, I fear Axiom on cygwin will be delayed till I have
| > time to work on GCL for cygwin (which will be none this year).
| >
|
| gcl-2.6.8pre compiles natively under Windows using MSYS/MinGW.
| cygwin is not necessary.
Well, I installed all I would find on mingw website. The
configure for GCL-2.6.8pre failed because it could not find
some of the basic standard headers.
I built GCL and AXIOMsys on Windows recently (last month). It
should still work. How to setup MSYS/MinGW to compile GCL and
Axiom is described here:

http://wiki.axiom-developer.org/BuildAxiom

Once built, GCL call be called from cygwin if you wish. This
would be necessary for example to build the parts of Axiom that
depend on pty (like sman) since there is no pty under native
windows.
Meanwhile, I learnt what -mno-cygwin can trick GCC to do.
I'm falling back to that.
Yes, I think it should be possible to cross-compile from cygwin
but that seems like a lot of trouble. If you get this to work
then please let me know.

Regards,
Bill Page.
Gabriel Dos Reis
2006-11-27 20:16:45 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

| On November 27, 2006 1:02 PM Gaby wrote:
| > ...
| > I've changed build-improvements to separate X11 stuff from
| > non-X11 stuff.
| >
|
| Is that separation of X11 stuff in the current build-improvements
| branch? (I could use that on some of the SourceForge compile farm
| machines.)

Yes, it is. (As part of my largest commit this week-end).

Configure detects (or can be told through --without-x) that X11 is not
available. Consequently we don't build the X11 parts in src/
(e.g. graph, hyper).
However, I just discover that I did not commit the corresponding
part for src/lib. Before you waste your time trying, let me correct
that part first.

| > The purpose of the exercise of building build-improvements on
| > cygwin (or mingw) is to see how effective the separation is,
| > and what remains to have an effective build.
|
| I would think that that main reason for using cygwin would be
| because you want X11 and pseudo terminals. No? But under native
| Windows this separation is necessary.

the reason I chosed to use cygwin for the host and build machine is
that it provides some POSIX/Unix functionalities that I'm not sure I
have under mingw, and on the other hand I can convince GCC to generate
mingw binaries from cygwin. It was not for the X11 stuff.

-- Gaby
Bill Page
2006-11-27 18:54:24 UTC
Permalink
Post by Gabriel Dos Reis
...
I've changed build-improvements to separate X11 stuff from
non-X11 stuff.
Is that separation of X11 stuff in the current build-improvements
branch? (I could use that on some of the SourceForge compile farm
machines.)
Post by Gabriel Dos Reis
The purpose of the exercise of building build-improvements on
cygwin (or mingw) is to see how effective the separation is,
and what remains to have an effective build.
I would think that that main reason for using cygwin would be
because you want X11 and pseudo terminals. No? But under native
Windows this separation is necessary.
Post by Gabriel Dos Reis
| If you are serious about building Axiom on Windows, I would
| be glad to repeat this to verify that it still works.
If you believe I was not or am not serious, we can terminate
this conversation.
Ok, lets lighten-up. I know I probably deserved that considering
my other thread. Peace. :-)

I am very happy that you are serious about building Axiom on
Windows and I will help however I can. A new version for Windows
is badly needed. Later today I will re-verify the my build of
gcl-2.6.8pre under Windows and let you know.

Building on cygwin is definitely an option if we want to get the
full functionality of Axiom as it exists on Linux, but it is
important to keep in mind that *most* Windows users are very
resistant to installing cygwin and even more negative about using
X11 applications under Windows because the look and feel is so
different. So I think we want at least AXIOMsys to run natively
without cygwin or the cygwin libraries.

Regards,
Bill Page.
Bill Page
2006-11-28 15:37:32 UTC
Permalink
Post by Gabriel Dos Reis
I installed mingw on yet another machine, restarted everything
from scratch, and now I can report that I was able to build
GCL-2.6.8pre on mingw and preliminary tests I made were working.
Great.

I did not complete my testing of a new MSYS/MinGW configuration
because I got bitten by another repository bug. I still can not
use svn (TortioseSVN) on Windows because the SourceForge site dies
with the well known read error and irrevocably locks the repository.
And I can not obtain build-improvements from Google Code because we
long ago exceeded the maximum allowed capacity of that site and it
is no longer being updated. So I opted for darcs only to discover
that at patch 150 the 'get' it dies because it says it cannot apply
the rename of SEGBIND.ps. And I haven't figured out how to stop
darcs from applying this patch. This is a new failure sense it used
to work *before* we decided to normalize the duplicate file names
to lower case. I wonder whether 'darcs get' of build-improvements
still works on MAC OSX?

Right now I am trying again using Mercurial.
Post by Gabriel Dos Reis
However, I was unable to build a working gcl on cygwin using the
mingw package. The generated executable does not use cygwin dll,
but raw_pre_gcl generated a stack overflow (my mingw/cygwin-fu is
not adequate to tackle a debug yet). Consequently, the link for
saved_gcl with for unresolved symbols (which are present in
libgcl.a!).
Adapting gcl memory management to windows is a big problem. Mike
Thomas did most of the work to make gcl run on Windows but he
specifically did not want to work with cygwin. I am not sure that
Mike is still here on this email list... but with Camm's help we
might be able to solve this problem for cygwin.
Post by Gabriel Dos Reis
Then I decided to try build-improvements on mingw to get a feeling.
That led to rethink some configure test. Pristine noweb cannot
be compiled on mingw because the archive contains symlinks.
The MSYS tools (including tar, I think) simulate symlinks via copy.
This works in most cases. In particular I am able to compile noweb
without changes.
Post by Gabriel Dos Reis
So untarred noweb using cygwin, modified the makefile using cygwin,
then invoked make [all install] under mingw.
Usually mixing cygwin and MSYS/MinGW tools is a *bad* idea because
they use quite different conventions when it comes to solving
compatibility problems with Windows.
Post by Gabriel Dos Reis
I had a functional noweb installed.
Next, the modified build-improvement's configure completed (under
mingw). Make stopped on bsdsignal.c. Which prompted me to look
into, and get out horrified: Axiom has a tendency of checking for
platforms, when in fact it is interested in functionalities most
of the time. I changed bsdsignal.c to test for availability of
sigaction(), SA_RESTART, SA_INTERRUPT, etc. The build proceeded
till func_key.c which has some calls to fork() and wait(). At
that point, I decided that I had enough experience for tonight
and tackle daytime job things.
I am surprised that bsdsignal.c compiled. I think Windows has fewer
and different signals than unix.

There is no fork() or wait() under native Windows so that was a
good place to suspend your testing. :-)
Post by Gabriel Dos Reis
I'll install a cross-compiling environment linux-x-mingw and
use that for further investigation.
I suspect that you will find this even less likely to succeed
but I am interested in seeing how far you get.
Post by Gabriel Dos Reis
And I'll tackle the mingw/cygwin issue much later when time
permits.
I keep hoping that some day the Axiom project will attract
a really experienced Windows Developer...

Regards,
Bill Page.
Gabriel Dos Reis
2006-11-28 16:06:01 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

[...]

| > So untarred noweb using cygwin, modified the makefile using cygwin,
| > then invoked make [all install] under mingw.
|
| Usually mixing cygwin and MSYS/MinGW tools is a *bad* idea because

I'm not so sure. The combination let me move through the
shortcomings of both individual systems that popped up. Of course,
you have to know the conventions.

| > I had a functional noweb installed.
| > Next, the modified build-improvement's configure completed (under
| > mingw). Make stopped on bsdsignal.c. Which prompted me to look
| > into, and get out horrified: Axiom has a tendency of checking for
| > platforms, when in fact it is interested in functionalities most
| > of the time. I changed bsdsignal.c to test for availability of
| > sigaction(), SA_RESTART, SA_INTERRUPT, etc. The build proceeded
| > till func_key.c which has some calls to fork() and wait(). At
^^^^^^^^^^

sorry, this should read fnct_key.c.

| > that point, I decided that I had enough experience for tonight
| > and tackle daytime job things.
|
| I am surprised that bsdsignal.c compiled. I think Windows has fewer
| and different signals than unix.

Well, onece you understand what bsdSignal was supposed to implement, it
is not hard to rewrite it in a way that it "works". As a matter of
fact, it is a straightforward variation on an example in the classical
"Advanced Programming in the Unix Environment" book by Stevens.

In particular for MSYS, bsdSinal was not (originally) supposed to do
anything terribly useful, except to systematically report an error.
Note however that MSYS has a minimal <signal.h> (old style) that could
be used. Again, this requires to abstract over the irritating testing
for platforms, when one is interested in functionbality.

| There is no fork() or wait() under native Windows so that was a
| good place to suspend your testing. :-)

A few other filess compile. bsdsignal.c does not use fork and wait.
it is fnct_key.c.

| > I'll install a cross-compiling environment linux-x-mingw and
| > use that for further investigation.
|
| I suspect that you will find this even less likely to succeed
| but I am interested in seeing how far you get.

Most definitely, I don't want to have to switch to
windows while I'm testing ideas on both linux (pirmary) and
mingw/cygwin (secondary).

| > And I'll tackle the mingw/cygwin issue much later when time
| > permits.
| >
|
| I keep hoping that some day the Axiom project will attract
| a really experienced Windows Developer...

I too hope -- because I have no intention to become an experienced
Windows developer (I don't have the time).

-- Gaby
Gabriel Dos Reis
2006-11-28 16:23:48 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:


[...]

| And I can not obtain build-improvements from Google Code because we
| long ago exceeded the maximum allowed capacity of that site

That was predictable.

-- Gaby
root
2006-11-28 16:30:25 UTC
Permalink
Post by Gabriel Dos Reis
| And I can not obtain build-improvements from Google Code because we
| long ago exceeded the maximum allowed capacity of that site
That was predictable.
I can see how you feel that way but I'm a bit puzzled.

Didn't google give us a full gig of disk space?

I thought SVN only copies files when they are changed
so branches should not be expensive since they store
only changes. I know that GIT only copies files when the
checksums differ so several copies of Axiom occupy almost
no additional space (or time, since GIT will not rsync
files that are duplicates). Surely Axiom will fit in a
gig of disk space.

t
Humberto Ortiz Zuazaga
2006-11-28 16:55:39 UTC
Permalink
Post by root
I thought SVN only copies files when they are changed
so branches should not be expensive since they store
only changes. I know that GIT only copies files when the
checksums differ so several copies of Axiom occupy almost
no additional space (or time, since GIT will not rsync
files that are duplicates). Surely Axiom will fit in a
gig of disk space.
I think I pulled the whole axiom tree with svk:

$ svk list /mirror/axiom
branches/
silver/
tags/
trunk/

$ du -sh ~/.svk/remote
364M /Users/humberto/.svk/remote
--
Humberto Ortiz-Zuazaga
Programmer-Archaeologist
UPR Bioinformatics Resource Center
http://www.hpcf.upr.edu/~humberto/
Alfredo Portes
2006-11-28 17:05:43 UTC
Permalink
Hi Bill,
Post by Humberto Ortiz Zuazaga
$ svk list /mirror/axiom
branches/
silver/
tags/
trunk/
In google, what is trunk and what is silver??? Isn't one of them redundant?
Sorry I lost track of the new schema for the repos.

Regards,

Alfredo
root
2006-11-28 16:57:44 UTC
Permalink
Post by Humberto Ortiz Zuazaga
Post by root
I thought SVN only copies files when they are changed
so branches should not be expensive since they store
only changes. I know that GIT only copies files when the
checksums differ so several copies of Axiom occupy almost
no additional space (or time, since GIT will not rsync
files that are duplicates). Surely Axiom will fit in a
gig of disk space.
$ svk list /mirror/axiom
branches/
silver/
tags/
trunk/
$ du -sh ~/.svk/remote
364M /Users/humberto/.svk/remote
That seems about right.
Arch axiom--main--1--patch-50 reports 305M
GIT for the same tree reports 135M

t
Gabriel Dos Reis
2006-11-28 18:18:13 UTC
Permalink
root <***@axiom-developer.org> writes:

| > | And I can not obtain build-improvements from Google Code because we
| > | long ago exceeded the maximum allowed capacity of that site
| >
| > That was predictable.
|
| I can see how you feel that way but I'm a bit puzzled.
|
| Didn't google give us a full gig of disk space?

I don'think they did.

| I thought SVN only copies files when they are changed
| so branches should not be expensive since they store
| only changes.

well, we have had many branches, some of them actually share nothing
in common (I think they must have 3 distinct copies of Axiom in the
repo), and there were lots of property changes on almost every
file.


-- Gaby
Humberto Ortiz Zuazaga
2006-11-28 16:47:23 UTC
Permalink
Post by Bill Page
I did not complete my testing of a new MSYS/MinGW configuration
because I got bitten by another repository bug. I still can not
use svn (TortioseSVN) on Windows because the SourceForge site dies
with the well known read error and irrevocably locks the repository.
And I can not obtain build-improvements from Google Code because we
long ago exceeded the maximum allowed capacity of that site and it
is no longer being updated. So I opted for darcs only to discover
that at patch 150 the 'get' it dies because it says it cannot apply
the rename of SEGBIND.ps. And I haven't figured out how to stop
darcs from applying this patch. This is a new failure sense it used
to work *before* we decided to normalize the duplicate file names
to lower case. I wonder whether 'darcs get' of build-improvements
still works on MAC OSX?
I haven't checked darcs today but I checked out a copy of
build-improvements today with svk on MacOS X 10.4.

svk sync mirrors the whole svn repository, but it's files are named
after the revision numbers. I guess it will work if you don't try to
check out a version with the name conflicts.
--
Humberto Ortiz-Zuazaga
Programmer-Archaeologist
UPR Bioinformatics Resource Center
http://www.hpcf.upr.edu/~humberto/
Bill Page
2006-11-28 17:16:19 UTC
Permalink
Post by root
Post by Gabriel Dos Reis
| And I can not obtain build-improvements from Google Code
| because we long ago exceeded the maximum allowed capacity
| of that site
That was predictable.
I can see how you feel that way but I'm a bit puzzled.
Didn't google give us a full gig of disk space?
No. They eventually grudgingly "compromised" at 750MBMbytes.
See email from Ben Collins-Sussman below.
Post by root
I thought SVN only copies files when they are changed
so branches should not be expensive since they store
only changes.
Yes, in theory. However when creating a mirror it seems that this
may or may not be the case. If the file systems are the same then
one can just rsync the files and the result is identical but
according to Ben, the version of SVN at Google Code uses some
new and different file system so simply copying the underlying
repository files is not an option. The last email I have from him
on this issue mentioned a possible problem with space allocation
or maybe a disk usage accounting error.

Also since then we have two new branches in the SourceForge SVN
repository. Apparently creating branches via the SVK mirror and
smerge commands does *not* conserve disk space.

And another thing is that since this is a repository and all
history is kept, everything we do takes more space. For example
each new version of gcl-2.6.8pre.tgz that we commit to the SVN
repository adds another 50 Mbytes since no attempt is made to do
binary diffs. (I think there are already 4 versions of gcl in the
build-improvements branch.)
Post by root
I know that GIT only copies files when the checksums differ so
several copies of Axiom occupy almost no additional space (or
time, since GIT will not rsync files that are duplicates).
Surely Axiom will fit in a gig of disk space.
"GIT only copies files when the checksums differ". Are you sure?
Are checksums that reliable? Can you give me a reference? But I
agree that GIT uses significantly less space than SVN (and Mercurial
uses even less).

Regards,
Bill Page.
Post by root
-----Original Message-----
Sent: October 9, 2006 2:14 PM
Subject: Re: [Axiom-developer] RE: [M#73697383] Re: Disk-quota Request
Post by Gabriel Dos Reis
...
Can we get the 1G and be done with it?
.. but then how would help debug the Google, svnsync, svk, and svn
programs? ;)
Enough is enough. You guys have been more than patient as
our guinea pigs!
I just rsync'd your sourceforge svn repository down to my local box
(165MB), made a dumpfile from that, then did an 'svnadmin load' in our
datacenter. Disturbingly, it still took two hours to load the history
and used up 381MB of space. I don't know whether our size-counting
code is buggy, or if our Bigtable storage schema is just really that
bad, I'll be investigating. It's not your problem. :-)
You're now live... all 176 revisions are up and happy. You're only
using 381 of 750MB of quota, so you should be fine for a good while.
Let me know if you have any more problems.
(Remember, do an https:// checkout if you intend to commit changes;
http:// checkouts produce read-only working copies.)
root
2006-11-28 17:26:56 UTC
Permalink
Post by Bill Page
"GIT only copies files when the checksums differ". Are you sure?
Are checksums that reliable? Can you give me a reference? But I
agree that GIT uses significantly less space than SVN (and Mercurial
uses even less).
GIT computes the MD5SUM of each file and each subdirectory and of the
whole tree. Only subtrees or files which have different MD5SUMs are
every stored, copied or transmitted. Thus creating a copy and checking
it in does almost nothing and takes almost no time to upload.

I know this works because I have switched to GIT for all of my
personal work. Multiple copies of gcl-2.6.8pre.tgz take no additional
space or time to transmit.

Arch seems to want to trade disk space for file transfer time.
If you check out an arch repository it makes a "pristine copy"
which it caches locally, thus taking twice the disk space. The
advantage is that you don't do any network traffic for some
operations because you have a local, unchanged copy. The disadvantage
is the huge disk space cost. GIT does not store a local copy but only
has to transmit the MD5SUM of the tree root or subdirectory to decide
if changes occur. I'm not sure what SVN does.

You can see it in the numbers I just sent. Arch takes 305M, GIT takes 135M
for the same copy of Axiom.

t
Gabriel Dos Reis
2006-11-28 18:22:20 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

[...]

| And another thing is that since this is a repository and all
| history is kept, everything we do takes more space. For example
| each new version of gcl-2.6.8pre.tgz that we commit to the SVN
| repository adds another 50 Mbytes since no attempt is made to do
| binary diffs. (I think there are already 4 versions of gcl in the
| build-improvements branch.)

Indeed. I'm of the opinion that keeping GCL as a tarball in the repo
is a mistake.

-- Gaby
Bill Page
2006-11-28 17:25:28 UTC
Permalink
Post by Alfredo Portes
Post by Humberto Ortiz Zuazaga
$ svk list /mirror/axiom
branches/
silver/
tags/
trunk/
In google, what is trunk and what is silver??? Isn't one of them
redundant? Sorry I lost track of the new schema for the repos.
'silver' is supposed to be an exact mirror of Tim's axiom--silver--1
repository.

'trunk' is the original trunk from the SourceForge SVN which was
copied from patch-50 with a few updates that were applied directly
to SVN but not backported to axiom--silver--1.

After the recent dispute over managing these repositories, I am
not certain of the status of these on SourceForge. The last
comment I recall from Tim was that he was bringing axiom-silver--1
up-to-date (and thus also 'silver' SVN at SourceForge because
there is a robot process that mirrors his changes) and that he
now considered 'silver' also up-to-date, so yes probably 'trunk'
is now redundant.

But because of the space problems on Google I am not certain that
the Google versions of these are identical to the SourceForge
versions.

In short the Google repository is a mess. We should probably
wipe it and try to start again. If you are inclined to help with
this, probably the best thing would be to contact Ben and discuss
it with him. Maybe he would be willing to manually re-populate
the Google repository again.

Regards,
Bill Page.
Waldek Hebisch
2006-11-28 19:56:14 UTC
Permalink
Post by Bill Page
Post by Alfredo Portes
Post by Humberto Ortiz Zuazaga
$ svk list /mirror/axiom
branches/
silver/
tags/
trunk/
In google, what is trunk and what is silver??? Isn't one of them
redundant? Sorry I lost track of the new schema for the repos.
'silver' is supposed to be an exact mirror of Tim's axiom--silver--1
repository.
'trunk' is the original trunk from the SourceForge SVN which was
copied from patch-50 with a few updates that were applied directly
to SVN but not backported to axiom--silver--1.
After the recent dispute over managing these repositories, I am
not certain of the status of these on SourceForge. The last
comment I recall from Tim was that he was bringing axiom-silver--1
up-to-date (and thus also 'silver' SVN at SourceForge because
there is a robot process that mirrors his changes) and that he
now considered 'silver' also up-to-date, so yes probably 'trunk'
is now redundant.
But because of the space problems on Google I am not certain that
the Google versions of these are identical to the SourceForge
versions.
Google shows revision 214. It looks that we exceeded Google quota by
creating 'silver'.
--
Waldek Hebisch
***@math.uni.wroc.pl
Page, Bill
2006-11-28 21:44:01 UTC
Permalink
Ben,

We are having some problems again with our Axiom repository on
Google Code. I am copying you on the email below as background.
To make a long story short: I think we need to ask you to delete
the current repository and rebuild it again via 'svnadmin load'
as you did back on October 9 (see copy of your email below).

Since that time the size of the Axiom repository on SourceForge
has grown from 165MB to about 364M due to the creation of two
new branches. Even accounting for the larger storage factor at
Google (roughly double), I think this should still fit within
our current 750MB quota.

If you have any questions about this please ask. Also if you can
give us an update on the status of the problem which you mentioned
in October 9 message, that would be great.

Regards,
Bill Page.
Post by Waldek Hebisch
...
Post by Bill Page
But because of the space problems on Google I am not certain
that the Google versions of these are identical to the
SourceForge versions.
Google shows revision 214. It looks that we exceeded Google quota
by creating 'silver'.
Yes, I think that is correct. Here is my attempt at post mortem:

Up to revision 199 (October 26), the 'svk smerge' transactions from
the axiom-developer.org server that are intended to synchronize the
Google repository with the SourceForge repository were properly
processed by Google. For reasons that I don't understand, the svk
smerge process did not automatically create the correct branch
structure at Google when we created 'silver' on SVN SourceForge.

Revision 214 on Google was the last of several transactions starting
with Revision 200 (including some errors) which I submitted to try
to create /silver with has the same structure as on SourceForge.
I ran a separate svk smerge job to try to transfer the contents
of 'silver' from SourceForge to Google. Revisions 200-210 appeared
to work but still resulted in wrong name and structure due to my
errors. Revision 210 was dated Sat Oct 28 03:44:24 2006.

After that I received notices each day from my svk smerge job on
the axiom-developer.org server showing that the storage quota was
exceeded on Google.

On November 15 I submitted 4 more revisons 211 to 214 which corrected
my errors by renaming and deleting unused branches. But this did not
have any positive effect at Google. Update transactions are still
not be processed.

Now I no longer even get any commit messages from Google at all
although the svk smerge job is still running each night and shows
that the local Google mirror on axiom-developer.org is being updated
successfully. It is not clear to me why I no longer seeing commit
attempts - either Google is not processing them or 'svk sync' is no
longer sending them. I have even recreated the svk google mirror
repository on axiom-developer in an attempt to re-synch. This worked
as expected but still no commit transactions seem to be attempted
at Google.

------

So I think the problems definitely were caused by the creation of
'silver'. But because we only have svn commit access to Google and
no svnadmin privileges, it seems impossible to be to correct this
problem without the intervention of people at Google.

Regards,
Bill Page.
Post by Waldek Hebisch
-----Original Message-----
Sent: Wednesday, November 15, 2006 2:27 PM
Subject: [axiom commit] r214 - silver_old
Author: synthesis.anikast.ca
Date: Wed Nov 15 10:40:08 2006
New Revision: 214
silver_old/
not needed
...
Post by Waldek Hebisch
-----Original Message-----
Sent: Friday, October 27, 2006 4:26 AM
Subject: [axiom commit] r200 - trunk2
Author: synthesis.anikast.ca
Date: Fri Oct 27 01:25:47 2006
New Revision: 200
trunk2/
/ (props changed)
Create /trunk2
-----Original Message-----
Sent: Thursday, October 26, 2006 4:29 AM
Subject: [axiom commit] r199 - branches/build-improvements/src/interp
Author: synthesis.anikast.ca
Date: Thu Oct 26 01:28:07 2006
New Revision: 199
/ (props changed)
branches/build-improvements/src/interp/ChangeLog.build-improvements
branches/build-improvements/src/interp/Makefile.in
branches/build-improvements/src/interp/Makefile.pamphlet
branches/build-improvements/src/interp/debugsys.lisp.pamphlet
* debugsys.lisp.pamphlet (build-interpsys): Adjust pathname to
files that are local to the current build directory.
* Makefile.pamphlet: Remove individual rules for making object
codes out of Boot pamphlet using bootsys.
(BOOT_TO_LISP, COMPILE_LISP): New.
(AXIOMsys_boot_sources): Likewise. List core Boot files here.
(<<extract source codes>>): New chunk. Abstract over special
individual rules to translate Boot to object code, using
bootsys.
* Makefile.in: Regenerate.
-----Original Message-----
Sent: October 9, 2006 2:14 PM
Subject: Re: [Axiom-developer] RE: [M#73697383] Re: Disk-quota Request
Post by Bill Page
...
Can we get the 1G and be done with it?
.. but then how would help debug the Google, svnsync, svk, and svn
programs? ;)
Enough is enough. You guys have been more than patient as
our guinea pigs!
I just rsync'd your sourceforge svn repository down to my local box
(165MB), made a dumpfile from that, then did an 'svnadmin load' in our
datacenter. Disturbingly, it still took two hours to load the history
and used up 381MB of space. I don't know whether our size-counting
code is buggy, or if our Bigtable storage schema is just really that
bad, I'll be investigating. It's not your problem. :-)
You're now live... all 176 revisions are up and happy. You're only
using 381 of 750MB of quota, so you should be fine for a good while.
Let me know if you have any more problems.
(Remember, do an https:// checkout if you intend to commit changes;
http:// checkouts produce read-only working copies.)
Gabriel Dos Reis
2006-11-28 22:16:38 UTC
Permalink
"Page, Bill" <***@drdc-rddc.gc.ca> writes:

[...]

| Up to revision 199 (October 26), the 'svk smerge' transactions from
| the axiom-developer.org server that are intended to synchronize the
| Google repository with the SourceForge repository were properly
| processed by Google. For reasons that I don't understand, the svk
| smerge process did not automatically create the correct branch
| structure at Google when we created 'silver' on SVN SourceForge.

I believe 'silver' was created from "scratch", so has not ancestry in
common with any other branch in the repository -- if my understand of
your famous picture is correct. It effectively is a
"repository" within the repository -- essentially, files are
_duplicated_ as opposed to being shared. For me, that explains the
sudden surge.

-- Gaby
Ben Collins-Sussman
2006-11-28 22:29:46 UTC
Permalink
On 28 Nov 2006 23:16:38 +0100, Gabriel Dos Reis
Post by Gabriel Dos Reis
[...]
| Up to revision 199 (October 26), the 'svk smerge' transactions from
| the axiom-developer.org server that are intended to synchronize the
| Google repository with the SourceForge repository were properly
| processed by Google. For reasons that I don't understand, the svk
| smerge process did not automatically create the correct branch
| structure at Google when we created 'silver' on SVN SourceForge.
I believe 'silver' was created from "scratch", so has not ancestry in
common with any other branch in the repository -- if my understand of
your famous picture is correct. It effectively is a
"repository" within the repository -- essentially, files are
_duplicated_ as opposed to being shared. For me, that explains the
sudden surge.
Yep, that would explain it. Branches are supposed to be cheap copies;
they're just a hardlink within the repository, so they're nearly
instantaneous to create and take essentially zero space when they're
first made. I don't know what you did on sourceforge, but it sure
wasn't a normal subversion branch!

I can reset the google repository to revision 0 if you'd like.
Gabriel Dos Reis
2006-11-28 23:00:13 UTC
Permalink
"Ben Collins-Sussman" <***@google.com> writes:

| On 28 Nov 2006 23:16:38 +0100, Gabriel Dos Reis
| <***@integrable-solutions.net> wrote:
| > "Page, Bill" <***@drdc-rddc.gc.ca> writes:
| >
| > [...]
| >
| > | Up to revision 199 (October 26), the 'svk smerge' transactions from
| > | the axiom-developer.org server that are intended to synchronize the
| > | Google repository with the SourceForge repository were properly
| > | processed by Google. For reasons that I don't understand, the svk
| > | smerge process did not automatically create the correct branch
| > | structure at Google when we created 'silver' on SVN SourceForge.
| >
| > I believe 'silver' was created from "scratch", so has not ancestry in
| > common with any other branch in the repository -- if my understand of
| > your famous picture is correct. It effectively is a
| > "repository" within the repository -- essentially, files are
| > _duplicated_ as opposed to being shared. For me, that explains the
| > sudden surge.
| >
|
| Yep, that would explain it. Branches are supposed to be cheap copies;
| they're just a hardlink within the repository, so they're nearly
| instantaneous to create and take essentially zero space when they're
| first made. I don't know what you did on sourceforge, but it sure
| wasn't a normal subversion branch!
|
| I can reset the google repository to revision 0 if you'd like.

That would be very much appreciated.

Bill, before your mirror to the next new repo, please let me make sure
that we don't store gcl and noweb as .tar.gz anymore.

-- Gaby
Page, Bill
2006-11-28 23:14:57 UTC
Permalink
Post by Ben Collins-Sussman
On 28 Nov 2006 23:16:38 +0100, Gabriel Dos Reis
Post by Gabriel Dos Reis
[...]
| Up to revision 199 (October 26), the 'svk smerge'
| transactions from the axiom-developer.org server that are
| intended to synchronize the Google repository with the
| SourceForge repository were properly processed by Google.
| For reasons that I don't understand, the svk smerge process
| did not automatically create the correct branch structure
at Google when we created 'silver' on SVN SourceForge.
I believe 'silver' was created from "scratch", so has not
ancestry in common with any other branch in the repository --
if my understand of your famous picture is correct. It
effectively is a "repository" within the repository --
essentially, files are _duplicated_ as opposed to being shared.
For me, that explains the sudden surge.
I don't understand "repository within repository" comment but
yes I agree that the large increase in size in not unexpected.
In fact we discussed exactly this before the creation of 'silver'
back in October. Do you recall our discuss of the consequences
of having two roots in the repository?

For me, the mystery is why 'svk smerge' did not successfully
mirror our creation of this new branch.
Post by Ben Collins-Sussman
Yep, that would explain it. Branches are supposed to be cheap
copies; they're just a hardlink within the repository, so they're
nearly instantaneous to create and take essentially zero space
when they're first made. I don't know what you did on
sourceforge, but it sure wasn't a normal subversion branch!
No it was not a normal subversion branch. Our problem has a long
history connected with different major Axiom developers using
different source code repository systems (arch, cvs and svn).
The original trunk in svn at SourceForge was created by the
SourceForge tool for converting cvs to svn. The contents of cvs
at SourceForge was obtained from a series of snapshots from the
arch master repository. Because of improperly flagged binary
files, the resulting svn repository was a bit of mess but that
was eventually corrected.

Meanwhile for a variety of reasons development continued on both
arch and svn independently with commits to both svn trunk and to
several svn branches as well as the original arch "silver". In
October we decided to try to merge the changes to svn trunk and
arch silver. My proposal was to automatically mirror the arch
silver repository as a new "trunk" on svn so that both sets of
developers could continue using their own repository systems.
This resulted in the duplication of files mentioned by Gaby.

To understand my motivation it is important to note that the svn
mirror of the silver repository contains the history of the
development in arch. While the existing trunk contains only the
snapshot history from cvs.

In mean time, we also implemented an automatic mirror process
that is supposed to keep the Axiom Google Code repository in
sync with the SourceForge repository. I guess in retrospect
creating 'silver' at SourceForge after implemented the automatic
mirror was a bad idea given that we are so severely limited in
disk quota at Google.

What I would really like to be able to do is "re-base" all of
the existing branches on svn on this new 'silver' so that they
are deltas of it instead of trunk. Then commits to the arch
silver master would still be properly reflected at SourceForge
and Google. But maybe this is a vane hope. And maybe the long
term continued use of arch is in doubt anyway.

To move forward, I think what we probably need to do is to replace
the trunk at SourceForge with the current contents of 'silver' at
SourceForge. I.e. compute the minimum delta from trunk to silver
and then apply it to trunk. And then scrap all of the history
related to 'silver' even though 'silver' actually has the more
complete history. I think we *might* be able to do this via
svnadmin at SourceForge... but I really need the help of someone
more knowledgeable than me in order to avoid yet another disk
space catastrophe.
Post by Ben Collins-Sussman
I can reset the google repository to revision 0 if you'd like.
You may recall that more than this was necessary since the way
the svk smerge mirror synchronization process works it seems to
result in much greater disk space ussage then just 'svnadmin load'.
So just reseting it to 0 may not be enough. We went around this
problem several times before until you decided to use svnadmin.

Regards,
Bill Page.
Page, Bill
2006-11-28 23:28:00 UTC
Permalink
Post by Gabriel Dos Reis
...
| I can reset the google repository to revision 0 if you'd like.
That would be very much appreciated.
Bill, before your mirror to the next new repo, please let me
make sure that we don't store gcl and noweb as .tar.gz anymore.
Yes, we discussed this previously and I had agreed to commit a
change so that both gcl and noweb appear as source in a new 'tools'
directory. I am sorry that I have not yet had time to do this.
I now think we should do this at the same time as merging silver
and trunk so that we can completely get rid of the zips directory.

But of course this is not the real cause of the disk space problems
which is really the duplication that you identified in your previous
message - it just makes it a bit worse.

So as a first step I will temporarily disable the cron job that runs
the 'svk smerge' to Google. Ok?

But I am still concerned that even after we get things cleaned up
in the SourceForge SVN, then we should not depend on 'svk smerge'
to initially re-populate the repository at Google since that is
likely to cause excessive disk space utilization again. However
I think that once populated via 'svnadmin load', using svk smerge
is probably ok provided I don't decide to do something stupid again...

Regards,
Bill Page.
Gabriel Dos Reis
2006-11-28 23:45:01 UTC
Permalink
"Page, Bill" <***@drdc-rddc.gc.ca> writes:

| On Tuesday, November 28, 2006 6:00 PM Gaby wrote:
|
| > Ben Collins-Sussman writes:
| > ...
| > | I can reset the google repository to revision 0 if you'd like.
| >
| > That would be very much appreciated.
| >
| > Bill, before your mirror to the next new repo, please let me
| > make sure that we don't store gcl and noweb as .tar.gz anymore.
| >
|
| Yes, we discussed this previously and I had agreed to commit a
| change so that both gcl and noweb appear as source in a new 'tools'
| directory. I am sorry that I have not yet had time to do this.
| I now think we should do this at the same time as merging silver
| and trunk so that we can completely get rid of the zips directory.
|
| But of course this is not the real cause of the disk space problems
| which is really the duplication that you identified in your previous
| message - it just makes it a bit worse.

Yes.

| So as a first step I will temporarily disable the cron job that runs
| the 'svk smerge' to Google. Ok?

I agree.

| But I am still concerned that even after we get things cleaned up
| in the SourceForge SVN, then we should not depend on 'svk smerge'
| to initially re-populate the repository at Google since that is
| likely to cause excessive disk space utilization again. However
| I think that once populated via 'svnadmin load', using svk smerge
| is probably ok provided I don't decide to do something stupid again...

I would think so; but I would very much welcome Ben's input here about
the best way to avoid the duplicate trees (maybe keep history only
after the repopulation?)

-- Gaby
Ben Collins-Sussman
2006-11-29 01:33:05 UTC
Permalink
On 29 Nov 2006 00:45:01 +0100, Gabriel Dos Reis
Post by Gabriel Dos Reis
I would think so; but I would very much welcome Ben's input here about
the best way to avoid the duplicate trees (maybe keep history only
after the repopulation?)
After reading the history of your project.... cvs, svn, svk, arch....
my head is truly spinning!

I hate to say this, but I think it's time to stop trying to use
multiple version control systems and multiple hosting services. If I
were running your project, I'd just 'stop the madness' and start
fresh. Completely start over. By that, I mean:

1. choose ONE hosting service
2. choose ONE version control system
3. import just the LATEST trunk code into that system (no history!)

Start coding against your latest codebase. If for some reason you
need to access history (look at an old branch or tag, or diff against
an old file), then dig out the older version control system and peer
into it.

Shocking statement: you really don't need your code history as much
as you think you do. In the scenario I'm advocating, your history
isn't even gone, it's just not super-convenient to access. Weigh that
slight inconvenience against the *monstrous* headaches you guys have
been going through trying to get svn/svk/arch to work together and
sync around. Your version control tool is supposed to be helpful for
collaboration and stay out of the way of your work.
Gabriel Dos Reis
2006-11-29 03:35:12 UTC
Permalink
"Ben Collins-Sussman" <***@google.com> writes:

| On 29 Nov 2006 00:45:01 +0100, Gabriel Dos Reis
| <***@integrable-solutions.net> wrote:
|
| > I would think so; but I would very much welcome Ben's input here about
| > the best way to avoid the duplicate trees (maybe keep history only
| > after the repopulation?)
|
| After reading the history of your project.... cvs, svn, svk, arch....
| my head is truly spinning!

I can understand that. Axiom is probably the only project I've worked on
that has more SCM than active developers :-/
(But, hey, I was the one who cannot work with arch; and we needed
something that can handle changeset, so CVS was no option).

Thanks!

-- Gaby
Gabriel Dos Reis
2006-11-29 20:07:32 UTC
Permalink
"Page, Bill" <***@drdc-rddc.gc.ca> writes:

| On Tuesday, November 28, 2006 6:00 PM Gaby wrote:
|
| > Ben Collins-Sussman writes:
| > ...
| > | I can reset the google repository to revision 0 if you'd like.
| >
| > That would be very much appreciated.
| >
| > Bill, before your mirror to the next new repo, please let me
| > make sure that we don't store gcl and noweb as .tar.gz anymore.
| >
|
| Yes, we discussed this previously and I had agreed to commit a
| change so that both gcl and noweb appear as source in a new 'tools'
| directory.

It should be done by now.

If SF/SVN server keeps its promises, you should experiment some
failures during the next update. You just have be patient ans start over
again until the update is finished. I would still appreciate if you
could wait more before starting the mirror to Google again.

-- Gaby

Waldek Hebisch
2006-11-29 02:11:55 UTC
Permalink
Post by Page, Bill
To understand my motivation it is important to note that the svn
mirror of the silver repository contains the history of the
development in arch. While the existing trunk contains only the
snapshot history from cvs.
AFAICS the current 'silver' was created as a big hunk in revision
208. Revisions 209 and 210 added a few changes (there are a few
other patches, but they reflect changes after cloning form arch)
-- I would not call this "history". Certainly 'trunk' contains
more information.
Post by Page, Bill
In mean time, we also implemented an automatic mirror process
that is supposed to keep the Axiom Google Code repository in
sync with the SourceForge repository. I guess in retrospect
creating 'silver' at SourceForge after implemented the automatic
mirror was a bad idea given that we are so severely limited in
disk quota at Google.
What I would really like to be able to do is "re-base" all of
the existing branches on svn on this new 'silver' so that they
are deltas of it instead of trunk. Then commits to the arch
silver master would still be properly reflected at SourceForge
and Google. But maybe this is a vane hope. And maybe the long
term continued use of arch is in doubt anyway.
To move forward, I think what we probably need to do is to replace
the trunk at SourceForge with the current contents of 'silver' at
SourceForge. I.e. compute the minimum delta from trunk to silver
and then apply it to trunk. And then scrap all of the history
related to 'silver' even though 'silver' actually has the more
complete history. I think we *might* be able to do this via
svnadmin at SourceForge... but I really need the help of someone
more knowledgeable than me in order to avoid yet another disk
space catastrophe.
I tried to check where disk space go. After looking at rsynced copy
of SF repository I see that most space is taken due to following
revisions:

201 80 Mb failed arcs clone ???
208 80 Mb current 'silver'
252
219
143
115
102
31 each 20Mb -- snapshots of gcl-2.6.8
--
Waldek Hebisch
***@math.uni.wroc.pl
Page, Bill
2006-11-29 03:19:17 UTC
Permalink
I've changed the subject to better reflect the subject and
omitted Ben from the Cc: list for now since he is probably
not interested in our detailed discussions on this subject.
Post by Waldek Hebisch
Post by Page, Bill
To understand my motivation it is important to note that the
svn mirror of the silver repository contains the history of
the development in arch. While the existing trunk contains
only the snapshot history from cvs.
AFAICS the current 'silver' was created as a big hunk in revision
208. Revisions 209 and 210 added a few changes (there are a few
other patches, but they reflect changes after cloning form arch)
-- I would not call this "history". Certainly 'trunk' contains
more information.
What I see from the commit emails are my failed attempts to
create 'silver' from revisions 205 to 215, then the real 'silver'
starting from revision 216, 217, and 218. But you are right no
history! :-( I suppose this must be because of the way that Tim
created axiom--silver--1 branch in arch. I had incorrectly assumed
that axiom--silver--1 had the same history as axiom--main--1 (gold)
but never checked until now. That is another mistake I made. Of
course if we had the real history from axiom--main--1 it would
contain at least 50 entrys - one for each patch set.

Now I am inclined to agree with Ben's recommendations in his last
email. I guess we can forget this history. Tim obviously didn't
think it was important anyway. But if we do move entirely from
SourceForge to Google Code are we really prepared to forget our
current history in SVN?
Gabriel Dos Reis
2006-11-29 03:45:37 UTC
Permalink
"Page, Bill" <***@drdc-rddc.gc.ca> writes:

[...]

| >
Waldek Hebisch
2006-11-29 11:02:58 UTC
Permalink
| How can we agree on a single host and a single repository system?
|
| > ...
| > I tried to check where disk space go. After looking at rsynced copy
| > of SF repository I see that most space is taken due to following
| >
| > 201 80 Mb failed arcs clone ???
| > 208 80 Mb current 'silver'
| > 252
| > 219
| > 143
| > 115
| > 102
| > 31 each 20Mb -- snapshots of gcl-2.6.8
| >
|
| 200-201 and 205-212 where both failed attempts. (I kept getting the
| name wrong in the Tailor configuration file.) I finally got it right
| in 213-218. Perhaps there are in fact 3 copies of the arch clone?
| Or did I somehow manage to rename one and re-use it?
|
AFAICS 208 was renamed two times: the final rename is 213.
| How can I clean-up such mistakes without leaving this mess in the
| repository?
|
I do not know how to clean SourceForge: it seems that their policy
forbids really deleting thing, so cleanup maybe impossible without
staff help. OTOH svn allow you to retrive changesets, so it should
be possible just to "replay" reasonable changes and create a new
somewhat clean repository -- but still containing all gcl madness
(and a lot of other trash).

I am affraid that the best thing we can do on SourceForge is to
update 'trunk' and delete 'silver'.
| Do you know the best way of merging 'silver' into 'trunk'? I think
| you once compared these versions and said there was really very
| little difference. Right?
Yes. I would retrive changesets from 'silver' and apply them to 'trunk'.
Then I would delete 'silver'. If you guys think that this is a good idea
I can do this.
diff compares files, not history or ancestry.
merge likes to have ancestry, so if you're doing star-merge and the
branches don't have common ancestry, typically SVK/SVK will first
delete then add (which you may think is silly but without ancestry,
that is a reasonable thing to do). That has the effect of
"duplicating" files.
Reasonable thing to do? I am not sure (GITs pack command factors files
into deltas even if they do not have common ancestry). But what you
wrote seem to explain part of disk usage.
| Are we prepared to cut the automatic connection with Tim's original
| axiom--silver--1? I believe Tim stated that he was no longer prepared
| to maintain this branch anyway. So I assume "yes".
|
IIRC Tim statement was about SourceForge. My understanding was that
he will put things in axiom--silver--1 when they are ready.
| By 252 to 31 do you mean there are 6 copies of the gcl-2.6.8pre
| tarball? I.e. 6 x 20 Mb = 120 Mb.
Yes. In fact, I think that we have 10 copies (giving us 5 or 6 versions)
of the gcl-2.6.8pre: the two copies of arch (201 and 208) probably
contain 2 copies (2 different versions) of gcl-2.6.8pre each.
Do we do that before or after my changes I'm testing? My changes
means we now have gcl/ and noweb/ toplevel subdirectories, and the
corresponding tarballs are gone; lsp/ is renamed to src/lisp.
It seems that you did not commit your changes. I belive that in
current sitation we should drop gcl from our repository and just
provide a script to fetch desired version form gcl repository.

I think it is time to start working on a 'dist' target for make.
I would imagine that such target should fetch dependencies and
build some files which may be problematic for users and that
pack everthing into a tarball.

FYI I recently succesfully recreated 142 viewports (all used in
HyperDoc) and almost all .pht files. ATM recreating graphic files
requires acccess to running X server. Also, the whole process is
somewhat fleaky, depending on little details it works or fails
-- it looks like random memory corruptions.

Also, I have a script that recreated many of book images.
--
Waldek Hebisch
***@math.uni.wroc.pl
Gabriel Dos Reis
2006-11-29 14:14:16 UTC
Permalink
Waldek Hebisch <***@math.uni.wroc.pl> writes:

[...]

| > diff compares files, not history or ancestry.
| > merge likes to have ancestry, so if you're doing star-merge and the
| > branches don't have common ancestry, typically SVK/SVK will first
| > delete then add (which you may think is silly but without ancestry,
| > that is a reasonable thing to do). That has the effect of
| > "duplicating" files.
| >
|
| Reasonable thing to do?

In the context of a system based on file ancestry, yes.

[...]

| > Do we do that before or after my changes I'm testing? My changes
| > means we now have gcl/ and noweb/ toplevel subdirectories, and the
| > corresponding tarballs are gone; lsp/ is renamed to src/lisp.
| >
|
| It seems that you did not commit your changes.

no, but after an hour, SVN said:

Commit into mirrored path: merging back directly.
Waiting for editor...
Merging back to mirror source https://svn.sourceforge.net/svnroot/axiom.
RA layer request failed: MERGE request failed on
'/svnroot/axiom/branches/build-improvements': MERGE of
'/svnroot/axiom/branches/build-improvements': 500 Internal Server
Error (https://svn.sourceforge.net)
Please sync mirrored path /axiom/branches/build-improvements first.

when in fact my tree is up-to-date. I'll merge bits and pieces.

-- Gaby
Page, Bill
2006-11-29 12:28:38 UTC
Permalink
Post by root
Post by Bill Page
"GIT only copies files when the checksums differ". Are you sure?
Are checksums that reliable? Can you give me a reference? But I
agree that GIT uses significantly less space than SVN (and
Mercurial uses even less).
Here is a reference:

http://en.wikipedia.org/wiki/Git_(software)

Mercurial is very similar:

http://www.selenic.com/mercurial/wiki/index.cgi/Design

except it also uses checksums to encode the object history.
Post by root
GIT computes the MD5SUM of each file and each subdirectory and of
the whole tree. Only subtrees or files which have different MD5SUMs
are every stored, copied or transmitted. Thus creating a copy and
checking it in does almost nothing and takes almost no time to
upload.
Actually SHA-1 hashes, not MD5, but the same principle applies.
Post by root
I know this works because I have switched to GIT for all of my
personal work. Multiple copies of gcl-2.6.8pre.tgz take no additional
space or time to transmit.
The multiple copies of gcl-2.6.8pre that we have been talking
about are nearly all slightly different pre-release versions. The
hash algorithm will ensure that each of these is treated as a new
and separate file, except for the extent to which GIT and Mercurial
can compute an efficient binary diff.
Post by root
...
GIT does not store a local copy but only has to transmit the
MD5SUM of the tree root or subdirectory to decide if changes
occur.
???

I don't think that is correct in most cases. Check the contents of
the .git directory in your local repository. Most of the space is
in .git/objects where the files and deltas are stored based on their
hash keys (as you said earlier).

GIT and Mercurial are distributed source code control systems so
they must necessarily distribute the data. In contrast SVN and CVS
require a centralized repository.
Post by root
I'm not sure what SVN does.
SVN also stores a local copy of the repository. SVK uses this to
allow distributed source code control.
Post by root
You can see it in the numbers I just sent. Arch takes 305M,
GIT takes 135M for the same copy of Axiom.
You are right that GIT and Mercurial repositories take about 1/2
the space of equivalent Arch and SVN repositories. Darcs is usually
intermediate between these two extremes.

Regards,
Bill Page.
Continue reading on narkive:
Loading...