Translate
IBU
Consol
|
|
Index
docs . freebsd . org / en / books / handbook / mirrors /
#external-mirrors
"A.4.1. Introduction
As of December 2020, FreeBSD uses git as the primary version
control system for storing all of FreeBSD's source code and
documentation. Changes from the git repo on the stable/11,
stable/12 and related releng branches are exported to the
subversion repository. This export will continue through the
life of these branches. From July 2012 to March 2021, FreeBSD
used Subversion as the only version control system for
storing all of FreeBSD's Ports Collection. As of April 2021,
FreeBSD uses git as the only version control system for
storing all of FreeBSD's Ports Collection."
Corrections Pending Integration @ 2019-09-14:
- Although CTM's etymology from first authour was "Cvs Tree
Mirror" that now misleads, per my http://www.berklix.org/ctm/#what
"Code Through Mail" might best summarise it at 2018-12
- CTM is now a means of delivering tree updates (Similar
result to to rdist6 & rsync & mirror, though those
rely on different functionality: end to end live pull using
ports some firewalls etc may block, whereas CTM just pushes
with SMTP, unlikely blocked)
- Most trees we deliver with CTM are src-* releases, + 1
ports, + 1 SVN, but surely CTM could just as readily deliver
a GIT tree as an SVN, as its already done for CVS too
-
2023-12-11 to 2024-01-04
src-13, src-14, ports-cur, src-cur :
Andre Albsmeier converting generator scripts
from svn to git, + new trees.
Julian added new mail lists
& removed old trees.
Andre's new deltas from 2023-12 onward honour
symlinks.
You need an up-to-date version of ctm:
-
2022-07-30
Status Thanks to Andre Albsmeier for reporting,
& Warner
Losh for fixing the up-stream git to svn
exporter that CTM depends on.
/pub/FreeBSD/development/CTM/svn-cur
1124 Jul 2 00:05 svn-cur.10151.xz
232956 Jul 30 00:06 svn-cur.10152.xz
/pub/FreeBSD/development/CTM/src-12
755 Jul 2 00:05 src-12.1491.gz
172904 Jul 30 00:07 src-12.1492.gz
-
2021-07-30 Status
ftp:// ftp . berklix . org / pub / FreeBSD /
development / CTM /
Not http:// ftp . berklix . org / pub /
FreeBSD / development / CTM / |
Date Of Last Delta |
Status
Scripts & mail list in place for
all
|
|
src-4/src-4.2355.gz |
Nov 15 2014 |
|
src-5/src-5.0862.gz |
Nov 15 2014 |
|
src-6/src-6.1661.gz |
Nov 12 2015 |
|
src-7/src-7.1772.gz |
Jul 1 2020 |
|
src-8/src-8.1980.gz |
Jul 1 2020 |
|
src-9/src-9.1930.gz |
Aug 11 2020 |
|
src-10/src-10.2188.gz |
Dec 17 2020 |
|
src-11/src-11.1937.gz |
Jul 28 08:11 |
Still being exported from git to svn |
|
src-12/src-12.1205.gz |
Jul 30 08:09 |
Still being exported from git to svn |
|
src-13/ |
|
The CTM scripts & mail list in place,
but I recall never exported from git to
svn |
|
src-14 Running 2024/ |
|
Running 2024 |
|
src-14r Started 2024-05-23/ |
|
Started 2024-05-23 |
|
src-cur/src-cur.14812.gz |
Dec 20 2020 |
No longer being exported from git to svn
? |
|
ports-cur/ports-cur.13927.gz |
Mar 31 16:34 |
No longer being exported from git to svn
? |
| svn-cur/svn-cur.09751.xz |
Jul 30 08:05 |
-
2021-01-05:
ftp:// ftp
. berklix . org / pub / git / README
To view directory: ftp:// ftp .
berklix . org / pub / git/
- NEW ADDITIONAL SERVICE: At 2019-08-16
04:00+0200
- 2019 August Outage, Status At 2019-08-16
04:00+0200 : On freebsd.org: all fixed now as
far as we know except ctm-src-cur still delayed by
moderation, & many others not checked if
moderated.
- OK: ctm-
src-11-fast, ctm-src-12, ctm-src-12-fast, ctm-src-cur-fast, ctm-svn-cur-fast, ctm-
ports-cur.
- Unchecked, likely
OK: ctm-
src-4, ctm-
src-4-fast, ctm-
src-5, ctm-
src-5-fast, ctm-
src-6, ctm-
src-6-fast, ctm-
src-7, ctm-
src-7-fast, ctm-
src-8, ctm-
src-8-fast, ctm-
src-9, ctm-
src-9-fast, ctm-
src-10, ctm-
src-10-fast, ctm-
src-11, ctm-
src-11-fast, ctm-
src-cur, ctm-svn-cur.
- The delta generator host
died.
- Generation moved to
ctm.berklix.org (which already had ftp
archives).
- Last backup of generator build
scripts was old,
- Generator build
scripts were
re-updated,
- Generator trees built
&
- Deltas generation
started.
- Briefly deltas were received
on lists
,
- When ctm-src-12 generation
started, it sent with From: owner-ctm-src-12[-fast]@mailman.berklix.org
, which was confusingly similar to
mailman standard alias ctm-src-12[-fast]-owner@
, so for ctm-src-12 it was
changed to From:
freebsd-ctm-bld@berklix
- Then although
ctm.berklix.org was mailing deltas
(other than ctm-src-12) to lists
on freebsd.org, after a bit, nothing
further appeared on lists
- Guessed that mail header
From: address did not match what
@freebsd .org lists
allowed.
- Guessed something had
happened the freebsd.org
end;
- The CTMownermail values in
the Mail
List Config Files that generate
the From: for the other lists, had not
changed, so those non ctm-src-12 lists
should have continued
working;
- Learnt how to specify the
From: field,
- The question was: what
address had been previously used ?
Various guesses failed experimenting
with old addresses including elements
cauchy, library, etc. .
- Gave up guessing, subscribed
a new From address
freebsd-ctm-bld@berklix to all ctm
lists.
- Guessed it would be by
default moderated.
-
Asked postmaster@freebsd to
un-moderate that address on all ctm
lists. Some or most were unmoderated
but at least one was not, as this was
seen for
ctm-src-cur:
- Created on
ctm.berklix.org UTC: Aug 12 14:08
src-cur.14144.gz
-
Outgoing tap to
test_address@:
- Date: Mon, 12 Aug
2019 14:08:02 GMT
- From:
freebsd-ctm-bld@berklix
- To: ctm-ctm-src-cur@mailman.berklix
- Subject: ctm-mail src-cur.14144.gz
1/1
-
Moderated:
- Subject: Your
message to ctm-ctm-src-cur
awaits moderator
approval
- From: ctm-ctm-src-cur-owner@freebsd
- Date: Mon, 12 Aug
2019 14:08:24
+0000
- To:
freebsd-ctm-bld@berklix
- Your mail to 'ctm-ctm-src-cur'
with the subject ctm-mail src-cur.14144.gz
1/1
- Is being held until
the list moderator can review
it for approval.
- Received from list on
jhs@ host UTC: Aug 12 14:10 src-cur.14144.gz
- ctm-svn-cur:
the tap on ctm.berklix.org mail
outgoing showed it ignoring the
directive set CTMownermail
freebsd-ctm-bld@berklix in
ctm.berklix.org/~sms/library/ctm/ctm_generation/ctm_conf.svn-cur
copy here:
www . berklix . com / ~jhs / src / bsd
/ fixes / freebsd / ports / gen / misc
/ ctm / files / ftp.berklix.org /
ctm_conf.svn-cur so freebsd.org
rejects those
Another script was fixed
.
-
Unchecked: A test was
run on all lists, to
ensure all have been unmoderated (as
some of the old lists rarely carry
traffic, so cannot be observed, but
might someday receive a delta we want
passed).
Content was produced by this
#!/bin/csh
foreach i ( `cat tests_lists.txt` )
echo "To: $i" > tests.now
cat tests_body.txt >> tests.now
sendmail -i -t < tests.now
echo Done $i
sleep 3
end
None of those test have
arrived back from any lists, so we
don't know if all lists have been
un-moderated.
|
- 2019-09-23 Removed tap to test_address@ from
ctm-src-cur & ctm-src-cur-fast
|
- jhs: Optimise crontab entries, & publicly document so
users know when it runs.
- Maximum length in kilobytes (KB) of a message body
max_message_size = 300 by default. Either jhs@ can increase
if it goes over limit, or jhs@ can consult andre@ what limits
should be ? Maybe higher limits for fast streams ?
- jhs: Backup of generator build
scripts
to be adjacent to Mail List Config
Files
-
Write a brief intro on how delta generation works, (in case
something happens to Stephen, Julian, or
ctm.berklix.org)
Odds About
SVN To Move:
-
Request a
CTM recipient to: add in to
manual
Why Use
CTM for Delivery Instead of
SVN or GIT ?
- (
CTM can also deliver you an
SVN tree, so you can still use
SVN locally.)
- If you have a slow net connection aka limited bandwidth,
or intermittent connectivity eg rarely reliably in one place
long enough to complete an
SVN update of whole trees. Or expensive data rates eg
mobile tariffs.
- If your corporate/ institutional firewall blocks you, but
allows mail.
- The patches are compressed, saving time & data
transfer charges.
- Imagine you'r travelling/ on holiday, you don't fancy
paying heavy data charges tethering via your mobile phone, so
stroll into a coffee shop with WLAN, or visit somewhere with
an incomprehensible maybe foreign language documented fire
walled WLAN, or if lucky encounter a town centre open WLAN
(there's a few in central Munich), but you have nowhere to
sit, no power socket, & you want it all downloaded as
fast as possible, not blocked on unknown ports, no thinking
on the hoof required, combined with mail pick up you know
won't be blocked:
CTM has it all rolled up ready for you. You don't even
have to stand with no seat, consuming battery, applying all
the deltas.
CTM deltas can be applied later where you do have a seat
& power supply.
- Imagine you'r in an airport departure lounge, with WLAN,
but may be called any time. So you just want those deltas
downloaded Fast. You can apply them later, perhaps even on
the plane/ train/ hotel with no net.
- A unique reproducible trees with eg /usr/svn/.ctm_status,
additional to the .svn status
- Pre extracted trees for all versions of src & ports
as well as
SVN tree available via
CTM delivery, so you can do local
SVN searches, exports etc.
- Suggestion: You may want to subscribe a different mail
address for
CTM delta reception, so you can first specify eg
fetchmail fetches your human mail.)
- Errors or Omissions ? Mail page author and/
or list ctm-users@freebsd.org
- "Code Through Mail" might best summarise it at
2018-12
-
man 1 ctm,
man 5 ctm,
man 1 ctm_smail,
man 1 ctm_rmail,
man 1 ctm_dequeue
- Source Code (patches, delivery & update)
Through Mail.
CTM is ideal for receiving source code updates on slow,
intermittent, or expensive tarif (eg mobile/cell) internet
connections. Used by & supported on FreeBSD (FreeBSD also supports
& is supported by other access methods eg
rsync);
CTM could easily be used by other BSDs, Linuxes &
beyond etc.
- Etymology: "Cvs Tree Mirror" as named by code author phk@ in
1994-04-11
(
CVS is a source code repository management system, as are
eg SCCS,
RCS,
SVN/Subversion.
BSD used SCCS,
FreeBSD used CVS, although FreeBSD now (2018) uses
SVN,
CTM continues to deliver FreeBSD Code Through Mail.
- Started by providing Deltas with public ftp access, (When
freebsd.org decided to cease to host the master deltas sent
from the generator site.
(& thus we also lost all out mirror sites).
- A set of
CTM Generator Build Scripts
& trees, ready to generate deltas
were copied in case anything failed at Missouri build
site.
- ctm-src-12 mail list created on
ctm.berlix.org
- 2019, The missouri master host died & production
moved to ctm.berlix.org
- Please do not overload ftp://ftp.berklix.org. There are
no mirrors. If you want to offer a public mirror, please do,
please contact contact host
master or ctm-users@.
- Please avoid heavy traffic during work days &
business hours in Germany (winter time zone +01:00, summer
+02:00). Time
now At 2019-08 it's a lightly loaded host, but that may
change, & I dont want to annoy the host.
- There is no point regularly ftp'ing latest deltas from here, as a purpose of
CTM is to provide Push technology, so you should be
receiving distribution updates by ctm-* mail lists.
If you want to regular fetch (which is Pull technology, not
push) then you should not be using
CTM, but be using
SVN or
Rsync direct form a freebsd.org mirror, nor from
berklix.org
Just use berklix.org ftp archives if you fail to receive a
delta via
ctm_rmail.
-
Mail Lists
- ctm-announce@freebsd
Not Used Now Stephen (ex generator of deltas)
prefered to use ctm-users@ so users could reply.
Ditto Julian. Andre has now succeeded Stephen. Archive
of announcements lists
. freebsd . org / pipermail / ctm-announce /
Join here:
lists . freebsd . org / mailman / listinfo /
ctm-announce
- ctm-users@freebsd
.org ( fallback http://
mailman.berklix.org/mailman/listinfo/ctm-users )
Archive of problems, solutions, announcements lists .
freebsd . org / pipermail / ctm-users/
Join here: lists
. freebsd . org / mailman / listinfo / ctm-users
-
- Contains dead link:
http://www.freebsd.org/handbook/synching.html#CTM
-
Some topics:
-
lists .
freebsd . org / mailman / listinfo has lots of CTM delta distribution
lists
- www
. berklix . com / ~jhs / dots / .procmailrc_lists
Examples of how to automatically process receipt from
these lists, applying the deltas to update a tree.
-
Documentation
- Dead link: http:// www . freebsd . org / handbook
/ synching . html # CTM
(in mail archive intro, not Href'd) URL redirects
to
Dead link: https:// www . freebsd . org / doc /
en_US . ISO8859 - 1 / books / handbook / synching .
html # CTM
- Dead link: http:// www . freebsd . org / doc /
en_US . ISO8859 - 1 / books / handbook / ctm.html @
2016-04-23 & 2016-09-01
-
svnweb . freebsd . org / base / head / usr.sbin / ctm
/ ctm / ctm.1
/usr/src usr.sbin/ctm/ctm/ctm.1 point to this page,
kindly committed by brueffer@
Mon 14 Dec 2016 svn rev 292212 from a patch jhs@ sent.
-
Deltas For FreeBSD
- Stephen Montgomery-Smith ran the incremental delta
generator.
- Andre Albsmeier is converting scripts from svn to
git
- ftp://
ftp . berklix . org / pub / FreeBSD / development / CTM
/
Deltas: 16.3 G (@ 2019-01-25. Includes base deltas
(names such as src-cur.13800xEmpty.gz ,
ports-cur.12800xEmpty.gz , ctm-svn-cur.07300xEmpty.xz ) as well as
incremental deltas (names such as src-cur.13882.gz ,
ports-cur.12833.gz , ctm-svn-cur.07398.xz ) of base deltas,
Updated regularly (@ 2019-01) with rsync from Master.
(Pre 2015, not sure if still the same) times were (TZ:
GMT(UTC)-06:00), starting at 23:00 GMT(UTC) =
(midnight German
(GMT(UTC)+01:00) time), ( maybe +/- an hour
when computers may implement Missouri time on master
& German summer times on slave mirror). (May
also vary later subject to traffic experience), (Not
directly hung after the master
CTM server runs (several times per day).
-
Patches For
CTM source itself
(Not to apply to generic large FreeBSD source trees of
src/ports/svn/) but patches to to apply to the small
CTM source itself. Patches to support newer releases
of FreeBSD
CTM streams.
(
CTM source lives within FreeBSD src/ & may (after
2018-10) migrate to ports/)
-
Generator Build Scripts &
Examples
-
berklix . com / ~jhs / src / bsd / fixes / freebsd / ports /
gen / misc / ctm / README.JHS
Full List of Delta Mail Lists
& FTP Archives on freebsd.org & ctm.berklix.org:
- Delta Generator converter, from Git Build Scripts, Andre Albsmeier
- Warner
Losh mainatining the up-stream git to svn exporter that
CTM depends on.
- Delta Generator maintainer, from SVN Build Scripts, Stephen
Montgomery-Smith @ Missouri.edu
- Delta Generator, maybe prior to Svn, Poul Henning Kamp I
recall ?
- Generator Site, FTP archive, ctm-src-13@ Mailman list: Julian H. Stacey @
Berklix.Org
Bandwidth & Hosting: IBU.DE
- Generator Site, FTP archive, ctm-src-12@ Mailman list: Julian H. Stacey @
Berklix.Org
Bandwidth & Hosting: IBU.DE
- Freebsd.org Distribution: ftpd & mailman (pre ctm-src-12), & security
recommendations 2015:
Peter Wemm @
FreeBSD.Org
- Maintainer of the GIT to SVN export server: Warner Losh
- Authors of
CTM: Code Poul-Henning Kamp see
his CTM
Saga ; Manual: Joerg
Wunsch & Patches by others
- Postmaster@@berklix.org for new delta mail
lists etc.
- Postmaster@@freebsd for old delta mail lists etc.
|
