HOW TO BECOME A MIRROR FOR THE APACHE SOFTWARE FOUNDATION

The Apache Software Foundation has mirror sites from all around the world, but we are always looking for more reliable and well connected sites that can help us distribute our software. We no longer encourage mirrors for the various Apache Software Foundation websites, since they have become too complicated to be easily replicated. Instead, we request mirrors that can aid in distributing the software itself by mirroring our main software distribution directory.

REQUIREMENTS

We have a few requirements for those wishing to run a mirror:

• At least 90 GB of available disk space. The current distribution directory is around 45 GB, but we wish to leave room for considerable expansion.
• You may mirror either the full distributions tree, or a reduced tree that excludes a few very resource-intensive (disk-wise and bandwidth-wise) projects.
• You must not trim or abridge the mirrored tree in any way.
• You must not modify the mirrored tree in any way. In particular, HEADER.html and README.html files must not be altered or removed ; see below for adding sponsor information.
• Your mirror must not be shown "inside" another site using, for instance, frames. 
• You must do an update-check at least once a day, and preferably twice a day. You may sync 4 times (max) per day, but only if you have a slow or poor connection, causing timeouts.
• You must subscribe to the mailing list for mirror maintainers (see below ).

In addition, we require that you run the Apache HTTP Server version 1.3.26 or 2.0.43 or later and use the following configuration for your web mirror to allow all the features of our download site to function optimally:

APACHE HTTP SERVER CONFIGURATION

  IndexOptions FancyIndexing NameWidth=* FoldersFirst ScanHTMLTitles DescriptionWidth=*
  HeaderName HEADER.html
  ReadmeName README.html
  AllowOverride FileInfo Indexes
  Options Indexes SymLinksIfOwnerMatch FollowSymLinks
  ErrorDocument 404 default

Please note that our HEADER.html files do not contain the HTML pre-amble, so it is important that the option SuppressHTMLPreamble is not enabled.

Please ensure that the server does not send a Content-Encoding header for any of the compressed archives. The hashes and signatures used to check downloads are created for the compressed archives so it is vital that the browser is not told to decompress them. For example, .tar.gz and .tgz files are compressed TAR files. They should have suitable Content-Type - e.g. application/x-gzip - but no Content-Encoding should be sent. If the server incorrectly sends Content-Encoding: x-gzip (for example), many browsers will automatically decompress the response. This produces a TAR file which will not verify when checked against the hashes or sigs.

We do add ftp mirrors to our list, but we much prefer http mirrors and we select only http mirrors as "preferred" mirrors.

MIRRORING TECHNIQUES

We only support rsync for updating mirrors.

You can find details on rsync at http://rsync.samba.org/. Update your mirror with :

rsync -avz --delete --safe-links rsync.apache.org::apache-dist /path/to/mirror

If you wish to exclude resource-intensive projects, replace ::apache-dist by ::apache-dist-most. Do not use --exclude.
(At the time of writing, the ::apache-dist module was 68GB and the ::apache-dist-most module a 35GB subset of it.)

If there is a problem with file/directory permissions, make sure you use a proper umask in your cronjob :

umask 022 ; rsync ...

Please don't rsync "on the hour" (cronjob minute 0) ; pick a random minute between 5 and 55. Never run cronjobs at minute 0 unless the nature of the job requires it.

Also note that you should run the job twice a day or at most 4 times a day

Please note: older versions of this page (June 2011 and before) said to use the options '-rtlzv'. This was wrong, because the '-p' (preserve permissions) option was missing. Please ensure you use '-avz'. The '-a' (archive mode) option includes the '-p' option.

Please ensure that you regularly check the output from the rsynch job so you can detect problems like lack of disk space or permission issues. Thanks.

SPONSOR INFORMATION

You can add sponsor information (only) as follows :

• edit the file /local/path/to/mirror/README.html
• add to your rsync options --exclude "/README.html"

The contents of README.html will appear near the bottom of your mirror's home page.

Please note that only the home-page (top-level) README.html may be customized in this way; all other README.html files must be mirrored unchanged. Don't omit the '/' in "/README.html".

Please limit the contents of the README.html to something like This mirror is donated by www.domain.com [url] to support the Apache open source community. Refering to specific services or products may be seen as PageRank manipulation by search engine companies and may have an adverse effect on your site's ranking.

It has been noted that certain sites redirect 404 pages as a form of extra advertising or traffic booster. We will remove mirrors that do this from our list and then send an email to the maintainers address. The top level /README.html file as mentioned above is the ONLY local alteration of any part of the mirror we accept for you to provide Sponsor Information.

TESTING YOUR MIRROR

We sometimes get requests to add mirrors that are not working correctly. Please check at least the following:

• page headers and footers are being displayed correctly
• check that all the top-level directories and files on the ASF mirror are present on your mirror
• download a couple of files to check that they work.
• trying to download a missing file should generate a 404 Not Found error. You can do this by modifying one of the valid URLs that you tested above.

INFORM US!

Once the site is configured, tested, and updating consistently, you should subscribe to the mirror maintainers mailing list by sending an mail to "[email protected]".

If possible, please use an alias like [email protected] for the contact address rather than a personal e-mail address. To subscribe the alias address, send the email to "mirrors-subscribe-admin=example.org@apache.org".

Then you can provide us with details of your mirror by submitting them to the INFRA Issue Tracker.

You'll need to sign up for an account, then create a new issue in project "Infrastructure (INFRA)" with component "Mirrors".

Please include the following information in the Description field:

• URL of mirror :
• [optional] rsync mirror : rsync://rsync.your.org/apache/
• Country where the mirror is located :
• Contact email address :
• Update frequency :
• Rsync repository used :
• I use these rsync options :
• I have configured my Apache HTTP server as requested (yes/no)
• I have subscribed the contact address to [email protected] : (yes/no)
• At the moment, I rsync from the IP address : (your IP address here)
• I am checking the output from the rsynch job on a regular basis

We only use the IP to check the logs before we add the mirror to the list. Once your mirror is on the list, you can change the IP of the mirror without notifying us.

link: http://www.apache.org/info/how-to-mirror.html