Instant update mirroring

Instant Update Mirroring

The future tier'ed CPAN mirroring will use File::Rsync::Mirror::Recent (also known as rrr). Currently for CPAN mirroring (as opposed to PAUSE-only mirroring...) a recent version from git is recommended.

This setup is just in testing for now; don't try it unless you want to help make sure it works!

Configuration

Email ask@perl.org for a username/password for the master mirror for testing and put it into the script below which you'll run to mirror CPAN "instantly". It will take a while to get synced up, but after that it'll get the latest changes every 20 seconds with very little overhead.

What to test

The goal of this is to find and fix any bugs in the rrr tools and make sure that all updates to CPAN comes through to your mirror the same as the traditional "full" rsync.

In production it'll likely be recommended to run a full rsync every few days in case rrr missed something; but for the testing please don't do that. Maybe it won't be necessary at all.

Client

In the not too far future this will be rrr-client; but for now you have to make your own script.

#!/usr/bin/env perl
use strict;
use warnings;
use local::lib;

my $user = '';
my $pass = '';
my $CPAN = '/local/ask/CPAN';
my $tmp  = '/local/ask/tmp';

$ENV{RSYNC_PASSWORD} = $pass;

use File::Rsync::Mirror::Recent;
my $rrr = File::Rsync::Mirror::Recent->new
  (
   ignore_link_stat_errors => 1,
   localroot => $CPAN,
   tempdir   => $tmp,
   remote => $user . '@cpan-rsync-master.perl.org::CPAN/RECENT.recent',
   max_files_per_connection => 5000,
   rsync_options => {
                     compress => 1,
                     links => 1,
                     'safe-links' => 1,
                     times => 1,
                     checksum => 0,
                    },
   verbose => 1,
   _runstatusfile => "recent-rmirror-state.yml",
   _logfilefordone => "recent-rmirror-donelog.log",
  );
$rrr->rmirror ( "skip-deletes" => 0, loop => 1 );