Difference between revisions of "QuantaStor Local Update Mirror"

From OSNEXUS Online Documentation Site
Jump to: navigation, search
m (Nginx + RSync Package Mirror Method)
m (Nginx + RSync Package Mirror Method)
 
(19 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
== Overview ==
 
== Overview ==
  
Often times QuantaStor systems are deployed in sites where there's no public internet access.  On these deployments the QuantaStor systems are unable to directly access packages.osnexus.com and in order for the systems to run upgrades they will need a local package mirror.  The local package mirror is basically a VM which has a copy of the QuantaStor and other package files from packages.osnexus.com so that the local DNS server can be configured to resolve packages.osnexus.com to the local mirror within the datacenter.
+
QuantaStor systems are sometimes deployed in sites where there's no public internet access.  On these deployments the QuantaStor systems are unable to directly access packages.osnexus.com and in order for the systems to run upgrades they will need a local package mirror.  
  
There are a couple of methods by which a package mirror repository can be setup, those are:
+
The local package mirror is usually a VM, which has a copy of the QuantaStor repository, and other package files from packages.osnexus.com, so that the local DNS server can be configured to resolve packages.osnexus.com to the local mirror within the datacenter.
  
* Nginx + rsync of packages.osnexus.com
+
Below is our supported method of creating a local repository.
* Apache + apt-mirror of packages.osnexus.com
+
  
== Apache + apt-mirror Package Mirror Method ==
+
== Nginx + RSync Package Mirror Method ==
  
Follow the steps below to create a local mirror of the OSNEXUS QuantaStor update repository using apt-mirror for systems with Ubuntu 14.04 as its base operating system. The below instructions assume you already have an local Ubuntu server currently installed. For each instruction, the label [QS] will be instructions for the QuantaStor appliance, and the label [MIR] will be instructions for the mirror server.
+
The Nginx + RSync method is the simplest and most common way to mirror the OSNexus package repository at packages.osnexus.com.
  
=== Instructions ===
+
Follow the steps below to create a local mirror of the OSNEXUS QuantaStor update repository.
  
[MIR] 1. Elevate as root (<code>sudo -i</code>)
+
1. Contact support@osnexus.com to request assistance with the OSNEXUS QuantaStor replication mirror for rsync access. Support can also assist with the below process and any questions you may have.
  
[MIR] 2. Update packages (<code>apt-get update</code>)
+
2. Setup a Linux VM with a web server, below is an example config file using nginx on Ubuntu 18.04 or newer:
  
[MIR] 3. Install and enable Apache Web Server (<code>apt install -y apache2 && systemctl enable apache2</code>)
+
Edit the /etc/nginx/sites-enabled/osnexus-mirror file. '''Please note. The file may be different depending on your nginx version or Linux OS. '''
  
[MIR] 4. Create a package directory and set the correct permissions (<code>mkdir -p /var/www/html/repo && chown www-data:www-data /var/www/html/repo</code>)
+
'''Replace''' yourserver.example.net with your web server DNS FQDN.
 
+
[MIR] 5. Install apt-mirror (<code>apt install -y apt-mirror</code>)
+
 
+
[MIR] 6. Make a backup of /etc/apt/mirror.list (<code>cp /etc/apt/mirror.list /etc/apt/mirror.list-bak</code>)
+
 
+
[MIR] 7. Edit /etc/apt/mirror.list to be like the below:
+
  
 
<pre>
 
<pre>
############# config ##################
+
 
#
+
set base_path    /var/www/html/repo
+
 
#
 
#
# set mirror_path  $base_path/mirror
+
# A virtual host entry for an OSNEXUS QuantaStor mirror
# set skel_path    $base_path/skel
+
# set var_path    $base_path/var
+
# set cleanscript $var_path/clean.sh
+
# set defaultarch  <running host architecture>
+
# set postmirror_script $var_path/postmirror.sh
+
# set run_postmirror 0
+
set nthreads    20
+
set _tilde 0
+
 
#
 
#
############# end config ##############
 
  
deb http://packages.osnexus.com/packages trusty main
+
server {
deb http://archive.ubuntu.com/ubuntu trusty main restricted universe multiverse
+
    listen      80;
deb http://archive.ubuntu.com/ubuntu trusty-security main restricted universe multiverse
+
    server_name  yourserver.example.net packages;
deb http://archive.ubuntu.com/ubuntu trusty-updates main restricted universe multiverse
+
    location / {
deb-src http://archive.ubuntu.com/ubuntu trusty main restricted universe multiverse
+
        root /var/lib/www/packages-root;
deb-src http://archive.ubuntu.com/ubuntu trusty-security main restricted universe multiverse
+
        autoindex on;
deb-src http://archive.ubuntu.com/ubuntu trusty-updates main restricted universe multiverse
+
    }
clean http://archive.ubuntu.com/ubuntu
+
}
clean http://packages.osnexus.com/packages
+
 
</pre>
 
</pre>
  
[MIR] 8. Create a var folder in the local repo (<code>mkdir -p /var/www/html/repo/var</code>)
+
Now restart and ensure it is enabled to startup automatically.
  
[MIR] 9. Copy /var/spool/apt-mirror/var/postmirror.sh to the var folder in the local repo (<code>mkdir -p cp /var/spool/apt-mirror/var/postmirror.sh /var/www/html/repo/var</code>)
+
<pre>
 +
systemctl enable nginx
 +
systemctl restart nginx
 +
</pre>
  
[MIR] 10. Start the mirroring process (<code>nohup apt-mirror &</code>)
+
3. ensure your ssh key is available for the root user as the id_rsa file under /root/.ssh/
  
'''Note:''' Depending on your internet speed, this may take some time, but you can monitor the process with <code>tail nohup.out</code>. When it's finished, continue with step 11.
+
4. Run an initial rsync to synchronize the QuantaStor update repo using the credentials and SSH key setup with the OSNEXUS support team in step 1.
  
[QS] 11. Elevate as root (<code>sudo -i</code>)
+
<pre>
 +
mkdir -p /var/lib/www/packages-root/
 +
RSYNC_PASSWORD=guest rsync -aPv rsync://guest@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/
 +
</pre>
  
[QS] 12. Make a copy of /etc/apt/sources.list and /etc/apt/sources.list.d/osnexus.list (<code>cp /etc/apt/sources.list /etc/apt/sources.list.bk && cp /etc/apt/sources.list.d/osnexus.list /etc/apt/sources.list.d/osnexus.list.bk</code>
+
5. Add a cron entry that will run every day to synchronize your local mirror with the QuantaStor repo. Note that this requires ssh access from your mirror server to pkgmirror.osnexus.com
 
+
[QS] 13. Edit /etc/apt/sources.list to be like the below:
+
  
 
<pre>
 
<pre>
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty-updates main
+
edit the /etc/cron.d/osnexus-mirror-sync file and add the below contents:
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty universe
+
0 02 * * * root RSYNC_PASSWORD=guest /usr/bin/rsync -aPv rsync://guest@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty-updates universe
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty multiverse
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty-updates multiverse
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty-security main restricted universe multiverse
+
 
</pre>
 
</pre>
  
'''Note:''' Due to some oddities with apt-mirror, we have to specify using 64-bit with <code>[arch=amd64]</code>. If you don't, you'll see errors related to not being able to find 32-bit (i386) packages.
+
6. Add your new local mirror to your QuantaStor systems for all updates:
  
[QS] 14. Edit /etc/apt/sources.list.d/osnexus.list to be like the below:
+
'''Replace''' yourserver.example.net with your web server DNS FQDN.
  
 
<pre>
 
<pre>
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/packages.osnexus.com/packages/ trusty main
+
# for focal based quantastor deployments
</pre>
+
echo 'deb http://yourserver.example.net quantastor-focal main' > /etc/apt/sources.list.d/osnexus.list
  
[QS] 15. Update packages and upgrade QuantaStor (<code>apt-get update && qs-upgrade</code>)
+
# for bionic based QuantaStor deployments
 +
echo 'deb http://yourserver.example.net quantastor-bionic main' > /etc/apt/sources.list.d/osnexus.list
 +
</pre>
  
'''Note:''' You may get stuck not being able to update the virt-what package. If this happens, you may either download (<code>wget</code>) and install (<code>dpkg -i</code>) the .deb package from a trusted source on the QuantaStor appliance, or download it to /var/www/html/repo/mirror/archive.ubuntu.com/ubuntu/pool/universe/v/virt-what.
+
If you need any further assistance, please contact support@osnexus.com.

Latest revision as of 09:14, 11 December 2023


Overview

QuantaStor systems are sometimes deployed in sites where there's no public internet access. On these deployments the QuantaStor systems are unable to directly access packages.osnexus.com and in order for the systems to run upgrades they will need a local package mirror.

The local package mirror is usually a VM, which has a copy of the QuantaStor repository, and other package files from packages.osnexus.com, so that the local DNS server can be configured to resolve packages.osnexus.com to the local mirror within the datacenter.

Below is our supported method of creating a local repository.

Nginx + RSync Package Mirror Method

The Nginx + RSync method is the simplest and most common way to mirror the OSNexus package repository at packages.osnexus.com.

Follow the steps below to create a local mirror of the OSNEXUS QuantaStor update repository.

1. Contact support@osnexus.com to request assistance with the OSNEXUS QuantaStor replication mirror for rsync access. Support can also assist with the below process and any questions you may have.

2. Setup a Linux VM with a web server, below is an example config file using nginx on Ubuntu 18.04 or newer:

Edit the /etc/nginx/sites-enabled/osnexus-mirror file. Please note. The file may be different depending on your nginx version or Linux OS.

Replace yourserver.example.net with your web server DNS FQDN.


#
# A virtual host entry for an OSNEXUS QuantaStor mirror
#

server {
    listen       80;
    server_name  yourserver.example.net packages;
    location / {
        root /var/lib/www/packages-root;
        autoindex on;
    }
}

Now restart and ensure it is enabled to startup automatically.

systemctl enable nginx
systemctl restart nginx

3. ensure your ssh key is available for the root user as the id_rsa file under /root/.ssh/

4. Run an initial rsync to synchronize the QuantaStor update repo using the credentials and SSH key setup with the OSNEXUS support team in step 1.

mkdir -p /var/lib/www/packages-root/
RSYNC_PASSWORD=guest rsync -aPv rsync://guest@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/

5. Add a cron entry that will run every day to synchronize your local mirror with the QuantaStor repo. Note that this requires ssh access from your mirror server to pkgmirror.osnexus.com

edit the /etc/cron.d/osnexus-mirror-sync file and add the below contents:
0 02 * * * root RSYNC_PASSWORD=guest /usr/bin/rsync -aPv rsync://guest@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/

6. Add your new local mirror to your QuantaStor systems for all updates:

Replace yourserver.example.net with your web server DNS FQDN.

# for focal based quantastor deployments
echo 'deb http://yourserver.example.net quantastor-focal main' > /etc/apt/sources.list.d/osnexus.list

# for bionic based QuantaStor deployments
echo 'deb http://yourserver.example.net quantastor-bionic main' > /etc/apt/sources.list.d/osnexus.list

If you need any further assistance, please contact support@osnexus.com.