Difference between revisions of "QuantaStor Local Update Mirror"

From OSNEXUS Online Documentation Site
Jump to: navigation, search
m
m (Nginx + RSync Package Mirror Method)
 
(23 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
== Overview ==
 
== Overview ==
  
If your QuantaStor appliance cannot connect to the internet to update packages, then you can use a local update mirror to update QuantaStor.  
+
QuantaStor systems are sometimes deployed in sites where there's no public internet access.  On these deployments the QuantaStor systems are unable to directly access packages.osnexus.com and in order for the systems to run upgrades they will need a local package mirror.  
  
== Systems with Ubuntu 14.04 Base ==
+
The local package mirror is usually a VM, which has a copy of the QuantaStor repository, and other package files from packages.osnexus.com, so that the local DNS server can be configured to resolve packages.osnexus.com to the local mirror within the datacenter.
  
Follow the steps below to create a local mirror of the OSNEXUS QuantaStor update repository using apt-mirror for systems with Ubuntu 14.04 as its base operating system. The below instructions assume you already have an local Ubuntu server currently installed. For each instruction, the label [QS] will be instructions for the QuantaStor appliance, and the label [MIR] will be instructions for the mirror server.
+
Below is our supported method of creating a local repository.
  
=== Instructions ===
+
== Nginx + RSync Package Mirror Method ==
  
[MIR] 1. Elevate as root (<code>sudo -i</code>)
+
The Nginx + RSync method is the simplest and most common way to mirror the OSNexus package repository at packages.osnexus.com.
 
+
[MIR] 2. Update packages (<code>apt-get update</code>)
+
 
+
[MIR] 3. Install and enable Apache Web Server (<code>apt install -y apache2 && systemctl enable apache2</code>)
+
 
+
[MIR] 4. Create a package directory and set the correct permissions (<code>mkdir -p /var/www/html/repo && chown www-data:www-data /var/www/html/repo</code>)
+
 
+
[MIR] 5. Install apt-mirror (<code>apt install -y apt-mirror</code>)
+
 
+
[MIR] 6. Make a backup of /etc/apt/mirror.list (<code>cp /etc/apt/mirror.list /etc/apt/mirror.list-bak</code>)
+
 
+
[MIR] 7. Edit /etc/apt/mirror.list to be like the below:
+
 
+
<pre>
+
############# config ##################
+
#
+
set base_path    /var/www/html/repo
+
#
+
# set mirror_path  $base_path/mirror
+
# set skel_path    $base_path/skel
+
# set var_path    $base_path/var
+
# set cleanscript $var_path/clean.sh
+
# set defaultarch  <running host architecture>
+
# set postmirror_script $var_path/postmirror.sh
+
# set run_postmirror 0
+
set nthreads    20
+
set _tilde 0
+
#
+
############# end config ##############
+
 
+
deb http://packages.osnexus.com/packages trusty main
+
deb http://archive.ubuntu.com/ubuntu trusty main restricted universe multiverse
+
deb http://archive.ubuntu.com/ubuntu trusty-security main restricted universe multiverse
+
deb http://archive.ubuntu.com/ubuntu trusty-updates main restricted universe multiverse
+
deb-src http://archive.ubuntu.com/ubuntu trusty main restricted universe multiverse
+
deb-src http://archive.ubuntu.com/ubuntu trusty-security main restricted universe multiverse
+
deb-src http://archive.ubuntu.com/ubuntu trusty-updates main restricted universe multiverse
+
clean http://archive.ubuntu.com/ubuntu
+
clean http://packages.osnexus.com/packages
+
</pre>
+
 
+
[MIR] 8. Create a var folder in the local repo (<code>mkdir -p /var/www/html/repo/var</code>)
+
 
+
[MIR] 9. Copy /var/spool/apt-mirror/var/postmirror.sh to the var folder in the local repo (<code>mkdir -p cp /var/spool/apt-mirror/var/postmirror.sh /var/www/html/repo/var</code>)
+
 
+
[MIR] 10. Start the mirroring process (<code>nohup apt-mirror &</code>)
+
 
+
'''Note:''' Depending on your internet speed, this may take some time, but you can monitor the process with <code>tail nohup.out</code>. When it's finished, continue with step 11.
+
 
+
[QS] 11. Elevate as root (<code>sudo -i</code>)
+
 
+
[QS] 12. Make a copy of /etc/apt/sources.list and /etc/apt/sources.list.d/osnexus.list (<code>cp /etc/apt/sources.list /etc/apt/sources.list.bk && cp /etc/apt/sources.list.d/osnexus.list /etc/apt/sources.list.d/osnexus.list.bk</code>
+
 
+
[QS] 13. Edit /etc/apt/sources.list to be like the below:
+
 
+
<pre>
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty-updates main
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty universe
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty-updates universe
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty multiverse
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty-updates multiverse
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/archive.ubuntu.com/ubuntu/ trusty-security main restricted universe multiverse
+
</pre>
+
 
+
'''Note:''' Due to some oddities with apt-mirror, we have to specify using 64-bit with <code>[arch=amd64]</code>. If you don't, you'll see errors related to not being able to find 32-bit (i386) packages.
+
 
+
[QS] 14. Edit /etc/apt/sources.list.d/osnexus.list to be like the below:
+
 
+
<pre>
+
deb [arch=amd64] http://<YOUR_MIR_IP>/repo/mirror/packages.osnexus.com/packages/ trusty main
+
</pre>
+
 
+
[QS] 15. Update packages and upgrade QuantaStor (<code>apt-get update && qs-upgrade</code>)
+
 
+
'''Note:''' You may get stuck not being able to update the virt-what package. If this happens, you may either download (<code>wget</code>) and install (<code>dpkg -i</code>) the .deb package from a trusted source on the QuantaStor appliance, or download it to /var/www/html/repo/mirror/archive.ubuntu.com/ubuntu/pool/universe/v/virt-what.
+
 
+
== Alternative Methods ==
+
  
 
Follow the steps below to create a local mirror of the OSNEXUS QuantaStor update repository.
 
Follow the steps below to create a local mirror of the OSNEXUS QuantaStor update repository.
  
1. Contact support@osnexus.com to request that your SSH public key be added to the OSNEXUS QuantaStor replication mirror for rsync access. Support can also assist with the below process and any questions you may have.
+
1. Contact support@osnexus.com to request assistance with the OSNEXUS QuantaStor replication mirror for rsync access. Support can also assist with the below process and any questions you may have.
  
 
2. Setup a Linux VM with a web server, below is an example config file using nginx on Ubuntu 18.04 or newer:
 
2. Setup a Linux VM with a web server, below is an example config file using nginx on Ubuntu 18.04 or newer:
Line 128: Line 51:
  
 
<pre>
 
<pre>
rsync -aPv rsync://YOURUSER@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/
+
mkdir -p /var/lib/www/packages-root/
 +
RSYNC_PASSWORD=guest rsync -aPv rsync://guest@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/
 
</pre>
 
</pre>
  
Line 135: Line 59:
 
<pre>
 
<pre>
 
edit the /etc/cron.d/osnexus-mirror-sync file and add the below contents:
 
edit the /etc/cron.d/osnexus-mirror-sync file and add the below contents:
0 02 * * * root /usr/bin/rsync -aPv rsync://YOURUSER@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/
+
0 02 * * * root RSYNC_PASSWORD=guest /usr/bin/rsync -aPv rsync://guest@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/
 
</pre>
 
</pre>
  
Line 143: Line 67:
  
 
<pre>
 
<pre>
echo 'deb http://yourserver.example.net/packages quantastor-bionic main' > /etc/apt/sources.list.d/osnexus.list
+
# for focal based quantastor deployments
 +
echo 'deb http://yourserver.example.net quantastor-focal main' > /etc/apt/sources.list.d/osnexus.list
 +
 
 +
# for bionic based QuantaStor deployments
 +
echo 'deb http://yourserver.example.net quantastor-bionic main' > /etc/apt/sources.list.d/osnexus.list
 
</pre>
 
</pre>
  
 
If you need any further assistance, please contact support@osnexus.com.
 
If you need any further assistance, please contact support@osnexus.com.

Latest revision as of 09:14, 11 December 2023


Overview

QuantaStor systems are sometimes deployed in sites where there's no public internet access. On these deployments the QuantaStor systems are unable to directly access packages.osnexus.com and in order for the systems to run upgrades they will need a local package mirror.

The local package mirror is usually a VM, which has a copy of the QuantaStor repository, and other package files from packages.osnexus.com, so that the local DNS server can be configured to resolve packages.osnexus.com to the local mirror within the datacenter.

Below is our supported method of creating a local repository.

Nginx + RSync Package Mirror Method

The Nginx + RSync method is the simplest and most common way to mirror the OSNexus package repository at packages.osnexus.com.

Follow the steps below to create a local mirror of the OSNEXUS QuantaStor update repository.

1. Contact support@osnexus.com to request assistance with the OSNEXUS QuantaStor replication mirror for rsync access. Support can also assist with the below process and any questions you may have.

2. Setup a Linux VM with a web server, below is an example config file using nginx on Ubuntu 18.04 or newer:

Edit the /etc/nginx/sites-enabled/osnexus-mirror file. Please note. The file may be different depending on your nginx version or Linux OS.

Replace yourserver.example.net with your web server DNS FQDN.


#
# A virtual host entry for an OSNEXUS QuantaStor mirror
#

server {
    listen       80;
    server_name  yourserver.example.net packages;
    location / {
        root /var/lib/www/packages-root;
        autoindex on;
    }
}

Now restart and ensure it is enabled to startup automatically.

systemctl enable nginx
systemctl restart nginx

3. ensure your ssh key is available for the root user as the id_rsa file under /root/.ssh/

4. Run an initial rsync to synchronize the QuantaStor update repo using the credentials and SSH key setup with the OSNEXUS support team in step 1.

mkdir -p /var/lib/www/packages-root/
RSYNC_PASSWORD=guest rsync -aPv rsync://guest@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/

5. Add a cron entry that will run every day to synchronize your local mirror with the QuantaStor repo. Note that this requires ssh access from your mirror server to pkgmirror.osnexus.com

edit the /etc/cron.d/osnexus-mirror-sync file and add the below contents:
0 02 * * * root RSYNC_PASSWORD=guest /usr/bin/rsync -aPv rsync://guest@pkgmirror.osnexus.com/packages /var/lib/www/packages-root/

6. Add your new local mirror to your QuantaStor systems for all updates:

Replace yourserver.example.net with your web server DNS FQDN.

# for focal based quantastor deployments
echo 'deb http://yourserver.example.net quantastor-focal main' > /etc/apt/sources.list.d/osnexus.list

# for bionic based QuantaStor deployments
echo 'deb http://yourserver.example.net quantastor-bionic main' > /etc/apt/sources.list.d/osnexus.list

If you need any further assistance, please contact support@osnexus.com.