Overview

This is a guide on setting up private apt repository that is accessible over a local network via HTTPS and is signed to avoid having to use –allow-unauthenticated to install packages.

This came about because of specific needs to provide a simple way to update Linux systems that didn’t have general Internet access and could only use specific versions of packages. I wanted to leverage existing packaging and distribution capabilities without having a complex system to manage and ultimately creating more work for myself.

Most of this information is available in the references, but none of the steps were a complete walk-through for my specific use case, so I put this guide together to document the various steps I had to do to build the solution. I am not going into all the details of why things work a certain way, read the reference links if you want some more detailed information on the why of everything.

For my use case I have two distributions of packages, they are production and test distributions. The packages in each distribution varies based on what I have approved to be used in a live/production environment versus a test environment. This is so that I can separate out packages that I am using for normal everyday use versus ones I am currently testing with and not ready to go live with. If you are only using one distribution modify the instructions accordingly.

Prerequisites
  • Computer running Ubuntu 16.04 LTS (other Linux variants may work but are not referenced here).
  • Permissions are setup correctly so the user can access the location of where you are putting the repository root folder.
Server / Repository Installation Steps

Step 1 – Create a GPG key pair.

You will need a digital signature to use to sign the release file. View the references to get more information on how this part all works, specifically https://ubuntuforums.org/showthread.php?t=1090731.  I am only going to document the specific steps I had to go through to enable this.

Generate the GPG key pair.

gpg --gen-key

Select the kind of key you want. Select the option for RSA (sign only), for me it was option 4.

Select the RSA key size. 4096.

Select how long the key should be valid. 0.

Verify if the length is correct. Y.

Specify user ID for the new key. I suggest reading the reference for suggestions on this one.

Real Name – <some name you want to use>
E-mail Address – leave blank
Comment – leave blank

Confirm. O.

Enter passphrase. This should be complex and kept secret. If you forget it you will lose access to your secret key and no longer able to generate digital signatures with it, so keep it written down somewhere sage.

Assuming it works correctly you will get information on your new key. You can list your keys at any time with the following command.

gpg --list-keys

The output of this command will look something like the blow with the key ID being where the bold x’s are in the output. You will need this key ID for the next step.

/home/someuser/.gnupg/pubring.gpg
----------------------------------
pub   4096R/xxxxxxxx 2018-02-15 [expires: 2068-02-03]
uid                  Apt Repository

Step 2 – Export the newly created Public Key.

Export the public key to a text file. Replace <your_key_id> with the key ID from the command above.

gpg --output pubkey-export-file --armor --export <your_key_id>

This create a file called pubkey-export-file. This file will need to be imported on every client machine that you want to install packages from the repository.

Step 3 – Install dpkg-dev

sudo apt-get install dpkg-dev

Step 4 – Create a directory that will be used to store the packages.

For this example I am using /data/aptrepo as the root of where I am putting everything. The stable packages will go to /data/aptrepo/stable and the testing packages will go to /data/aptrepo/testing to store all the packages I need.

mkdir -p /data/aptrepo
mkdir -p /data/aptrepo/production
mkdir -p /data/aptrepo/test

Step 5 – Create a release configuration file for each suite.

Create a configuration file per suite located in the /data/aptrepo directory. This is used by the script to populate information into Release file that is generated.

nano /data/aptrepo/distribution.production

Do this for each suite, in my example I would end up having a distribution.production and a distribution.test file. Here is an example of the information for the distribution file.

Origin: My_Local_Repo
Label: My_Local_Repo
Codename: xenial
Architectures: amd64
Components: main
Description: My local APT repository
SignWith: <your_key_id_from_step_1>

I also put in the same value for Origin and Label as I did for the Real Name from Step 1.

Step 6 – Create a script to update and sign information about the packages in the package directory.

Use a text editor of your choice, I use nano for the example, and create a script file that will run the command dpkg-scanpackages to regenerate the package information. Anytime you add or remove packages you would run this script.

sudo nano /bin/update-aptrepo

Add these lines to the script and save it. Replace the /data/aptrepo variable with your package root directory.

#!/bin/bash

# Define root directory for the repository
DIR=/data/aptrepo

# test to see if user entered in a distro variable, if not display a use message
if [ -z "$1" ] 
   then
      echo -e "No argument supplied.\n"
      echo -e "usage: `basename $0` distro\n"
      echo -e "DISTRO is the distribution description (i.e. production, test, etc).\n"
      echo -e"The way to use this script is to do the changes to the target repo first, i.e. delete or copy in the .deb file to $DIR/production or $DIR/test, and then run this script.\n"
      echo -e "This script can be run as an unprivileged user - root is not needed so long as your user can write to the local repository directory.\n"
      exit 1
fi

# test to see if the directory exists, if not exit
if [ ! -d $DIR/$1 ] 
   then 
      echo -e "Directory $DIR/$1 does not exist, please create and rerun the script.\n"
      exit 1
fi

cd $DIR/$1

# Generate the Packages file
dpkg-scanpackages . /dev/null > Packages
gzip --keep --force -9 Packages

# Generate the Release file
cat ../distribution.$1 > Release

# The Date: field has the same format as the Debian package changelog entries,
# that is, RFC 2822 with time zone +0000
echo -e "Date: `LANG=C date -Ru`" >> Release

# Release must contain MD5 sums of all repository files (in a simple repo just the Packages and Packages.gz files)
echo -e 'MD5Sum:' >> Release
printf ' '$(md5sum Packages.gz | cut --delimiter=' ' --fields=1)' %16d Packages.gz' $(wc --bytes Packages.gz | cut --delimiter=' ' --fields=1) >> Release
printf '\n '$(md5sum Packages | cut --delimiter=' ' --fields=1)' %16d Packages' $(wc --bytes Packages | cut --delimiter=' ' --fields=1) >> Release

# Release must contain SHA256 sums of all repository files (in a simple repo just the Packages and Packages.gz files)
echo -e '\nSHA256:' >> Release
printf ' '$(sha256sum Packages.gz | cut --delimiter=' ' --fields=1)' %16d Packages.gz' $(wc --bytes Packages.gz | cut --delimiter=' ' --fields=1) >> Release
printf '\n '$(sha256sum Packages | cut --delimiter=' ' --fields=1)' %16d Packages' $(wc --bytes Packages | cut --delimiter=' ' --fields=1) >> Release

# Clearsign the Release file (that is, sign it without encrypting it)
gpg --clearsign --digest-algo SHA512 -o InRelease Release

# Release.gpg only need for older apt versions
# gpg -abs --digest-algo SHA512 -o Release.gpg Release

Keep in mind if you are cutting and pasting this into a text editor like nano, the line breaks may get screwy so it you have errors with the script, go through and make sure to edit the script to remove any wonky line breaks.

Also make sure to set proper permission on the script file once created

sudo chmod 755 /bin/update-aptrepo

The script will run as the current user, so make sure you run this script as the same user that has the key generated. I ran into three issues trying to get this fully automated, one is anytime I specified the user option (–local-user) in the gpg command it couldn’t find the secret key. Even if I ran it as the same user. The second is it will prompt for the passphrase. The third and final issue is it prompts to overwrite any existing InRelease file that may exist.  I may look at solving those later, but for now it just means I have to run this manually when I update the repositories and need to run it as the same user.

Step 7 – Setup a web server

Next you need to install a web server to provide access to the packages. I am using Nginx. The setup of Nginx is standard, so I am not going into much detail on it. I will include some references inline to help with the basics, you can refer to those for more details.

Install Nginx. https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-16-04.

sudo apt-get install nginx

If using a firewall like ufw, enable HTTPS traffic to pass through it.

sudo ufw allow 'Nginx HTTPS'

To configure Nginx for HTTPS you need an SSL certificate. You can either create a self-signed certificate such as listed at https://www.digitalocean.com/community/tutorials/how-to-create-a-self-signed-ssl-certificate-for-nginx-in-ubuntu-16-04 or use a trusted third party like Digicert https://www.digicert.com/csr-ssl-installation/nginx-openssl.htm#create_csr_openssl.

After obtaining and installing the certificate, create a configuration file in /etc/nginx/sites-available/ to setup the website. I am using the site name aptrepo.conf for this example.

sudo nano /etc/nginx/sites-available/aptrepo.yourdomain.com

Your configuration file will look something like this.

server {
  listen 443;
  server_name aptrepo.yourdomain.com;

  access_log /var/log/nginx/packages-access.log;
  error_log /var/log/nginx/packages-error.log;

  location / {
    root /data/aptrepro;
    index index.html;
  }

  location ~\.(production|test)$ {
    deny all;
    error_page 403 =404 /;
  }
}

Link the server block from sites-available to sites-enabled to make the new site active. https://www.digitalocean.com/community/tutorials/how-to-set-up-nginx-server-blocks-virtual-hosts-on-ubuntu-16-04.

sudo ln -s /etc/nginx/sites-available/aptrep.yourdomain.com /etc/nginx/sites-enabled

Test the website configuration to make sure no problems.

sudo nginx -t

Fix any problems you might have. If no problems, restart Nginx.

sudo systemctl restart nginx

Connect to the new web server to make sure everything is working.

Step 8 – Make your public key available to clients

Copy the public key from Step 2 to the root directory of the website. This will allow you to retrieve the public key for the clients easily using wget. I explain the client portion later, for now we are just making the public key available.

mv /home/<user>/pubkey-export-file /data/aptrepo
How to update the apt repository

Simply add or remove the package files to the folders, which in my case is either /data/aptrepo/production or /data/aptrepo/test and then run the update-aptrepo script. Don’t forget you may need your secret key passphrase.

Client Installation Steps

Once the server / repository is setup, you need to configure the client systems to connect to the new repository and use it as one of the apt sources. In my specific use it will be the only apt source set on the client devices.

Step 1 – Import the public key

The public key should be in the root folder of the website repository. Use wget to retrieve the file and then add it via apt-key.

wget -O https://aptrep.yourdomain.com/pubkey-export-file | sudo apt-key add pubkey-export-file

If all correct the public key should appear with the apt-key command.

apt-key list

Step 2 – Update /etc/apt/sources.list

Edit the /etc/apt/sources.list file to point to the apt repository you setup.

sudo nano /etc/apt/sources.list

Add a new line to the file pointing to the location of your apt repository. For this example the link I am using is the HTTPS connection I setup earlier on the server using the production packages.

deb https://aptrepo.yourdomain.com/production /

Once the /etc/apt/sources.list file is updated, update apt to retrieve the list of packages.

sudo apt-get update
References

https://help.ubuntu.com/community/Repositories/Personal

https://askubuntu.com/questions/170348/how-to-create-a-local-apt-repository

https://ubuntuforums.org/showthread.php?t=1090731

https://help.ubuntu.com/community/CreateAuthenticatedRepository

https://help.ubuntu.com/community/GnuPrivacyGuardHowto

https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-16-04

https://www.digitalocean.com/community/tutorials/how-to-set-up-nginx-server-blocks-virtual-hosts-on-ubuntu-16-04


Leave a Reply

%d bloggers like this: