Understanding How Applications are Managed

Forget [most] everything you knew about software installation from traditional desktops. This increasingly holds true regardless of OS, since other operating systems are moving toward managed installation like Linux has had for a decade, and everyone is moving toward contained installers.

Whether you manage just one system or one hundred systems, it's important to understand how software is installed, un-installed, and upgraded on a Slackware system. Generically, this is referred to as “package management”. For large scale installs, there is administrative software (such as Puppet) but it's still important to understand how it all happens on an instance-by-instance basis.

GNU and Linux systems have become known for having a progressive and unique method of installing software, referred usually as the “repository” or “package management” systems. Debian and Red Hat Linux(es) popularised this idea when they began using databases to keep track of all software installed on a computer. The database could be queried and compared to packages avalailable on distribution servers to discover newer versions of installed software.

Slackware uses its own, simpler version of package management, but the concept is the same, and indeed conceptually the problem being solved is the same across all operating systems: a user gets the base system for an OS but wants to load personal software on the computer. The question is, how can an OS keep itself up-to-date while also not breaking the applications that the user has installed on top of it all?

The method for this in traditional unix is to install software to a “local” directory in the operating system. Within this “local” directory, a user can maintain private versions of support libraries and applications, without depending on anything but the most basic and common libraries existing in the lower levels of the OS.

In practise, that can be redundant and quite a lot of work, so what happens normally is that the operating system either gets updated frequently, and all of the user's applications along with it, or the operating system sits mostly untouched for years at a time while the user works with what they have installed. Since major revisions of software which depend major renovations of underlying libraries don't actually happen all that often, there is stability and consistency.

Slackware (and longterm releases of other distributions) opts for the latter.

Slackware Package Tools

Slackware provides, for all practical purposes, four primary methods of managing the software installed on a system:

  1. pkgtool and related tools like installpkg and removepkg, slackpkg, and rpm2tgz
  2. Slackbuilds
  3. Source code and makepkg (technically a part of the pkgtool set, but meant for compiling source code)
  4. Binaries or custom packages

Each of these offers a variety of choices for how a user manages the software on a computer; none of them impose update schedules, automatic dependency installation and removal, or demand a complex system of database calls be learned.

Pre-Built Packages

Quickstart

Install a Slackware package:

  1. Find a Slackware package from a trusted site such as http://studioware.org or http://slackware.org.uk/people/alien/restricted_slackbuilds and download it and its the md5 file, if available to your computer.
  2. If an md5 sum was available, verify the authenticity of the package with md5sum /path/to/package | cut -f1 -d“ ” && cat /path/to/md5sum
  3. Assuming the numbers match, open a terminal and type su -c “installpkg /path/to/package” to install.

Remove a Slackware package:

  1. su -c “removepkg /path/to/package” un-installs the application, but not your personal data relating to that application (nor any configuration files that have changed since having been installed).

Look at the contents of a package:

tar -tvf /path/to/package | more

Package Tools

The package tools that ship with Slackware are designed to help the user install, upgrade, and remove pre-built Slackware packages.

Slackware packages are ordinary compressed tar files. They contain, quite simply, a collection of pre-built files that have been archived in a minimalised mirror image of a Slackware system; they usually have the file extension .tgz, .tbz, or .txz.

Slackware packages are generic archives, but not all .tgz, .tbz, and .txz are Slackware packages! In other words, Slackware packages are a subset of archives; compressed tarballs are used to store all kinds of files, so just because you find a .t?z archive it doesn't mean that you can install it.

Most Slackware packages follow a conventional naming scheme like appname-version-arch-tag.t?z (for example, planter-0.5.0-noarch-SMi.tgz is a Slackware package, while planter-master.tgz is not).

If you are ever unsure, use tar -tvf name-of-file.t?z to look into the archive without opening it.

Those that are Slackware packages contain, generally speaking, these files:

  • slack-desc: A description of the application this package will install.
  • doinst.sh: A shell script doing post-install tasks, like adding support in one program for the application just installed.
  • /usr, /usr/share, and other folders into which the executable application itself exists and from which it will be copied out into the rest of the system.

A traditional package, therefore, is a mimicry of your Linux system; it contains a few descriptive files or scripts to do a few specialized tasks, but mainly it is simply an isolated /usr, /etc, and whatever else is required for the application to run, with all the compiled files in place.

When a Slackware package gets installed, all of these files are imposed over your existing system. Once the files are copied from the package to your system, you can run the application.

Slackware provides tools to automate all aspects of installing, tracking, and removing packages:

  • installpkg installs and logs packages
  • upgradepkg upgrades packages and updates log entries
  • removepkg removes packages and package log entries
  • pkgtool a menu system for managing packages

installpkg is a simple front-end to the process of decompressing, copying files from, and logging the contents of a Slackware package.

$ su -c 'installpkg /media/cdrom/extra/foo/foo-*.txz'

Once a package is installed, all of the files that have been placed on your system are noted in /var/log/packages. You may see a list of every package currently installed on your system:

ls /var/log/packages | less

You can also see every file contained in each one of those packages:

less /var/log/packages/foo

Using this information, two other scripts can upgrade packages, or remove packages.

$ su -c 'upgradepkg /var/log/packages/foo-version.txz foo-newVersion.txz'
$ su -c 'removepkg /var/log/packages/foo-version.txz'

If you have a difficult time remembering commands, you can use pkgtool, a graphical tool that runs in the terminal. Its use is fairly self-explanatory; choose whether you want to install packages from a directory, remove packages that are already installed, or view a list of installed files associated with a package.

Since Slackware itself uses Slackware packages to install itself in the first place, you can also run several initial install scripts again, should you need to re-configure.

Slackpkg

The first Slackware packages you'll deal with are the ones installed onto your system during the normal installation process. You won't interact with these directly, but they are always available for re-installation, either from your install media or via the internet.

Aside from the default packages installed with a full install, there is an /extra directory containing additional packages. To install them from the install DVD, use installpkg:

# installpkg /path/to/dvdrom/extra/foo-0.1.tgz

If you don't have the install media, use the internet.

To install a package from the internet, you could download the .txz file from a Slackware mirror and then use installpkg or pkgtool, but there is a simple script that can make the process easier, called slackpkg.

To configure slackpkg so that it knows where to look online for packages, open /etc/slackpkg/mirrors (a text file) as root and find the mirror that is associated with your Slackware version and that is geographically close to you. Uncomment it by removing the leading hash mark.

The list of mirrors offers both ftp and http access, but you must uncomment only one.

In terms of slackpkg, there is no functional difference between ftp and http. If you use a network that restricts, for instance, ftp traffic, then http would have the obvious advantage for you.

Update the list of available packages by issuing the initial command slackpkg update; this updates the list of packages on the server, it does not upgrade any of your software.

The repository can be searched by using the command slackpkg search foo as root, and an application can be installed with slackpkg install foo. As usual, reading the manual page for slackpkg reveals a full listing of associated commands.

Once a package has been installed with slackpkg, it can be removed or upgraded using pkgtool and the other package management commands.

Since a full install of Slackware contains nearly everything that slackpkg has access to anyway, the most common use for slackpkg in Slackermedia will be for security updates. You should perform a periodic update of slackpkg followed by

$ su -c 'slackpkg upgrade-all'

Look through the list of available updates, and update as you see fit.

Only essential security updates are distributed via slackpkg, so even in-production machines are safe to patch.

Finding Additional Packages

If you installed the full Slackware install base, then the only additional packages that slackpkg provides are applications found in the /extra and / testing directories (also available on your install disc). Aside from these directories, everything else on the server or disc has been installed on your system.

Distributors of Slack Packages

Slackware packages can be found elsewhere online, but you should make sure you trust the source of those packages, just as you would when considering installing applications on any other operating system.

A few reliable places with good reputations:

Studioware

http://studioware.org/packages is full of current builds of all kinds of multimedia tools. There are a wealth of new production toys to be discovered here, so it will probably take you a few days to explore and test what you find.

AlienBob

http://slackware.com/~alien/slackbuilds is a repository of useful applications, including Scribus, the OpenJDK Java dev kit and runtime, the Chromium web browser, and the Opus audio codec. Not all of the applications he offers are always terribly up-to-date, and some are, so take that into consideration before deciding what you want to install from this server.

http://slackware.org.uk/people/alien/restricted_slackbuilds is a repository of useful applications that include codecs or code that cannot be distributed on USA-based servers. Most notably this includes VLC and HandBrake. Not all of the applications he offers are up-to-date, and some are, so take that into consideration before deciding what you want to install from this server.

Slacky.eu

http://slacky.eu is a long-standing and vibrant community of Slackware users in Italy. The site has both 32-bit and 64-bit repositories with useful software spanning all categories.

Slackermedia

http://slackermedia.info/slackbuilds offers useful (and some not-so-useful) packages, including JSampler, butt audio streamer, and most importantly the Slackermedia-approved, everything-plus-the-kitchen-sink compile of ffmpeg.

Not everything is always up-to-date, but they are the versions of software that the author of Slackermedia is using.

rpm2tgz

The command rpm2tgz, which converts a pre-built package for Red Hat or Fedora Linux into a slackpkg (ie, a .t?z file) that can then be installed via installpkg or pkgtool. This is similar to finding a pre-built .tgz on any given software's download page, except that rpm files are ubiquitous to Linux package formats and the likelihood of finding a .rpm file often exceeds the likelihood of happening across a Slackware package.

There are differences between how a Red Hat system is built and how Slackware is built, however, so there is some chance that the converted package will not work, but it's usually worth trying.

Reliable sites for RPM packages include:

  • Official Red Hat (and related) mirrors. This maze can be difficult to navigate, so choose your architecture and then choose a nearby server to use; the packages are located from the webroot of the mirror: pub/fedora/linux/releases/??/Everything/x86_64/os/Packages/ (where ?? is a version number, and the architecture is 64bit).

Once you've downloaded the .rpm file, use rpm2tgz to convert it:

# rpm2tgz foobar-x.x.xx.rpm

The result is a .tgz file, so after the conversion is finished the original .rpm can be discarded and pkgtool used to install. Ensure that the computer contains the dependency code for the application to function; this means that you might need to install other software before installing the application you really want to use. For instance, the Red Hat documentation tool publican is distributed as an .rpm and it will install, but in order for it to actually work there are over twenty perl modules that must be installed first.

The user should also trust the source of the package such that they can be sure there is no broken, disabled, or malicious code present.

Source Code

Slackware users often prefer to compile the software they install themselves; that is, they take the raw text files containing the application's raw code and process it through the GNU gcc compiler, and then move the resulting binary (computer executable files) into their system. This has a wide variety of advantages:

  • It allows the user to control where the executable binary (the application) is installed.
  • It allows the user to enable or disable features of the application before the application is built.
  • It allows the user to optimize the application for a specific processor.
  • It allows the user to review the code that goes into the building of an application, should the user wish to do so.
  • It allows the user to change the code that goes into the building of an application, should the user wish to do so.
  • It ensures the availability of applications to all users on any GNU, Linux, or unix-like system; source code is universal in that if an application was written for a GNU Linux system then it will compile and run on any GNU Linux system regardless of distribution.

The general process of compiling is in theory simple. Obtaining it is much like obtaining any downloadable software; you go to a website or a code management site (like Sourceforge or Github or Gitlab) and download the source code of an application. Typically, free software is available in both a work-in-progress (“current” or “nightly”) form as well as a packaged “stable” release version. Slackermedia recommends, with few exceptions, downloading the stable version, which indicates that the code has been tested to some degree and that the programmers of the application feel confident enough in the code to package it into a .zip or .tar.gz or .tar.bz2 archive, give it an official number and sometimes a release name, and offer it for download to the general non-programmer public.

Once the code has been downloaded, it is uncompressed, and a series of two or three commands are issued which starts GNU gcc, which compiles the code and then sends the binaries into the system (viz. it is “installed”).

How to Compile Software from Source

Before you compile from source code, you should ask yourself why you are compiling from source code. Common reasons include:

  • You cannot find an existing Slack package or Slackbuild script for the application you are about to compile. There are also no .rpm versions available, or if there are, they do not install or work as required.
  • You need new features found only in a fresh release. You have tried to modify an existing Slackbuild script but it fails to compile the software for you, so you want to give it a go yourself.
    This can be difficult, because usually if an existing buildscript has been broken by a new release, then the code has either changed so drastically or else it (or its build system) are broken, and serious hacking will be required. Still, it's free to try.
  • You just want to build from source code.

There is not just one way to compile software. There is a prevalent way, but any programmer may choose a different method to manage how their software is compiled. The compilation itself is usually performed by the GNU C Compiler, gcc. The way that gcc knows what steps to take during the compile is from instructions placed in a Makefile, or some configuration list; that is, a document written in a syntax specific to the compiler, which informs the compiler what procedures and where to find supportive code to use while building the application. It may contain pointers to other code that is expected to already exists on the user's system, or to other parts of code within that application's project, and it may contain special instructions for installing the software after it has been built as well as instructions on how to un-install that application should the user decide it is no longer needed.

The source code downloaded may or may not include all of the code it actually requires to compile and run. If, for example, Developer Foo has developed code for a graphics application that requires heavily on Developer Bar's opus codec, then Developer Foo is obviously not going to distribute libopus along with the Foo code, because libopus is maintained by Developer Bar. Even though opus is open source and may be able to be distributed, it arguably wouldn't make sense to bundle something that a user may already have installed.

It is up the user, therefore, to read the README file (or similar) packaged along with Developer Foo's application and find out what other code is required to be present on the computer for Application Foo to work (this is called “dependency resolution”) and what build system the project uses, and what steps must be followed for the code to compile as expected.

The general steps in compiling code, abstracted away from any specific build system, are:

  1. Prepare the build scripts for the exact configuration of your system.
  2. Start the compiler to build the code, and then link that code to libraries it needs in order to run.
  3. Copy the new files to places that your operating system expects to find them. Icons are moved to the system icon directory, documentation to the system doc location, libraries to the system library path, executables to one of the system's application directories, and launchers to the system launcher location.

Things can go wrong, of course, so here are common problems that arise in even a simple build.

Common Compilation Problems

configure, cmake, etc.

By far the most common problem during the configuration step are missing libraries. If an application requires libfoo to compile (because the developer has borrowed some code from libfoo, so counts on that library being on your computer), and it cannot find it, then it is the job of the configuration script to alert you.

The solution is to install the missing libraries, and then run the make command again. If there is a make clean command, run it first to clear out any progress prior to the error.

Another potential solution is to look at configuration options (usually done by adding a –help flag to the configure command) to see if the missing library is indeed absolutely necessary. Sometimes it is possible to build without a library by using a flag like –without-libfoo or similar.

Of course, you would not want to do this arbitrarily; usually the correct solution is to install the missing library.

make, build

A problem during the actual compiling of code can mean that either the configuration step made an error, or that there is an error in the code itself.

If the configuration step made an error, then the compiler is likely looking in the wrong location for libraries that it needs. This can usually be solved by either manually correcting configuration files, or by overriding configuration files with environment variables. Unless the compiler error tells you what to look for, or you can debug the code yourself, your best chance of fixing such a problem would be to do an Internet search on the error.

If there is a bug in the code that is preventing it from compiling, you can either fix it (if you are a skilled programmer) or you can do an Internet search on the error to see if anyone else has encountered and fixed the problem.

make install, install

This step fails less often than earlier steps, since all it should be doing is copying files to your system. If it does fail, it is probably because you do not have permission to write files to system folders. Re-try the command with root privileges.

The problem with compiling and installing source code in this traditional way is that tracking what you have installed is nearly impossible. Any given application might install several libraries, icons, sets of documentation, and binary executables. This makes it extremely difficult to accurately and safely update components of your system when, for instance, an application requires you to update multiple libraries which in turn are being used by other applications which may be effected by the update, and so on.

For this reason, Slackware has a built-in method of wrapping up a compiled application as a custom-made Slack package: makepkg.

Makepkg

To use makepkg, you do need to understand what build system the developer is using so that you can redirect the install process (usually the final step, after the code has been compiled) to a directory of your choosing, rather than to the root of your system. Run makepkg on that directory, and you have a Slack package.

Here is an overview of common ways of defining a new installation directory:

Autotools

Autotools is a collection of tools within the GNU build tools. Being the original freely-available build system in the world, it is by far the most common.

To redirect an install, you run the usual commands, with one small addition to the install command. Therefore the first two steps are the same as always:

$ ./configure
$ make

The final step intercepts the default install path and forces make to treat a DESTDIR of your choosing as if it were the root of your filesystem:

$ mkdir /tmp/pkg-foo; make install DESTDIR=/tmp/pkg-foo

Move into the DESTDIR and use makepkg to create your Slack package:

$ cd /tmp/pkg-foo
$ su -c 'makepkg -l y -c n /tmp/foo-version-$(uname -m)_SMi.txz'

Install the package with installpkg. A Slackware package can also be used on other Slackware machines, so you may wish to store it somewhere for later use or re-distribution.

Cmake

Cmake is a build system that features a rich interactive GUI and a highly customisable build environment. In practise, the difference between Cmake and Automake is in the configuration step, so in practise using makepkg with it is the essentially the same as with Autotools.

The first two steps are the usual cmake commands:

$ mkdir build ; cd build; cmake ..
$ make

The final step intercepts the default install path and forces make to treat a DESTDIR of your choosing as if it were the root of your filesystem:

$ mkdir /tmp/pkg-foo; make install DESTDIR=/tmp/pkg-foo

Move into the DESTDIR and use makepkg to create your Slack package:

$ cd /tmp/pkg-foo
$ su -c 'makepkg -l y -c n /tmp/foo-version-$(uname -m)_SMi.txz'

Install it with installpkg. A Slackware package can also be used on other Slackware machines, so you may wish to store it somewhere for later use or re-distribution.

Python Distutils

Distutils is the default method for installing Python applications, since it is the distribution tool that ships with Python itself. Generally, it places all configuration and installation actions in a setup.py file. It is, however, highly customisable, so be sure to read any documentation about installation that the developer of the code provides.

To re-direct the destination of the installable files, use the –root flag when launching the setup script.

$ python setup.py install --root=/tmp/pkg-foo

Move into the DESTDIR and use makepkg to create your Slack package:

$ cd /tmp/pkg-foo
$ makepkg -l y -c n /tmp/foo-version-$(uname -m)_SMi.tgz

Install it with installpkg, and of course you can store this Slack package somewhere for later use or for re-distribution, if needed.

There are many more build systems out there, and there is even the remote possibility that entirely custom install commands or no install commands could turn up. In the event of non-standard build systems, you can read the documentation included with the code and attempt a build nevertheless. If you can't figure out how to redirect the install process, you can choose to install without creating a Slack package; theoretically, that will be harder to un-install or update, but it might be a trade-off worth making, depending on the application and how badly you want it.

As a last resort, you can contact Slackermedia for assistance; we may be able to build a package for you. Generally, this can be done for free, since it's very likely that if you are attempting to build and use a multimedia application, then other Slackermedia users may also be interested in trying it. If it is critical that a package gets created promptly, or if the package proves difficult to build even for us, then we might have to provide you with a price quote in order to get the job done to your specifications or else politely admit defeat.

The good news is that many applications already have scripts that configure, compile, and create a Slack package for you. These are called Slackbuilds (because they are simple scripts that you run in order to automate the steps of compiling).

The SlackBuild system allows you to get the best of both of worlds: the ability to compile from raw source code and transparently take the intermediate step of creating a Slackware Package that will be logged in /var/log/packages along with everything else installed on your system, but without doing any of the leg work. Slackbuild scripts are published in many places, but the largest repository of them is http://slackbuilds.org.

Read how to use SlackBuilds in the next section.

R S Q