ART-DECOR building and testing

This article or section is in the middle of an expansion or major restructuring and is not yet ready for use. You are welcome to assist in its construction by editing it as well.

This page will document how to build and test ART-DECOR in a automated process.

Building ART-DECOR

The following steps are taken during an ART-DECOR build:

• install eXist-db
• Create new ART-DECOR packages for installation in eXist-db database
• Create new vue build
• Restoring data

eXist-db

Install eXist-db from installer

The part will install the eXist-db database. A Unix-like OS is assumed, but the main part of the work is done inside eXist-db.

Notes on the installation

• Make sure JAVA is installed
• Mac and Linux can use the .jar installer: java -jar INSTALLER_FILE.jar
• Please note ART-DECOR uses a specific eXist-db installer which contains configuration for ART-DECOR. Details about the configuration can be found at: https://bitbucket.org/art-decor/art-decor-backend/src/exist-db-5/exist5/
• Start the database: sudo INSTALLATION_FOLDER/bin/startup.sh or use wrapper scripts for linux (see below)
• See also the installation process for Linux for older versions: ART-DECOR_installation_on_linux
• See also: https://exist-db.org/exist/apps/doc/basic-installation

Install eXist-db on linux

Installation procedure for Linux, tested on CentOS.

• Make sure JAVA is installed
• Setup the directory structure

sudo mkdir /opt/art-decor-linux;
cd /opt/art-decor-linux
sudo mkdir tooling
sudo mkdir tooling/backups
sudo mkdir tooling/packages
sudo mkdir tooling/packages_archives
sudo mkdir tooling/scripts_archives
sudo chown -R root:wheel /opt/art-decor-linux/
cd /opt/art-decor-linux/tooling/packages

• Run the eXist-db installer

sudo su -l root
# stop previous version of eXist on your system
systemctl stop exist-db.service
cd /opt/art-decor-linux/tooling/packages
# Set the version:
VERSION=5.3.0
VERSIONSHORT=530
# download the ART-DECOR eXist-db installer
wget https://decor.test-nictiz.nl/pub/exist_art_decor_${VERSION}/exist-installer-${VERSION}_artdecor.jar --no-check-certificate
# or curl https://decor.test-nictiz.nl/pub/exist_art_decor_${VERSION}/exist-installer-${VERSION}_artdecor.jar -o exist-installer-${VERSION}_artdecor.jar

# start the eXist-db installer:
java -jar exist-installer-${VERSION}_artdecor.jar -console
# output/input during the installer:
# installation path contains VERSIONSHORT_CURRENTDATE
Select the installation path:  [/usr/local/exist_atp_520_20200130]

# Use default choice for installation packages (these are eXist-db packages, not ART-DECOR)
Select Installation Packages -> 0

# Usually not sensible to create shortcuts on a headless system:
    ----------------------------------
    Create shortcuts in the Start Menu
    ----------------------------------
    Enter Y for Yes, N for No:
    N
    ------------------------------------------
    Create additional shortcuts on the desktop
    ------------------------------------------
    Enter Y for Yes, N for No:
    N

• create backup of empty eXist-db

This is handy for when you need to re-deploy the database. Another option for re-deploy is just running the installer again (then this step can be skipped)

mkdir -p /opt/art-decor-linux/tooling/backups
cd /opt/art-decor-linux/tooling/backups
tar zcvf exist_without_artdecor_${VERSIONSHORT}_$(date '+%Y%m%d%H%M').tgz /usr/local/exist_atp_${VERSIONSHORT}_*

• Create symlink

cd /usr/local
ln -s  exist_atp_${VERSIONSHORT}_* exist_atp_${VERSIONSHORT}

• adduser existdb

adduser --system --shell /sbin/nologin --comment "eXist-db Service Account" existdb
chown -h existdb:existdb /usr/local/exist_atp_${VERSIONSHORT}
chown -R existdb:existdb /usr/local/exist_atp_${VERSIONSHORT}_20*

• check that there is no other eXist-db currently running

systemctl status exist-db-${VERSIONSHORT}.service
ps aux |grep -i exist
netstat -anp|grep :8080
netstat -anp|grep :8

• create a wrapper script for systemd

More info: https://exist-db.org/exist/apps/doc/advanced-installation#headless

location_symlink=/usr/local/exist_atp_${VERSIONSHORT}
FILE=/etc/systemd/system/exist-db-${VERSIONSHORT}.service
cat > ${FILE} << _EOF_
[Unit]
Description=eXist-db ${VERSIONSHORT} Server
Documentation=
After=syslog.target

[Service]
Type=simple
User=existdb
Group=existdb
ExecStart=${location_symlink}/bin/startup.sh

[Install]
WantedBy=multi-user.target
_EOF_

• configure systemd and set permissions

cd /etc/systemd/system
chown existdb exist-db-${VERSIONSHORT}.service
chgrp existdb exist-db-${VERSIONSHORT}.service
echo eXist-db can be started/stopped/status with:
echo sudo systemctl status exist-db-${VERSIONSHORT}.service
echo sudo systemctl start exist-db-${VERSIONSHORT}.service
exit

# as normal user, start eXist-db
sudo systemctl status exist-db-${VERSIONSHORT}.service
sudo systemctl start exist-db-${VERSIONSHORT}.service

# If you have replaced the version of exist-db on this server:
sudo systemctl daemon-reload

• Note that for development environments, the systemd wrapper could be skipped and instead:

Start eXist by navigating in a command window to the newly installed folder, bin directory and run:

cd /usr/local/exist_atp_${VERSIONSHORT}/bin; sudo nohup ./startup.sh &

(no privileges required, eXist runs on a non-protected port, 8877).

• check status and logging files

sudo ps aux |grep -i exist
sudo netstat -anp |grep :8877

sudo updatedb
sudo locate exist.log
sudo tail -f /usr/local/exist_atp_${VERSIONSHORT}/logs/*.log
# or just tail: exist.log

eXist-db ART-DECOR package installation

In this step we will install the ART-DECOR packages into eXist-db.

Perform the following steps:

• If needed: Create SSH-tunnel to the server. localhost:8877 to server:8877
• Check the eXist-db logs while you are installing packages below
• Navigate in a browser to http://localhost:8877
• Access the package manager:

http://localhost:8877/exist/apps/dashboard/admin#/launcher http://localhost:8877/exist/apps/dashboard/admin#/packagemanager

• Login as the admin user
• Start the Package Manager and update all packages that are marked as updateable (from the eXist-db repository). Expected are: Dashboard, eXide, shared*

• Change the repository configuration to the ART-DECOR repository

Browse to eXide: http://localhost:8877/exist/apps/eXide/index.html

• Open this file: by clicking on directory and clicking to this location:

/db/apps/packageservice/configuration.xml The default configuration allows you to install/update eXist-db packages from exist-db.org .xar package repository. Note that eXist-db can only handle one repository.

• Change the repository stanza so that it points to the ART-DECOR .xar package repository (and comment the exist-db.org line out):

<settings>
    <repositories>
      <!--  <repository active="true" default="true">http://exist-db.org/exist/apps/public-repo</repository> -->
        <repository active="true" default="true">http://decor.nictiz.nl/apps/public-repo-dev5</repository>

• File, Save

• Access the package manager:

http://localhost:8877/exist/apps/dashboard/admin#/packagemanager

• Install ART-DECOR packages from the ART-DECOR repository with the package manager

install, in the given order:

1. Advanced Requirements Tooling
2. DECOR core files
3. DECOR services
4. ART-DECOR System Services
5. Terminology Applications
6. DECOR example files
7. ART-DECOR API
8. OID Registry tools

• API documentation:

See: http://localhost:8877/exist/apps/api/api.html

• (optional) create users, demo users with which you can test ART-DECOR:

Author: decor, editor, issues, terminology Project admin: decor, decor-admin, editor, issues, terminology

• create backup of eXist-db which has ART-DECOR installed

VERSIONSHORT=(version you are running)
sudo systemctl stop exist-db-${VERSIONSHORT}.service
sudo systemctl status exist-db-${VERSIONSHORT}.service
mkdir -p /opt/art-decor-linux/tooling/backups
cd /opt/art-decor-linux/tooling/backups
sudo tar zcvf exist_installed_artdecor_${VERSIONSHORT}_$(date '+%Y%m%d%H%M').tgz /usr/local/exist_atp_${VERSIONSHORT}_*
sudo systemctl start exist-db-${VERSIONSHORT}.service

eXist-db troubleshooting

A couple pointers if issues occur:

• For problems with tokens/logging in. Make sure eXist-db EXPath Cryptographic library and JWT are installed and updated to the latest version. JWT relies on Crypto, so update Crypto first. Note that you may have to stop/start eXist-db after performing these upgrades.

eXist-db ART-DECOR code update

Commit code

Code is committed in a software repository, see ART_developers_manual#ART-DECOR_Next_generation_repository

Build .xar packages

During this step code from the code repository is converted into .xar packages.

Documentation on how to run the script for updating the repository can be found here: TODO

The result (development) repository is viewable here: http://decor.nictiz.nl/apps/public-repo-dev5/retrieve.html

Note: two other routes to get your own sourcecode into a running eXist-db database:

• First locally, create a .xar package by running ant xar and install that package into the database: at localhost:8877, through the package manager. Or:
• connect to eXist-db and save your code into the database. For instance at localhost:8877, through eXide. Please note that during creation of the .xar package any variables such as @project.title@ in expath-pkg.xml.tmpl, but maybe also in api.json are replaced by the values that are set in build.xml. Therefore the literal code in the git repository is different than the code created in the .xar package (which will be installed into eXist-db).

Installing .xars into a ART-DECOR instance

Documentation can be found here: ART-DECOR_building_and_testing#eXist-db_ART-DECOR_package_installation

Restoring data

Test instances or production instances of ART-DECOR might need data restored into their database. For example:

• Make sure you have backup from NEW installation (so you can rollback)
• Copy data from old installation. Note that this works for version 5 -> version 5 eXist-db, but might not work between other major eXist-db versions.

OLD=/usr/local/exist_atp_511
NEW=/usr/local/exist_atp_530
sudo rm -Rf $NEW/data
sudo cp -r --preserve=all $OLD/data $NEW

Vue

Vue background information

More info on vue: https://vuetifyjs.com/en/getting-started/quick-start/

Vue installation for user

For users that want to install their own ART-DECOR instance:

• Install node-js
• For front-end installation follow the README.md https://bitbucket.org/art-decor/art-decor

Vue installation for developers

• Install node-js
• For front-end installation follow the README.md https://bitbucket.org/art-decor/art-decor
  • For Linux users the documentation is at: Install vue linux
• Visual studio code, free version. Install node-js separate because the version that comes with Visual studio is older

Vue debugging

https://vuejs.org/v2/cookbook/debugging-in-vscode.html

Chrome

• Chrome DevTools: what requests are being sent to the back-end. Shows queries/responses.
• extension: Vue Devtools and other debugging information.

Testing ART-DECOR

Tests that can be run against a ART-DECOR instance are documented at: ART_developers_manual#ART-DECOR_testscenarios