ART-DECOR building and testing

This article or section is in the middle of an expansion or major restructuring and is not yet ready for use. You are welcome to assist in its construction by editing it as well.

This page will document how to build and test ART-DECOR in a automated process.

Building ART-DECOR

The following steps are taken during an ART-DECOR build:

• install eXist-db
• Create new ART-DECOR packages for installation in eXist-db database
• Create new vue build
• Restoring data

eXist-db

Install eXist-db from installer

The part will install the eXist-db database. First download the ART-DECOR eXist-db installer .jar from: https://decor.test-nictiz.nl/pub/exist_art_decor_5.1.1/

Notes on the installation

• Make sure JAVA is installed
• Mac and Linux can use the .jar installer: java -jar INSTALLER_FILE.jar
• Please note we build a specific eXist-db installer which contains configuration for ART-DECOR. Details about the configuration can be found at: https://bitbucket.org/art-decor/art-decor-backend/src/exist-db-5/exist5/
• Start the database: sudo INSTALLATION_FOLDER/bin/startup.sh or use wrapper scripts for linux (see below)
• See also the installation process for Linux for older versions: ART-DECOR_installation_on_linux
• See also: https://exist-db.org/exist/apps/doc/basic-installation

Install eXist-db on linux

Installation procedure for Linux, tested on CentOS.

• Make sure JAVA is installed
• Setup the directory structure

sudo mkdir /opt/art-decor-linux;
cd /opt/art-decor-linux
sudo mkdir tooling
sudo mkdir tooling/backups
sudo mkdir tooling/packages
sudo mkdir tooling/packages_archives
sudo mkdir tooling/scripts_archives
sudo chown -R root:wheel /opt/art-decor-linux/
cd /opt/art-decor-linux/tooling/packages

• Run the eXist-db installer

sudo su -l root
# stop previous version of eXist on your system
systemctl stop exist-db.service
cd /opt/art-decor-linux/tooling/packages
VERSION=5.1.1
VERSIONSHORT=511
# Or run another SNAPSHOT version:
VERSION=5.3.0-SNAPSHOT
VERSIONSHORT=530S
# download the ART-DECOR eXist-db installer
wget https://decor.test-nictiz.nl/pub/exist_art_decor_${VERSION}/exist-installer-${VERSION}_artdecor.jar --no-check-certificate

# start the eXist-db installer:
java -jar exist-installer-${VERSION}_artdecor.jar -console
# output/input during the installer:
# installation path contains VERSIONSHORT_CURRENTDATE
Select the installation path:  [/usr/local/exist_atp_520_20200130]

# Use default choice for installation packages (these are eXist-db packages, not ART-DECOR)
Select Installation Packages -> 0

# Usually not sensible to create shortcuts on a headless system:
    ----------------------------------
    Create shortcuts in the Start Menu
    ----------------------------------
    Enter Y for Yes, N for No:
    N
    ------------------------------------------
    Create additional shortcuts on the desktop
    ------------------------------------------
    Enter Y for Yes, N for No:
    N

• create backup of empty eXist-db

This is handy for when you need to re-deploy the database. Another option for re-deploy is just running the installer again (then this step can be skipped)

mkdir -p /opt/art-decor-linux/tooling/backups
cd /opt/art-decor-linux/tooling/backups
tar zcvf exist_without_artdecor_${VERSIONSHORT}_$(date '+%Y%m%d%H%M').tgz /usr/local/exist_atp_${VERSIONSHORT}_*

• Create symlink

cd /usr/local
ln -s  exist_atp_${VERSIONSHORT}_* exist_atp_${VERSIONSHORT}

• adduser existdb

adduser --system --shell /sbin/nologin --comment "eXist-db Service Account" existdb
chown -h existdb:existdb /usr/local/exist_atp_${VERSIONSHORT}
chown -R existdb:existdb /usr/local/exist_atp_${VERSIONSHORT}_20*

• check that there is no other eXist-db currently running

systemctl status exist-db.service
ps aux |grep -i exist
netstat -anp|grep :8080
netstat -anp|grep :8

• create a wrapper script for systemd

More info: https://exist-db.org/exist/apps/doc/advanced-installation#headless

location_symlink=/usr/local/exist_atp_${VERSIONSHORT}
FILE=/etc/systemd/system/exist-db-${VERSIONSHORT}.service
cat > ${FILE} << _EOF_
[Unit]
Description=eXist-db ${VERSIONSHORT} Server
Documentation=
After=syslog.target

[Service]
Type=simple
User=existdb
Group=existdb
ExecStart=${location_symlink}/bin/startup.sh

[Install]
WantedBy=multi-user.target
_EOF_

• configure systemd and set permissions

cd /etc/systemd/system
chown existdb exist-db.service
chgrp existdb exist-db.service
echo eXist-db can be started/stopped/status with:
echo sudo systemctl status exist-db-${VERSIONSHORT}.service
echo sudo systemctl start exist-db-${VERSIONSHORT}.service
exit

# as normal user, start eXist-db
sudo systemctl status exist-db-${VERSIONSHORT}.service
sudo systemctl start exist-db-${VERSIONSHORT}.service

# If you have replaced the version of exist-db on this server:
sudo systemctl daemon-reload

• check status and logging files

sudo ps aux |grep -i exist
sudo netstat -anp |grep :8877

sudo updatedb
sudo locate exist.log
sudo tail -f /usr/local/exist_atp_${VERSIONSHORT}/logs/*.log
# or just tail: exist.log

eXist-db ART-DECOR package installation

In this step we will install the ART-DECOR packages into eXist-db

• If needed: Create SSH-tunnel to the server. localhost:8877 to server:8877
• access eXist-db on http://localhost:8877 with your browser
• check the eXist-db logs while you are installing packages below

• Update eXist-db packages:

http://localhost:8877/exist/apps/dashboard/admin#/launcher http://localhost:8877/exist/apps/dashboard/admin#/packagemanager

Update if there are updates (from the eXist-db repository): Dashboard, eXide, shared

• Install packages from ART-DECOR repository:

Via exide: http://localhost:8877/exist/apps/eXide/index.html

Open this file: /db/apps/packageservice/configuration.xml

The default configuration allows you to install/update eXist-db packages from exist-db.org .xar package repository. Note that eXist-db can only handle one repository.

Change it so that it points to the ART-DECOR .xar package repository:

<settings>
    <repositories>
      <!--  <repository active="true" default="true">http://exist-db.org/exist/apps/public-repo</repository> -->
        <repository active="true" default="true">http://decor.nictiz.nl/apps/public-repo-dev5</repository>

Save

• Install ART-DECOR packages with the package manager

First install: eXist Module for Schematron validation with SchXslt

Then continue with the other packages as described here: https://art-decor.org/mediawiki/index.php?title=ART-DECOR_installation_on_linux#Install_ART-DECOR_software_.26_data

So: Advanced Requirements Tooling, ..

• API documentation:

See: http://localhost:8877/exist/apps/api/api.html

• (optional) create users, demo users with which you can test ART-DECOR:

Author: decor, editor, issues, terminology Project admin: decor, decor-admin, editor, issues, terminology

• create backup of eXist-db which has ART-DECOR installed

VERSIONSHORT=(version you are running)
sudo systemctl stop exist-db-${VERSIONSHORT}.service
sudo systemctl status exist-db-${VERSIONSHORT}.service
mkdir -p /opt/art-decor-linux/tooling/backups
cd /opt/art-decor-linux/tooling/backups
sudo tar zcvf exist_installed_artdecor_${VERSIONSHORT}_$(date '+%Y%m%d%H%M').tgz /usr/local/exist_atp_${VERSIONSHORT}_*
sudo systemctl start exist-db-${VERSIONSHORT}.service

eXist-db ART-DECOR code update

Commit code

Code is committed in a software repository, see ART_developers_manual#ART-DECOR_Next_generation_repository

Build .xar packages

During this step code from the code repository is converted into .xar packages.

Documentation on how to run the script for updating the repository can be found here: TODO

The result (development) repository is viewable here: http://decor.nictiz.nl/apps/public-repo-dev5/retrieve.html

Note: two other routes to get your own sourcecode into a running eXist-db database:

• First locally, create a .xar package by running ant xar and install that package into the database: at localhost:8877, through the package manager. Or:
• connect to eXist-db and save your code into the database. For instance at localhost:8877, through eXide. Please note that during creation of the .xar package any variables such as @project.title@ in expath-pkg.xml.tmpl, but maybe also in api.json are replaced by the values that are set in build.xml. Therefore the literal code in the git repository is different than the code created in the .xar package (which will be installed into eXist-db).

Installing .xars into a ART-DECOR instance

Documentation can be found here: ART-DECOR_building_and_testing#eXist-db_ART-DECOR_package_installation

Restoring data

Test instances or production instances of ART-DECOR might need data restored into their database. For example:

• Make sure you have backup from NEW installation (so you can rollback)
• Copy data from old installation. Note that this works for version 5 -> version 5 eXist-db, but might not work between other major eXist-db versions.

OLD=/usr/local/exist_atp_511
NEW=/usr/local/exist_atp_530
sudo rm -Rf $NEW/data
sudo cp -r --preserve=all $OLD/data $NEW

Vue

Vue background information

More info on vue: https://vuetifyjs.com/en/getting-started/quick-start/

Vue installation for user

For users that want to install their own ART-DECOR instance:

• Install node-js
• For front-end installation follow the README.md https://bitbucket.org/art-decor/art-decor

Vue installation for developers

• Install node-js
• For front-end installation follow the README.md https://bitbucket.org/art-decor/art-decor
  • For Linux users the documentation is at: Install vue linux
• Visual studio code, free version. Install node-js separate because the version that comes with Visual studio is older

Vue debugging

https://vuejs.org/v2/cookbook/debugging-in-vscode.html

Chrome

• Chrome DevTools: what requests are being sent to the back-end. Shows queries/responses.
• extension: Vue Devtools and other debugging information.

Testing ART-DECOR

Tests that can be run against a ART-DECOR instance are documented at: ART_developers_manual#ART-DECOR_testscenarios