ART-DECOR installation on linux

This article or section is in the middle of an expansion or major restructuring and is not yet ready for use. You are welcome to assist in its construction by editing it as well.

Introduction

This page contains a complete installation manual for ART-DECOR in production environments. This setup has been tested on CentOS 64 bits servers. It uses the somewhat older eXist-db 2.1 installer which will be updated in Q1 2015. It assumes you already have a user with sudo permissions. The installation process will assume you are are running scripts as a normal user (unless otherwise explicitly noted). It assumes you use yum as package installation manager. This script assumes you want to install:

• the ART-DECOR installation script into: /opt/art-decor-linux/tooling
• Apache tomcat at: /usr/share/tomcat
• eXist-db at: /usr/local/exist_atp

Note: this installation page now downloads the trunk/developer version of installation scripts. This will be changed to stable when slightly more mature.

Download installation scripts

Here we setup a structure for installation scripts and packages, and then download the ART-DECOR installation scripts.

# setup the directory structure
sudo mkdir /opt/art-decor-linux;
cd /opt/art-decor-linux
sudo mkdir tooling
sudo mkdir tooling/backups
sudo mkdir tooling/packages
sudo mkdir tooling/packages_archives
sudo mkdir tooling/scripts_archives
sudo chown -R root:wheel /opt/art-decor-linux/
cd /opt/art-decor-linux/tooling
# get installation scripts from svn
sudo yum install svn
sudo svn checkout svn://svn.code.sf.net/p/artdecor/code-0/trunk/utilities/art_decor_installers/art-decor-linux/tooling/scripts scripts

This action should have downloaded the following files: (in /opt/art-decor-linux/tooling/scripts)

• settings: a configuration file to store setting: java location and location for installation scripts
• install_tomcat.sh: Tomcat installation script
• install_exist.sh: eXist-db installer helper script

Install Sun java JDK

We want to use Sun/Oracle java JDK. First place the JDK rpm in /opt/art-decor-linux/tooling/packages

Visit: http://www.oracle.com/technetwork/java/javase/downloads/index.html

Download a JDK version, for instance: jdk-7u71-linux-x64.rpm (or a later version). One way to get the rpm to the server is to first download it to your desktop and then place it on the server with scp/winscp.

# Check the current version of java
java -version

Output might look somewhat like (this is the OpenJDK default version we don't want to use):

java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.11.90) (rhel-1.62.1.11.11.90.el6_4-x86_64)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

Start the java installation:

# sudo to a root shell
sudo su -l root
mkdir /usr/java; cd /usr/java
# install sun jdk (next to default installed openJDK)
chmod +x /opt/art-decor-linux/tooling/packages/jdk-7u*.rpm
rpm -Uvh /opt/art-decor-linux/tooling/packages/jdk-7u*.rpm
# check which version java the systeem now uses:
java -version

Output should look somewhat like:

java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)

Check the java installation:

# if java -version does still report the OpenJDK version update java alternatives:
alternatives --install /usr/bin/java java /usr/java/latest/jre/bin/java 20000

# Check where java is installed?
ls /usr/java/

Output should look somewhat like:

default  jdk1.7.0_71  latest

Change the java version in scripts/settings

# exit the root shell
exit
# Change the java version in scripts/settings
cd /opt/art-decor-linux/tooling/scripts; sudo vi settings

In settings change the line with java_location to your current version: Example:

# set your java version here
# for centos
java_location=/usr/java/jdk1.7.0_71

Exit vi and save the file with: <esc> :wq

Amount of memory for your server

Check the amount of RAM memory on your server:

cat /proc/meminfo|grep MemTotal 
MemTotal:       24602824 kB

This output shows this server has 24GB total RAM memory.

Information pages at exist-db.org:

• http://exist-db.org/exist/apps/doc/quickstart.xml
• http://exist-db.org/exist/apps/doc/tuning.xml

We would use the following setup:

amount of MB	used for
8192	Apache Tomcat
8192	eXist-db max mem = wrapper.java.maxmemory
2048	eXist-db cache (approx. 1/3 of exist-db max mem)
2048	eXist-db db-connection cacheSize (approx. 1/3 of exist-db max mem)
1024	eXist-db db-connection collectionCache (1/2 of db-connection cacheSize)

eXist-db installation

Install eXist-db from installer

The part will install the eXist-db database. First download the ART-DECOR eXist-db installer .jar from: http://sourceforge.net/projects/artdecor/files/eXist-db/

Place eXist-db-setup-2.1-rev0000.jar in /opt/art-decor-linux/tooling/packages

Start the installation:

cd /opt/art-decor-linux/tooling/scripts
# run the installation script:
sudo ./install_exist.sh

Output will be similar to the following:

Starting exist-db installer. Press enter if settings are correct!
Select target path [/usr/local/exist_atp_21_20141216]

press 1 to continue, 2 to quit, 3 to redisplay
1
Set Data Directory
Please select a directory where eXist-db will keep its data files. On Windows, this should be outside the 'Program Files' directory. Please make sure eXist can write to the directory it is installed in.
Data dir:  [webapp/WEB-INF/data]

press 1 to continue, 2 to quit, 3 to redisplay
1
Set Admin Password and Configure Memory
Enter password:  []
<password>
Enter password:  []
<password>
------------------------------------------

press 1 to continue, 2 to quit, 3 to redisplay
1
[ Starting to unpack ]
[ Processing package: Core (1/6) ]
[ Processing package: Sources (2/6) ]
[ Processing package: Apps (3/6) ]
[ Processing package: shared (4/6) ]
[ Processing package: dashboard (5/6) ]
[ Processing package: eXide (6/6) ]
[ Unpacking finished ]
[ Starting processing ]
Starting process Setting admin password ... (1/1)
--- Starting embedded database instance ---
Dec 16, 2014 10:11:13 PM org.expath.pkg.repo.util.Logger info
INFO: Create a new repository with storage: File system storage in /usr/local/exist_atp_21_20141216/webapp/WEB-INF/data/expathrepo
Setting admin user password...
--- Initialization complete. Shutdown embedded database instance ---
[ Console installation done ]

Note: The installation script installs eXist-db to: /usr/local/exist_atp_21_<date>

The installation script creates a symlink at /usr/local/exist_atp pointing to the actual installation path.

eXist-db post installation configuration

We need to check the following:

• is the correct java linked from eXist-db
• edit the memory settings for eXist-db

# is the correct java linked from eXist-db?
sudo vi /usr/local/exist_atp/tools/wrapper/conf/wrapper.conf

The file should contain a line like:

wrapper.java.command=/usr/lib/jvm/jdk1.7.0_71/jre/bin/java

Still in wrapper.conf, edit the memory settings. Rationale is that the wrapper script determines the maximum amount of memory available for the eXist-db java instance, but is still set to 1024MB after installation.

Replace the old configuration with this one: (see Amount of memory for your server)

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=8192

Still in wrapper.conf, edit the wrapper startup timeout. Rationale is that when you have the SNOMED CT terminology package installed, reindexing while starting the server can take 30 minutes and will otherwise time out. Add the following line at the end:

# set wrapper startup timeout to 50 minutes
wrapper.startup.timeout=60000

Exit vi and save the file with: <esc> :wq

Next edit eXist-settings.sh. By setting the java minimum amount of memory and maximum amount of memory to the same number, the garbage collector has to do less work.

sudo vi  /usr/local/exist_atp/bin/functions.d/eXist-settings.sh

At set_java_options, set Xms and Xmx to the same number as wrapper.java.maxmemory:

set_java_options() {
  if [ -z "${JAVA_OPTIONS}" ]; then
    JAVA_OPTIONS="-Xms8192m -Xmx8192m -Dfile.encoding=UTF-8";

Exit vi and save the file with: <esc> :wq

Next increase cacheSize up to approx. 1/3 of the main memory available to Java (we use 1/4)

sudo vi /usr/local/exist_atp/conf.xml

Set the cacheSize and collectionCache to the numbers in Amount of memory for your server:

<db-connection cacheSize="2048M" collectionCache="1024M" database="native"
      files="webapp/WEB-INF/data" pageSize="4096" nodesBuffer="1000"
      doc-ids="default">

Still in conf.xml: add a eXist-db cron scheduler to make sudo nightly backups. Insert this just above </scheduler>:

<!-- edit for ART-DECOR: do automated full backups, daily 03:23 to webapp/WEB-INF/data/export/ -->
<job type="system" class="org.exist.storage.ConsistencyCheckTask" cron-trigger="0 23 3 ? * *">
   <!-- the output directory. paths are relative to the data dir -->
   <parameter name="output" value="export"/>
   <parameter name="backup" value="yes"/>
   <parameter name="incremental" value="no"/>
   <parameter name="incremental-check" value="no"/>
</job>

Exit vi and save the file with: <esc> :wq

Do not start exist on server startup:

$ sudo chkconfig --list exist_atp

Our installation script has created a service script: /etc/init.d/exist_atp

We don't want to start eXist-db at server startup, since starting eXist-db (especially reindexing SNOMED CT) will prevent us from logging in with ssh while eXist-db is starting. Since we did not configure the exist_atp service to start at server startup the output should be similar to:

service exist supports chkconfig, but is not referenced in any runlevel (run 'chkconfig --add exist_atp')

All eXist-db config files outside the database are configured now. Create a backup of eXist-db before adding ART-DECOR packages. This helps speed up the installation process if you need to repeat it later, for instance if you want to restart with a clean database (of the installer has not changed in the meantime).

cd /opt/art-decor-linux/tooling/backups
# create a zipped tar archive
sudo tar zcvf exist_without_artdecor_$(date '+%Y%m%d%H%M').tgz /usr/local/exist_atp_21_20141216/

Starting eXist-db

Our installation script has created a service script: /etc/init.d/exist_atp

Start the eXist-db database with:

sudo service exist_atp start
Starting eXist Native XML Database...
Waiting for eXist Native XML Database.......
running: PID:32004

Check whether the server is running:

sudo service exist_atp status

Output should be similar to:

eXist-db Native XML Database is running: PID:32004, Wrapper:STARTED, Java:STARTED

Check that eXist-db is listening on ports 8877 (and 8477):

$ sudo netstat -anp|grep :8
tcp        0      0 :::8477    :::*      LISTEN      32004/java
tcp        0      0 :::8877    :::*      LISTEN      32004/java

eXist-db logging

The configuration file that determines where eXist-db output is logged to:

/usr/local/exist_atp/tools/wrapper/conf/log4j.xml

In our setup the eXist-db logging is located at:

/usr/local/exist_atp/webapp/WEB-INF/logs/exist.log

After starting output should be similar to:

2014-12-16 03:00:46,864 [WrapperListener_start_runner] INFO  (JettyStart.java [lifeCycleStarted]:417) - Jetty server started.
2014-12-16 03:00:46,865 [WrapperListener_start_runner] INFO  (JettyStart.java [run]:231) - -----------------------------------------------------
2014-12-16 03:00:46,865 [WrapperListener_start_runner] INFO  (JettyStart.java [run]:232) - Server has started on ports 8877 8477. Configured contexts:
2014-12-16 03:00:46,865 [WrapperListener_start_runner] INFO  (JettyStart.java [run]:241) - '/exist'
2014-12-16 03:00:46,865 [WrapperListener_start_runner] INFO  (JettyStart.java [run]:279) - -----------------------------------------------------

When you experience eXist-db startup problems, check the startup wrapper logging:

/usr/local/exist_atp/tools/wrapper/logs/wrapper.log

Apache Tomcat installation

Download and install ART-DECOR Orbeon web archive

Download the art-decor.war file from: http://sourceforge.net/projects/artdecor/files/Orbeon/art-decor.war/download

Unzip.

Follow the instructions in the readme file: http://sourceforge.net/projects/artdecor/files/Orbeon/README.txt/download