ART-DECOR installation on linux

This article or section is in the middle of an expansion or major restructuring and is not yet ready for use. You are welcome to assist in its construction by editing it as well.

Introduction

This page contains a complete installation manual for ART-DECOR in production environments. This setup has been tested on CentOS 64 bits servers. It uses the somewhat older eXist-db 2.1 installer which will be updated in Q1 2015. It assumes you already have a user with sudo permissions. The installation process will assume you are are running scripts as a normal user (unless otherwise explicitly noted). It assumes you use yum as package installation manager. This script assumes you want to install:

• the ART-DECOR installation script into: /opt/art-decor-linux/tooling
• Apache tomcat at: /usr/share/tomcat
• eXist-db at: /usr/local/exist_atp

Note: this installation page now downloads the trunk/developer version of installation scripts. This will be changed to stable when slightly more mature.

Download installation scripts

Here we setup a structure for installation scripts and packages, and then download the ART-DECOR installation scripts.

# setup the directory structure
sudo mkdir /opt/art-decor-linux;
cd /opt/art-decor-linux
sudo mkdir tooling
sudo mkdir tooling/backups
sudo mkdir tooling/packages
sudo mkdir tooling/packages_archives
sudo mkdir tooling/scripts_archives
sudo chown -R root:wheel /opt/art-decor-linux/
cd /opt/art-decor-linux/tooling
# get installation scripts from svn
sudo yum install svn
sudo svn checkout svn://svn.code.sf.net/p/artdecor/code-0/trunk/utilities/art_decor_installers/art-decor-linux/tooling/scripts scripts

This action should have downloaded the following files: (in /opt/art-decor-linux/tooling/scripts)

• settings: a configuration file to store setting: java location and location for installation scripts
• install_tomcat.sh: Tomcat installation script
• install_exist.sh: eXist-db installer helper script

Install Sun java JDK

We want to use Sun/Oracle java JDK. First place the JDK rpm in /opt/art-decor-linux/tooling/packages

Visit: http://www.oracle.com/technetwork/java/javase/downloads/index.html

Download a JDK version, for instance: jdk-7u71-linux-x64.rpm (or a later version). One way to get the rpm to the server is to first download it to your desktop and then place it on the server with scp/winscp.

# Check the current version of java
java -version

Output might look somewhat like (this is the OpenJDK default version we don't want to use):

java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.11.90) (rhel-1.62.1.11.11.90.el6_4-x86_64)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

Start the java installation:

# sudo to a root shell
sudo su -l root
mkdir /usr/java; cd /usr/java
# install sun jdk (next to default installed openJDK)
chmod +x /opt/art-decor-linux/tooling/packages/jdk-7u*.rpm
rpm -Uvh /opt/art-decor-linux/tooling/packages/jdk-7u*.rpm
# check which version java the systeem now uses:
java -version

Output should look somewhat like:

java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)

Check the java installation:

# if java -version does still report the OpenJDK version update java alternatives:
alternatives --install /usr/bin/java java /usr/java/latest/jre/bin/java 20000

# Check where java is installed?
ls /usr/java/

Output should look somewhat like:

default  jdk1.7.0_71  latest

Change the java version in scripts/settings

# exit the root shell
exit
# Change the java version in scripts/settings
cd /opt/art-decor-linux/tooling/scripts; sudo vi settings

In settings change the line with java_location to your current version: Example:

# set your java version here
# for centos
java_location=/usr/java/jdk1.7.0_71

Exit vi and save the file with: <esc> :wq

Amount of memory for your server

Check the amount of RAM memory on your server:

cat /proc/meminfo|grep MemTotal 
MemTotal:       24602824 kB

This output shows this server has 24GB total RAM memory.

Information pages at exist-db.org:

• http://exist-db.org/exist/apps/doc/quickstart.xml
• http://exist-db.org/exist/apps/doc/tuning.xml

We would use the following setup:

amount of MB	used for
8192	Apache Tomcat
8192	eXist-db max mem = wrapper.java.maxmemory
2048	eXist-db cache (approx. 1/3 of exist-db max mem)
2048	eXist-db db-connection cacheSize (approx. 1/3 of exist-db max mem)
1024	eXist-db db-connection collectionCache (1/2 of db-connection cacheSize)

eXist-db installation

Install eXist-db from installer

The part will install the eXist-db database. First download the ART-DECOR eXist-db installer .jar from: http://sourceforge.net/projects/artdecor/files/eXist-db/

Place eXist-db-setup-2.1-rev0000.jar in /opt/art-decor-linux/tooling/packages

Start the installation:

cd /opt/art-decor-linux/tooling/scripts
# run the installation script:
sudo ./install_exist.sh

Output will be similar to the following:

Starting exist-db installer. Press enter if settings are correct!
Select target path [/usr/local/exist_atp_21_20141216]

press 1 to continue, 2 to quit, 3 to redisplay
1
Set Data Directory
Please select a directory where eXist-db will keep its data files. On Windows, this should be outside the 'Program Files' directory. Please make sure eXist can write to the directory it is installed in.
Data dir:  [webapp/WEB-INF/data]

press 1 to continue, 2 to quit, 3 to redisplay
1
Set Admin Password and Configure Memory
Enter password:  []
<password>
Enter password:  []
<password>
------------------------------------------

press 1 to continue, 2 to quit, 3 to redisplay
1
[ Starting to unpack ]
[ Processing package: Core (1/6) ]
[ Processing package: Sources (2/6) ]
[ Processing package: Apps (3/6) ]
[ Processing package: shared (4/6) ]
[ Processing package: dashboard (5/6) ]
[ Processing package: eXide (6/6) ]
[ Unpacking finished ]
[ Starting processing ]
Starting process Setting admin password ... (1/1)
--- Starting embedded database instance ---
Dec 16, 2014 10:11:13 PM org.expath.pkg.repo.util.Logger info
INFO: Create a new repository with storage: File system storage in /usr/local/exist_atp_21_20141216/webapp/WEB-INF/data/expathrepo
Setting admin user password...
--- Initialization complete. Shutdown embedded database instance ---
[ Console installation done ]

Note: The installation script installs eXist-db to: /usr/local/exist_atp_21_<date>

The installation script creates a symlink at /usr/local/exist_atp pointing to the actual installation path.

eXist-db post installation configuration

We need to check the following:

• is the correct java linked from eXist-db
• edit the memory settings for eXist-db

# is the correct java linked from eXist-db?
sudo vi /usr/local/exist_atp/tools/wrapper/conf/wrapper.conf

The file should contain a line like:

wrapper.java.command=/usr/lib/jvm/jdk1.7.0_71/jre/bin/java

Still in wrapper.conf, edit the memory settings. Rationale is that the wrapper script determines the maximum amount of memory available for the eXist-db java instance, but is still set to 1024MB after installation.

Replace the old configuration with this one: (see Amount of memory for your server)

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=8192

Still in wrapper.conf, edit the wrapper startup timeout. Rationale is that when you have the SNOMED CT terminology package installed, reindexing while starting the server can take 30 minutes and will otherwise time out. Add the following line at the end:

# set wrapper startup timeout to 50 minutes
wrapper.startup.timeout=60000

Exit vi and save the file with: <esc> :wq

Next edit eXist-settings.sh. By setting the java minimum amount of memory and maximum amount of memory to the same number, the garbage collector has to do less work.

sudo vi  /usr/local/exist_atp/bin/functions.d/eXist-settings.sh

At set_java_options, set Xms and Xmx to the same number as wrapper.java.maxmemory:

set_java_options() {
  if [ -z "${JAVA_OPTIONS}" ]; then
    JAVA_OPTIONS="-Xms8192m -Xmx8192m -Dfile.encoding=UTF-8";

Exit vi and save the file with: <esc> :wq

Next increase cacheSize up to approx. 1/3 of the main memory available to Java (we use 1/4)

sudo vi /usr/local/exist_atp/conf.xml

Set the cacheSize and collectionCache to the numbers in Amount of memory for your server:

<db-connection cacheSize="2048M" collectionCache="1024M" database="native"
      files="webapp/WEB-INF/data" pageSize="4096" nodesBuffer="1000"
      doc-ids="default">

Still in conf.xml: add a eXist-db cron scheduler to make sudo nightly backups. Insert this just above </scheduler>:

<!-- edit for ART-DECOR: do automated full backups, daily 03:23 to webapp/WEB-INF/data/export/ -->
<job type="system" class="org.exist.storage.ConsistencyCheckTask" cron-trigger="0 23 3 ? * *">
   <!-- the output directory. paths are relative to the data dir -->
   <parameter name="output" value="export"/>
   <parameter name="backup" value="yes"/>
   <parameter name="incremental" value="no"/>
   <parameter name="incremental-check" value="no"/>
</job>

Exit vi and save the file with: <esc> :wq

Do not start exist on server startup:

$ sudo chkconfig --list exist_atp

Our installation script has created a service script: /etc/init.d/exist_atp

We don't want to start eXist-db at server startup, since starting eXist-db (especially reindexing SNOMED CT) will prevent us from logging in with ssh while eXist-db is starting. Since we did not configure the exist_atp service to start at server startup the output should be similar to:

service exist supports chkconfig, but is not referenced in any runlevel (run 'chkconfig --add exist_atp')

All eXist-db config files outside the database are configured now. Create a backup of eXist-db before adding ART-DECOR packages. This helps speed up the installation process if you need to repeat it later, for instance if you want to restart with a clean database (of the installer has not changed in the meantime).

cd /opt/art-decor-linux/tooling/backups
# create a zipped tar archive
sudo tar zcvf exist_without_artdecor_$(date '+%Y%m%d%H%M').tgz /usr/local/exist_atp_21_20141216/

Starting eXist-db

Our installation script has created a service script: /etc/init.d/exist_atp

Start the eXist-db database with:

sudo service exist_atp start
Starting eXist Native XML Database...
Waiting for eXist Native XML Database.......
running: PID:32004

Check whether the server is running:

sudo service exist_atp status

Output should be similar to:

eXist-db Native XML Database is running: PID:32004, Wrapper:STARTED, Java:STARTED

Check that eXist-db is listening on ports 8877 (and 8477):

$ sudo netstat -anp|grep :8
tcp        0      0 :::8477    :::*      LISTEN      32004/java
tcp        0      0 :::8877    :::*      LISTEN      32004/java

eXist-db logging

The configuration file that determines where eXist-db output is logged to:

/usr/local/exist_atp/tools/wrapper/conf/log4j.xml

In our setup the eXist-db logging is located at:

/usr/local/exist_atp/webapp/WEB-INF/logs/exist.log

After starting output should be similar to:

2014-12-16 03:00:46,864 [WrapperListener_start_runner] INFO  (JettyStart.java [lifeCycleStarted]:417) - Jetty server started.
2014-12-16 03:00:46,865 [WrapperListener_start_runner] INFO  (JettyStart.java [run]:231) - -----------------------------------------------------
2014-12-16 03:00:46,865 [WrapperListener_start_runner] INFO  (JettyStart.java [run]:232) - Server has started on ports 8877 8477. Configured contexts:
2014-12-16 03:00:46,865 [WrapperListener_start_runner] INFO  (JettyStart.java [run]:241) - '/exist'
2014-12-16 03:00:46,865 [WrapperListener_start_runner] INFO  (JettyStart.java [run]:279) - -----------------------------------------------------

When you experience eXist-db startup problems, check the startup wrapper logging:

/usr/local/exist_atp/tools/wrapper/logs/wrapper.log

Apache Tomcat installation

Download Apache Tomcat

Download Apache Tomcat:

• Select a version Tomcat that supports the installed Sun/Oracle java JDK version: http://tomcat.apache.org/whichversion.html
• After choosing a version under 'Download', select the core, tar.gz binary distribution.
• Place the Tomcat tar.gz in /opt/art-decor-linux/tooling/packages

Download ART-DECOR Orbeon web archive

• Download the art-decor.war file from: http://sourceforge.net/projects/artdecor/files/Orbeon/art-decor.war/download
• Place the art-decor.war in /opt/art-decor-linux/tooling/packages

Install Tomcat and deploy ART-DECOR Orbeon web archive

Run the Tomcat installer

Next we will run a script that does the following:

• Install prerequisites: gcc gcc-c++ make
• Caution: the installer script will remove any previous installation of Tomcat
• Unpack Tomcat and move it to /usr/share/tomcat-<version>
• Set up jsvc so that Tomcat can be run as a daemon
• Set up permissions
• Add an /etc/init.d/tomcat script to start Tomcat when the server starts
• Set (java and tomcat) environment variables in /etc/profile.d/env.sh
• Copy the ART-DECOR Orbeon web archive to tomcat and list it in the Tomcat config file

To perform these actions: run the Tomcat installer script:

# sudo to a root shell
sudo su -l root
cd /opt/art-decor-linux/tooling/scripts
# run the Tomcat installer script
./install_tomcat.sh
# exit the root shell
exit

Check that tomcat will start after server reboot (the Tomcat script should have handled this):

sudo chkconfig --list tomcat

Output should be similar to the following (for runlevel 3 to 5 tomcat should be listed as on)

tomcat          0:off   1:off   2:on    3:on    4:on    5:on    6:off

Configure the amount of memory for Tomcat

Next define the amount of memory that should be allocated to Tomcat:

• See how much memory you need for Tomcat: Amount of memory for your server
• Create a configuration file to set the amount of memory for Tomcat:

sudo vi /usr/share/tomcat/bin/setenv.sh

With the following content (change the amount of memory to your setup):

#!/bin/sh
#
# ENVARS for Tomcat environment
#

CATALINA_OPTS="-Xmx8192M -Xms8192M"
# EOF

Exit vi and save the file with: <esc> :wq

Set the proper permissions:

sudo chmod u+x /usr/share/tomcat/bin/setenv.sh
sudo chown tomcat:root /usr/share/tomcat/bin/setenv.sh

Extract ART-DECOR Orbeon web archive in Tomcat folder

We need to start Tomcat for a brief period to extract the ART-DECOR Orbeon web archive, so that we can change the configuration files for ART-DECOR Orbeon

• Start Tomcat

sudo service tomcat start

• Check that the ART-DECOR Orbeon war is extracted

ls /usr/share/tomcat/webapps/art-decor/

Output should show these folders:

META-INF  WEB-INF

• Stop tomcat

sudo service tomcat stop

• Check that Tomcat is stopped or kill it

sudo ps aux |grep tomcat

If it is still running you can kill the Tomcat process with:

sudo kill -15 <PID>

Edit ART-DECOR Orbeon configuration

Now we can edit the ART-DECOR Orbeon configuration files.

• Change the default Orbeon password

sudo vi /usr/share/tomcat/webapps/art-decor/WEB-INF/resources/config/properties-local.xml

Change the default password:

<!-- The password is already changed from the Orbeon default, it should be changed for each installation -->
   <property as="xs:string" name="oxf.xforms.password" value=".....”/>

• In the same file: Change the variables that Tomcat passes to client browsers with links to ART-DECOR content (assumes we will use a HTTPS certificate later on):

<property as="xs:anyURI" name="art.external.exist.url" value="http://localhost:8877/art"/>

Change each property that has external in it's name to point to the FQDN of your server:

<property as="xs:anyURI" name="art.external.exist.url" value="https://art-decor.org/art"/>

Exit vi and save the file with: <esc> :wq

• (optional) If you want to change to default home page:

sudo vi /usr/share/tomcat/webapps/art-decor/WEB-INF/resources/page-flow.xml

Comment or uncomment the homepage you want to use: home or terminology-home. To use the default ART-DECOR homepage, leave as is:

  <!-- Welcome page, change to use a different homepage -->
   <page id="home" path-info="/home" view="http://localhost:8877/art/modules/get-form.xq?form=home"/>
<!--   <page id="home" path-info="/home" view="http://localhost:8877/art/modules/get-form.xq?form=terminology-home"/>-->

Start Tomcat

sudo service tomcat start

Output should be similar to the following:

Using CATALINA_BASE:   /usr/share/tomcat
Using CATALINA_HOME:   /usr/share/tomcat
Using CATALINA_TMPDIR: /usr/share/tomcat/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_71/jre
Using CLASSPATH:       /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar

• Check that the Tomcat process is running:

sudo ps aux|grep ^tomcat

Output should be similar to the following:

tomcat   14042  0.1  7.6 14868812 3759704 ?    Sl   Nov25  66:36 /usr/java/jdk1.7.0_71/jre/bin/java -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx8192M -Xms8192M -Djava.endorsed.dirs=/usr/share/tomcat/endorsed -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.io.tmpdir=/usr/share/tomcat/temp org.apache.catalina.startup.Bootstrap start

• Check that the Tomcat process is listening on port 8080:

sudo netstat -anp| grep :8

Output should be similar to the following:

tcp        0      0 ::ffff:127.0.0.1:8005       :::*                        LISTEN      14042/java
tcp        0      0 :::8009                     :::*                        LISTEN      14042/java
tcp        0      0 :::8080                     :::*                        LISTEN      14042/java