Overview

This document is made up of four sections, describing installation , post-installation work , performance and the admin interface

Installation

Prerequisites

  • The icat distribution: icat.server-4.9.1-distro.zip
  • Java 8
  • A suitable deployed container. The installation scripts support Glassfish/Payara and to a lesser extent WildFly. Testing has been carried out with Payara41.Glassfish/Payara installation instructions are available.
  • A database as described in Database installation instructions installed on the server
  • Deployed ICAT authenticators.
  • A deployed icat.lucene server it you plan to use TopCAT
  • Python (version 2.4 to 2.7) and the python-suds package installed on the server.

Summary of steps

  1. Upgrade the database schema if you already have an ICAT installation.
  2. If you wish to install multiple servers each running an ICAT connected to the same database please see Installing a group of ICATs
  3. Please follow the generic installation instructions
  4. See if it works.

Installing a group of ICATs

If your facility depends upon a single ICAT instance then ingestion of data can be held up by a user making an expensive query.

To avoid this it is suggested that you install multiple servers each running a Glassfish with an ICAT but all sharing one database and one icat.lucene server. It is recommended that you install the machines individually first and make sure that each works before tying them together. Finally ingestion can be directed to one node and the other nodes can be load balanced for user access by, for example, an Apache web server.

All machines must use the same database and icat.lucene server

Authentication can either be carried out on one machine - which has the advantage that you only have authenticator logs building up on that machine or each machine can host its own authenticators. If an icat.server is configured with the new style restful authenticators then more than one equivalent authenticator server may be specified to make the system more robust. If you use the old style where JNDI settings must be specified it is recommended for each icat.server to have its own authenticators.

The icat.servers are linked by specifying a cluster parameter in the run.properties file for each machine. This parameter may be the same for each machine as the software tries to avoid sending messages to itself.

You could then set up an Apache front end to do load balancing. This will probably just connect to the satellites leaving the central machine to handle ingestion of data. See Apache front end for one way of doing this.

Schema upgrade

Any existing lucene database should be removed. The location of this would have been specified in the previous icat.properties file. Ensure that the directory specified there is empty.

The database schema must be upgraded in steps depending upon how old your icat installation is

Upgrade 4.2.5 schema to 4.3.x

This is for upgrading a 4.2.5 schema to 4.3.2. If you have already upgraded to 4.3.x skip this step. Do not attempt to use this procedure on a 4.3.x schema!

  1. Back up the database in case it should get into a state from which recovery is impractical.
  2. Run the get_rules program to save the rules in the format accepted by icat-setup. This must be run as somebody who has read access to the rules. For example: ./get_rules.py https://example.com:8181 db username root password password The program should report how many rules it has saved and where.
  3. Ensure that nobody tries to use ICAT while it is being upgraded - the simplest approach is to undeploy the old one which can be done from the command line or by using a web browser and connecting on the admin port (typically 4848) and undeploying from there.
  4. For MySQL edit username, password, schema and dbhost at the top of the file ./upgrade_mysql_4_2_5.py and run it or or for Oracle, edit username , password and db at the top of the file ./upgrade_oracle_4_2_5.py and run that. Note that the procedure has been tested on ICAT 4.2.5 but should work for earlier 4.2 versions. The script will first check that everything should go work. If it reports problems fix them and try again. Once it gets past the checking stage it starts the conversion which can take a long time (many hours for a production system). At the end you should have a 4.3.2 database. Any indices which had been created manually will have been removed.
  5. Install the new icat
  6. Restore the rules using the icat-setup tool. For example:
    icat-setup -f rules.authz https://example.com:8181 db username root password secret

    This assumes that you are in the directory where you ran get_rules.py which will have created a file rules.authz. The credentials (keyword value pairs following the authenticator mnemonic) should be those of one of the users specified in the rootUserNames of the icat.properties file.

    Please check the rules.authz first as it will not work if it references entities that no longer exist. For example "Group" must be replaced by "Grouping" and InputDatafile is no longer part of ICAT. Also because some problems have been found with conditions containing dots (such as "InvestigationUser [user.name='fred']") in rules these must now be re-expressed without dots.

Upgrade 4.3.x schema to 4.4.0

You may increase the size of the "what" column of the Rule table to 1024 to match the size the column has on a brand new installation and you must modify the INVESTIGATIONUSER table as role values may no longer be null. Choose a name to use for the default role - for example 'member' and for MySQL:

UPDATE INVESTIGATIONUSER SET ROLE = 'member' WHERE ROLE IS NULL;
ALTER TABLE INVESTIGATIONUSER MODIFY COLUMN ROLE varchar(255) NOT NULL;
ALTER TABLE INVESTIGATIONUSER DROP FOREIGN KEY FK_INVESTIGATIONUSER_USER_ID;
ALTER TABLE INVESTIGATIONUSER DROP INDEX UNQ_INVESTIGATIONUSER_0;
ALTER TABLE INVESTIGATIONUSER ADD CONSTRAINT UNQ_INVESTIGATIONUSER_0 UNIQUE (USER_ID, INVESTIGATION_ID, ROLE);
ALTER TABLE INVESTIGATIONUSER ADD CONSTRAINT FK_INVESTIGATIONUSER_USER_ID FOREIGN KEY (USER_ID) REFERENCES USER_ (ID)

or for Oracle:

UPDATE INVESTIGATIONUSER SET ROLE = 'member' WHERE ROLE IS NULL;
ALTER TABLE INVESTIGATIONUSER MODIFY (ROLE varchar2(255) NOT NULL);
ALTER TABLE DROP CONSTRAINT UNQ_INVESTIGATIONUSER_0;
ALTER TABLE INVESTIGATIONUSER ADD CONSTRAINT UNQ_INVESTIGATIONUSER_0 UNIQUE (USER_ID, INVESTIGATION_ID, ROLE);

Upgrade 4.4.0 schema to 4.5.0

The mechanism for assigning unique id values for each entity in ICAT has been changed. Previously a sequence table (called SEQUENCE) was used to hold the last value used. This has been changed to make use of the native DBMS mechanism which is making the id columns AUTO_INCREMENT for MySQL or using a Sequence (rather than a table called SEQUENCE) in the case of Oracle. Do not omit this step otherwise the id values for new rows will not be set correctly and you will run into problems with duplicate values. For MySQL run mysql -u icat -p icat < upgrade_mysql_4_4.sql or for Oracle run sqlplus icat @upgrade_oracle_4_4.sql where in both cases it is assumed that the tables are owned by user "icat". The MySQL script is simply a list of alter table statements for each table in the 4.4.0 schema. The oracle script takes the last sequence number from the SEQUENCE table and uses this to initialize a sequence. Note that the increment for the sequence must be exactly 50 and the start value must be at least 51 more than the number in the old SEQUENCE table.

Upgrade 4.5.x or 4.6.x schema to 4.7.0

The Rule and DataCollection tables have changed. First dump the existing rules with: ./rules.py dump https://example.com:8181 db username root password secret > rules.ie i.e. run the rules.py script in the unpacked distribution directory with the "dump" parameter, the url of your icat and then the authenticator plugin and credentials to identify a user specified in the rootUserNames list in icat.properties. This will redirect the dump into a file called rules.ie. Then undeploy the existing icat ./setup uninstall -v . Edit the rules.ie file to remove rules (probably two of them) referring to the Log table.

Then for MySQL run mysql -u icat -p icat < upgrade_mysql_4_7.sql or for Oracle run sqlplus icat @upgrade_oracle_4_7.sql where in both cases it is assumed that the tables are owned by user "icat". The script will drop the Rule table and add a DOI column to the DataCollection table. Next install the new ICAT (which will recreate an empty Rule table) Finally run ./rules.py load https://example.com:8181 db username root password secret < rules.ie which will populate the Rule table. Only run this script once or you will get duplicate entries in the table.

In addition the Log table is no longer used and may be dropped after you have extracted any information from it that you need.

Upgrade 4.7.0 schema to 4.8.0

A column must be added to the RULE_ table. This can be done or MySQL or MariaDB by mysql -u icat -p icat < upgrade_mysql_4_8.sql or for Oracle run sqlplus icat @upgrade_oracle_4_8.sql where in both cases it is assumed that the tables are owned by user "icat". This can be done be done while icat.server 4.7.0 is running as the old code is unaware of the addition of the new column.

The setup.properties file

container
Values must be chosen from: TargetServer Though only Glassfish is working properly at the moment.
home
is the top level of the container installation. For Glassfish it must contain "glassfish/domains" and for JBoss (wildfly) it must contain jboss-modules.jar.
port
is the administration port of the container which is typically 4848 for Glassfish and 9990 for JBoss.
secure
must be set to true or false. If true then only https and not http connections will be allowed.
db.driver
is the name of the jdbc driver which must match the jar file installed in the container and matching your database.
db.url
url to connect to your database. For example: jdbc:mysql://localhost:3306/icat
db.username
username to connect to your database.
db.password
password to connect to your database.
db.target
This is optional and may be used to control the SQL generated by the JPA. Values must be chosen from: TargetDatabase
db.logging
This is optional and if set to one of the values in Eclipse Link logging.level controls the logging of JPA generated SQL statements.

The logback.xml file

If you wish to modify the provided logging levels then rename logback.xml.example to logback.xml and edit to suit your needs.

The run.properties file

lifetimeMinutes
Defines the lifetime of an ICAT sessionid. You should avoid making it have a long duration as this increases the risk if it is intercepted, lost or stolen.
rootUserNames
Is a space separated list of user identifiers having full access to all tables. The format of the user identifier is determined by the chosen authentication plugin. The authn_db and authn_ldap plugins may be configured to either return the simple user name or to prepend it with a name identifying the mechanism. For example if there is a an entry "root" in the database then if the authn_db authenticator is configured without a mechanism then the user name to consider will be just "root", however if it has been configured with a mechanism of "db" then the string "db/root" must be specified.
maxEntities
Restrict total number of entities to return in a search or get call. This should be set as small as possible to protect the server from running out of memory. However if you set it too small it may prevent users from doing reasonable things.
maxIdsInQuery
For handling INCLUDEs, ICAT may generate queries which are not acceptable to the database system. To avoid this problem such queries are broken down. This is the maximum size of each chunk which must not exceed 1000 for Oracle.
importCacheSize
the size of a cache used during import to avoid an excessive number of calls to the database. The cache is dropped after each call to import to ensure that authorization rules are enforced. As the cache is short-lived, modifications to ICAT are unlikely to result in stale information being used from the cache.
exportCacheSize
the size of a cache used during export to avoid an excessive number of calls to the database. The cache is dropped after each call to export to ensure that authorization rules are enforced. As the cache is short-lived, modifications to ICAT are unlikely to result in stale information being used from the cache.
authn.list
is a space separated set of mnemonics for a user to select the authenticator in the login call. Authenticators are separate applications which may be "Remote EJBs", identified by jndi, if installed in the same container or they may be restful services identified by url. The list must not reference Remote EJBs which are not installed as these are checked when ICAT performs its initialisation; their absence will cause ICAT to not start.
authn.<mnemonic>.url
is a space separated list of the urls of machines with a restful authenticator service. This will take the form: https://example.com:443 There must be one such entry for each restful authenticator. If more than one url is provided the services referenced must be functionally equivalent. An invalid URL syntax will cause the ICAT server not to start. This must not be set for an EJB Authenticator.
authn.<mnemonic>.jndi
is the jndi name to locate an EJB authenticator. When you installed the authenticator a message would have appeared in the server.log stating the JNDI names. The name will start: java:global/ There must be one such entry for each EJB authenticator. This must not be set for a restful Authenticator.
authn.<mnemonic>.friendly
is optional. It gives a name that a tool might use to label the plugin.
authn.<mnemonic>.admin
is optional. Set to true if you wish to indicate that this authenticator should only be advertised to administration tools.
notification.list
is optional. It is a space separated set of Entity names for which you with to generate notifications. For each one there must be another line saying under what conditions you wish to generate a notification for the entity.
notification.<entity name>
a string of letters taken from the set "C" and "U" indicating for which operations (create and update) you wish to be notified for that kind of operation on the entity.
log.list
is optional. If present it specifies a set of call types to log via JMS calls. The types are specified by a space separated list of values taken from READ, WRITE, SESSION, INFO.
lucene.url
This is optional. It is the machine url of the icat.lucene server if needed. It is needed for TopCAT to work.
lucene.populateBlockSize
This is ignored if lucene.url is not set. The number of entries to batch off to the lucene server when using lucenePopulate.
lucene.directory
This is ignored if lucene.url is not set. Path of a directory holding files for requests that are queued to go the icat.lucene server.
lucene.backlogHandlerIntervalSeconds
This is ignored if lucene.url is not set. How often to check the backlog file.
lucene.enqueuedRequestIntervalSecond
This is ignored if lucene.url is not set. How often to transmit lucene requests to the icat.lucene server.
jms.topicConnectionFactory
This is optional and may be used to override the default value of java:comp/DefaultJMSConnectionFactory
key
This is optional but if there is an IDS server in use and it has a key for digest protection of Datafile.location then this key value must be identical.

Check that ICAT works

A small test program, testicat, will have been installed for you. This is a python script which requires that the suds client is available. This connects as one of the root users you defined as 'rootUserNames' in the icat.properties file. Invoke the script specifying the url of the machine on which the ICAT service is deployed (something like https://example.com:8181), the mnemonic for the chosen authentication plugin followed by the credentials for one of the root user names supported by that plugin. These credentials should be passed in as pairs of parameters with key followed by value. For example: testicat https://example.com:8181 db username root password secret

It should report:

Logged in as ... with 119.9... minutes to go
Login, search, create, delete and logout operations were all successful.

This script can be run at any time as it is almost harmless - it simply creates a "Group" with an unlikely name and removes it again.

In case of problems, first erase the directory /tmp/suds and try the testicat again. If it still fails, look at the log files: server.log and icat.log which can both be found in the logs directory below your domain. Look also at the relevant authenticator log.

Post-installation work

Fresh Install

If this is a fresh install then you can use the import facility to do the initial icat population or you could use the icat manager to create rules, a Facility and other high level entities.

If you are using Oracle the type NUMBER(38, 19) will have been used for all floating point numbers. This constrains the values that can be stored - they may be truncated or rejected. To fix this please execute the SQL statements in fix_floats_oracle.sql

In all cases

Populate the lucene index by using the icatadmin tool.

Performance

To improve performance:
  • Consider creating the indices defined in indices.sql. Indices can make a huge difference to the database performance but there is also a small cost for each index.
  • Make entities readable by anyone if they contain no sensitive information. This is generally the case for those entities that implement an many-to-many relationship. For example InvestigationUser relates Investigation to User but has no attributes. By making it world readable no access to Investigation or User is granted. An in memory cache of world readable entities is maintained by ICAT.
  • Add entries to PublicStep to allow the INCLUDE mechanism to be less costly. PublicStep is explained in the ICAT Java Client User Manual. Its contents are also held in an in-memory cache for performance.

The icatadmin tool

Administration operations have been added to the ICAT API and are accessible via the icatadmin tool which will have been installed by the setup.py script. It should be invoked as:

icatadmin <url> <plugin> <credentials>... -- <command> <args>...

to run a single command or

icatadmin <url> <plugin> <credentials>...

to be prompted for a series of commands as shown below. In either case if you specify '-' as the password you will be prompted for it. Note that in the single command case the "--" marker is needed to terminate the list of credentials. For example:

icatadmin https://example.com:8181 db username root password secret -- properties
Only users mentioned in the rootUserNames of the icat.properties file are authorized to use this command.
populate [<entity name>]
re-populates lucene for the specified entity name. This is useful if the database has been modified directly rather than by using the ICAT API. This call is asynchronous and simply places the request in a set of entity types to be populated. When the request is processed all lucene entries of the specified entity type are first cleared then the corresponding icat entries are scanned to re-populate lucene. To find what it is doing please use the "populating" operation described below. It may also be run without an entity name in which case it will process all entities. The new lucene index will not be seen until it is completely rebuilt. While the index is being rebuilt ICAT can be used as normal as any lucene updates are stored to be applied later.
populating
returns a list of entity types to be processed for populating lucene. Normally the first item returned will be being processed currently.
commit
instructs lucene to update indices. Normally this is not needed as it is will be done periodically according to the value of lucene.commitSeconds
clear
stops any population and clears all the lucene indices.