Installation
Prerequisites
-
The icat distribution:
icat.ear-4.3.0-distro.zip
-
A suitable deployed container (here assumed to be glassfish though tomcat should be sufficient) to support a web
application.
Testing has been carried out with Glassfish 3.1.2.
Glassfish installation instructions
are available.
-
A database as described in
Database installation instructions
installed on the server
-
Deployed ICAT authenticators.
-
Python (version 2.4 to 2.7) and the python-suds package installed on the server.
- MySQL-python must be installed if you have an MySQL 4.2.x schema you need to upgrade.
- cx_Oracle must be installed (in addition to an Oracle client) if you have an Oracle 4.2.x schema you need to upgrade.
-
icat-setup
version 1.1.x or higher
must be installed if you have a 4.2.x schema you need to upgrade and you wish to preserve existing
rules but is also generally
a useful component to install.
-
ice
, the ICAT editor will make it easier to configure your ICAT.
Summary of steps
- Upgrade the database schema if you already have a 4.2.x version which you wish to use with ICAT 4.3.0
-
Please follow
the generic installation instructions
- See if it works.
Schema upgrade
- Back up the database in case it should get into a state from which recovery is impractical.
-
Run the get_rules program to save the rules in the format accepted by icat-setup. This must be run as somebody who has read
access to the rules. For example:
./get_rules.py https://example.com:8181 db username root password password
The program should report how many rules it has saved and where.
- Ensure that nobody tries to use ICAT while it is being upgraded - the simplest approach is to undeploy the old one which can
be done from the command line or by using a web browser and connecting on the admin port (typically 4848) and undeploying from
there.
-
For MySQL edit username, password, schema and dbhost at the top of the file
./upgrade_mysql_4_2_5.py
and run it or
or for Oracle, edit username , password and db at the top of the file
./upgrade_oracle_4_2_5.py
and run that.
Note that the procedure has been tested on ICAT 4.2.5 but should work for earlier 4.2 versions. The script
will
first check that everything should go work. If it reports problems fix them and try again. Once it gets past the checking
stage it starts the conversion which can take a long time (many hours for a production system). At the end you should have a
4.3.0 database. Any indices which had
been created manually will have been removed.
The icat.setup.properties file
- driver
- is the name of the jdbc driver and must match the jar file for your database that you stored in the previous step.
- icatProperties
- identifies the icat database and how to connect to it.
- glassfish
- is the top level of the glassfish installation. It must contain "glassfish/domains", and will be referred to here as
GLASSFISH_HOME as if an environment variable had been set.
- port
- is the administration port of the chosen glassfish domain which is typically 4848.
For a local oracle-xe installation the following values of driver and icatProperties should be good except for
the user and password
values:
driver=oracle.jdbc.pool.OracleDataSource
icatProperties=url="'"jdbc:oracle:thin:@//localhost:1521/XE"'":ImplicitCachingEnabled=true:MaxStatements=200:user=icat:password=secret
Note the
"'"
which is needed because the url contains colons which also separate individual properties.
For MySQL:
driver=com.mysql.jdbc.jdbc2.optional.MysqlDataSource
icatProperties=user=icat:password=secret:databaseName=icat
The icat.properties and icat.log4j.properties files
If you wish to modify the provided logging
levels then rename
icat.log4j.properties.example to icat.log4j.properties and update the
icat.properties
file to reference it as
explained below. The icat.properties file may need
other changes:
- lifetimeMinutes
- Defines the lifetime of an ICAT sessionid. You should avoid making it have a long duration as this increases the risk if it
is
intercepted, lost or stolen.
- rootUserNames
- Is a space separated list of user identifiers having full access to the authorization tables: Rule, Group, User and UserGroup.
The format of the user identifier is determined by the chosen authentication plugin. The authn_db and authn_ldap plugins may
be configured to either return the simple user name or to prepend it with a name identifying the mechanism. For example if
there is a an entry "root" in the database then if the authn_db authenticator is configured without a mechanism then the user
name to consider will be just "root", however if it has been configured with a mechanism of "db" then the string "db/root"
must be specified.
- authn.list
-
is a space separated set of mnemonics for user to select the plugin in the login call. This must not reference plugins which
are not installed as plugins are checked when ICAT performs its initialisation; if plugins are missing ICAT will not start.
-
authn.<mnemonic>.jndi
- is the jndi name to locate the plugin. When you installed the plugin a message would have appeared in the server.log stating
the JNDI names. For example for authn_db you would expect to see
java:global/authn_db.ear-1.0.0/authn_db.ejb-1.0.0/DB_Authenticator. There must be one such entry for each
plugin.
- notification.list
- is a space separated set of Entity names for which you with to generate notifications. For each one there must be another line
saying under what conditions you wish to generate a notification for the entity.
-
notification.<entity name>
- a string of letters taken from the set "C", "U" and "D" indicating for which operations (create, update and delete) you wish
to be notified for that kind of operation on the entity.
- log.list
- is a space separated set of call logging destinations - currently valid destinations are "table" and "file". For each one
there
must
be another line
saying under what group of calls you wish to log.
- log.<destination>
- a string of letters taken from the set "S", "R" and "W" indicating for which group of calls you wish to generate a log entry.
"S" denotes session calls (login, refresh and logout); "R" is used for the various read calls and "W" is used for all write
calls (create, update and delete).
- log4j.properties
- This is optional. If present it must specify the path to a log4j.properties file. The path may be absolute or relative to
the
config directory.
- lucene.directory
- This is optional. It is the path to a directory (whose parent must exist) in which to store the lucene index. If this is
specified then
lucene.commitSeconds must also be specified. If it is omitted then lucene indices will not be created and the
searchText()
call will return nothing.
- lucene.commitSeconds
- the interval in seconds between committing lucene changes to disk and updating the index. If you set it to 300 then
searchText() calls will see what was available at some time in the past (up to 5 minutes ago) and which is also currently
present.
- lucene.commitCount
- the number of changes to accumulate before committing them to disk. If the number is too high there can be memory
problems.
Currently this is only used by the call to lucenePopulate.
Check that ICAT works
A small test program,
testicat,
will have been installed for you.
This is a python script which requires that the suds client is available. This connects as one of
the root users you defined as 'rootUserNames' in
the icat.properties file. Invoke the
script specifying the url of the machine on
which the
ICAT service is deployed
(something like
https://example.com:8181), the
mnemonic for the chosen authentication plugin
followed by the
credentials for one of the root user names
supported
by that plugin.
These
credentials should be passed in as pairs of
parameters with
key followed by value. For example:
testicat https://example.com:8181 db username root password secret
It should report:
Logged in with 119.9... minutes to go
Login, search, create, delete and logout operations were all successful.
This script can be run at any time as it is almost harmless - it simply creates a "Group" with an unlikely name and removes it
again.
In case of problems, first erase the directory /tmp/suds and try the testicat again. If it still fails, look at the log files:
server.log and icat.log which can both be found in the logs directory below your domain. Look also at the relevant authenticator
log.
Post-installation work
Fresh Install
If this is a fresh install then you can use
icat-setup
to add rules to ICAT, then use ice to create a Facility and other high level
entities.
If you are using Oracle the type NUMBER(38, 19) will have been used for all floating point numbers. This constrains the values
that can be stored - they may be truncated or rejected. To fix this please execute the SQL statements in
fix_floats_oracle.sql
Upgrade
If you have performed a schema upgrade then you should restore the rules using the
icat-setup
tool. For example:
icat-setup -f rules.authz https://example.com:8181 db username root password secret
This assumes that you are in the directory where ran get_rules.py which will created a file rules.authz. The credentials (keyword
value
pairs following the authenticator mnemonic) should be those
of one of the users specified in the rootUserNames of the
icat.properties file.
Please check the rules.authz first as it will not work if it references entities that no longer exist. For example "Group" must be
replaced by "Grouping" and InputDatafile is no longer part of ICAT. Also because some problems have been found with conditions
containing dots (such as "InvestigationUser [user.name='fred']") in rules these must now be re-expressed without dots.
Finally you should run the icatadmin populate command to populate lucene for all existing data as in the example below:
icatadmin https://example.com:8181 db username root password secret -- populate --timeout 3600
Performance
To improve performance:
- Consider creating the indices defined in indices.sql. Indices can make a huge difference to the database performance but there
is
also a small cost for each index.
- Make entities readable by anyone if they contain no sensitive information. This is generally the case for those entities that
implement an many-to-many relationship. For example InvestigationUser relates Investigation to User but has no attributes. By
making it world readable no access to Investigation or User is granted. An in memory cache of world readable entities is
maintained by ICAT.
- Add entries to PublicStep to allow the INCLUDE mechanism to be less costly. PublicStep is
explained in the ICAT Java
Client User
Manual. Its contents are also held in an in-memory cache for performance.
The icatadmin tool
Administration operations have been added to the ICAT API and are accessible via the icatadmin tool which will have been installed by
the
setup.py
script. It should be invoked as:
icatadmin <url> <plugin> <credentials>... -- <command> <args>...
to run a single command or
icatadmin <url> <plugin> <credentials>...
to be prompted for a series of commands as shown below. In either case if you specify '-' as the password you will be prompted for
it.
Note that in the single command case the "--" marker is needed to terminate the list of credentials. For example:
icatadmin https://example.com:8181 db username root password secret -- properties
. Only users mentioned in the rootUserNames of the icat.properties file are authorized to use this command.
- properties
- lists the active contents of the icat.properties file. It does this by examining the properties after they have been read in
so any
superfluous definitions in the original properties file will not be seen.
- populate [<entity name>]
-
re-populates lucene for the specified entity name. This is
useful if the database has been modified directly rather than by using
the ICAT API. Note that because of caching, ICAT should
be
reloaded after any direct database modifications are made. This call is
asynchronous and simply places the request in a set of
entity types to be populated. When the request is processed all lucene
entries of the specified entity type are first cleared
then the corresponding icat entries are scanned to re-populate lucene. To
find what it is doing please use the "populating" operation desribed below.
It may also be run without an entity name in which case
it will process all entities.
- populating
- returns a list of entity types to be processed for populating lucene. Normally the first
item returned
will be being processed
currently.
- commit
- instructs lucene to update indices. Normally this is not needed as it is will be done periodically according to the value of
lucene.commitSeconds
- clear
- clears all the lucene indices - but does not commit.
- search <query> [<maxCount> [<entityName>]]
-
searches lucene indices and returns a list of entity_name:entity_id values. Arguments are
- query
- a lucene query which should be enclosed in quotes if it contains more than one word. Queries can contain AND and OR in
upper case as well as parentheses. The default operator is OR. Wildcards of * and ? are also supported. Other features
are
described in the user documentation for the searchText call.
- maxCount
- maximum number of values to return (defaults to 100)
- entityName
- if specified, this restrict results to entities with this name