Migrate TermWeb 3.x to TermWeb 4.0 (Linux)

Owned by Aleksei Melnikov
Last updated: Nov 28, 2023
7 min read

Prepare for migration

  • Prior to start migration of TermWeb 3 it should be updated to latest version of 3.18.

  • With migration to TermWeb 4 it keeps everything from previous version you won’t loose any history or metadata. Only old Grid View data will be dropped. Before starting migration if some users using Grid View, they must publish or discard their changes.

  • Install Oracle JDK 8 (latest free version is 8u202) or OpenJDK 8 (we're using and testing with Amazon Corretto 8). Don’t install version higher than 8 TermWeb 4 won’t work with it.

For better understanding on install process you may refer to TermWeb 4 install on Linux

Update and start migration

  • You may use MySQL, Tomcat and Elasticsearch on same or different hosts.

  • Update or Install MySQL to version 8.0.x

  • Create new scheme and user with next script on MySQL 8.0.x instance:

    create_scheme_and_user.sql

    CREATE DATABASE `termweb4db` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; CREATE USER 'termwebuser'@'%' IDENTIFIED BY '<your_password_goes_here>'; GRANT ALL PRIVILEGES ON `termweb4db`.* TO 'termwebuser'@'%';

     

  • Import old backup into new scheme on MySQL 8.0.x instance

  • Install Elasticsearch 7.16.x (On linux much better to install it via rpm repository)
    https://www.elastic.co/guide/en/elasticsearch/reference/7.x/rpm.html

    In most cases for Elasticsearch process will be enough 1GB of RAM.
    Only staging areas rely on elasticsearch, so there is no need to create failover setup with two nodes.

    Install plugin for it bin/elasticsearch-plugin install analysis-icu

  • Configure new termweb.properties in TermWeb 4 home folder (it should have been copied with TermWeb 3 folder), add here connection properties to new MySQL 8.0.x instance and Elasticsearch
    Disable use of SSL if you're sure about security of connection between Tomcat and MySQL by adding useSSL=false property

    termweb.properties

    // DB connection settings spring.datasource.driverClassName=com.mysql.jdbc.Driver spring.datasource.url=jdbc:mysql://mysql80host/termweb4db?autoReconnect=true&useUnicode=true&character_set_server=utf8mb4&connectionCollation=utf8mb4_general_ci&useSSL=false spring.datasource.username=dbuser spring.datasource.password=dbpassword // old TermWeb3 properties termweb.home = comment this property, now it should be specified in java options for tomcat work.dir = comment this property, work directory now is located in termweb.home folder base.url = set new value if server address will be different # cluster name used mostly in logs for identification termweb.elasticsearch.cluster.name=termwebes-cluster termweb.elasticsearch.index-prefix=termweb4 termweb.elasticsearch.address=172.1.1.1 termweb.elasticsearch.port=9200

     

  • Update Apache httpd if it's too old, configure serving of angular application (termweb-ui.zip file from distributive), deploy termweb-ui (contents of dist folder) application into your httpd web directory, i.e /var/www/termweb4

    As example how to deploy you can check out our script for deployment. If your linux distribution is using SELinux security mechanism, then you need to run restorecon command too, otherwise Apache httpd won't be able to serve new files from /var/www/termweb4 folder.

    Deploy web static fiels

     

  • Configure SSL certificate for your VirtualHost or do it later.

  • Prepare database for first run by executing next sql script:

    schema_version.sql

     

  • Some of old databases may have problems with migration scripts, you may encounter them or no. Check values in tw_userproperties table:

    If some of the values length of column properties bigger than 65535 then you need to update it. Safest way is to delete xml elements visitedTermOids from properties value.

    Sometimes migration may fail with wrong constraints for table or incompatible values, to be sure in successful result execute next script:

     

  • Start tomcat as service and check the logs. TermWeb 4 should run migration scripts automatically. It can take a while (for 2GB database it can take about 1 hour)

    Run and wait

     

  • Wait for message:    [           main] com.nescit.term.log.CustomLogger         : TermWeb started

  • If log file contains errors check Known Issues or contact Tech support.

Known issues during migration:

  • Warning about missing tables can be ignored.



  • Problem with creation of Elasticsearch index:


    Occurs because of similar field names in dictionary, like 'Usage note' and 'usage note'. Can be solved by giving new name to one of the fields. And later by pressing 'Reindex' in dictionary list in AdminView.