Loading and Transformation Ref...

LAT Packaging and Installation

The LAT supports multiple types of packages for deployment and installation:

  •  .deb package
  •  .rpm package
  • Raw .tar.gz tar folder
  • Direct .jar
  •  container

Contact Ocient Support for the latest package for a deployment method.

Prerequisites

  •  17 or greater
    • Can either be system-level Java, or installed somewhere accessible by the user.

The LAT must run on Ocient Loader Nodes where you installed the Ocient RPM to provide access to numerical libraries, ensuring the correct loading of geospatial data types. If this condition is not met, some geospatial objects might load inaccurately.

Debian/CentOS Packages

1. Install/Upgrade

Debian: sudo dpkg -i lat_$VERSION_all.deb

CentOS: sudo rpm -i lat-$VERSION-1.noarch.rpm

2. Setup Java

If the default system Java is version 17 or greater, no action is required. If not, edit the service file to contain a JAVA_HOME environment variable pointing to a Java version 17 or greater installation.

  1. Run: systemctl edit lat
  2. Insert the following
Shell


3. Configure

Modify the Source Configuration and Log4J Configuration (see Package Contents for file locations).

It is recommended to use the .conf file for most LAT configuration. However, LAT_MEMORY and LAT_MEMORY_DIRECT can only be set by using environment variable. For these use the same process as setting JAVA_HOME as detailed in step 2.

4. Start/Stop

Start: systemctl start lat

Stop: systemctl stop lat

Package Contents

  • Jar: /opt/lat/lat-$VERSION.jar
  • Service File: /usr/lib/systemd/system/lat.service
  • Configuration: /etc/lat/
    • Server Configuration: lat.conf
    • Logging Configuration: log4j2.xml
  • Logs: /var/log/lat/lat.log
  • LAT data directory: /opt/lat/.lat-data

Tar Folder

1. Extract file in your preferred location

Run: tar -xzf lat-$VERSION.tar.gz

2. Setup Java

If the default system Java is version 17 or greater, no action is required. If not, set the $JAVA_HOME environment variable to a Java installation with version 17 or greater:

export JAVA_HOME="PATH_TO_JAVA_HOME"

3. Configure

Modify the Source Configuration and Log4J Configuration (see Folder Contents for file locations).

It is recommended to use the .conf file for most LAT configuration. However, LAT_MEMORY and LAT_MEMORY_DIRECT can only be set using environment variable. For these use the same process as setting JAVA_HOME as detailed in step 2.

4. Start/Stop

Start: ./lat.sh [path], where path is an optional parameter for the path to a server configuration file (e.g., lat.conf). If path is not present, it will default to config/lat.conf.

Stop: Stop the running lat.sh script with ctrl-c.

Folder Contents

  • Jar: lat-$VERSION.jar
  • Configuration: config/
    • Server Configuration: lat.conf
    • Logging Configuration: log4j2.xml
  • Start Script: lat.sh
  • LAT data directory: .lat-data

Jar Direct

The LAT is a Java jar that can be deployed on any machine running Java 17+.

To run the LAT use the following command line statement:

Shell

  • log4j-config-path: REQUIRED : path to the log4j configuration file
  • lat-jar-path: REQUIRED : path to the LAT jar
  • lat-config-path: OPTIONAL : path to a LAT service configuration file

Recommended Java Command Line Options

See the Java command line documentation for options.

Max Direct Memory Size

Option:

-XX:MaxDirectMemorySize

Recommended:

32G

Example:

-XX:MaxDirectMemorySize=32G

Minimum Heap Size

Option:

-Xms

Recommended:

32G

Example:

-Xms=32G

Maximum Heap Size

Option:

-Xmx

Recommended:

32G

Example:

-Xmx=32G

Example Java Command Line

Shell


Docker Container

Ocient provides a Docker container for the LAT for testing workloads.

The container is not recommended for production workloads and should be used for testing only.

Configuration

The only mode of configuring the LAT within a Docker container is by using the service configuration environment variables. A lat.conf file is not supported.

See Service Configuration for the list of supported environment variables.

It is not recommended to override the following environment variables.

  • LAT_DATA_PATH
  • LAT_API_PORT

Logging

Within the Docker container the LAT logs stdout. The user can use the docker logs ... command to view the LAT logs.

The following environment variables are available to modify the log level of various loggers. See Log4J Configuration for descriptions of the loggers and log levels.

ROOT_LOG_LEVEL

Logger:

root

Default:

debug

BINDERS_LOG_LEVEL

Logger:

com.ocient.lat.sink.ocient.Binders.Binder

Default:

info

HTTP_LOG_LEVEL

Logger:

org.eclipse.jetty.server.HttpChannel

Default:

debug

KAFKA_LOG_LEVEL

Logger:

org.apache.kafka

Default:

info

Java

The JAVA_OPTS environment variable is available for controlling settings. Container defaults to default JVM settings.

See the Java command line documentation for detail on the command line options.

Example:

Shell


Data Volumes

The LAT in the docker container writes persistent data to /lat/data. It is recommended to mount a volume to this path so that the data can outlive the container instance. The Error Log File is also written to this path if configured. If viewing this error file is necessary, a volume must be used.

Do not use the LAT_DATA_PATH configuration to override the data path inside the docker container. It will cause the LAT to write data and error logs into different locations and will require two volumes.

If using the Local File Source, the user must mount the data directory as a bind mount inside the container. It is recommended to mount to a location under the /lat/ directory. When configuring the local file source the bind mount path should be configured with a file group.

Ports

By default the LAT API listens on port 8080. It is not recommended to override the LAT_API_PORT environment variable and instead use Docker’s port mapping functionality.

Examples

Docker

Shell


Docker Compose

YAML


Related Links

Load Data