Loading and Transformation Ref...

Service Configuration

The LAT service is configurable using Service Configuration.

This configuration is derived from three sources, with the following precedence order:

  1. Environment Variables
  2. Properties File
  3. Code Defaults

Configuration File Backup

When you install a new LAT instance or upgrade the existing instance, LAT creates a backup file of the /etc/lat/lat.conf as /etc/lat/override.conf and creates a new version of the /etc/lat/lat.conf file. The /etc/lat/overrirde.conf file does not control the LAT configuration. If you made prior changes to lat.conf, you must review override.conf and add any custom values into the revised lat.conf.

Environment Variables

LAT_DEFAULT_WORKERS

Default number of workers used by a newly configured pipeline.

Environment Variable:

LAT_DEFAULT_WORKERS

Property:

lat.default.workers

Default:

Server core count

LAT_SHUTDOWN_DURATION

Duration, in milliseconds, to wait prior to termination on shutdown.

Environment Variable:

LAT_SHUTDOWN_DURATION

Property:

lat.shutdown.duration

Default:

120000

LAT_UDT_PATH

Directory containing zero or more user defined transformation (UDT) jars. Ensure that the specified path can be accessed by the LAT (e.g.,

 does not allow the LAT to access the home directory of an administrative user).

Environment Variable:

LAT_UDT_PATH

Property:

lat.udt.path

Default:

null

LAT_API_PORT

REST API and Metrics port.

Environment Variable:

LAT_API_PORT

Property:

lat.api.port

Default:

8080

LAT_DATA_PATH

Path to the LAT data directory. This directory is used to store persistent data related to pipelines (pipeline/) and sinks (sink/). In almost all cases, using the LAT Client Command Line Interface is preferred over editing the files in this directory manually. Ensure that the specified path can be accessed by the LAT (e.g., CentOS does not allow the LAT to access the home directory of an administrative user).

Environment Variable:

LAT_DATA_PATH

Property:

lat.data.path

Default:

.lat-data

LAT_AUTO_START

On LAT startup, automatically starts running a Pipeline if one is present. This only supports pipelines with

 sources; File sources will not automatically start even if this configuration is set to true.

Environment Variable:

LAT_AUTO_START

Property:

lat.auto.start

Default:

false

LAT_ALLOW_LOG_ORIGINAL_RECORDS

Allow the original records that are being loaded to be logged when errors occur. This setting must be enabled for log_original_records to be enabled.

This setting only affects pipelines that log errors to the error log file.

Environment Variable:

LAT_ALLOW_LOG_ORIGINAL_RECORDS

Property:

lat.allow.log.original.records

Default:

false

LAT_MAX_RETRIEVABLE_ERRORS

A limit on the number of errors that you can retrieve using the pipeline errors command of the LAT client.

Environment Variable:

LAT_MAX_RETRIEVABLE_ERRORS

Property:

lat.max.retrievable.errors

Default:

1000

LAT_API_PROXY_HOST

API proxy host for

 authorization requests.

Environment Variable:

LAT_API_PROXY_HOST

Property:

lat.api.proxy.host

Default:

null

LAT_API_PROXY_PORT

API proxy port for Okta authorization requests.

Environment Variable:

LAT_API_PROXY_PORT

Property:

lat.api.proxy.PORT

Default:

null

LAT_API_OAUTH_DOMAIN

Okta authorization domain.

Environment Variable:

LAT_API_OAUTH_DOMAIN

Property:

lat.api.oauth.domain

Default:

null

LAT_API_OAUTH_ENDPOINTS

A list of LAT API endpoints that Okta should apply authentication on. To override, provide a comma-delimited list, e.g., /endpoint1,/endpoint2,/endpoint3.

Environment Variable:

LAT_API_OAUTH_ENDPOINTS

Property:

lat.api.oauth.endpoints

Default:

null

LAT_API_OAUTH_SERVER

Okta authorization server.

Environment Variable:

LAT_API_OAUTH_SERVER

Property:

lat.api.oauth.server

Default:

null

LAT_API_CERT_PATH

API cert file path. Ensure that the specified path can be accessed by the LAT (e.g.,

 does not allow the LAT to access the home directory of an administrative user).

Environment Variable:

LAT_API_CERT_PATH

Property:

lat.api.cert.path

Default:

null

LAT_DNS_CACHE_TTL

The time-to-live (TTL) in seconds that the LAT should use for its DNS cache. This configuration is only supported as an Environment Variable or as Default.

Environment Variable:

LAT_DNS_CACHE_TTL

Property:

N/A

Default:

30

LAT_MEMORY

Maximum JVM® heap memory. This configuration is only supported as an Environment Variable or as Default when running the LAT through the provided package service file or through the provided lat.sh script. If directly running the Jar JVM heap memory should be set through command line arguments.

Environment Variable:

LAT_MEMORY

Property:

N/A

Default:

32G

LAT_MEMORY_DIRECT

Maximum JVM direct memory. This configuration is only supported as an Environment Variable or as Default when running the LAT through the provided package service file or through the provided lat.sh script. If directly running the Jar JVM heap memory should be set through command line arguments.

Environment Variable:

LAT_MEMORY_DIRECT

Property:

N/A

Default:

32G

Override Environment Variables

The recommended way to manage the production service configuration when LAT is running as a daemon is using environment variables or the /etc/systemd/system/lat.service.d/override.conf file.

To override LAT environment variables as a systemd service, there are two recommended commands that edit the etc/systemd/system/lat.service.d/override.conf file.

  1. Run sudo vim etc/systemd/system/lat.service.d/override.conf or sudo systemctl edit lat.
  2. Insert the environment variables to override in the file. Save and exit the editor.

Configuration File Example:

Shell


Related Links

Load Data

Updated 19 Jun 2024
Did this page help you?