Alter Default Data Pipeline Behavior

An administrator might configure some parameters that control the behavior of data pipelines to modify the default behavior across the ﻿ System or specific nodes. You can use the ALTER SYSTEM ALTER CONFIG SQL statements to control the configuration parameters.
Syntax
SQL
ALTER SYSTEM ALTER CONFIG SET 'sql.pipelineparameters.parameter_name' 'parameter_value';
ALTER SYSTEM ALTER CONFIG SET 'sql.pipelineparameters.parameter_name' 'parameter_value';
﻿
Parameter
Data Type
Description
parameter_name
string
The name of the pipeline parameter, either general or specific to the BINARY format.
parameter_value
string
The value for the pipeline parameter.
General Parameters
General Parameter Name
Default
Data Type
Required Value and Description
sql.pipelineparameters.targetTimePerExtractorTask
600
UINT64_T
﻿
Target execution time in seconds for run_extractor tasks for file loads.
sql.pipelineparameters.defaultPartitionsPerExtractorTask
-1
INT
Number of partitions per task.
If you set this parameter to -1, the number is based on available CPU cores.
sql.pipelineparameters.defaultExtractorCores
-1
INT
Number of cores to use in a task.
If you set this parameter to -1, the number is based on available CPU cores.
Binary Format Parameters
You can modify these parameters to change the default behavior of the BINARY data format extraction.
Binary Format Parameter Name
Default
Data Type
Required Value and Description
sql.pipelineparameters.extract.binary.defaultEndianness
big
string
The endianness of transforms when you do not specify it in another way.
Supported values are big or little.
sql.pipelineparameters.extract.binary.defaultCharset
ibm1047
string
Determines the default character set to use when you load data in the BINARY format.
sql.pipelineparameters.extract.binary.defaultAutoTrimPadding
true
Boolean
Determines the default for whether a padding character should be trimmed when decoding data in the BINARY format to a string.
sql.pipelineparameters.extract.binary.defaultPaddingCharacter
' '
CHAR
Determines the default padding character to trim from the end of a string when decoding data in the BINARY format.
This parameter is effective only if the extract.binary.defaultAutoTrimPadding parameter is set to true.
Examples
Set the target execution time for the extractor task to 600 seconds.
SQL
ALTER SYSTEM ALTER CONFIG SET 'sql.pipelineparameters.targetTimePerExtractorTask' 600;
ALTER SYSTEM ALTER CONFIG SET 'sql.pipelineparameters.targetTimePerExtractorTask' 600;
﻿
Set the default endianness of transforms to little endian for the BINARY extract.
SQL
ALTER SYSTEM ALTER CONFIG SET 'sql.pipelineparameters.extract.binary.defaultEndianness' 'little';
ALTER SYSTEM ALTER CONFIG SET 'sql.pipelineparameters.extract.binary.defaultEndianness' 'little';
﻿
The configuration values only go into effect after the ALTER SQL statement succeeds and you restart the affected nodes.
Related Links
﻿Load Data﻿﻿
﻿Data Types for Data Pipelines﻿﻿
﻿Data Formats for Data Pipelines﻿﻿
﻿Configuration Settings for Data Pipelines﻿ 

Parameter	Data Type	Description
parameter_name	string	The name of the pipeline parameter, either general or specific to the BINARY format.
parameter_value	string	The value for the pipeline parameter.

General Parameter Name	Default	Data Type	Required Value and Description
sql.pipelineparameters.targetTimePerExtractorTask	600	UINT64_T	Target execution time in seconds for run_extractor tasks for file loads.
sql.pipelineparameters.defaultPartitionsPerExtractorTask	-1	INT	Number of partitions per task. If you set this parameter to -1, the number is based on available CPU cores.
sql.pipelineparameters.defaultExtractorCores	-1	INT	Number of cores to use in a task. If you set this parameter to -1, the number is based on available CPU cores.

Binary Format Parameter Name	Default	Data Type	Required Value and Description
sql.pipelineparameters.extract.binary.defaultEndianness	big	string	The endianness of transforms when you do not specify it in another way. Supported values are big or little.
sql.pipelineparameters.extract.binary.defaultCharset	ibm1047	string	Determines the default character set to use when you load data in the BINARY format.
sql.pipelineparameters.extract.binary.defaultAutoTrimPadding	true	Boolean	Determines the default for whether a padding character should be trimmed when decoding data in the BINARY format to a string.
sql.pipelineparameters.extract.binary.defaultPaddingCharacter	' '	CHAR	Determines the default padding character to trim from the end of a string when decoding data in the BINARY format. This parameter is effective only if the extract.binary.defaultAutoTrimPadding parameter is set to true.

Multiple Storage Clusters for Loading Data

Configuration Settings for Data Pipelines