Articles on: Connectors

Function Expressions - Arrays

Overview


Objects represent a special data type that is mutable and can be used to store a collection of data. Arrays are a special type of variable that is also mutable and can also be used to store a list of values.

List of Functions


Join


Concatenates all the items of an array into a string, using the specified separator between each item
Syntax: join(array, separator)


Length


Returns the number of item in an array
Syntax: length(array)


Keys


Returns an array of a given objects or arrays properties
Syntax: keys(object)


Slice


Returns a new array containing only selected items
Syntax: slice(array, start, end)


Merge


Merges two arrays into one array.
Syntax: merge(array1, array2)


Keymove


Returns an array by moving a key value to a child or parent node.
Syntax: keyMove(array, source, target)


Contains


Verify if an array contains the value.
Syntax: arraycontains(array, value)


ContainsExtended


Verify if an array of objects contains the value.
Syntax: arraycontainsextended(array, key, value)



Remove


Removes value specified in the parameters of an array. Effective only on primitive arrays of text or numbers
Syntax: remove(array, [value1,value2,...])


Add


Add value specified in the parameters to an array and returns that array
Syntax: adds(array, [value1,value2,...])


AddDic


Add a key object to all the item of an array of objects.
Syntax: AddDic(array, key, value)


getColumn


Return a column from an array. <Column> is the name or index or the target column in the array.
Syntax: getColumn(array, column)

----------
arr = ['111', '222', '3333']
getColumn(arr, 0)
output:
---------
['111', '222', '3333']
arr = [['111', '222', '3333'], ['aaa', 'bbb', 'ccc'], ['xxx', 'yyy']]
getColumn(arr, 0)
getColumn(arr, 2)
output
-----------
["111","aaa","xxx"]
["3333","ccc",""]
arr = [{'country': 'australia', 'population': "26M"}, {'country': 'usa', 'population': "400M"}]
getColumn(arr, 'country')
output:
----------
["australia","usa"]


columnLabels


Return the column labels of an array
Syntax: columnLabels(array)


Map


Returns a primitive array containing object picked from a complex array. single source key with empty target key will return the selected source key values at root position.
Syntax: map(complex array, source key paths separated by comma, target key paths separated by comma)

map an array schema with another schema structure


Array of Objects --> to Array of Objects


map(input_array, ['name','address.country','contacts'], ['name','country','contacts'])
output:
----------
TBD

OR if you leave the target empty, the same nested position of the source keys will be returned
map(input_array, ['name','address.country','contacts'], [])
output:
----------
TBD


Array of Objects --> Single key at root


if it's a single source key and you leave the target empty then the source key values will return at the root level of the array
map(input_array, ['name'], [])
output
---------
TBD


Array of Objects --> to Array of Arrays


Define the input array path to the array of arrays and keep the source + target empty\or with single value
map(input_array.path, [], [])
output:
----------
[['value1', 'value2'], ['value3', 'value4']]
map(input_array.path, [], ['keyTarget'])
output:
----------
{'ketTarget': [['value1', 'value2'], ['value3', 'value4']]}


GetNameValue


Returns a value from an array of key name string objects. Example: [{'keyName' : ' filterName', 'keyvalue', 'filterValue'}]
Syntax: getnamevalue(array, KeyName, FilterName, KeyValue)


GetKeyValue


Returns a value from an array of key/value objects. Example: [{'name' : 'value'}]
Syntax: getkeyvalue(array, KeyName)


IndexOf


Returns item from array at index
Syntax: IndexofArray(Array, index)


Sort


Sorts values of an array. Order is either 'asc' or 'desc'. Use 'Key' argument to access properties inside complex objects. Use raw variable names for keys. To access nested properties, use dot notation. First item in an array is index 1
Syntax: sort(array, [order], [key])


Distinct


Removes duplicates inside an array. Use 'key' argument to access properties inside complex objects. To access nested properties, use dot notation, First item in an array is index 1
Syntax: distinct(array, [key])


Reverse


Reverse items of an array.
Syntax: reverse(array)


Substruct Array


Substruct items of an array
Syntax: substructarray(<source array>, <source key>, <compare array>, <compare key>)


Filter


Filter array by value
Syntax: filterArray(<source array>, <value>, <operator: ANY,ALL,EXACT>)


Filter array by key value
Key Filter
Syntax: filterArrayByKey(<source array>,<source key>, <value>, <operator: ANY,ALL,EXACT>)



Join Arrays


join Arrays with a database-style. Merge Data Frame or named Series objects with a database-style.
Syntax: joinArray(<left array>, <right array>, <how: inner, left, right, outer>, <left: index or keyName>, <right: index or keyName>)

join https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.merge.html

how: inner, left, right, outer
left & right: the index of the field to join on in case of array of array, or the key name in case of array of object
NB: always the result will return as array of array

Join Arrays


join Arrays with a database-style. Merge Data Frame or named Series objects with a database-style.
Syntax: joinArray(<left array>, <right array>, <how: inner, left, right, outer>, <left: index or keyName>, <right: index or keyName>)


Array of Array Case
data1 = [['20210401', '1', '2', '3'], ['20210402', '11', '22', '33']]
data2 = [['20210401', '111', '222', '333'], ['20210402', '1111', '2222', '3333']]
merge by date -->
joinArray(data1, data2, inner, 0, 0)
results:
[["20210401","1","2","3","111","222","333"],
["20210402","11","22","33","1111","2222","3333"]]


Concat Array


join 2 array based on row index style
Syntax: concatArray(<array1>,<array2>)




concatArray(?,?)

ability to concat "array of arrays" and "array of objects" together
output: array of arrays

concatArray(<array1>,<array2>)
arr1 = [[1, 2, 3], [11, 22, 33], [111, 222, 333]]
arr_obj1 = arr_obj = [{"col1": "20210413", "col2": "2", "col3": "3"}, {"col1": "20210414", "col2": "22", "col3": "33"}, {"col1": "20210415", "col2": "222", "col3": "333"}]

concatArray(arr1, arr_obj1)

output:
---------
[[1,2,3,"20210413","2","3"],[11,22,33,"20210414","22","33"],
[111,222,333,"20210415","222","333"]]


Add Index


Add an incremental index column to the array. e.g: 0,1,2,3.....

Syntax: AddArrayIndex(<array>)


CombineColumns


Combining two columns in a new column that contains an element for each corresponding elements from both columns.

Syntax: combineColumns(<column1>,<column2>,<separator>)


column1 = ['A', 'B', 'C']
column2 = ['11', '22', '33']
separator = '_'
combineColumns(column1, column2, separator)
output:
---------
['A_11', 'B_22', 'C_33']


Split Column


Splits the string in a column from the beginning, at the specified delimiter string.

Syntax: splitColumn(<column>,<delimiter>)


column = ['A_11', 'B_22', 'C_33']
delimiter = '_'
splitColumn(column, delimiter)
output:
---------
[['A', '11'],['B', '22'], ['C', '33']]


dropColumn


Remove column by specifying by specifying directly index or column name.

Syntax: dropColumn(<array>,<column>)


array = ['A', 'B', 'C']
column = 0
dropColumn(array, column)
output:
---------
['B', 'C']
array = [['A', 'AA'], ['B', 'BB'], ['C', 'CC']]
column = 0
dropColumn(array, column)

output:
---------
[['B', 'BB'], ['C', 'CC']]
array = [{'name': 'A', 'value': '11'}, {'name': 'B', 'value': '22'}]
column = 'name'
dropColumn(array, column)

output:
---------
['value': '11'}, {'value': '22'}]


Function Arrays


Suppose we are going to implement function to the below Jason payload. we will call it input_array
[
{
'name': 'Peter',
'age': 41,
'address': {
  ' country': 'Australia',
   'city': 'Sydney'
},
 'contacts': [111,222,3333],
'jobs': [
{
'role': 'architect',
'position': 'senior'
},
{
   'role': 'manager',
   'position': 'CEO'
      }
  ]
 },
{
        'name': 'Maya',
        'age': 35,
        'address': {
            'country': 'Lebanon',
            'city': 'Beirut'
        },
        'contacts': [4444,55555,66666],
        'jobs': []
      }
    ]


csvTojson


Convert csv to json array. sourcetype=file|plain|binary. skipheader=0|1

Syntax: csvTojson(<sourcetype>, <data>, <delimiter>, <skipheader>)


jsonTocsv


Convert array of json json to csv
Syntax: Syntax: jsonTocsv(<source>, <separator>)

Updated on: 25/07/2022

Was this article helpful?

Share your feedback

Cancel

Thank you!