CloudWatchLogs / Client / test_transformer

test_transformer#

CloudWatchLogs.Client.test_transformer(**kwargs)#

Use this operation to test a log transformer. You enter the transformer configuration and a set of log events to test with. The operation responds with an array that includes the original log events and the transformed versions.

Request Syntax

response = client.test_transformer(
    transformerConfig=[
        {
            'addKeys': {
                'entries': [
                    {
                        'key': 'string',
                        'value': 'string',
                        'overwriteIfExists': True|False
                    },
                ]
            },
            'copyValue': {
                'entries': [
                    {
                        'source': 'string',
                        'target': 'string',
                        'overwriteIfExists': True|False
                    },
                ]
            },
            'csv': {
                'quoteCharacter': 'string',
                'delimiter': 'string',
                'columns': [
                    'string',
                ],
                'source': 'string'
            },
            'dateTimeConverter': {
                'source': 'string',
                'target': 'string',
                'targetFormat': 'string',
                'matchPatterns': [
                    'string',
                ],
                'sourceTimezone': 'string',
                'targetTimezone': 'string',
                'locale': 'string'
            },
            'deleteKeys': {
                'withKeys': [
                    'string',
                ]
            },
            'grok': {
                'source': 'string',
                'match': 'string'
            },
            'listToMap': {
                'source': 'string',
                'key': 'string',
                'valueKey': 'string',
                'target': 'string',
                'flatten': True|False,
                'flattenedElement': 'first'|'last'
            },
            'lowerCaseString': {
                'withKeys': [
                    'string',
                ]
            },
            'moveKeys': {
                'entries': [
                    {
                        'source': 'string',
                        'target': 'string',
                        'overwriteIfExists': True|False
                    },
                ]
            },
            'parseCloudfront': {
                'source': 'string'
            },
            'parseJSON': {
                'source': 'string',
                'destination': 'string'
            },
            'parseKeyValue': {
                'source': 'string',
                'destination': 'string',
                'fieldDelimiter': 'string',
                'keyValueDelimiter': 'string',
                'keyPrefix': 'string',
                'nonMatchValue': 'string',
                'overwriteIfExists': True|False
            },
            'parseRoute53': {
                'source': 'string'
            },
            'parsePostgres': {
                'source': 'string'
            },
            'parseVPC': {
                'source': 'string'
            },
            'parseWAF': {
                'source': 'string'
            },
            'renameKeys': {
                'entries': [
                    {
                        'key': 'string',
                        'renameTo': 'string',
                        'overwriteIfExists': True|False
                    },
                ]
            },
            'splitString': {
                'entries': [
                    {
                        'source': 'string',
                        'delimiter': 'string'
                    },
                ]
            },
            'substituteString': {
                'entries': [
                    {
                        'source': 'string',
                        'from': 'string',
                        'to': 'string'
                    },
                ]
            },
            'trimString': {
                'withKeys': [
                    'string',
                ]
            },
            'typeConverter': {
                'entries': [
                    {
                        'key': 'string',
                        'type': 'boolean'|'integer'|'double'|'string'
                    },
                ]
            },
            'upperCaseString': {
                'withKeys': [
                    'string',
                ]
            }
        },
    ],
    logEventMessages=[
        'string',
    ]
)

Parameters:

transformerConfig (list) –
[REQUIRED]

This structure contains the configuration of this log transformer that you want to test. A log transformer is an array of processors, where each processor applies one type of transformation to the log events that are ingested.
- (dict) –
  
  This structure contains the information about one processor in a log transformer.
  - addKeys (dict) –
    
    Use this parameter to include the addKeys processor in your transformer.
    - entries (list) – [REQUIRED]
      
      An array of objects, where each object contains the information about one key to add to the log event.
      - (dict) –
        
        This object defines one key that will be added with the addKeys processor.
        
        key (string) – [REQUIRED]
        
        The key of the new entry to be added to the log event
        
        value (string) – [REQUIRED]
        
        The value of the new entry to be added to the log event
        
        overwriteIfExists (boolean) –
        
        Specifies whether to overwrite the value if the key already exists in the log event. If you omit this, the default is false.
  - copyValue (dict) –
    
    Use this parameter to include the copyValue processor in your transformer.
    - entries (list) – [REQUIRED]
      
      An array of CopyValueEntry objects, where each object contains the information about one field value to copy.
      - (dict) –
        
        This object defines one value to be copied with the copyValue processor.
        
        source (string) – [REQUIRED]
        
        The key to copy.
        
        target (string) – [REQUIRED]
        
        The key of the field to copy the value to.
        
        overwriteIfExists (boolean) –
        
        Specifies whether to overwrite the value if the destination key already exists. If you omit this, the default is false.
  - csv (dict) –
    
    Use this parameter to include the CSV processor in your transformer.
    - quoteCharacter (string) –
      
      The character used used as a text qualifier for a single column of data. If you omit this, the double quotation mark " character is used.
    - delimiter (string) –
      
      The character used to separate each column in the original comma-separated value log event. If you omit this, the processor looks for the comma , character as the delimiter.
    - columns (list) –
      
      An array of names to use for the columns in the transformed log event.
      
      If you omit this, default column names ( [column_1, column_2 ...]) are used.
      - (string) –
    - source (string) –
      
      The path to the field in the log event that has the comma separated values to be parsed. If you omit this value, the whole log message is processed.
  - dateTimeConverter (dict) –
    
    Use this parameter to include the datetimeConverter processor in your transformer.
    - source (string) – [REQUIRED]
      
      The key to apply the date conversion to.
    - target (string) – [REQUIRED]
      
      The JSON field to store the result in.
    - targetFormat (string) –
      
      The datetime format to use for the converted data in the target field.
      
      If you omit this, the default of yyyy-MM-dd'T'HH:mm:ss.SSS'Z is used.
    - matchPatterns (list) – [REQUIRED]
      
      A list of patterns to match against the source field.
      - (string) –
    - sourceTimezone (string) –
      
      The time zone of the source field. If you omit this, the default used is the UTC zone.
    - targetTimezone (string) –
      
      The time zone of the target field. If you omit this, the default used is the UTC zone.
    - locale (string) –
      
      The locale of the source field. If you omit this, the default of locale.ROOT is used.
  - deleteKeys (dict) –
    
    Use this parameter to include the deleteKeys processor in your transformer.
    - withKeys (list) – [REQUIRED]
      
      The list of keys to delete.
      - (string) –
  - grok (dict) –
    
    Use this parameter to include the grok processor in your transformer.
    - source (string) –
      
      The path to the field in the log event that you want to parse. If you omit this value, the whole log message is parsed.
    - match (string) – [REQUIRED]
      
      The grok pattern to match against the log event. For a list of supported grok patterns, see Supported grok patterns.
  - listToMap (dict) –
    
    Use this parameter to include the listToMap processor in your transformer.
    - source (string) – [REQUIRED]
      
      The key in the log event that has a list of objects that will be converted to a map.
    - key (string) – [REQUIRED]
      
      The key of the field to be extracted as keys in the generated map
    - valueKey (string) –
      
      If this is specified, the values that you specify in this parameter will be extracted from the source objects and put into the values of the generated map. Otherwise, original objects in the source list will be put into the values of the generated map.
    - target (string) –
      
      The key of the field that will hold the generated map
    - flatten (boolean) –
      
      A Boolean value to indicate whether the list will be flattened into single items. Specify true to flatten the list. The default is false
    - flattenedElement (string) –
      
      If you set flatten to true, use flattenedElement to specify which element, first or last, to keep.
      
      You must specify this parameter if flatten is true
  - lowerCaseString (dict) –
    
    Use this parameter to include the lowerCaseString processor in your transformer.
    - withKeys (list) – [REQUIRED]
      
      The array caontaining the keys of the fields to convert to lowercase.
      - (string) –
  - moveKeys (dict) –
    
    Use this parameter to include the moveKeys processor in your transformer.
    - entries (list) – [REQUIRED]
      
      An array of objects, where each object contains the information about one key to move.
      - (dict) –
        
        This object defines one key that will be moved with the moveKey processor.
        
        source (string) – [REQUIRED]
        
        The key to move.
        
        target (string) – [REQUIRED]
        
        The key to move to.
        
        overwriteIfExists (boolean) –
        
        Specifies whether to overwrite the value if the destination key already exists. If you omit this, the default is false.
  - parseCloudfront (dict) –
    
    Use this parameter to include the parseCloudfront processor in your transformer.
    
    If you use this processor, it must be the first processor in your transformer.
    - source (string) –
      
      Omit this parameter and the whole log message will be processed by this processor. No other value than @message is allowed for source.
  - parseJSON (dict) –
    
    Use this parameter to include the parseJSON processor in your transformer.
    - source (string) –
      
      Path to the field in the log event that will be parsed. Use dot notation to access child fields. For example, store.book
    - destination (string) –
      
      The location to put the parsed key value pair into. If you omit this parameter, it is placed under the root node.
  - parseKeyValue (dict) –
    
    Use this parameter to include the parseKeyValue processor in your transformer.
    - source (string) –
      
      Path to the field in the log event that will be parsed. Use dot notation to access child fields. For example, store.book
    - destination (string) –
      
      The destination field to put the extracted key-value pairs into
    - fieldDelimiter (string) –
      
      The field delimiter string that is used between key-value pairs in the original log events. If you omit this, the ampersand & character is used.
    - keyValueDelimiter (string) –
      
      The delimiter string to use between the key and value in each pair in the transformed log event.
      
      If you omit this, the equal = character is used.
    - keyPrefix (string) –
      
      If you want to add a prefix to all transformed keys, specify it here.
    - nonMatchValue (string) –
      
      A value to insert into the value field in the result, when a key-value pair is not successfully split.
    - overwriteIfExists (boolean) –
      
      Specifies whether to overwrite the value if the destination key already exists. If you omit this, the default is false.
  - parseRoute53 (dict) –
    
    Use this parameter to include the parseRoute53 processor in your transformer.
    
    If you use this processor, it must be the first processor in your transformer.
    - source (string) –
      
      Omit this parameter and the whole log message will be processed by this processor. No other value than @message is allowed for source.
  - parsePostgres (dict) –
    
    Use this parameter to include the parsePostGres processor in your transformer.
    
    If you use this processor, it must be the first processor in your transformer.
    - source (string) –
      
      Omit this parameter and the whole log message will be processed by this processor. No other value than @message is allowed for source.
  - parseVPC (dict) –
    
    Use this parameter to include the parseVPC processor in your transformer.
    
    If you use this processor, it must be the first processor in your transformer.
    - source (string) –
      
      Omit this parameter and the whole log message will be processed by this processor. No other value than @message is allowed for source.
  - parseWAF (dict) –
    
    Use this parameter to include the parseWAF processor in your transformer.
    
    If you use this processor, it must be the first processor in your transformer.
    - source (string) –
      
      Omit this parameter and the whole log message will be processed by this processor. No other value than @message is allowed for source.
  - renameKeys (dict) –
    
    Use this parameter to include the renameKeys processor in your transformer.
    - entries (list) – [REQUIRED]
      
      An array of RenameKeyEntry objects, where each object contains the information about a single key to rename.
      - (dict) –
        
        This object defines one key that will be renamed with the renameKey processor.
        
        key (string) – [REQUIRED]
        
        The key to rename
        
        renameTo (string) – [REQUIRED]
        
        The string to use for the new key name
        
        overwriteIfExists (boolean) –
        
        Specifies whether to overwrite the existing value if the destination key already exists. The default is false
  - splitString (dict) –
    
    Use this parameter to include the splitString processor in your transformer.
    - entries (list) – [REQUIRED]
      
      An array of SplitStringEntry objects, where each object contains the information about one field to split.
      - (dict) –
        
        This object defines one log field that will be split with the splitString processor.
        
        source (string) – [REQUIRED]
        
        The key of the field to split.
        
        delimiter (string) – [REQUIRED]
        
        The separator characters to split the string entry on.
  - substituteString (dict) –
    
    Use this parameter to include the substituteString processor in your transformer.
    - entries (list) – [REQUIRED]
      
      An array of objects, where each object contains the information about one key to match and replace.
      - (dict) –
        
        This object defines one log field key that will be replaced using the substituteString processor.
        
        source (string) – [REQUIRED]
        
        The key to modify
        
        from (string) – [REQUIRED]
        
        The regular expression string to be replaced. Special regex characters such as [ and ] must be escaped using \ when using double quotes and with when using single quotes. For more information, see Class Pattern on the Oracle web site.
        
        to (string) – [REQUIRED]
        
        The string to be substituted for each match of from
  - trimString (dict) –
    
    Use this parameter to include the trimString processor in your transformer.
    - withKeys (list) – [REQUIRED]
      
      The array containing the keys of the fields to trim.
      - (string) –
  - typeConverter (dict) –
    
    Use this parameter to include the typeConverter processor in your transformer.
    - entries (list) – [REQUIRED]
      
      An array of TypeConverterEntry objects, where each object contains the information about one field to change the type of.
      - (dict) –
        
        This object defines one value type that will be converted using the typeConverter processor.
        
        key (string) – [REQUIRED]
        
        The key with the value that is to be converted to a different type.
        
        type (string) – [REQUIRED]
        
        The type to convert the field value to. Valid values are integer, double, string and boolean.
  - upperCaseString (dict) –
    
    Use this parameter to include the upperCaseString processor in your transformer.
    - withKeys (list) – [REQUIRED]
      
      The array of containing the keys of the field to convert to uppercase.
      - (string) –
logEventMessages (list) –
[REQUIRED]

An array of the raw log events that you want to use to test this transformer.
- (string) –

Return type:

dict

Returns:

Response Syntax

{
    'transformedLogs': [
        {
            'eventNumber': 123,
            'eventMessage': 'string',
            'transformedEventMessage': 'string'
        },
    ]
}

Response Structure

(dict) –
- transformedLogs (list) –
  
  An array where each member of the array includes both the original version and the transformed version of one of the log events that you input.
  - (dict) –
    
    This structure contains information for one log event that has been processed by a log transformer.
    - eventNumber (integer) –
      
      The event number.
    - eventMessage (string) –
      
      The original log event message before it was transformed.
    - transformedEventMessage (string) –
      
      The log event message after being transformed.

Exceptions

CloudWatchLogs.Client.exceptions.InvalidParameterException
CloudWatchLogs.Client.exceptions.ServiceUnavailableException
CloudWatchLogs.Client.exceptions.InvalidOperationException