Glue / Client / list_table_optimizer_runs

list_table_optimizer_runs¶

Glue.Client.list_table_optimizer_runs(**kwargs)¶

Lists the history of previous optimizer runs for a specific table.

Request Syntax

response = client.list_table_optimizer_runs(
    CatalogId='string',
    DatabaseName='string',
    TableName='string',
    Type='compaction'|'retention'|'orphan_file_deletion',
    MaxResults=123,
    NextToken='string'
)

Parameters:

CatalogId (string) –
[REQUIRED]

The Catalog ID of the table.
DatabaseName (string) –
[REQUIRED]

The name of the database in the catalog in which the table resides.
TableName (string) –
[REQUIRED]

The name of the table.
Type (string) –
[REQUIRED]

The type of table optimizer.
MaxResults (integer) – The maximum number of optimizer runs to return on each call.
NextToken (string) – A continuation token, if this is a continuation call.

Return type:

dict

Returns:

Response Syntax

{
    'CatalogId': 'string',
    'DatabaseName': 'string',
    'TableName': 'string',
    'NextToken': 'string',
    'TableOptimizerRuns': [
        {
            'eventType': 'starting'|'completed'|'failed'|'in_progress',
            'startTimestamp': datetime(2015, 1, 1),
            'endTimestamp': datetime(2015, 1, 1),
            'metrics': {
                'NumberOfBytesCompacted': 'string',
                'NumberOfFilesCompacted': 'string',
                'NumberOfDpus': 'string',
                'JobDurationInHour': 'string'
            },
            'error': 'string',
            'compactionMetrics': {
                'IcebergMetrics': {
                    'NumberOfBytesCompacted': 123,
                    'NumberOfFilesCompacted': 123,
                    'DpuHours': 123.0,
                    'NumberOfDpus': 123,
                    'JobDurationInHour': 123.0
                }
            },
            'compactionStrategy': 'binpack'|'sort'|'z-order',
            'retentionMetrics': {
                'IcebergMetrics': {
                    'NumberOfDataFilesDeleted': 123,
                    'NumberOfManifestFilesDeleted': 123,
                    'NumberOfManifestListsDeleted': 123,
                    'DpuHours': 123.0,
                    'NumberOfDpus': 123,
                    'JobDurationInHour': 123.0
                }
            },
            'orphanFileDeletionMetrics': {
                'IcebergMetrics': {
                    'NumberOfOrphanFilesDeleted': 123,
                    'DpuHours': 123.0,
                    'NumberOfDpus': 123,
                    'JobDurationInHour': 123.0
                }
            }
        },
    ]
}

Response Structure

(dict) –
- CatalogId (string) –
  
  The Catalog ID of the table.
- DatabaseName (string) –
  
  The name of the database in the catalog in which the table resides.
- TableName (string) –
  
  The name of the table.
- NextToken (string) –
  
  A continuation token for paginating the returned list of optimizer runs, returned if the current segment of the list is not the last.
- TableOptimizerRuns (list) –
  
  A list of the optimizer runs associated with a table.
  - (dict) –
    
    Contains details for a table optimizer run.
    - eventType (string) –
      
      An event type representing the status of the table optimizer run.
    - startTimestamp (datetime) –
      
      Represents the epoch timestamp at which the compaction job was started within Lake Formation.
    - endTimestamp (datetime) –
      
      Represents the epoch timestamp at which the compaction job ended.
    - metrics (dict) –
      
      A RunMetrics object containing metrics for the optimizer run.
      
      This member is deprecated. See the individual metric members for compaction, retention, and orphan file deletion.
      - NumberOfBytesCompacted (string) –
        
        The number of bytes removed by the compaction job run.
      - NumberOfFilesCompacted (string) –
        
        The number of files removed by the compaction job run.
      - NumberOfDpus (string) –
        
        The number of DPUs consumed by the job, rounded up to the nearest whole number.
      - JobDurationInHour (string) –
        
        The duration of the job in hours.
    - error (string) –
      
      An error that occured during the optimizer run.
    - compactionMetrics (dict) –
      
      A CompactionMetrics object containing metrics for the optimizer run.
      - IcebergMetrics (dict) –
        
        A structure containing the Iceberg compaction metrics for the optimizer run.
        
        NumberOfBytesCompacted (integer) –
        
        The number of bytes removed by the compaction job run.
        
        NumberOfFilesCompacted (integer) –
        
        The number of files removed by the compaction job run.
        
        DpuHours (float) –
        
        The number of DPU hours consumed by the job.
        
        NumberOfDpus (integer) –
        
        The number of DPUs consumed by the job, rounded up to the nearest whole number.
        
        JobDurationInHour (float) –
        
        The duration of the job in hours.
    - compactionStrategy (string) –
      
      The strategy used for the compaction run. Indicates which algorithm was applied to determine how files were selected and combined during the compaction process. Valid values are:
      - binpack: Combines small files into larger files, typically targeting sizes over 100MB, while applying any pending deletes. This is the recommended compaction strategy for most use cases.
      - sort: Organizes data based on specified columns which are sorted hierarchically during compaction, improving query performance for filtered operations. This strategy is recommended when your queries frequently filter on specific columns. To use this strategy, you must first define a sort order in your Iceberg table properties using the sort_order table property.
      - z-order: Optimizes data organization by blending multiple attributes into a single scalar value that can be used for sorting, allowing efficient querying across multiple dimensions. This strategy is recommended when you need to query data across multiple dimensions simultaneously. To use this strategy, you must first define a sort order in your Iceberg table properties using the sort_order table property.
    - retentionMetrics (dict) –
      
      A RetentionMetrics object containing metrics for the optimizer run.
      - IcebergMetrics (dict) –
        
        A structure containing the Iceberg retention metrics for the optimizer run.
        
        NumberOfDataFilesDeleted (integer) –
        
        The number of data files deleted by the retention job run.
        
        NumberOfManifestFilesDeleted (integer) –
        
        The number of manifest files deleted by the retention job run.
        
        NumberOfManifestListsDeleted (integer) –
        
        The number of manifest lists deleted by the retention job run.
        
        DpuHours (float) –
        
        The number of DPU hours consumed by the job.
        
        NumberOfDpus (integer) –
        
        The number of DPUs consumed by the job, rounded up to the nearest whole number.
        
        JobDurationInHour (float) –
        
        The duration of the job in hours.
    - orphanFileDeletionMetrics (dict) –
      
      An OrphanFileDeletionMetrics object containing metrics for the optimizer run.
      - IcebergMetrics (dict) –
        
        A structure containing the Iceberg orphan file deletion metrics for the optimizer run.
        
        NumberOfOrphanFilesDeleted (integer) –
        
        The number of orphan files deleted by the orphan file deletion job run.
        
        DpuHours (float) –
        
        The number of DPU hours consumed by the job.
        
        NumberOfDpus (integer) –
        
        The number of DPUs consumed by the job, rounded up to the nearest whole number.
        
        JobDurationInHour (float) –
        
        The duration of the job in hours.

Exceptions

Glue.Client.exceptions.EntityNotFoundException
Glue.Client.exceptions.AccessDeniedException
Glue.Client.exceptions.InvalidInputException
Glue.Client.exceptions.ValidationException
Glue.Client.exceptions.InternalServiceException
Glue.Client.exceptions.ThrottlingException