GlueDataBrew

Table of Contents

Client

class GlueDataBrew.Client

A low-level client representing AWS Glue DataBrew

AWS Glue DataBrew is a visual, cloud-scale data-preparation service. DataBrew simplifies data preparation tasks, targeting data issues that are hard to spot and time-consuming to fix. DataBrew empowers users of all technical levels to visualize the data and perform one-click data transformations, with no coding required.

import boto3

client = boto3.client('databrew')

These are the available methods:

batch_delete_recipe_version(**kwargs)

Deletes one or more versions of a recipe at a time.

See also: AWS API Documentation

Request Syntax

response = client.batch_delete_recipe_version(
    Name='string',
    RecipeVersions=[
        'string',
    ]
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the recipe to be modified.

  • RecipeVersions (list) --

    [REQUIRED]

    An array of version identifiers to be deleted.

    • (string) --
Return type

dict

Returns

Response Syntax

{
    'Name': 'string',
    'Errors': [
        {
            'ErrorCode': 'string',
            'ErrorMessage': 'string',
            'RecipeVersion': 'string'
        },
    ]
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the recipe that was modified.

    • Errors (list) --

      Errors, if any, that were encountered when deleting the recipe versions.

      • (dict) --

        Represents any errors encountered when attempting to delete multiple recipe versions.

        • ErrorCode (string) --

          The HTTP status code for the error.

        • ErrorMessage (string) --

          The text of the error message.

        • RecipeVersion (string) --

          The identifier for the recipe version associated with this error.

Exceptions

  • GlueDataBrew.Client.exceptions.ConflictException
  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
can_paginate(operation_name)

Check if an operation can be paginated.

Parameters
operation_name (string) -- The operation name. This is the same name as the method name on the client. For example, if the method name is create_foo, and you'd normally invoke the operation as client.create_foo(**kwargs), if the create_foo operation can be paginated, you can use the call client.get_paginator("create_foo").
Returns
True if the operation can be paginated, False otherwise.
create_dataset(**kwargs)

Creates a new AWS Glue DataBrew dataset for this AWS account.

See also: AWS API Documentation

Request Syntax

response = client.create_dataset(
    Name='string',
    FormatOptions={
        'Json': {
            'MultiLine': True|False
        },
        'Excel': {
            'SheetNames': [
                'string',
            ],
            'SheetIndexes': [
                123,
            ]
        }
    },
    Input={
        'S3InputDefinition': {
            'Bucket': 'string',
            'Key': 'string'
        },
        'DataCatalogInputDefinition': {
            'CatalogId': 'string',
            'DatabaseName': 'string',
            'TableName': 'string',
            'TempDirectory': {
                'Bucket': 'string',
                'Key': 'string'
            }
        }
    },
    Tags={
        'string': 'string'
    }
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the dataset to be created.

  • FormatOptions (dict) --

    Options that define how Microsoft Excel input is to be interpreted by DataBrew.

    • Json (dict) --

      Options that define how JSON input is to be interpreted by DataBrew.

      • MultiLine (boolean) --

        A value that specifies whether JSON input contains embedded new line characters.

    • Excel (dict) --

      Options that define how Excel input is to be interpreted by DataBrew.

      • SheetNames (list) --

        Specifies one or more named sheets in the Excel file, which will be included in the dataset.

        • (string) --
      • SheetIndexes (list) --

        Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.

        • (integer) --
  • Input (dict) --

    [REQUIRED]

    Information on how AWS Glue DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.

    • S3InputDefinition (dict) --

      The Amazon S3 location where the data is stored.

      • Bucket (string) -- [REQUIRED]

        The S3 bucket name.

      • Key (string) --

        The unique name of the object in the bucket.

    • DataCatalogInputDefinition (dict) --

      The AWS Glue Data Catalog parameters for the data.

      • CatalogId (string) --

        The unique identifier of the AWS account that holds the Data Catalog that stores the data.

      • DatabaseName (string) -- [REQUIRED]

        The name of a database in the Data Catalog.

      • TableName (string) -- [REQUIRED]

        The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.

      • TempDirectory (dict) --

        An Amazon location that AWS Glue Data Catalog can use as a temporary directory.

        • Bucket (string) -- [REQUIRED]

          The S3 bucket name.

        • Key (string) --

          The unique name of the object in the bucket.

  • Tags (dict) --

    Metadata tags to apply to this dataset.

    • (string) --
      • (string) --
Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the dataset that you created.

Exceptions

  • GlueDataBrew.Client.exceptions.AccessDeniedException
  • GlueDataBrew.Client.exceptions.ConflictException
  • GlueDataBrew.Client.exceptions.ServiceQuotaExceededException
  • GlueDataBrew.Client.exceptions.ValidationException
create_profile_job(**kwargs)

Creates a new job to profile an AWS Glue DataBrew dataset that exists in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.create_profile_job(
    DatasetName='string',
    EncryptionKeyArn='string',
    EncryptionMode='SSE-KMS'|'SSE-S3',
    Name='string',
    LogSubscription='ENABLE'|'DISABLE',
    MaxCapacity=123,
    MaxRetries=123,
    OutputLocation={
        'Bucket': 'string',
        'Key': 'string'
    },
    RoleArn='string',
    Tags={
        'string': 'string'
    },
    Timeout=123
)
Parameters
  • DatasetName (string) --

    [REQUIRED]

    The name of the dataset that this job is to act upon.

  • EncryptionKeyArn (string) -- The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
  • EncryptionMode (string) --

    The encryption mode for the job, which can be one of the following:

    • SSE-KMS - para>``SSE-KMS`` - server-side encryption with AWS KMS-managed keys.
    • SSE-S3 - Server-side encryption with keys managed by Amazon S3.
  • Name (string) --

    [REQUIRED]

    The name of the job to be created.

  • LogSubscription (string) -- A value that enables or disables Amazon CloudWatch logging for the current AWS account. If logging is enabled, CloudWatch writes one log stream for each job run.
  • MaxCapacity (integer) -- The maximum number of nodes that DataBrew can use when the job processes data.
  • MaxRetries (integer) -- The maximum number of times to retry the job after a job run fails.
  • OutputLocation (dict) --

    [REQUIRED]

    An Amazon S3 location (bucket name an object key) where DataBrew can read input data, or write output from a job.

    • Bucket (string) -- [REQUIRED]

      The S3 bucket name.

    • Key (string) --

      The unique name of the object in the bucket.

  • RoleArn (string) --

    [REQUIRED]

    The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.

  • Tags (dict) --

    Metadata tags to apply to this job.

    • (string) --
      • (string) --
  • Timeout (integer) -- The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .
Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the job that was created.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
create_project(**kwargs)

Creates a new AWS Glue DataBrew project in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.create_project(
    DatasetName='string',
    Name='string',
    RecipeName='string',
    Sample={
        'Size': 123,
        'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
    },
    RoleArn='string',
    Tags={
        'string': 'string'
    }
)
Parameters
  • DatasetName (string) --

    [REQUIRED]

    The name of the dataset to associate this project with.

  • Name (string) --

    [REQUIRED]

    A unique name for the new project.

  • RecipeName (string) --

    [REQUIRED]

    The name of an existing recipe to associate with the project.

  • Sample (dict) --

    Represents the sample size and sampling type for AWS Glue DataBrew to use for interactive data analysis.

    • Size (integer) --

      The number of rows in the sample.

    • Type (string) -- [REQUIRED]

      The way in which DataBrew obtains rows from a dataset.

  • RoleArn (string) --

    [REQUIRED]

    The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.

  • Tags (dict) --

    Metadata tags to apply to this project.

    • (string) --
      • (string) --
Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the project that you created.

Exceptions

  • GlueDataBrew.Client.exceptions.ConflictException
  • GlueDataBrew.Client.exceptions.InternalServerException
  • GlueDataBrew.Client.exceptions.ServiceQuotaExceededException
  • GlueDataBrew.Client.exceptions.ValidationException
create_recipe(**kwargs)

Creates a new AWS Glue DataBrew recipe for the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.create_recipe(
    Description='string',
    Name='string',
    Steps=[
        {
            'Action': {
                'Operation': 'string',
                'Parameters': {
                    'string': 'string'
                }
            },
            'ConditionExpressions': [
                {
                    'Condition': 'string',
                    'Value': 'string',
                    'TargetColumn': 'string'
                },
            ]
        },
    ],
    Tags={
        'string': 'string'
    }
)
Parameters
  • Description (string) -- A description for the recipe.
  • Name (string) --

    [REQUIRED]

    A unique name for the recipe.

  • Steps (list) --

    [REQUIRED]

    An array containing the steps to be performed by the recipe. Each recipe step consists of one recipe action and (optionally) an array of condition expressions.

    • (dict) --

      Represents a single step to be performed in an AWS Glue DataBrew recipe.

      • Action (dict) -- [REQUIRED]

        The particular action to be performed in the recipe step.

        • Operation (string) -- [REQUIRED]

          The name of a valid DataBrew transformation to be performed on the data.

        • Parameters (dict) --

          Contextual parameters for the transformation.

          • (string) --
            • (string) --
      • ConditionExpressions (list) --

        One or more conditions that must be met, in order for the recipe step to succeed.

        Note

        All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.

        • (dict) --

          Represents an individual condition that evaluates to true or false.

          Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.

          If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.

          • Condition (string) -- [REQUIRED]

            A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .

          • Value (string) --

            A value that the condition must evaluate to for the condition to succeed.

          • TargetColumn (string) -- [REQUIRED]

            A column to apply this condition to, within an AWS Glue DataBrew dataset.

  • Tags (dict) --

    Metadata tags to apply to this recipe.

    • (string) --
      • (string) --
Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the recipe that you created.

Exceptions

  • GlueDataBrew.Client.exceptions.ConflictException
  • GlueDataBrew.Client.exceptions.ServiceQuotaExceededException
  • GlueDataBrew.Client.exceptions.ValidationException
create_recipe_job(**kwargs)

Creates a new job for an existing AWS Glue DataBrew recipe in the current AWS account. You can create a standalone job using either a project, or a combination of a recipe and a dataset.

See also: AWS API Documentation

Request Syntax

response = client.create_recipe_job(
    DatasetName='string',
    EncryptionKeyArn='string',
    EncryptionMode='SSE-KMS'|'SSE-S3',
    Name='string',
    LogSubscription='ENABLE'|'DISABLE',
    MaxCapacity=123,
    MaxRetries=123,
    Outputs=[
        {
            'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
            'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
            'PartitionColumns': [
                'string',
            ],
            'Location': {
                'Bucket': 'string',
                'Key': 'string'
            },
            'Overwrite': True|False
        },
    ],
    ProjectName='string',
    RecipeReference={
        'Name': 'string',
        'RecipeVersion': 'string'
    },
    RoleArn='string',
    Tags={
        'string': 'string'
    },
    Timeout=123
)
Parameters
  • DatasetName (string) -- The name of the dataset that this job processes.
  • EncryptionKeyArn (string) -- The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
  • EncryptionMode (string) --

    The encryption mode for the job, which can be one of the following:

    • SSE-KMS - Server-side encryption with AWS KMS-managed keys.
    • SSE-S3 - Server-side encryption with keys managed by Amazon S3.
  • Name (string) --

    [REQUIRED]

    A unique name for the job.

  • LogSubscription (string) -- A value that enables or disables Amazon CloudWatch logging for the current AWS account. If logging is enabled, CloudWatch writes one log stream for each job run.
  • MaxCapacity (integer) -- The maximum number of nodes that DataBrew can consume when the job processes data.
  • MaxRetries (integer) -- The maximum number of times to retry the job after a job run fails.
  • Outputs (list) --

    [REQUIRED]

    One or more artifacts that represent the output from running the job.

    • (dict) --

      Represents individual output from a particular job run.

      • CompressionFormat (string) --

        The compression algorithm used to compress the output text of the job.

      • Format (string) --

        The data format of the output of the job.

      • PartitionColumns (list) --

        The names of one or more partition columns for the output of the job.

        • (string) --
      • Location (dict) -- [REQUIRED]

        The location in Amazon S3 where the job writes its output.

        • Bucket (string) -- [REQUIRED]

          The S3 bucket name.

        • Key (string) --

          The unique name of the object in the bucket.

      • Overwrite (boolean) --

        A value that, if true, means that any data in the location specified for output is overwritten with new output.

  • ProjectName (string) -- Either the name of an existing project, or a combination of a recipe and a dataset to associate with the recipe.
  • RecipeReference (dict) --

    Represents all of the attributes of an AWS Glue DataBrew recipe.

    • Name (string) -- [REQUIRED]

      The name of the recipe.

    • RecipeVersion (string) --

      The identifier for the version for the recipe.

  • RoleArn (string) --

    [REQUIRED]

    The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.

  • Tags (dict) --

    Metadata tags to apply to this job dataset.

    • (string) --
      • (string) --
  • Timeout (integer) -- The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .
Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the job that you created.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
create_schedule(**kwargs)

Creates a new schedule for one or more AWS Glue DataBrew jobs. Jobs can be run at a specific date and time, or at regular intervals.

See also: AWS API Documentation

Request Syntax

response = client.create_schedule(
    JobNames=[
        'string',
    ],
    CronExpression='string',
    Tags={
        'string': 'string'
    },
    Name='string'
)
Parameters
  • JobNames (list) --

    The name or names of one or more jobs to be run.

    • (string) --
  • CronExpression (string) --

    [REQUIRED]

    The date or dates and time or times, in cron format, when the jobs are to be run.

  • Tags (dict) --

    Metadata tags to apply to this schedule.

    • (string) --
      • (string) --
  • Name (string) --

    [REQUIRED]

    A unique name for the schedule.

Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the schedule that was created.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
delete_dataset(**kwargs)

Deletes a dataset from AWS Glue DataBrew.

See also: AWS API Documentation

Request Syntax

response = client.delete_dataset(
    Name='string'
)
Parameters
Name (string) --

[REQUIRED]

The name of the dataset to be deleted.

Return type
dict
Returns
Response Syntax
{
    'Name': 'string'
}

Response Structure

  • (dict) --
    • Name (string) --

      The name of the dataset that you deleted.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
delete_job(**kwargs)

Deletes the specified AWS Glue DataBrew job from the current AWS account. The job can be for a recipe or for a profile.

See also: AWS API Documentation

Request Syntax

response = client.delete_job(
    Name='string'
)
Parameters
Name (string) --

[REQUIRED]

The name of the job to be deleted.

Return type
dict
Returns
Response Syntax
{
    'Name': 'string'
}

Response Structure

  • (dict) --
    • Name (string) --

      The name of the job that you deleted.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
delete_project(**kwargs)

Deletes an existing AWS Glue DataBrew project from the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.delete_project(
    Name='string'
)
Parameters
Name (string) --

[REQUIRED]

The name of the project to be deleted.

Return type
dict
Returns
Response Syntax
{
    'Name': 'string'
}

Response Structure

  • (dict) --
    • Name (string) --

      The name of the project that you deleted.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
delete_recipe_version(**kwargs)

Deletes a single version of an AWS Glue DataBrew recipe.

See also: AWS API Documentation

Request Syntax

response = client.delete_recipe_version(
    Name='string',
    RecipeVersion='string'
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the recipe to be deleted.

  • RecipeVersion (string) --

    [REQUIRED]

    The version of the recipe to be deleted.

Return type

dict

Returns

Response Syntax

{
    'Name': 'string',
    'RecipeVersion': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the recipe that was deleted.

    • RecipeVersion (string) --

      The version of the recipe that was deleted.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
delete_schedule(**kwargs)

Deletes the specified AWS Glue DataBrew schedule from the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.delete_schedule(
    Name='string'
)
Parameters
Name (string) --

[REQUIRED]

The name of the schedule to be deleted.

Return type
dict
Returns
Response Syntax
{
    'Name': 'string'
}

Response Structure

  • (dict) --
    • Name (string) --

      The name of the schedule that was deleted.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
describe_dataset(**kwargs)

Returns the definition of a specific AWS Glue DataBrew dataset that is in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.describe_dataset(
    Name='string'
)
Parameters
Name (string) --

[REQUIRED]

The name of the dataset to be described.

Return type
dict
Returns
Response Syntax
{
    'CreatedBy': 'string',
    'CreateDate': datetime(2015, 1, 1),
    'Name': 'string',
    'FormatOptions': {
        'Json': {
            'MultiLine': True|False
        },
        'Excel': {
            'SheetNames': [
                'string',
            ],
            'SheetIndexes': [
                123,
            ]
        }
    },
    'Input': {
        'S3InputDefinition': {
            'Bucket': 'string',
            'Key': 'string'
        },
        'DataCatalogInputDefinition': {
            'CatalogId': 'string',
            'DatabaseName': 'string',
            'TableName': 'string',
            'TempDirectory': {
                'Bucket': 'string',
                'Key': 'string'
            }
        }
    },
    'LastModifiedDate': datetime(2015, 1, 1),
    'LastModifiedBy': 'string',
    'Source': 'S3'|'DATA-CATALOG',
    'Tags': {
        'string': 'string'
    },
    'ResourceArn': 'string'
}

Response Structure

  • (dict) --
    • CreatedBy (string) --

      The identifier (user name) of the user who created the dataset.

    • CreateDate (datetime) --

      The date and time that the dataset was created.

    • Name (string) --

      The name of the dataset.

    • FormatOptions (dict) --

      Options that define how Microsoft Excel input is to be interpreted by DataBrew.

      • Json (dict) --

        Options that define how JSON input is to be interpreted by DataBrew.

        • MultiLine (boolean) --

          A value that specifies whether JSON input contains embedded new line characters.

      • Excel (dict) --

        Options that define how Excel input is to be interpreted by DataBrew.

        • SheetNames (list) --

          Specifies one or more named sheets in the Excel file, which will be included in the dataset.

          • (string) --
        • SheetIndexes (list) --

          Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.

          • (integer) --
    • Input (dict) --

      Information on how AWS Glue DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.

      • S3InputDefinition (dict) --

        The Amazon S3 location where the data is stored.

        • Bucket (string) --

          The S3 bucket name.

        • Key (string) --

          The unique name of the object in the bucket.

      • DataCatalogInputDefinition (dict) --

        The AWS Glue Data Catalog parameters for the data.

        • CatalogId (string) --

          The unique identifier of the AWS account that holds the Data Catalog that stores the data.

        • DatabaseName (string) --

          The name of a database in the Data Catalog.

        • TableName (string) --

          The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.

        • TempDirectory (dict) --

          An Amazon location that AWS Glue Data Catalog can use as a temporary directory.

          • Bucket (string) --

            The S3 bucket name.

          • Key (string) --

            The unique name of the object in the bucket.

    • LastModifiedDate (datetime) --

      The date and time that the dataset was last modified.

    • LastModifiedBy (string) --

      The identifier (user name) of the user who last modified the dataset.

    • Source (string) --

      The location of the data for this dataset, Amazon S3 or the AWS Glue Data Catalog.

    • Tags (dict) --

      Metadata tags associated with this dataset.

      • (string) --
        • (string) --
    • ResourceArn (string) --

      The Amazon Resource Name (ARN) of the dataset.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
describe_job(**kwargs)

Returns the definition of a specific AWS Glue DataBrew job that is in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.describe_job(
    Name='string'
)
Parameters
Name (string) --

[REQUIRED]

The name of the job to be described.

Return type
dict
Returns
Response Syntax
{
    'CreateDate': datetime(2015, 1, 1),
    'CreatedBy': 'string',
    'DatasetName': 'string',
    'EncryptionKeyArn': 'string',
    'EncryptionMode': 'SSE-KMS'|'SSE-S3',
    'Name': 'string',
    'Type': 'PROFILE'|'RECIPE',
    'LastModifiedBy': 'string',
    'LastModifiedDate': datetime(2015, 1, 1),
    'LogSubscription': 'ENABLE'|'DISABLE',
    'MaxCapacity': 123,
    'MaxRetries': 123,
    'Outputs': [
        {
            'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
            'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
            'PartitionColumns': [
                'string',
            ],
            'Location': {
                'Bucket': 'string',
                'Key': 'string'
            },
            'Overwrite': True|False
        },
    ],
    'ProjectName': 'string',
    'RecipeReference': {
        'Name': 'string',
        'RecipeVersion': 'string'
    },
    'ResourceArn': 'string',
    'RoleArn': 'string',
    'Tags': {
        'string': 'string'
    },
    'Timeout': 123
}

Response Structure

  • (dict) --
    • CreateDate (datetime) --

      The date and time that the job was created.

    • CreatedBy (string) --

      The identifier (user name) of the user associated with the creation of the job.

    • DatasetName (string) --

      The dataset that the job acts upon.

    • EncryptionKeyArn (string) --

      The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.

    • EncryptionMode (string) --

      The encryption mode for the job, which can be one of the following:

      • SSE-KMS - Server-side encryption with AWS KMS-managed keys.
      • SSE-S3 - Server-side encryption with keys managed by Amazon S3.
    • Name (string) --

      The name of the job.

    • Type (string) --

      The job type, which must be one of the following:

      • PROFILE - The job analyzes the dataset to determine its size, data types, data distribution, and more.
      • RECIPE - The job applies one or more transformations to a dataset.
    • LastModifiedBy (string) --

      The identifier (user name) of the user who last modified the job.

    • LastModifiedDate (datetime) --

      The date and time that the job was last modified.

    • LogSubscription (string) --

      A value that indicates whether Amazon CloudWatch logging is enabled for this job.

    • MaxCapacity (integer) --

      The maximum number of nodes that AWS Glue DataBrew can consume when the job processes data.

    • MaxRetries (integer) --

      The maximum number of times to retry the job after a job run fails.

    • Outputs (list) --

      One or more artifacts that represent the output from running the job.

      • (dict) --

        Represents individual output from a particular job run.

        • CompressionFormat (string) --

          The compression algorithm used to compress the output text of the job.

        • Format (string) --

          The data format of the output of the job.

        • PartitionColumns (list) --

          The names of one or more partition columns for the output of the job.

          • (string) --
        • Location (dict) --

          The location in Amazon S3 where the job writes its output.

          • Bucket (string) --

            The S3 bucket name.

          • Key (string) --

            The unique name of the object in the bucket.

        • Overwrite (boolean) --

          A value that, if true, means that any data in the location specified for output is overwritten with new output.

    • ProjectName (string) --

      The DataBrew project associated with this job.

    • RecipeReference (dict) --

      Represents all of the attributes of an AWS Glue DataBrew recipe.

      • Name (string) --

        The name of the recipe.

      • RecipeVersion (string) --

        The identifier for the version for the recipe.

    • ResourceArn (string) --

      The Amazon Resource Name (ARN) of the job.

    • RoleArn (string) --

      The ARN of the AWS Identity and Access Management (IAM) role that was assumed for this request.

    • Tags (dict) --

      Metadata tags associated with this job.

      • (string) --
        • (string) --
    • Timeout (integer) --

      The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
describe_project(**kwargs)

Returns the definition of a specific AWS Glue DataBrew project that is in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.describe_project(
    Name='string'
)
Parameters
Name (string) --

[REQUIRED]

The name of the project to be described.

Return type
dict
Returns
Response Syntax
{
    'CreateDate': datetime(2015, 1, 1),
    'CreatedBy': 'string',
    'DatasetName': 'string',
    'LastModifiedDate': datetime(2015, 1, 1),
    'LastModifiedBy': 'string',
    'Name': 'string',
    'RecipeName': 'string',
    'ResourceArn': 'string',
    'Sample': {
        'Size': 123,
        'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
    },
    'RoleArn': 'string',
    'Tags': {
        'string': 'string'
    },
    'SessionStatus': 'ASSIGNED'|'FAILED'|'INITIALIZING'|'PROVISIONING'|'READY'|'RECYCLING'|'ROTATING'|'TERMINATED'|'TERMINATING'|'UPDATING',
    'OpenedBy': 'string',
    'OpenDate': datetime(2015, 1, 1)
}

Response Structure

  • (dict) --
    • CreateDate (datetime) --

      The date and time that the project was created.

    • CreatedBy (string) --

      The identifier (user name) of the user who created the project.

    • DatasetName (string) --

      The dataset associated with the project.

    • LastModifiedDate (datetime) --

      The date and time that the project was last modified.

    • LastModifiedBy (string) --

      The identifier (user name) of the user who last modified the project.

    • Name (string) --

      The name of the project.

    • RecipeName (string) --

      The recipe associated with this job.

    • ResourceArn (string) --

      The Amazon Resource Name (ARN) of the project.

    • Sample (dict) --

      Represents the sample size and sampling type for AWS Glue DataBrew to use for interactive data analysis.

      • Size (integer) --

        The number of rows in the sample.

      • Type (string) --

        The way in which DataBrew obtains rows from a dataset.

    • RoleArn (string) --

      The ARN of the AWS Identity and Access Management (IAM) role that was assumed for this request.

    • Tags (dict) --

      Metadata tags associated with this project.

      • (string) --
        • (string) --
    • SessionStatus (string) --

      Describes the current state of the session:

      • PROVISIONING - allocating resources for the session.
      • INITIALIZING - getting the session ready for first use.
      • ASSIGNED - the session is ready for use.
    • OpenedBy (string) --

      The identifier (user name) of the user that opened the project for use.

    • OpenDate (datetime) --

      The date and time when the project was opened.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
describe_recipe(**kwargs)

Returns the definition of a specific AWS Glue DataBrew recipe that is in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.describe_recipe(
    Name='string',
    RecipeVersion='string'
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the recipe to be described.

  • RecipeVersion (string) -- The recipe version identifier. If this parameter isn't specified, then the latest published version is returned.
Return type

dict

Returns

Response Syntax

{
    'CreatedBy': 'string',
    'CreateDate': datetime(2015, 1, 1),
    'LastModifiedBy': 'string',
    'LastModifiedDate': datetime(2015, 1, 1),
    'ProjectName': 'string',
    'PublishedBy': 'string',
    'PublishedDate': datetime(2015, 1, 1),
    'Description': 'string',
    'Name': 'string',
    'Steps': [
        {
            'Action': {
                'Operation': 'string',
                'Parameters': {
                    'string': 'string'
                }
            },
            'ConditionExpressions': [
                {
                    'Condition': 'string',
                    'Value': 'string',
                    'TargetColumn': 'string'
                },
            ]
        },
    ],
    'Tags': {
        'string': 'string'
    },
    'ResourceArn': 'string',
    'RecipeVersion': 'string'
}

Response Structure

  • (dict) --

    • CreatedBy (string) --

      The identifier (user name) of the user who created the recipe.

    • CreateDate (datetime) --

      The date and time that the recipe was created.

    • LastModifiedBy (string) --

      The identifier (user name) of the user who last modified the recipe.

    • LastModifiedDate (datetime) --

      The date and time that the recipe was last modified.

    • ProjectName (string) --

      The name of the project associated with this recipe.

    • PublishedBy (string) --

      The identifier (user name) of the user who last published the recipe.

    • PublishedDate (datetime) --

      The date and time when the recipe was last published.

    • Description (string) --

      The description of the recipe.

    • Name (string) --

      The name of the recipe.

    • Steps (list) --

      One or more steps to be performed by the recipe. Each step consists of an action, and the conditions under which the action should succeed.

      • (dict) --

        Represents a single step to be performed in an AWS Glue DataBrew recipe.

        • Action (dict) --

          The particular action to be performed in the recipe step.

          • Operation (string) --

            The name of a valid DataBrew transformation to be performed on the data.

          • Parameters (dict) --

            Contextual parameters for the transformation.

            • (string) --
              • (string) --
        • ConditionExpressions (list) --

          One or more conditions that must be met, in order for the recipe step to succeed.

          Note

          All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.

          • (dict) --

            Represents an individual condition that evaluates to true or false.

            Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.

            If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.

            • Condition (string) --

              A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .

            • Value (string) --

              A value that the condition must evaluate to for the condition to succeed.

            • TargetColumn (string) --

              A column to apply this condition to, within an AWS Glue DataBrew dataset.

    • Tags (dict) --

      Metadata tags associated with this project.

      • (string) --
        • (string) --
    • ResourceArn (string) --

      The ARN of the recipe.

    • RecipeVersion (string) --

      The recipe version identifier.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
describe_schedule(**kwargs)

Returns the definition of a specific AWS Glue DataBrew schedule that is in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.describe_schedule(
    Name='string'
)
Parameters
Name (string) --

[REQUIRED]

The name of the schedule to be described.

Return type
dict
Returns
Response Syntax
{
    'CreateDate': datetime(2015, 1, 1),
    'CreatedBy': 'string',
    'JobNames': [
        'string',
    ],
    'LastModifiedBy': 'string',
    'LastModifiedDate': datetime(2015, 1, 1),
    'ResourceArn': 'string',
    'CronExpression': 'string',
    'Tags': {
        'string': 'string'
    },
    'Name': 'string'
}

Response Structure

  • (dict) --
    • CreateDate (datetime) --

      The date and time that the schedule was created.

    • CreatedBy (string) --

      The identifier (user name) of the user who created the schedule.

    • JobNames (list) --

      The name or names of one or more jobs to be run by using the schedule.

      • (string) --
    • LastModifiedBy (string) --

      The identifier (user name) of the user who last modified the schedule.

    • LastModifiedDate (datetime) --

      The date and time that the schedule was last modified.

    • ResourceArn (string) --

      The Amazon Resource Name (ARN) of the schedule.

    • CronExpression (string) --

      The date or dates and time or times, in cron format, when the jobs are to be run for the schedule.

    • Tags (dict) --

      Metadata tags associated with this schedule.

      • (string) --
        • (string) --
    • Name (string) --

      The name of the schedule.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
generate_presigned_url(ClientMethod, Params=None, ExpiresIn=3600, HttpMethod=None)

Generate a presigned url given a client, its method, and arguments

Parameters
  • ClientMethod (string) -- The client method to presign for
  • Params (dict) -- The parameters normally passed to ClientMethod.
  • ExpiresIn (int) -- The number of seconds the presigned url is valid for. By default it expires in an hour (3600 seconds)
  • HttpMethod (string) -- The http method to use on the generated url. By default, the http method is whatever is used in the method's model.
Returns

The presigned url

get_paginator(operation_name)

Create a paginator for an operation.

Parameters
operation_name (string) -- The operation name. This is the same name as the method name on the client. For example, if the method name is create_foo, and you'd normally invoke the operation as client.create_foo(**kwargs), if the create_foo operation can be paginated, you can use the call client.get_paginator("create_foo").
Raises OperationNotPageableError
Raised if the operation is not pageable. You can use the client.can_paginate method to check if an operation is pageable.
Return type
L{botocore.paginate.Paginator}
Returns
A paginator object.
get_waiter(waiter_name)

Returns an object that can wait for some condition.

Parameters
waiter_name (str) -- The name of the waiter to get. See the waiters section of the service docs for a list of available waiters.
Returns
The specified waiter object.
Return type
botocore.waiter.Waiter
list_datasets(**kwargs)

Lists all of the AWS Glue DataBrew datasets for the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.list_datasets(
    MaxResults=123,
    NextToken='string'
)
Parameters
  • MaxResults (integer) -- The maximum number of results to return in this request.
  • NextToken (string) -- A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.
Return type

dict

Returns

Response Syntax

{
    'Datasets': [
        {
            'AccountId': 'string',
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'Name': 'string',
            'FormatOptions': {
                'Json': {
                    'MultiLine': True|False
                },
                'Excel': {
                    'SheetNames': [
                        'string',
                    ],
                    'SheetIndexes': [
                        123,
                    ]
                }
            },
            'Input': {
                'S3InputDefinition': {
                    'Bucket': 'string',
                    'Key': 'string'
                },
                'DataCatalogInputDefinition': {
                    'CatalogId': 'string',
                    'DatabaseName': 'string',
                    'TableName': 'string',
                    'TempDirectory': {
                        'Bucket': 'string',
                        'Key': 'string'
                    }
                }
            },
            'LastModifiedDate': datetime(2015, 1, 1),
            'LastModifiedBy': 'string',
            'Source': 'S3'|'DATA-CATALOG',
            'Tags': {
                'string': 'string'
            },
            'ResourceArn': 'string'
        },
    ],
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • Datasets (list) --

      A list of datasets that are defined in the current AWS account.

      • (dict) --

        Represents a dataset that can be processed by AWS Glue DataBrew.

        • AccountId (string) --

          The ID of the AWS account that owns the dataset.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the dataset.

        • CreateDate (datetime) --

          The date and time that the dataset was created.

        • Name (string) --

          The unique name of the dataset.

        • FormatOptions (dict) --

          Options that define how DataBrew interprets the data in the dataset.

          • Json (dict) --

            Options that define how JSON input is to be interpreted by DataBrew.

            • MultiLine (boolean) --

              A value that specifies whether JSON input contains embedded new line characters.

          • Excel (dict) --

            Options that define how Excel input is to be interpreted by DataBrew.

            • SheetNames (list) --

              Specifies one or more named sheets in the Excel file, which will be included in the dataset.

              • (string) --
            • SheetIndexes (list) --

              Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.

              • (integer) --
        • Input (dict) --

          Information on how DataBrew can find the dataset, in either the AWS Glue Data Catalog or Amazon S3.

          • S3InputDefinition (dict) --

            The Amazon S3 location where the data is stored.

            • Bucket (string) --

              The S3 bucket name.

            • Key (string) --

              The unique name of the object in the bucket.

          • DataCatalogInputDefinition (dict) --

            The AWS Glue Data Catalog parameters for the data.

            • CatalogId (string) --

              The unique identifier of the AWS account that holds the Data Catalog that stores the data.

            • DatabaseName (string) --

              The name of a database in the Data Catalog.

            • TableName (string) --

              The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.

            • TempDirectory (dict) --

              An Amazon location that AWS Glue Data Catalog can use as a temporary directory.

              • Bucket (string) --

                The S3 bucket name.

              • Key (string) --

                The unique name of the object in the bucket.

        • LastModifiedDate (datetime) --

          The last modification date and time of the dataset.

        • LastModifiedBy (string) --

          The identifier (the user name) of the user who last modified the dataset.

        • Source (string) --

          The location of the data for the dataset, either Amazon S3 or the AWS Glue Data Catalog.

        • Tags (dict) --

          Metadata tags that have been applied to the dataset.

          • (string) --
            • (string) --
        • ResourceArn (string) --

          The unique Amazon Resource Name (ARN) for the dataset.

    • NextToken (string) --

      A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To obtain the next set of pages, pass in the NextToken from the response object of the previous page call.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
list_job_runs(**kwargs)

Lists all of the previous runs of a particular AWS Glue DataBrew job in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.list_job_runs(
    Name='string',
    MaxResults=123,
    NextToken='string'
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the job.

  • MaxResults (integer) -- The maximum number of results to return in this request.
  • NextToken (string) -- A token generated by AWS Glue DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.
Return type

dict

Returns

Response Syntax

{
    'JobRuns': [
        {
            'Attempt': 123,
            'CompletedOn': datetime(2015, 1, 1),
            'DatasetName': 'string',
            'ErrorMessage': 'string',
            'ExecutionTime': 123,
            'JobName': 'string',
            'RunId': 'string',
            'State': 'STARTING'|'RUNNING'|'STOPPING'|'STOPPED'|'SUCCEEDED'|'FAILED'|'TIMEOUT',
            'LogSubscription': 'ENABLE'|'DISABLE',
            'LogGroupName': 'string',
            'Outputs': [
                {
                    'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
                    'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
                    'PartitionColumns': [
                        'string',
                    ],
                    'Location': {
                        'Bucket': 'string',
                        'Key': 'string'
                    },
                    'Overwrite': True|False
                },
            ],
            'RecipeReference': {
                'Name': 'string',
                'RecipeVersion': 'string'
            },
            'StartedBy': 'string',
            'StartedOn': datetime(2015, 1, 1)
        },
    ],
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • JobRuns (list) --

      A list of job runs that have occurred for the specified job.

      • (dict) --

        Represents one run of an AWS Glue DataBrew job.

        • Attempt (integer) --

          The number of times that DataBrew has attempted to run the job.

        • CompletedOn (datetime) --

          The date and time when the job completed processing.

        • DatasetName (string) --

          The name of the dataset for the job to process.

        • ErrorMessage (string) --

          A message indicating an error (if any) that was encountered when the job ran.

        • ExecutionTime (integer) --

          The amount of time, in seconds, during which a job run consumed resources.

        • JobName (string) --

          The name of the job being processed during this run.

        • RunId (string) --

          The unique identifier of the job run.

        • State (string) --

          The current state of the job run entity itself.

        • LogSubscription (string) --

          The current status of Amazon CloudWatch logging for the job run.

        • LogGroupName (string) --

          The name of an Amazon CloudWatch log group, where the job writes diagnostic messages when it runs.

        • Outputs (list) --

          One or more output artifacts from a job run.

          • (dict) --

            Represents individual output from a particular job run.

            • CompressionFormat (string) --

              The compression algorithm used to compress the output text of the job.

            • Format (string) --

              The data format of the output of the job.

            • PartitionColumns (list) --

              The names of one or more partition columns for the output of the job.

              • (string) --
            • Location (dict) --

              The location in Amazon S3 where the job writes its output.

              • Bucket (string) --

                The S3 bucket name.

              • Key (string) --

                The unique name of the object in the bucket.

            • Overwrite (boolean) --

              A value that, if true, means that any data in the location specified for output is overwritten with new output.

        • RecipeReference (dict) --

          The set of steps processed by the job.

          • Name (string) --

            The name of the recipe.

          • RecipeVersion (string) --

            The identifier for the version for the recipe.

        • StartedBy (string) --

          The identifier (the user name) of the user who initiated the job run.

        • StartedOn (datetime) --

          The date and time when the job run began.

    • NextToken (string) --

      A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To obtain the next set of pages, pass in the NextToken from the response object of the previous page call.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
list_jobs(**kwargs)

Lists the AWS Glue DataBrew jobs in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.list_jobs(
    DatasetName='string',
    MaxResults=123,
    NextToken='string',
    ProjectName='string'
)
Parameters
  • DatasetName (string) -- The name of a dataset. Using this parameter indicates to return only those jobs that act on the specified dataset.
  • MaxResults (integer) -- The maximum number of results to return in this request.
  • NextToken (string) -- A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.
  • ProjectName (string) -- The name of a project. Using this parameter indicates to return only those jobs that are associated with the specified project.
Return type

dict

Returns

Response Syntax

{
    'Jobs': [
        {
            'AccountId': 'string',
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'DatasetName': 'string',
            'EncryptionKeyArn': 'string',
            'EncryptionMode': 'SSE-KMS'|'SSE-S3',
            'Name': 'string',
            'Type': 'PROFILE'|'RECIPE',
            'LastModifiedBy': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'LogSubscription': 'ENABLE'|'DISABLE',
            'MaxCapacity': 123,
            'MaxRetries': 123,
            'Outputs': [
                {
                    'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
                    'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
                    'PartitionColumns': [
                        'string',
                    ],
                    'Location': {
                        'Bucket': 'string',
                        'Key': 'string'
                    },
                    'Overwrite': True|False
                },
            ],
            'ProjectName': 'string',
            'RecipeReference': {
                'Name': 'string',
                'RecipeVersion': 'string'
            },
            'ResourceArn': 'string',
            'RoleArn': 'string',
            'Timeout': 123,
            'Tags': {
                'string': 'string'
            }
        },
    ],
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • Jobs (list) --

      A list of jobs that are defined in the current AWS account.

      • (dict) --

        Represents all of the attributes of an AWS Glue DataBrew job.

        • AccountId (string) --

          The ID of the AWS account that owns the job.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the job.

        • CreateDate (datetime) --

          The date and time that the job was created.

        • DatasetName (string) --

          A dataset that the job is to process.

        • EncryptionKeyArn (string) --

          The Amazon Resource Name (ARN) of an encryption key that is used to protect a job.

        • EncryptionMode (string) --

          The encryption mode for the job, which can be one of the following:

          • SSE-KMS - Server-side encryption with AWS KMS-managed keys.
          • SSE-S3 - Server-side encryption with keys managed by Amazon S3.
        • Name (string) --

          The unique name of the job.

        • Type (string) --

          The job type of the job, which must be one of the following:

          • PROFILE - A job to analyze a dataset, to determine its size, data types, data distribution, and more.
          • RECIPE - A job to apply one or more transformations to a dataset.
        • LastModifiedBy (string) --

          The identifier (the user name) of the user who last modified the job.

        • LastModifiedDate (datetime) --

          The modification date and time of the job.

        • LogSubscription (string) --

          The current status of Amazon CloudWatch logging for the job.

        • MaxCapacity (integer) --

          The maximum number of nodes that can be consumed when the job processes data.

        • MaxRetries (integer) --

          The maximum number of times to retry the job after a job run fails.

        • Outputs (list) --

          One or more artifacts that represent output from running the job.

          • (dict) --

            Represents individual output from a particular job run.

            • CompressionFormat (string) --

              The compression algorithm used to compress the output text of the job.

            • Format (string) --

              The data format of the output of the job.

            • PartitionColumns (list) --

              The names of one or more partition columns for the output of the job.

              • (string) --
            • Location (dict) --

              The location in Amazon S3 where the job writes its output.

              • Bucket (string) --

                The S3 bucket name.

              • Key (string) --

                The unique name of the object in the bucket.

            • Overwrite (boolean) --

              A value that, if true, means that any data in the location specified for output is overwritten with new output.

        • ProjectName (string) --

          The name of the project that the job is associated with.

        • RecipeReference (dict) --

          A set of steps that the job runs.

          • Name (string) --

            The name of the recipe.

          • RecipeVersion (string) --

            The identifier for the version for the recipe.

        • ResourceArn (string) --

          The unique Amazon Resource Name (ARN) for the job.

        • RoleArn (string) --

          The Amazon Resource Name (ARN) of the role that will be assumed for this job.

        • Timeout (integer) --

          The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .

        • Tags (dict) --

          Metadata tags that have been applied to the job.

          • (string) --
            • (string) --
    • NextToken (string) --

      A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To obtain the next set of pages, pass in the NextToken from the response object of the previous page call.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
list_projects(**kwargs)

Lists all of the DataBrew projects in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.list_projects(
    NextToken='string',
    MaxResults=123
)
Parameters
  • NextToken (string) -- A pagination token that can be used in a subsequent request.
  • MaxResults (integer) -- The maximum number of results to return in this request.
Return type

dict

Returns

Response Syntax

{
    'Projects': [
        {
            'AccountId': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'CreatedBy': 'string',
            'DatasetName': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'LastModifiedBy': 'string',
            'Name': 'string',
            'RecipeName': 'string',
            'ResourceArn': 'string',
            'Sample': {
                'Size': 123,
                'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
            },
            'Tags': {
                'string': 'string'
            },
            'RoleArn': 'string',
            'OpenedBy': 'string',
            'OpenDate': datetime(2015, 1, 1)
        },
    ],
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • Projects (list) --

      A list of projects that are defined in the current AWS account.

      • (dict) --

        Represents all of the attributes of an AWS Glue DataBrew project.

        • AccountId (string) --

          The ID of the AWS account that owns the project.

        • CreateDate (datetime) --

          The date and time that the project was created.

        • CreatedBy (string) --

          The identifier (the user name) of the user who crated the project.

        • DatasetName (string) --

          The dataset that the project is to act upon.

        • LastModifiedDate (datetime) --

          The last modification date and time for the project.

        • LastModifiedBy (string) --

          The identifier (user name) of the user who last modified the project.

        • Name (string) --

          The unique name of a project.

        • RecipeName (string) --

          The name of a recipe that will be developed during a project session.

        • ResourceArn (string) --

          The Amazon Resource Name (ARN) for the project.

        • Sample (dict) --

          The sample size and sampling type to apply to the data. If this parameter isn't specified, then the sample will consiste of the first 500 rows from the dataset.

          • Size (integer) --

            The number of rows in the sample.

          • Type (string) --

            The way in which DataBrew obtains rows from a dataset.

        • Tags (dict) --

          Metadata tags that have been applied to the project.

          • (string) --
            • (string) --
        • RoleArn (string) --

          The Amazon Resource Name (ARN) of the role that will be assumed for this project.

        • OpenedBy (string) --

          The identifier (the user name) of the user that opened the project for use.

        • OpenDate (datetime) --

          The date and time when the project was opened.

    • NextToken (string) --

      A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
list_recipe_versions(**kwargs)

Lists all of the versions of a particular AWS Glue DataBrew recipe in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.list_recipe_versions(
    MaxResults=123,
    NextToken='string',
    Name='string'
)
Parameters
  • MaxResults (integer) -- The maximum number of results to return in this request.
  • NextToken (string) -- A pagination token that can be used in a subsequent request.
  • Name (string) --

    [REQUIRED]

    The name of the recipe for which to return version information.

Return type

dict

Returns

Response Syntax

{
    'NextToken': 'string',
    'Recipes': [
        {
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'LastModifiedBy': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'ProjectName': 'string',
            'PublishedBy': 'string',
            'PublishedDate': datetime(2015, 1, 1),
            'Description': 'string',
            'Name': 'string',
            'ResourceArn': 'string',
            'Steps': [
                {
                    'Action': {
                        'Operation': 'string',
                        'Parameters': {
                            'string': 'string'
                        }
                    },
                    'ConditionExpressions': [
                        {
                            'Condition': 'string',
                            'Value': 'string',
                            'TargetColumn': 'string'
                        },
                    ]
                },
            ],
            'Tags': {
                'string': 'string'
            },
            'RecipeVersion': 'string'
        },
    ]
}

Response Structure

  • (dict) --

    • NextToken (string) --

      A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.

    • Recipes (list) --

      A list of versions for the specified recipe.

      • (dict) --

        Represents one or more actions to be performed on an AWS Glue DataBrew dataset.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the recipe.

        • CreateDate (datetime) --

          The date and time that the recipe was created.

        • LastModifiedBy (string) --

          The identifier (user name) of the user who last modified the recipe.

        • LastModifiedDate (datetime) --

          The last modification date and time of the recipe.

        • ProjectName (string) --

          The name of the project that the recipe is associated with.

        • PublishedBy (string) --

          The identifier (the user name) of the user who published the recipe.

        • PublishedDate (datetime) --

          The date and time when the recipe was published.

        • Description (string) --

          The description of the recipe.

        • Name (string) --

          The unique name for the recipe.

        • ResourceArn (string) --

          The Amazon Resource Name (ARN) for the recipe.

        • Steps (list) --

          A list of steps that are defined by the recipe.

          • (dict) --

            Represents a single step to be performed in an AWS Glue DataBrew recipe.

            • Action (dict) --

              The particular action to be performed in the recipe step.

              • Operation (string) --

                The name of a valid DataBrew transformation to be performed on the data.

              • Parameters (dict) --

                Contextual parameters for the transformation.

                • (string) --
                  • (string) --
            • ConditionExpressions (list) --

              One or more conditions that must be met, in order for the recipe step to succeed.

              Note

              All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.

              • (dict) --

                Represents an individual condition that evaluates to true or false.

                Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.

                If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.

                • Condition (string) --

                  A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .

                • Value (string) --

                  A value that the condition must evaluate to for the condition to succeed.

                • TargetColumn (string) --

                  A column to apply this condition to, within an AWS Glue DataBrew dataset.

        • Tags (dict) --

          Metadata tags that have been applied to the recipe.

          • (string) --
            • (string) --
        • RecipeVersion (string) --

          The identifier for the version for the recipe.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
list_recipes(**kwargs)

Lists all of the AWS Glue DataBrew recipes in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.list_recipes(
    MaxResults=123,
    NextToken='string',
    RecipeVersion='string'
)
Parameters
  • MaxResults (integer) -- The maximum number of results to return in this request.
  • NextToken (string) -- A pagination token that can be used in a subsequent request.
  • RecipeVersion (string) -- A version identifier. Using this parameter indicates to return only those recipes that have this version identifier.
Return type

dict

Returns

Response Syntax

{
    'Recipes': [
        {
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'LastModifiedBy': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'ProjectName': 'string',
            'PublishedBy': 'string',
            'PublishedDate': datetime(2015, 1, 1),
            'Description': 'string',
            'Name': 'string',
            'ResourceArn': 'string',
            'Steps': [
                {
                    'Action': {
                        'Operation': 'string',
                        'Parameters': {
                            'string': 'string'
                        }
                    },
                    'ConditionExpressions': [
                        {
                            'Condition': 'string',
                            'Value': 'string',
                            'TargetColumn': 'string'
                        },
                    ]
                },
            ],
            'Tags': {
                'string': 'string'
            },
            'RecipeVersion': 'string'
        },
    ],
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • Recipes (list) --

      A list of recipes that are defined in the current AWS account.

      • (dict) --

        Represents one or more actions to be performed on an AWS Glue DataBrew dataset.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the recipe.

        • CreateDate (datetime) --

          The date and time that the recipe was created.

        • LastModifiedBy (string) --

          The identifier (user name) of the user who last modified the recipe.

        • LastModifiedDate (datetime) --

          The last modification date and time of the recipe.

        • ProjectName (string) --

          The name of the project that the recipe is associated with.

        • PublishedBy (string) --

          The identifier (the user name) of the user who published the recipe.

        • PublishedDate (datetime) --

          The date and time when the recipe was published.

        • Description (string) --

          The description of the recipe.

        • Name (string) --

          The unique name for the recipe.

        • ResourceArn (string) --

          The Amazon Resource Name (ARN) for the recipe.

        • Steps (list) --

          A list of steps that are defined by the recipe.

          • (dict) --

            Represents a single step to be performed in an AWS Glue DataBrew recipe.

            • Action (dict) --

              The particular action to be performed in the recipe step.

              • Operation (string) --

                The name of a valid DataBrew transformation to be performed on the data.

              • Parameters (dict) --

                Contextual parameters for the transformation.

                • (string) --
                  • (string) --
            • ConditionExpressions (list) --

              One or more conditions that must be met, in order for the recipe step to succeed.

              Note

              All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.

              • (dict) --

                Represents an individual condition that evaluates to true or false.

                Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.

                If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.

                • Condition (string) --

                  A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .

                • Value (string) --

                  A value that the condition must evaluate to for the condition to succeed.

                • TargetColumn (string) --

                  A column to apply this condition to, within an AWS Glue DataBrew dataset.

        • Tags (dict) --

          Metadata tags that have been applied to the recipe.

          • (string) --
            • (string) --
        • RecipeVersion (string) --

          The identifier for the version for the recipe.

    • NextToken (string) --

      A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
list_schedules(**kwargs)

Lists the AWS Glue DataBrew schedules in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.list_schedules(
    JobName='string',
    MaxResults=123,
    NextToken='string'
)
Parameters
  • JobName (string) -- The name of the job that these schedules apply to.
  • MaxResults (integer) -- The maximum number of results to return in this request.
  • NextToken (string) -- A pagination token that can be used in a subsequent request.
Return type

dict

Returns

Response Syntax

{
    'Schedules': [
        {
            'AccountId': 'string',
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'JobNames': [
                'string',
            ],
            'LastModifiedBy': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'ResourceArn': 'string',
            'CronExpression': 'string',
            'Tags': {
                'string': 'string'
            },
            'Name': 'string'
        },
    ],
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • Schedules (list) --

      A list of schedules in the current AWS account.

      • (dict) --

        Represents one or more dates and times when a job is to run.

        • AccountId (string) --

          The ID of the AWS account that owns the schedule.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the schedule.

        • CreateDate (datetime) --

          The date and time that the schedule was created.

        • JobNames (list) --

          A list of jobs to be run, according to the schedule.

          • (string) --
        • LastModifiedBy (string) --

          The identifier (the user name) of the user who last modified the schedule.

        • LastModifiedDate (datetime) --

          The date and time when the schedule was last modified.

        • ResourceArn (string) --

          The Amazon Resource Name (ARN) of the schedule.

        • CronExpression (string) --

          The date(s) and time(s), in cron format, when the job will run.

        • Tags (dict) --

          Metadata tags that have been applied to the schedule.

          • (string) --
            • (string) --
        • Name (string) --

          The name of the schedule.

    • NextToken (string) --

      A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
list_tags_for_resource(**kwargs)

Lists all the tags for an AWS Glue DataBrew resource.

See also: AWS API Documentation

Request Syntax

response = client.list_tags_for_resource(
    ResourceArn='string'
)
Parameters
ResourceArn (string) --

[REQUIRED]

The Amazon Resource Name (ARN) string that uniquely identifies the DataBrew resource.

Return type
dict
Returns
Response Syntax
{
    'Tags': {
        'string': 'string'
    }
}

Response Structure

  • (dict) --
    • Tags (dict) --

      A list of tags associated with the DataBrew resource.

      • (string) --
        • (string) --

Exceptions

  • GlueDataBrew.Client.exceptions.InternalServerException
  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
publish_recipe(**kwargs)

Publishes a new major version of an AWS Glue DataBrew recipe that exists in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.publish_recipe(
    Description='string',
    Name='string'
)
Parameters
  • Description (string) -- A description of the recipe to be published, for this version of the recipe.
  • Name (string) --

    [REQUIRED]

    The name of the recipe to be published.

Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the recipe that you published.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
send_project_session_action(**kwargs)

Performs a recipe step within an interactive AWS Glue DataBrew session that's currently open.

See also: AWS API Documentation

Request Syntax

response = client.send_project_session_action(
    Preview=True|False,
    Name='string',
    RecipeStep={
        'Action': {
            'Operation': 'string',
            'Parameters': {
                'string': 'string'
            }
        },
        'ConditionExpressions': [
            {
                'Condition': 'string',
                'Value': 'string',
                'TargetColumn': 'string'
            },
        ]
    },
    StepIndex=123,
    ClientSessionId='string',
    ViewFrame={
        'StartColumnIndex': 123,
        'ColumnRange': 123,
        'HiddenColumns': [
            'string',
        ]
    }
)
Parameters
  • Preview (boolean) -- Returns the result of the recipe step, without applying it. The result isn't added to the view frame stack.
  • Name (string) --

    [REQUIRED]

    The name of the project to apply the action to.

  • RecipeStep (dict) --

    Represents a single step to be performed in an AWS Glue DataBrew recipe.

    • Action (dict) -- [REQUIRED]

      The particular action to be performed in the recipe step.

      • Operation (string) -- [REQUIRED]

        The name of a valid DataBrew transformation to be performed on the data.

      • Parameters (dict) --

        Contextual parameters for the transformation.

        • (string) --
          • (string) --
    • ConditionExpressions (list) --

      One or more conditions that must be met, in order for the recipe step to succeed.

      Note

      All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.

      • (dict) --

        Represents an individual condition that evaluates to true or false.

        Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.

        If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.

        • Condition (string) -- [REQUIRED]

          A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .

        • Value (string) --

          A value that the condition must evaluate to for the condition to succeed.

        • TargetColumn (string) -- [REQUIRED]

          A column to apply this condition to, within an AWS Glue DataBrew dataset.

  • StepIndex (integer) -- The index from which to preview a step. This index is used to preview the result of steps that have already been applied, so that the resulting view frame is from earlier in the view frame stack.
  • ClientSessionId (string) -- A unique identifier for an interactive session that's currently open and ready for work. The action will be performed on this session.
  • ViewFrame (dict) --

    Represents the data being being transformed during an AWS Glue DataBrew project session.

    • StartColumnIndex (integer) -- [REQUIRED]

      The starting index for the range of columns to return in the view frame.

    • ColumnRange (integer) --

      The number of columns to include in the view frame, beginning with the StartColumnIndex value and ignoring any columns in the HiddenColumns list.

    • HiddenColumns (list) --

      A list of columns to hide in the view frame.

      • (string) --
Return type

dict

Returns

Response Syntax

{
    'Result': 'string',
    'Name': 'string',
    'ActionId': 123
}

Response Structure

  • (dict) --

    • Result (string) --

      A message indicating the result of performing the action.

    • Name (string) --

      The name of the project that was affected by the action.

    • ActionId (integer) --

      A unique identifier for the action that was performed.

Exceptions

  • GlueDataBrew.Client.exceptions.ConflictException
  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
start_job_run(**kwargs)

Runs an AWS Glue DataBrew job that exists in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.start_job_run(
    Name='string'
)
Parameters
Name (string) --

[REQUIRED]

The name of the job to be run.

Return type
dict
Returns
Response Syntax
{
    'RunId': 'string'
}

Response Structure

  • (dict) --
    • RunId (string) --

      A system-generated identifier for this particular job run.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
start_project_session(**kwargs)

Creates an interactive session, enabling you to manipulate an AWS Glue DataBrew project.

See also: AWS API Documentation

Request Syntax

response = client.start_project_session(
    Name='string',
    AssumeControl=True|False
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the project to act upon.

  • AssumeControl (boolean) -- A value that, if true, enables you to take control of a session, even if a different client is currently accessing the project.
Return type

dict

Returns

Response Syntax

{
    'Name': 'string',
    'ClientSessionId': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the project to be acted upon.

    • ClientSessionId (string) --

      A system-generated identifier for the session.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
stop_job_run(**kwargs)

Stops the specified job from running in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.stop_job_run(
    Name='string',
    RunId='string'
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the job to be stopped.

  • RunId (string) --

    [REQUIRED]

    The ID of the job run to be stopped.

Return type

dict

Returns

Response Syntax

{
    'RunId': 'string'
}

Response Structure

  • (dict) --

    • RunId (string) --

      The ID of the job run that you stopped.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
tag_resource(**kwargs)

Adds metadata tags to an AWS Glue DataBrew resource, such as a dataset, job, project, or recipe.

See also: AWS API Documentation

Request Syntax

response = client.tag_resource(
    ResourceArn='string',
    Tags={
        'string': 'string'
    }
)
Parameters
  • ResourceArn (string) --

    [REQUIRED]

    The DataBrew resource to which tags should be added. The value for this parameter is an Amazon Resource Name (ARN). For DataBrew, you can tag a dataset, a job, a project, or a recipe.

  • Tags (dict) --

    [REQUIRED]

    One or more tags to be assigned to the resource.

    • (string) --
      • (string) --
Return type

dict

Returns

Response Syntax

{}

Response Structure

  • (dict) --

Exceptions

  • GlueDataBrew.Client.exceptions.InternalServerException
  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
untag_resource(**kwargs)

Removes metadata tags from an AWS Glue DataBrew resource.

See also: AWS API Documentation

Request Syntax

response = client.untag_resource(
    ResourceArn='string',
    TagKeys=[
        'string',
    ]
)
Parameters
  • ResourceArn (string) --

    [REQUIRED]

    An DataBrew resource from which you want to remove a tag or tags. The value for this parameter is an Amazon Resource Name (ARN).

  • TagKeys (list) --

    [REQUIRED]

    The tag keys (names) of one or more tags to be removed.

    • (string) --
Return type

dict

Returns

Response Syntax

{}

Response Structure

  • (dict) --

Exceptions

  • GlueDataBrew.Client.exceptions.InternalServerException
  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
update_dataset(**kwargs)

Modifies the definition of an existing AWS Glue DataBrew dataset in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.update_dataset(
    Name='string',
    FormatOptions={
        'Json': {
            'MultiLine': True|False
        },
        'Excel': {
            'SheetNames': [
                'string',
            ],
            'SheetIndexes': [
                123,
            ]
        }
    },
    Input={
        'S3InputDefinition': {
            'Bucket': 'string',
            'Key': 'string'
        },
        'DataCatalogInputDefinition': {
            'CatalogId': 'string',
            'DatabaseName': 'string',
            'TableName': 'string',
            'TempDirectory': {
                'Bucket': 'string',
                'Key': 'string'
            }
        }
    }
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the dataset to be updated.

  • FormatOptions (dict) --

    Options that define how Microsoft Excel input is to be interpreted by DataBrew.

    • Json (dict) --

      Options that define how JSON input is to be interpreted by DataBrew.

      • MultiLine (boolean) --

        A value that specifies whether JSON input contains embedded new line characters.

    • Excel (dict) --

      Options that define how Excel input is to be interpreted by DataBrew.

      • SheetNames (list) --

        Specifies one or more named sheets in the Excel file, which will be included in the dataset.

        • (string) --
      • SheetIndexes (list) --

        Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.

        • (integer) --
  • Input (dict) --

    [REQUIRED]

    Information on how AWS Glue DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.

    • S3InputDefinition (dict) --

      The Amazon S3 location where the data is stored.

      • Bucket (string) -- [REQUIRED]

        The S3 bucket name.

      • Key (string) --

        The unique name of the object in the bucket.

    • DataCatalogInputDefinition (dict) --

      The AWS Glue Data Catalog parameters for the data.

      • CatalogId (string) --

        The unique identifier of the AWS account that holds the Data Catalog that stores the data.

      • DatabaseName (string) -- [REQUIRED]

        The name of a database in the Data Catalog.

      • TableName (string) -- [REQUIRED]

        The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.

      • TempDirectory (dict) --

        An Amazon location that AWS Glue Data Catalog can use as a temporary directory.

        • Bucket (string) -- [REQUIRED]

          The S3 bucket name.

        • Key (string) --

          The unique name of the object in the bucket.

Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the dataset that you updated.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
update_profile_job(**kwargs)

Modifies the definition of an existing AWS Glue DataBrew job in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.update_profile_job(
    EncryptionKeyArn='string',
    EncryptionMode='SSE-KMS'|'SSE-S3',
    Name='string',
    LogSubscription='ENABLE'|'DISABLE',
    MaxCapacity=123,
    MaxRetries=123,
    OutputLocation={
        'Bucket': 'string',
        'Key': 'string'
    },
    RoleArn='string',
    Timeout=123
)
Parameters
  • EncryptionKeyArn (string) -- The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
  • EncryptionMode (string) --

    The encryption mode for the job, which can be one of the following:

    • SSE-KMS - Server-side encryption with AWS KMS-managed keys.
    • SSE-S3 - Server-side encryption with keys managed by Amazon S3.
  • Name (string) --

    [REQUIRED]

    The name of the job to be updated.

  • LogSubscription (string) -- A value that enables or disables Amazon CloudWatch logging for the current AWS account. If logging is enabled, CloudWatch writes one log stream for each job run.
  • MaxCapacity (integer) -- The maximum number of nodes that DataBrew can use when the job processes data.
  • MaxRetries (integer) -- The maximum number of times to retry the job after a job run fails.
  • OutputLocation (dict) --

    [REQUIRED]

    An Amazon S3 location (bucket name an object key) where DataBrew can read input data, or write output from a job.

    • Bucket (string) -- [REQUIRED]

      The S3 bucket name.

    • Key (string) --

      The unique name of the object in the bucket.

  • RoleArn (string) --

    [REQUIRED]

    The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.

  • Timeout (integer) -- The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .
Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the job that was updated.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
update_project(**kwargs)

Modifies the definition of an existing AWS Glue DataBrew project in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.update_project(
    Sample={
        'Size': 123,
        'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
    },
    RoleArn='string',
    Name='string'
)
Parameters
  • Sample (dict) --

    Represents the sample size and sampling type for AWS Glue DataBrew to use for interactive data analysis.

    • Size (integer) --

      The number of rows in the sample.

    • Type (string) -- [REQUIRED]

      The way in which DataBrew obtains rows from a dataset.

  • RoleArn (string) --

    [REQUIRED]

    The Amazon Resource Name (ARN) of the IAM role to be assumed for this request.

  • Name (string) --

    [REQUIRED]

    The name of the project to be updated.

Return type

dict

Returns

Response Syntax

{
    'LastModifiedDate': datetime(2015, 1, 1),
    'Name': 'string'
}

Response Structure

  • (dict) --

    • LastModifiedDate (datetime) --

      The date and time that the project was last modified.

    • Name (string) --

      The name of the project that you updated.

Exceptions

  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
  • GlueDataBrew.Client.exceptions.ValidationException
update_recipe(**kwargs)

Modifies the definition of the latest working version of an AWS Glue DataBrew recipe in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.update_recipe(
    Description='string',
    Name='string',
    Steps=[
        {
            'Action': {
                'Operation': 'string',
                'Parameters': {
                    'string': 'string'
                }
            },
            'ConditionExpressions': [
                {
                    'Condition': 'string',
                    'Value': 'string',
                    'TargetColumn': 'string'
                },
            ]
        },
    ]
)
Parameters
  • Description (string) -- A description of the recipe.
  • Name (string) --

    [REQUIRED]

    The name of the recipe to be updated.

  • Steps (list) --

    One or more steps to be performed by the recipe. Each step consists of an action, and the conditions under which the action should succeed.

    • (dict) --

      Represents a single step to be performed in an AWS Glue DataBrew recipe.

      • Action (dict) -- [REQUIRED]

        The particular action to be performed in the recipe step.

        • Operation (string) -- [REQUIRED]

          The name of a valid DataBrew transformation to be performed on the data.

        • Parameters (dict) --

          Contextual parameters for the transformation.

          • (string) --
            • (string) --
      • ConditionExpressions (list) --

        One or more conditions that must be met, in order for the recipe step to succeed.

        Note

        All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.

        • (dict) --

          Represents an individual condition that evaluates to true or false.

          Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.

          If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.

          • Condition (string) -- [REQUIRED]

            A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .

          • Value (string) --

            A value that the condition must evaluate to for the condition to succeed.

          • TargetColumn (string) -- [REQUIRED]

            A column to apply this condition to, within an AWS Glue DataBrew dataset.

Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the recipe that was updated.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
  • GlueDataBrew.Client.exceptions.ResourceNotFoundException
update_recipe_job(**kwargs)

Modifies the definition of an existing AWS Glue DataBrew recipe job in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.update_recipe_job(
    EncryptionKeyArn='string',
    EncryptionMode='SSE-KMS'|'SSE-S3',
    Name='string',
    LogSubscription='ENABLE'|'DISABLE',
    MaxCapacity=123,
    MaxRetries=123,
    Outputs=[
        {
            'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
            'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
            'PartitionColumns': [
                'string',
            ],
            'Location': {
                'Bucket': 'string',
                'Key': 'string'
            },
            'Overwrite': True|False
        },
    ],
    RoleArn='string',
    Timeout=123
)
Parameters
  • EncryptionKeyArn (string) -- The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
  • EncryptionMode (string) --

    The encryption mode for the job, which can be one of the following:

    • SSE-KMS - Server-side encryption with AWS KMS-managed keys.
    • SSE-S3 - Server-side encryption with keys managed by Amazon S3.
  • Name (string) --

    [REQUIRED]

    The name of the job to update.

  • LogSubscription (string) -- A value that enables or disables Amazon CloudWatch logging for the current AWS account. If logging is enabled, CloudWatch writes one log stream for each job run.
  • MaxCapacity (integer) -- The maximum number of nodes that DataBrew can consume when the job processes data.
  • MaxRetries (integer) -- The maximum number of times to retry the job after a job run fails.
  • Outputs (list) --

    [REQUIRED]

    One or more artifacts that represent the output from running the job.

    • (dict) --

      Represents individual output from a particular job run.

      • CompressionFormat (string) --

        The compression algorithm used to compress the output text of the job.

      • Format (string) --

        The data format of the output of the job.

      • PartitionColumns (list) --

        The names of one or more partition columns for the output of the job.

        • (string) --
      • Location (dict) -- [REQUIRED]

        The location in Amazon S3 where the job writes its output.

        • Bucket (string) -- [REQUIRED]

          The S3 bucket name.

        • Key (string) --

          The unique name of the object in the bucket.

      • Overwrite (boolean) --

        A value that, if true, means that any data in the location specified for output is overwritten with new output.

  • RoleArn (string) --

    [REQUIRED]

    The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.

  • Timeout (integer) -- The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .
Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the job that you updated.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException
update_schedule(**kwargs)

Modifies the definition of an existing AWS Glue DataBrew schedule in the current AWS account.

See also: AWS API Documentation

Request Syntax

response = client.update_schedule(
    JobNames=[
        'string',
    ],
    CronExpression='string',
    Name='string'
)
Parameters
  • JobNames (list) --

    The name or names of one or more jobs to be run for this schedule.

    • (string) --
  • CronExpression (string) --

    [REQUIRED]

    The date or dates and time or times, in cron format, when the jobs are to be run.

  • Name (string) --

    [REQUIRED]

    The name of the schedule to update.

Return type

dict

Returns

Response Syntax

{
    'Name': 'string'
}

Response Structure

  • (dict) --

    • Name (string) --

      The name of the schedule that was updated.

Exceptions

  • GlueDataBrew.Client.exceptions.ValidationException

Paginators

The available paginators are:

class GlueDataBrew.Paginator.ListDatasets
paginator = client.get_paginator('list_datasets')
paginate(**kwargs)

Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_datasets().

See also: AWS API Documentation

Request Syntax

response_iterator = paginator.paginate(
    PaginationConfig={
        'MaxItems': 123,
        'PageSize': 123,
        'StartingToken': 'string'
    }
)
Parameters
PaginationConfig (dict) --

A dictionary that provides parameters to control pagination.

  • MaxItems (integer) --

    The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.

  • PageSize (integer) --

    The size of each page.

  • StartingToken (string) --

    A token to specify where to start paginating. This is the NextToken from a previous response.

Return type
dict
Returns
Response Syntax
{
    'Datasets': [
        {
            'AccountId': 'string',
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'Name': 'string',
            'FormatOptions': {
                'Json': {
                    'MultiLine': True|False
                },
                'Excel': {
                    'SheetNames': [
                        'string',
                    ],
                    'SheetIndexes': [
                        123,
                    ]
                }
            },
            'Input': {
                'S3InputDefinition': {
                    'Bucket': 'string',
                    'Key': 'string'
                },
                'DataCatalogInputDefinition': {
                    'CatalogId': 'string',
                    'DatabaseName': 'string',
                    'TableName': 'string',
                    'TempDirectory': {
                        'Bucket': 'string',
                        'Key': 'string'
                    }
                }
            },
            'LastModifiedDate': datetime(2015, 1, 1),
            'LastModifiedBy': 'string',
            'Source': 'S3'|'DATA-CATALOG',
            'Tags': {
                'string': 'string'
            },
            'ResourceArn': 'string'
        },
    ],

}

Response Structure

  • (dict) --
    • Datasets (list) --

      A list of datasets that are defined in the current AWS account.

      • (dict) --

        Represents a dataset that can be processed by AWS Glue DataBrew.

        • AccountId (string) --

          The ID of the AWS account that owns the dataset.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the dataset.

        • CreateDate (datetime) --

          The date and time that the dataset was created.

        • Name (string) --

          The unique name of the dataset.

        • FormatOptions (dict) --

          Options that define how DataBrew interprets the data in the dataset.

          • Json (dict) --

            Options that define how JSON input is to be interpreted by DataBrew.

            • MultiLine (boolean) --

              A value that specifies whether JSON input contains embedded new line characters.

          • Excel (dict) --

            Options that define how Excel input is to be interpreted by DataBrew.

            • SheetNames (list) --

              Specifies one or more named sheets in the Excel file, which will be included in the dataset.

              • (string) --
            • SheetIndexes (list) --

              Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.

              • (integer) --
        • Input (dict) --

          Information on how DataBrew can find the dataset, in either the AWS Glue Data Catalog or Amazon S3.

          • S3InputDefinition (dict) --

            The Amazon S3 location where the data is stored.

            • Bucket (string) --

              The S3 bucket name.

            • Key (string) --

              The unique name of the object in the bucket.

          • DataCatalogInputDefinition (dict) --

            The AWS Glue Data Catalog parameters for the data.

            • CatalogId (string) --

              The unique identifier of the AWS account that holds the Data Catalog that stores the data.

            • DatabaseName (string) --

              The name of a database in the Data Catalog.

            • TableName (string) --

              The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.

            • TempDirectory (dict) --

              An Amazon location that AWS Glue Data Catalog can use as a temporary directory.

              • Bucket (string) --

                The S3 bucket name.

              • Key (string) --

                The unique name of the object in the bucket.

        • LastModifiedDate (datetime) --

          The last modification date and time of the dataset.

        • LastModifiedBy (string) --

          The identifier (the user name) of the user who last modified the dataset.

        • Source (string) --

          The location of the data for the dataset, either Amazon S3 or the AWS Glue Data Catalog.

        • Tags (dict) --

          Metadata tags that have been applied to the dataset.

          • (string) --
            • (string) --
        • ResourceArn (string) --

          The unique Amazon Resource Name (ARN) for the dataset.

class GlueDataBrew.Paginator.ListJobRuns
paginator = client.get_paginator('list_job_runs')
paginate(**kwargs)

Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_job_runs().

See also: AWS API Documentation

Request Syntax

response_iterator = paginator.paginate(
    Name='string',
    PaginationConfig={
        'MaxItems': 123,
        'PageSize': 123,
        'StartingToken': 'string'
    }
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the job.

  • PaginationConfig (dict) --

    A dictionary that provides parameters to control pagination.

    • MaxItems (integer) --

      The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.

    • PageSize (integer) --

      The size of each page.

    • StartingToken (string) --

      A token to specify where to start paginating. This is the NextToken from a previous response.

Return type

dict

Returns

Response Syntax

{
    'JobRuns': [
        {
            'Attempt': 123,
            'CompletedOn': datetime(2015, 1, 1),
            'DatasetName': 'string',
            'ErrorMessage': 'string',
            'ExecutionTime': 123,
            'JobName': 'string',
            'RunId': 'string',
            'State': 'STARTING'|'RUNNING'|'STOPPING'|'STOPPED'|'SUCCEEDED'|'FAILED'|'TIMEOUT',
            'LogSubscription': 'ENABLE'|'DISABLE',
            'LogGroupName': 'string',
            'Outputs': [
                {
                    'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
                    'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
                    'PartitionColumns': [
                        'string',
                    ],
                    'Location': {
                        'Bucket': 'string',
                        'Key': 'string'
                    },
                    'Overwrite': True|False
                },
            ],
            'RecipeReference': {
                'Name': 'string',
                'RecipeVersion': 'string'
            },
            'StartedBy': 'string',
            'StartedOn': datetime(2015, 1, 1)
        },
    ],

}

Response Structure

  • (dict) --

    • JobRuns (list) --

      A list of job runs that have occurred for the specified job.

      • (dict) --

        Represents one run of an AWS Glue DataBrew job.

        • Attempt (integer) --

          The number of times that DataBrew has attempted to run the job.

        • CompletedOn (datetime) --

          The date and time when the job completed processing.

        • DatasetName (string) --

          The name of the dataset for the job to process.

        • ErrorMessage (string) --

          A message indicating an error (if any) that was encountered when the job ran.

        • ExecutionTime (integer) --

          The amount of time, in seconds, during which a job run consumed resources.

        • JobName (string) --

          The name of the job being processed during this run.

        • RunId (string) --

          The unique identifier of the job run.

        • State (string) --

          The current state of the job run entity itself.

        • LogSubscription (string) --

          The current status of Amazon CloudWatch logging for the job run.

        • LogGroupName (string) --

          The name of an Amazon CloudWatch log group, where the job writes diagnostic messages when it runs.

        • Outputs (list) --

          One or more output artifacts from a job run.

          • (dict) --

            Represents individual output from a particular job run.

            • CompressionFormat (string) --

              The compression algorithm used to compress the output text of the job.

            • Format (string) --

              The data format of the output of the job.

            • PartitionColumns (list) --

              The names of one or more partition columns for the output of the job.

              • (string) --
            • Location (dict) --

              The location in Amazon S3 where the job writes its output.

              • Bucket (string) --

                The S3 bucket name.

              • Key (string) --

                The unique name of the object in the bucket.

            • Overwrite (boolean) --

              A value that, if true, means that any data in the location specified for output is overwritten with new output.

        • RecipeReference (dict) --

          The set of steps processed by the job.

          • Name (string) --

            The name of the recipe.

          • RecipeVersion (string) --

            The identifier for the version for the recipe.

        • StartedBy (string) --

          The identifier (the user name) of the user who initiated the job run.

        • StartedOn (datetime) --

          The date and time when the job run began.

class GlueDataBrew.Paginator.ListJobs
paginator = client.get_paginator('list_jobs')
paginate(**kwargs)

Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_jobs().

See also: AWS API Documentation

Request Syntax

response_iterator = paginator.paginate(
    DatasetName='string',
    ProjectName='string',
    PaginationConfig={
        'MaxItems': 123,
        'PageSize': 123,
        'StartingToken': 'string'
    }
)
Parameters
  • DatasetName (string) -- The name of a dataset. Using this parameter indicates to return only those jobs that act on the specified dataset.
  • ProjectName (string) -- The name of a project. Using this parameter indicates to return only those jobs that are associated with the specified project.
  • PaginationConfig (dict) --

    A dictionary that provides parameters to control pagination.

    • MaxItems (integer) --

      The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.

    • PageSize (integer) --

      The size of each page.

    • StartingToken (string) --

      A token to specify where to start paginating. This is the NextToken from a previous response.

Return type

dict

Returns

Response Syntax

{
    'Jobs': [
        {
            'AccountId': 'string',
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'DatasetName': 'string',
            'EncryptionKeyArn': 'string',
            'EncryptionMode': 'SSE-KMS'|'SSE-S3',
            'Name': 'string',
            'Type': 'PROFILE'|'RECIPE',
            'LastModifiedBy': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'LogSubscription': 'ENABLE'|'DISABLE',
            'MaxCapacity': 123,
            'MaxRetries': 123,
            'Outputs': [
                {
                    'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
                    'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
                    'PartitionColumns': [
                        'string',
                    ],
                    'Location': {
                        'Bucket': 'string',
                        'Key': 'string'
                    },
                    'Overwrite': True|False
                },
            ],
            'ProjectName': 'string',
            'RecipeReference': {
                'Name': 'string',
                'RecipeVersion': 'string'
            },
            'ResourceArn': 'string',
            'RoleArn': 'string',
            'Timeout': 123,
            'Tags': {
                'string': 'string'
            }
        },
    ],

}

Response Structure

  • (dict) --

    • Jobs (list) --

      A list of jobs that are defined in the current AWS account.

      • (dict) --

        Represents all of the attributes of an AWS Glue DataBrew job.

        • AccountId (string) --

          The ID of the AWS account that owns the job.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the job.

        • CreateDate (datetime) --

          The date and time that the job was created.

        • DatasetName (string) --

          A dataset that the job is to process.

        • EncryptionKeyArn (string) --

          The Amazon Resource Name (ARN) of an encryption key that is used to protect a job.

        • EncryptionMode (string) --

          The encryption mode for the job, which can be one of the following:

          • SSE-KMS - Server-side encryption with AWS KMS-managed keys.
          • SSE-S3 - Server-side encryption with keys managed by Amazon S3.
        • Name (string) --

          The unique name of the job.

        • Type (string) --

          The job type of the job, which must be one of the following:

          • PROFILE - A job to analyze a dataset, to determine its size, data types, data distribution, and more.
          • RECIPE - A job to apply one or more transformations to a dataset.
        • LastModifiedBy (string) --

          The identifier (the user name) of the user who last modified the job.

        • LastModifiedDate (datetime) --

          The modification date and time of the job.

        • LogSubscription (string) --

          The current status of Amazon CloudWatch logging for the job.

        • MaxCapacity (integer) --

          The maximum number of nodes that can be consumed when the job processes data.

        • MaxRetries (integer) --

          The maximum number of times to retry the job after a job run fails.

        • Outputs (list) --

          One or more artifacts that represent output from running the job.

          • (dict) --

            Represents individual output from a particular job run.

            • CompressionFormat (string) --

              The compression algorithm used to compress the output text of the job.

            • Format (string) --

              The data format of the output of the job.

            • PartitionColumns (list) --

              The names of one or more partition columns for the output of the job.

              • (string) --
            • Location (dict) --

              The location in Amazon S3 where the job writes its output.

              • Bucket (string) --

                The S3 bucket name.

              • Key (string) --

                The unique name of the object in the bucket.

            • Overwrite (boolean) --

              A value that, if true, means that any data in the location specified for output is overwritten with new output.

        • ProjectName (string) --

          The name of the project that the job is associated with.

        • RecipeReference (dict) --

          A set of steps that the job runs.

          • Name (string) --

            The name of the recipe.

          • RecipeVersion (string) --

            The identifier for the version for the recipe.

        • ResourceArn (string) --

          The unique Amazon Resource Name (ARN) for the job.

        • RoleArn (string) --

          The Amazon Resource Name (ARN) of the role that will be assumed for this job.

        • Timeout (integer) --

          The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .

        • Tags (dict) --

          Metadata tags that have been applied to the job.

          • (string) --
            • (string) --

class GlueDataBrew.Paginator.ListProjects
paginator = client.get_paginator('list_projects')
paginate(**kwargs)

Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_projects().

See also: AWS API Documentation

Request Syntax

response_iterator = paginator.paginate(
    PaginationConfig={
        'MaxItems': 123,
        'PageSize': 123,
        'StartingToken': 'string'
    }
)
Parameters
PaginationConfig (dict) --

A dictionary that provides parameters to control pagination.

  • MaxItems (integer) --

    The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.

  • PageSize (integer) --

    The size of each page.

  • StartingToken (string) --

    A token to specify where to start paginating. This is the NextToken from a previous response.

Return type
dict
Returns
Response Syntax
{
    'Projects': [
        {
            'AccountId': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'CreatedBy': 'string',
            'DatasetName': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'LastModifiedBy': 'string',
            'Name': 'string',
            'RecipeName': 'string',
            'ResourceArn': 'string',
            'Sample': {
                'Size': 123,
                'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
            },
            'Tags': {
                'string': 'string'
            },
            'RoleArn': 'string',
            'OpenedBy': 'string',
            'OpenDate': datetime(2015, 1, 1)
        },
    ],

}

Response Structure

  • (dict) --
    • Projects (list) --

      A list of projects that are defined in the current AWS account.

      • (dict) --

        Represents all of the attributes of an AWS Glue DataBrew project.

        • AccountId (string) --

          The ID of the AWS account that owns the project.

        • CreateDate (datetime) --

          The date and time that the project was created.

        • CreatedBy (string) --

          The identifier (the user name) of the user who crated the project.

        • DatasetName (string) --

          The dataset that the project is to act upon.

        • LastModifiedDate (datetime) --

          The last modification date and time for the project.

        • LastModifiedBy (string) --

          The identifier (user name) of the user who last modified the project.

        • Name (string) --

          The unique name of a project.

        • RecipeName (string) --

          The name of a recipe that will be developed during a project session.

        • ResourceArn (string) --

          The Amazon Resource Name (ARN) for the project.

        • Sample (dict) --

          The sample size and sampling type to apply to the data. If this parameter isn't specified, then the sample will consiste of the first 500 rows from the dataset.

          • Size (integer) --

            The number of rows in the sample.

          • Type (string) --

            The way in which DataBrew obtains rows from a dataset.

        • Tags (dict) --

          Metadata tags that have been applied to the project.

          • (string) --
            • (string) --
        • RoleArn (string) --

          The Amazon Resource Name (ARN) of the role that will be assumed for this project.

        • OpenedBy (string) --

          The identifier (the user name) of the user that opened the project for use.

        • OpenDate (datetime) --

          The date and time when the project was opened.

class GlueDataBrew.Paginator.ListRecipeVersions
paginator = client.get_paginator('list_recipe_versions')
paginate(**kwargs)

Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_recipe_versions().

See also: AWS API Documentation

Request Syntax

response_iterator = paginator.paginate(
    Name='string',
    PaginationConfig={
        'MaxItems': 123,
        'PageSize': 123,
        'StartingToken': 'string'
    }
)
Parameters
  • Name (string) --

    [REQUIRED]

    The name of the recipe for which to return version information.

  • PaginationConfig (dict) --

    A dictionary that provides parameters to control pagination.

    • MaxItems (integer) --

      The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.

    • PageSize (integer) --

      The size of each page.

    • StartingToken (string) --

      A token to specify where to start paginating. This is the NextToken from a previous response.

Return type

dict

Returns

Response Syntax

{
    'Recipes': [
        {
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'LastModifiedBy': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'ProjectName': 'string',
            'PublishedBy': 'string',
            'PublishedDate': datetime(2015, 1, 1),
            'Description': 'string',
            'Name': 'string',
            'ResourceArn': 'string',
            'Steps': [
                {
                    'Action': {
                        'Operation': 'string',
                        'Parameters': {
                            'string': 'string'
                        }
                    },
                    'ConditionExpressions': [
                        {
                            'Condition': 'string',
                            'Value': 'string',
                            'TargetColumn': 'string'
                        },
                    ]
                },
            ],
            'Tags': {
                'string': 'string'
            },
            'RecipeVersion': 'string'
        },
    ]
}

Response Structure

  • (dict) --

    • Recipes (list) --

      A list of versions for the specified recipe.

      • (dict) --

        Represents one or more actions to be performed on an AWS Glue DataBrew dataset.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the recipe.

        • CreateDate (datetime) --

          The date and time that the recipe was created.

        • LastModifiedBy (string) --

          The identifier (user name) of the user who last modified the recipe.

        • LastModifiedDate (datetime) --

          The last modification date and time of the recipe.

        • ProjectName (string) --

          The name of the project that the recipe is associated with.

        • PublishedBy (string) --

          The identifier (the user name) of the user who published the recipe.

        • PublishedDate (datetime) --

          The date and time when the recipe was published.

        • Description (string) --

          The description of the recipe.

        • Name (string) --

          The unique name for the recipe.

        • ResourceArn (string) --

          The Amazon Resource Name (ARN) for the recipe.

        • Steps (list) --

          A list of steps that are defined by the recipe.

          • (dict) --

            Represents a single step to be performed in an AWS Glue DataBrew recipe.

            • Action (dict) --

              The particular action to be performed in the recipe step.

              • Operation (string) --

                The name of a valid DataBrew transformation to be performed on the data.

              • Parameters (dict) --

                Contextual parameters for the transformation.

                • (string) --
                  • (string) --
            • ConditionExpressions (list) --

              One or more conditions that must be met, in order for the recipe step to succeed.

              Note

              All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.

              • (dict) --

                Represents an individual condition that evaluates to true or false.

                Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.

                If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.

                • Condition (string) --

                  A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .

                • Value (string) --

                  A value that the condition must evaluate to for the condition to succeed.

                • TargetColumn (string) --

                  A column to apply this condition to, within an AWS Glue DataBrew dataset.

        • Tags (dict) --

          Metadata tags that have been applied to the recipe.

          • (string) --
            • (string) --
        • RecipeVersion (string) --

          The identifier for the version for the recipe.

class GlueDataBrew.Paginator.ListRecipes
paginator = client.get_paginator('list_recipes')
paginate(**kwargs)

Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_recipes().

See also: AWS API Documentation

Request Syntax

response_iterator = paginator.paginate(
    RecipeVersion='string',
    PaginationConfig={
        'MaxItems': 123,
        'PageSize': 123,
        'StartingToken': 'string'
    }
)
Parameters
  • RecipeVersion (string) -- A version identifier. Using this parameter indicates to return only those recipes that have this version identifier.
  • PaginationConfig (dict) --

    A dictionary that provides parameters to control pagination.

    • MaxItems (integer) --

      The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.

    • PageSize (integer) --

      The size of each page.

    • StartingToken (string) --

      A token to specify where to start paginating. This is the NextToken from a previous response.

Return type

dict

Returns

Response Syntax

{
    'Recipes': [
        {
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'LastModifiedBy': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'ProjectName': 'string',
            'PublishedBy': 'string',
            'PublishedDate': datetime(2015, 1, 1),
            'Description': 'string',
            'Name': 'string',
            'ResourceArn': 'string',
            'Steps': [
                {
                    'Action': {
                        'Operation': 'string',
                        'Parameters': {
                            'string': 'string'
                        }
                    },
                    'ConditionExpressions': [
                        {
                            'Condition': 'string',
                            'Value': 'string',
                            'TargetColumn': 'string'
                        },
                    ]
                },
            ],
            'Tags': {
                'string': 'string'
            },
            'RecipeVersion': 'string'
        },
    ],

}

Response Structure

  • (dict) --

    • Recipes (list) --

      A list of recipes that are defined in the current AWS account.

      • (dict) --

        Represents one or more actions to be performed on an AWS Glue DataBrew dataset.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the recipe.

        • CreateDate (datetime) --

          The date and time that the recipe was created.

        • LastModifiedBy (string) --

          The identifier (user name) of the user who last modified the recipe.

        • LastModifiedDate (datetime) --

          The last modification date and time of the recipe.

        • ProjectName (string) --

          The name of the project that the recipe is associated with.

        • PublishedBy (string) --

          The identifier (the user name) of the user who published the recipe.

        • PublishedDate (datetime) --

          The date and time when the recipe was published.

        • Description (string) --

          The description of the recipe.

        • Name (string) --

          The unique name for the recipe.

        • ResourceArn (string) --

          The Amazon Resource Name (ARN) for the recipe.

        • Steps (list) --

          A list of steps that are defined by the recipe.

          • (dict) --

            Represents a single step to be performed in an AWS Glue DataBrew recipe.

            • Action (dict) --

              The particular action to be performed in the recipe step.

              • Operation (string) --

                The name of a valid DataBrew transformation to be performed on the data.

              • Parameters (dict) --

                Contextual parameters for the transformation.

                • (string) --
                  • (string) --
            • ConditionExpressions (list) --

              One or more conditions that must be met, in order for the recipe step to succeed.

              Note

              All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.

              • (dict) --

                Represents an individual condition that evaluates to true or false.

                Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.

                If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.

                • Condition (string) --

                  A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .

                • Value (string) --

                  A value that the condition must evaluate to for the condition to succeed.

                • TargetColumn (string) --

                  A column to apply this condition to, within an AWS Glue DataBrew dataset.

        • Tags (dict) --

          Metadata tags that have been applied to the recipe.

          • (string) --
            • (string) --
        • RecipeVersion (string) --

          The identifier for the version for the recipe.

class GlueDataBrew.Paginator.ListSchedules
paginator = client.get_paginator('list_schedules')
paginate(**kwargs)

Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_schedules().

See also: AWS API Documentation

Request Syntax

response_iterator = paginator.paginate(
    JobName='string',
    PaginationConfig={
        'MaxItems': 123,
        'PageSize': 123,
        'StartingToken': 'string'
    }
)
Parameters
  • JobName (string) -- The name of the job that these schedules apply to.
  • PaginationConfig (dict) --

    A dictionary that provides parameters to control pagination.

    • MaxItems (integer) --

      The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.

    • PageSize (integer) --

      The size of each page.

    • StartingToken (string) --

      A token to specify where to start paginating. This is the NextToken from a previous response.

Return type

dict

Returns

Response Syntax

{
    'Schedules': [
        {
            'AccountId': 'string',
            'CreatedBy': 'string',
            'CreateDate': datetime(2015, 1, 1),
            'JobNames': [
                'string',
            ],
            'LastModifiedBy': 'string',
            'LastModifiedDate': datetime(2015, 1, 1),
            'ResourceArn': 'string',
            'CronExpression': 'string',
            'Tags': {
                'string': 'string'
            },
            'Name': 'string'
        },
    ],

}

Response Structure

  • (dict) --

    • Schedules (list) --

      A list of schedules in the current AWS account.

      • (dict) --

        Represents one or more dates and times when a job is to run.

        • AccountId (string) --

          The ID of the AWS account that owns the schedule.

        • CreatedBy (string) --

          The identifier (the user name) of the user who created the schedule.

        • CreateDate (datetime) --

          The date and time that the schedule was created.

        • JobNames (list) --

          A list of jobs to be run, according to the schedule.

          • (string) --
        • LastModifiedBy (string) --

          The identifier (the user name) of the user who last modified the schedule.

        • LastModifiedDate (datetime) --

          The date and time when the schedule was last modified.

        • ResourceArn (string) --

          The Amazon Resource Name (ARN) of the schedule.

        • CronExpression (string) --

          The date(s) and time(s), in cron format, when the job will run.

        • Tags (dict) --

          Metadata tags that have been applied to the schedule.

          • (string) --
            • (string) --
        • Name (string) --

          The name of the schedule.