Table of Contents
A low-level client representing AWS Glue DataBrew
AWS Glue DataBrew is a visual, cloud-scale data-preparation service. DataBrew simplifies data preparation tasks, targeting data issues that are hard to spot and time-consuming to fix. DataBrew empowers users of all technical levels to visualize the data and perform one-click data transformations, with no coding required.
import boto3
client = boto3.client('databrew')
These are the available methods:
Deletes one or more versions of a recipe at a time.
See also: AWS API Documentation
Request Syntax
response = client.batch_delete_recipe_version(
Name='string',
RecipeVersions=[
'string',
]
)
[REQUIRED]
The name of the recipe to be modified.
[REQUIRED]
An array of version identifiers to be deleted.
dict
Response Syntax
{
'Name': 'string',
'Errors': [
{
'ErrorCode': 'string',
'ErrorMessage': 'string',
'RecipeVersion': 'string'
},
]
}
Response Structure
(dict) --
Name (string) --
The name of the recipe that was modified.
Errors (list) --
Errors, if any, that were encountered when deleting the recipe versions.
(dict) --
Represents any errors encountered when attempting to delete multiple recipe versions.
ErrorCode (string) --
The HTTP status code for the error.
ErrorMessage (string) --
The text of the error message.
RecipeVersion (string) --
The identifier for the recipe version associated with this error.
Exceptions
Check if an operation can be paginated.
Creates a new AWS Glue DataBrew dataset for this AWS account.
See also: AWS API Documentation
Request Syntax
response = client.create_dataset(
Name='string',
FormatOptions={
'Json': {
'MultiLine': True|False
},
'Excel': {
'SheetNames': [
'string',
],
'SheetIndexes': [
123,
]
}
},
Input={
'S3InputDefinition': {
'Bucket': 'string',
'Key': 'string'
},
'DataCatalogInputDefinition': {
'CatalogId': 'string',
'DatabaseName': 'string',
'TableName': 'string',
'TempDirectory': {
'Bucket': 'string',
'Key': 'string'
}
}
},
Tags={
'string': 'string'
}
)
[REQUIRED]
The name of the dataset to be created.
Options that define how Microsoft Excel input is to be interpreted by DataBrew.
Options that define how JSON input is to be interpreted by DataBrew.
A value that specifies whether JSON input contains embedded new line characters.
Options that define how Excel input is to be interpreted by DataBrew.
Specifies one or more named sheets in the Excel file, which will be included in the dataset.
Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.
[REQUIRED]
Information on how AWS Glue DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.
The Amazon S3 location where the data is stored.
The S3 bucket name.
The unique name of the object in the bucket.
The AWS Glue Data Catalog parameters for the data.
The unique identifier of the AWS account that holds the Data Catalog that stores the data.
The name of a database in the Data Catalog.
The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.
An Amazon location that AWS Glue Data Catalog can use as a temporary directory.
The S3 bucket name.
The unique name of the object in the bucket.
Metadata tags to apply to this dataset.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the dataset that you created.
Exceptions
Creates a new job to profile an AWS Glue DataBrew dataset that exists in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.create_profile_job(
DatasetName='string',
EncryptionKeyArn='string',
EncryptionMode='SSE-KMS'|'SSE-S3',
Name='string',
LogSubscription='ENABLE'|'DISABLE',
MaxCapacity=123,
MaxRetries=123,
OutputLocation={
'Bucket': 'string',
'Key': 'string'
},
RoleArn='string',
Tags={
'string': 'string'
},
Timeout=123
)
[REQUIRED]
The name of the dataset that this job is to act upon.
The encryption mode for the job, which can be one of the following:
[REQUIRED]
The name of the job to be created.
[REQUIRED]
An Amazon S3 location (bucket name an object key) where DataBrew can read input data, or write output from a job.
The S3 bucket name.
The unique name of the object in the bucket.
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.
Metadata tags to apply to this job.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the job that was created.
Exceptions
Creates a new AWS Glue DataBrew project in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.create_project(
DatasetName='string',
Name='string',
RecipeName='string',
Sample={
'Size': 123,
'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
},
RoleArn='string',
Tags={
'string': 'string'
}
)
[REQUIRED]
The name of the dataset to associate this project with.
[REQUIRED]
A unique name for the new project.
[REQUIRED]
The name of an existing recipe to associate with the project.
Represents the sample size and sampling type for AWS Glue DataBrew to use for interactive data analysis.
The number of rows in the sample.
The way in which DataBrew obtains rows from a dataset.
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.
Metadata tags to apply to this project.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the project that you created.
Exceptions
Creates a new AWS Glue DataBrew recipe for the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.create_recipe(
Description='string',
Name='string',
Steps=[
{
'Action': {
'Operation': 'string',
'Parameters': {
'string': 'string'
}
},
'ConditionExpressions': [
{
'Condition': 'string',
'Value': 'string',
'TargetColumn': 'string'
},
]
},
],
Tags={
'string': 'string'
}
)
[REQUIRED]
A unique name for the recipe.
[REQUIRED]
An array containing the steps to be performed by the recipe. Each recipe step consists of one recipe action and (optionally) an array of condition expressions.
Represents a single step to be performed in an AWS Glue DataBrew recipe.
The particular action to be performed in the recipe step.
The name of a valid DataBrew transformation to be performed on the data.
Contextual parameters for the transformation.
One or more conditions that must be met, in order for the recipe step to succeed.
Note
All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.
Represents an individual condition that evaluates to true or false.
Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.
If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.
A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .
A value that the condition must evaluate to for the condition to succeed.
A column to apply this condition to, within an AWS Glue DataBrew dataset.
Metadata tags to apply to this recipe.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the recipe that you created.
Exceptions
Creates a new job for an existing AWS Glue DataBrew recipe in the current AWS account. You can create a standalone job using either a project, or a combination of a recipe and a dataset.
See also: AWS API Documentation
Request Syntax
response = client.create_recipe_job(
DatasetName='string',
EncryptionKeyArn='string',
EncryptionMode='SSE-KMS'|'SSE-S3',
Name='string',
LogSubscription='ENABLE'|'DISABLE',
MaxCapacity=123,
MaxRetries=123,
Outputs=[
{
'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
'PartitionColumns': [
'string',
],
'Location': {
'Bucket': 'string',
'Key': 'string'
},
'Overwrite': True|False
},
],
ProjectName='string',
RecipeReference={
'Name': 'string',
'RecipeVersion': 'string'
},
RoleArn='string',
Tags={
'string': 'string'
},
Timeout=123
)
The encryption mode for the job, which can be one of the following:
[REQUIRED]
A unique name for the job.
[REQUIRED]
One or more artifacts that represent the output from running the job.
Represents individual output from a particular job run.
The compression algorithm used to compress the output text of the job.
The data format of the output of the job.
The names of one or more partition columns for the output of the job.
The location in Amazon S3 where the job writes its output.
The S3 bucket name.
The unique name of the object in the bucket.
A value that, if true, means that any data in the location specified for output is overwritten with new output.
Represents all of the attributes of an AWS Glue DataBrew recipe.
The name of the recipe.
The identifier for the version for the recipe.
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.
Metadata tags to apply to this job dataset.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the job that you created.
Exceptions
Creates a new schedule for one or more AWS Glue DataBrew jobs. Jobs can be run at a specific date and time, or at regular intervals.
See also: AWS API Documentation
Request Syntax
response = client.create_schedule(
JobNames=[
'string',
],
CronExpression='string',
Tags={
'string': 'string'
},
Name='string'
)
The name or names of one or more jobs to be run.
[REQUIRED]
The date or dates and time or times, in cron format, when the jobs are to be run.
Metadata tags to apply to this schedule.
[REQUIRED]
A unique name for the schedule.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the schedule that was created.
Exceptions
Deletes a dataset from AWS Glue DataBrew.
See also: AWS API Documentation
Request Syntax
response = client.delete_dataset(
Name='string'
)
[REQUIRED]
The name of the dataset to be deleted.
{
'Name': 'string'
}
Response Structure
The name of the dataset that you deleted.
Exceptions
Deletes the specified AWS Glue DataBrew job from the current AWS account. The job can be for a recipe or for a profile.
See also: AWS API Documentation
Request Syntax
response = client.delete_job(
Name='string'
)
[REQUIRED]
The name of the job to be deleted.
{
'Name': 'string'
}
Response Structure
The name of the job that you deleted.
Exceptions
Deletes an existing AWS Glue DataBrew project from the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.delete_project(
Name='string'
)
[REQUIRED]
The name of the project to be deleted.
{
'Name': 'string'
}
Response Structure
The name of the project that you deleted.
Exceptions
Deletes a single version of an AWS Glue DataBrew recipe.
See also: AWS API Documentation
Request Syntax
response = client.delete_recipe_version(
Name='string',
RecipeVersion='string'
)
[REQUIRED]
The name of the recipe to be deleted.
[REQUIRED]
The version of the recipe to be deleted.
dict
Response Syntax
{
'Name': 'string',
'RecipeVersion': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the recipe that was deleted.
RecipeVersion (string) --
The version of the recipe that was deleted.
Exceptions
Deletes the specified AWS Glue DataBrew schedule from the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.delete_schedule(
Name='string'
)
[REQUIRED]
The name of the schedule to be deleted.
{
'Name': 'string'
}
Response Structure
The name of the schedule that was deleted.
Exceptions
Returns the definition of a specific AWS Glue DataBrew dataset that is in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.describe_dataset(
Name='string'
)
[REQUIRED]
The name of the dataset to be described.
{
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'Name': 'string',
'FormatOptions': {
'Json': {
'MultiLine': True|False
},
'Excel': {
'SheetNames': [
'string',
],
'SheetIndexes': [
123,
]
}
},
'Input': {
'S3InputDefinition': {
'Bucket': 'string',
'Key': 'string'
},
'DataCatalogInputDefinition': {
'CatalogId': 'string',
'DatabaseName': 'string',
'TableName': 'string',
'TempDirectory': {
'Bucket': 'string',
'Key': 'string'
}
}
},
'LastModifiedDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'Source': 'S3'|'DATA-CATALOG',
'Tags': {
'string': 'string'
},
'ResourceArn': 'string'
}
Response Structure
The identifier (user name) of the user who created the dataset.
The date and time that the dataset was created.
The name of the dataset.
Options that define how Microsoft Excel input is to be interpreted by DataBrew.
Options that define how JSON input is to be interpreted by DataBrew.
A value that specifies whether JSON input contains embedded new line characters.
Options that define how Excel input is to be interpreted by DataBrew.
Specifies one or more named sheets in the Excel file, which will be included in the dataset.
Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.
Information on how AWS Glue DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.
The Amazon S3 location where the data is stored.
The S3 bucket name.
The unique name of the object in the bucket.
The AWS Glue Data Catalog parameters for the data.
The unique identifier of the AWS account that holds the Data Catalog that stores the data.
The name of a database in the Data Catalog.
The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.
An Amazon location that AWS Glue Data Catalog can use as a temporary directory.
The S3 bucket name.
The unique name of the object in the bucket.
The date and time that the dataset was last modified.
The identifier (user name) of the user who last modified the dataset.
The location of the data for this dataset, Amazon S3 or the AWS Glue Data Catalog.
Metadata tags associated with this dataset.
The Amazon Resource Name (ARN) of the dataset.
Exceptions
Returns the definition of a specific AWS Glue DataBrew job that is in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.describe_job(
Name='string'
)
[REQUIRED]
The name of the job to be described.
{
'CreateDate': datetime(2015, 1, 1),
'CreatedBy': 'string',
'DatasetName': 'string',
'EncryptionKeyArn': 'string',
'EncryptionMode': 'SSE-KMS'|'SSE-S3',
'Name': 'string',
'Type': 'PROFILE'|'RECIPE',
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'LogSubscription': 'ENABLE'|'DISABLE',
'MaxCapacity': 123,
'MaxRetries': 123,
'Outputs': [
{
'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
'PartitionColumns': [
'string',
],
'Location': {
'Bucket': 'string',
'Key': 'string'
},
'Overwrite': True|False
},
],
'ProjectName': 'string',
'RecipeReference': {
'Name': 'string',
'RecipeVersion': 'string'
},
'ResourceArn': 'string',
'RoleArn': 'string',
'Tags': {
'string': 'string'
},
'Timeout': 123
}
Response Structure
The date and time that the job was created.
The identifier (user name) of the user associated with the creation of the job.
The dataset that the job acts upon.
The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
The encryption mode for the job, which can be one of the following:
The name of the job.
The job type, which must be one of the following:
The identifier (user name) of the user who last modified the job.
The date and time that the job was last modified.
A value that indicates whether Amazon CloudWatch logging is enabled for this job.
The maximum number of nodes that AWS Glue DataBrew can consume when the job processes data.
The maximum number of times to retry the job after a job run fails.
One or more artifacts that represent the output from running the job.
Represents individual output from a particular job run.
The compression algorithm used to compress the output text of the job.
The data format of the output of the job.
The names of one or more partition columns for the output of the job.
The location in Amazon S3 where the job writes its output.
The S3 bucket name.
The unique name of the object in the bucket.
A value that, if true, means that any data in the location specified for output is overwritten with new output.
The DataBrew project associated with this job.
Represents all of the attributes of an AWS Glue DataBrew recipe.
The name of the recipe.
The identifier for the version for the recipe.
The Amazon Resource Name (ARN) of the job.
The ARN of the AWS Identity and Access Management (IAM) role that was assumed for this request.
Metadata tags associated with this job.
The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .
Exceptions
Returns the definition of a specific AWS Glue DataBrew project that is in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.describe_project(
Name='string'
)
[REQUIRED]
The name of the project to be described.
{
'CreateDate': datetime(2015, 1, 1),
'CreatedBy': 'string',
'DatasetName': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'Name': 'string',
'RecipeName': 'string',
'ResourceArn': 'string',
'Sample': {
'Size': 123,
'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
},
'RoleArn': 'string',
'Tags': {
'string': 'string'
},
'SessionStatus': 'ASSIGNED'|'FAILED'|'INITIALIZING'|'PROVISIONING'|'READY'|'RECYCLING'|'ROTATING'|'TERMINATED'|'TERMINATING'|'UPDATING',
'OpenedBy': 'string',
'OpenDate': datetime(2015, 1, 1)
}
Response Structure
The date and time that the project was created.
The identifier (user name) of the user who created the project.
The dataset associated with the project.
The date and time that the project was last modified.
The identifier (user name) of the user who last modified the project.
The name of the project.
The recipe associated with this job.
The Amazon Resource Name (ARN) of the project.
Represents the sample size and sampling type for AWS Glue DataBrew to use for interactive data analysis.
The number of rows in the sample.
The way in which DataBrew obtains rows from a dataset.
The ARN of the AWS Identity and Access Management (IAM) role that was assumed for this request.
Metadata tags associated with this project.
Describes the current state of the session:
The identifier (user name) of the user that opened the project for use.
The date and time when the project was opened.
Exceptions
Returns the definition of a specific AWS Glue DataBrew recipe that is in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.describe_recipe(
Name='string',
RecipeVersion='string'
)
[REQUIRED]
The name of the recipe to be described.
dict
Response Syntax
{
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'ProjectName': 'string',
'PublishedBy': 'string',
'PublishedDate': datetime(2015, 1, 1),
'Description': 'string',
'Name': 'string',
'Steps': [
{
'Action': {
'Operation': 'string',
'Parameters': {
'string': 'string'
}
},
'ConditionExpressions': [
{
'Condition': 'string',
'Value': 'string',
'TargetColumn': 'string'
},
]
},
],
'Tags': {
'string': 'string'
},
'ResourceArn': 'string',
'RecipeVersion': 'string'
}
Response Structure
(dict) --
CreatedBy (string) --
The identifier (user name) of the user who created the recipe.
CreateDate (datetime) --
The date and time that the recipe was created.
LastModifiedBy (string) --
The identifier (user name) of the user who last modified the recipe.
LastModifiedDate (datetime) --
The date and time that the recipe was last modified.
ProjectName (string) --
The name of the project associated with this recipe.
PublishedBy (string) --
The identifier (user name) of the user who last published the recipe.
PublishedDate (datetime) --
The date and time when the recipe was last published.
Description (string) --
The description of the recipe.
Name (string) --
The name of the recipe.
Steps (list) --
One or more steps to be performed by the recipe. Each step consists of an action, and the conditions under which the action should succeed.
(dict) --
Represents a single step to be performed in an AWS Glue DataBrew recipe.
Action (dict) --
The particular action to be performed in the recipe step.
Operation (string) --
The name of a valid DataBrew transformation to be performed on the data.
Parameters (dict) --
Contextual parameters for the transformation.
ConditionExpressions (list) --
One or more conditions that must be met, in order for the recipe step to succeed.
Note
All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.
(dict) --
Represents an individual condition that evaluates to true or false.
Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.
If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.
Condition (string) --
A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .
Value (string) --
A value that the condition must evaluate to for the condition to succeed.
TargetColumn (string) --
A column to apply this condition to, within an AWS Glue DataBrew dataset.
Tags (dict) --
Metadata tags associated with this project.
ResourceArn (string) --
The ARN of the recipe.
RecipeVersion (string) --
The recipe version identifier.
Exceptions
Returns the definition of a specific AWS Glue DataBrew schedule that is in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.describe_schedule(
Name='string'
)
[REQUIRED]
The name of the schedule to be described.
{
'CreateDate': datetime(2015, 1, 1),
'CreatedBy': 'string',
'JobNames': [
'string',
],
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'ResourceArn': 'string',
'CronExpression': 'string',
'Tags': {
'string': 'string'
},
'Name': 'string'
}
Response Structure
The date and time that the schedule was created.
The identifier (user name) of the user who created the schedule.
The name or names of one or more jobs to be run by using the schedule.
The identifier (user name) of the user who last modified the schedule.
The date and time that the schedule was last modified.
The Amazon Resource Name (ARN) of the schedule.
The date or dates and time or times, in cron format, when the jobs are to be run for the schedule.
Metadata tags associated with this schedule.
The name of the schedule.
Exceptions
Generate a presigned url given a client, its method, and arguments
The presigned url
Create a paginator for an operation.
Returns an object that can wait for some condition.
Lists all of the AWS Glue DataBrew datasets for the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.list_datasets(
MaxResults=123,
NextToken='string'
)
dict
Response Syntax
{
'Datasets': [
{
'AccountId': 'string',
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'Name': 'string',
'FormatOptions': {
'Json': {
'MultiLine': True|False
},
'Excel': {
'SheetNames': [
'string',
],
'SheetIndexes': [
123,
]
}
},
'Input': {
'S3InputDefinition': {
'Bucket': 'string',
'Key': 'string'
},
'DataCatalogInputDefinition': {
'CatalogId': 'string',
'DatabaseName': 'string',
'TableName': 'string',
'TempDirectory': {
'Bucket': 'string',
'Key': 'string'
}
}
},
'LastModifiedDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'Source': 'S3'|'DATA-CATALOG',
'Tags': {
'string': 'string'
},
'ResourceArn': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Datasets (list) --
A list of datasets that are defined in the current AWS account.
(dict) --
Represents a dataset that can be processed by AWS Glue DataBrew.
AccountId (string) --
The ID of the AWS account that owns the dataset.
CreatedBy (string) --
The identifier (the user name) of the user who created the dataset.
CreateDate (datetime) --
The date and time that the dataset was created.
Name (string) --
The unique name of the dataset.
FormatOptions (dict) --
Options that define how DataBrew interprets the data in the dataset.
Json (dict) --
Options that define how JSON input is to be interpreted by DataBrew.
MultiLine (boolean) --
A value that specifies whether JSON input contains embedded new line characters.
Excel (dict) --
Options that define how Excel input is to be interpreted by DataBrew.
SheetNames (list) --
Specifies one or more named sheets in the Excel file, which will be included in the dataset.
SheetIndexes (list) --
Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.
Input (dict) --
Information on how DataBrew can find the dataset, in either the AWS Glue Data Catalog or Amazon S3.
S3InputDefinition (dict) --
The Amazon S3 location where the data is stored.
Bucket (string) --
The S3 bucket name.
Key (string) --
The unique name of the object in the bucket.
DataCatalogInputDefinition (dict) --
The AWS Glue Data Catalog parameters for the data.
CatalogId (string) --
The unique identifier of the AWS account that holds the Data Catalog that stores the data.
DatabaseName (string) --
The name of a database in the Data Catalog.
TableName (string) --
The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.
TempDirectory (dict) --
An Amazon location that AWS Glue Data Catalog can use as a temporary directory.
Bucket (string) --
The S3 bucket name.
Key (string) --
The unique name of the object in the bucket.
LastModifiedDate (datetime) --
The last modification date and time of the dataset.
LastModifiedBy (string) --
The identifier (the user name) of the user who last modified the dataset.
Source (string) --
The location of the data for the dataset, either Amazon S3 or the AWS Glue Data Catalog.
Tags (dict) --
Metadata tags that have been applied to the dataset.
ResourceArn (string) --
The unique Amazon Resource Name (ARN) for the dataset.
NextToken (string) --
A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To obtain the next set of pages, pass in the NextToken from the response object of the previous page call.
Exceptions
Lists all of the previous runs of a particular AWS Glue DataBrew job in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.list_job_runs(
Name='string',
MaxResults=123,
NextToken='string'
)
[REQUIRED]
The name of the job.
dict
Response Syntax
{
'JobRuns': [
{
'Attempt': 123,
'CompletedOn': datetime(2015, 1, 1),
'DatasetName': 'string',
'ErrorMessage': 'string',
'ExecutionTime': 123,
'JobName': 'string',
'RunId': 'string',
'State': 'STARTING'|'RUNNING'|'STOPPING'|'STOPPED'|'SUCCEEDED'|'FAILED'|'TIMEOUT',
'LogSubscription': 'ENABLE'|'DISABLE',
'LogGroupName': 'string',
'Outputs': [
{
'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
'PartitionColumns': [
'string',
],
'Location': {
'Bucket': 'string',
'Key': 'string'
},
'Overwrite': True|False
},
],
'RecipeReference': {
'Name': 'string',
'RecipeVersion': 'string'
},
'StartedBy': 'string',
'StartedOn': datetime(2015, 1, 1)
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
JobRuns (list) --
A list of job runs that have occurred for the specified job.
(dict) --
Represents one run of an AWS Glue DataBrew job.
Attempt (integer) --
The number of times that DataBrew has attempted to run the job.
CompletedOn (datetime) --
The date and time when the job completed processing.
DatasetName (string) --
The name of the dataset for the job to process.
ErrorMessage (string) --
A message indicating an error (if any) that was encountered when the job ran.
ExecutionTime (integer) --
The amount of time, in seconds, during which a job run consumed resources.
JobName (string) --
The name of the job being processed during this run.
RunId (string) --
The unique identifier of the job run.
State (string) --
The current state of the job run entity itself.
LogSubscription (string) --
The current status of Amazon CloudWatch logging for the job run.
LogGroupName (string) --
The name of an Amazon CloudWatch log group, where the job writes diagnostic messages when it runs.
Outputs (list) --
One or more output artifacts from a job run.
(dict) --
Represents individual output from a particular job run.
CompressionFormat (string) --
The compression algorithm used to compress the output text of the job.
Format (string) --
The data format of the output of the job.
PartitionColumns (list) --
The names of one or more partition columns for the output of the job.
Location (dict) --
The location in Amazon S3 where the job writes its output.
Bucket (string) --
The S3 bucket name.
Key (string) --
The unique name of the object in the bucket.
Overwrite (boolean) --
A value that, if true, means that any data in the location specified for output is overwritten with new output.
RecipeReference (dict) --
The set of steps processed by the job.
Name (string) --
The name of the recipe.
RecipeVersion (string) --
The identifier for the version for the recipe.
StartedBy (string) --
The identifier (the user name) of the user who initiated the job run.
StartedOn (datetime) --
The date and time when the job run began.
NextToken (string) --
A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To obtain the next set of pages, pass in the NextToken from the response object of the previous page call.
Exceptions
Lists the AWS Glue DataBrew jobs in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.list_jobs(
DatasetName='string',
MaxResults=123,
NextToken='string',
ProjectName='string'
)
dict
Response Syntax
{
'Jobs': [
{
'AccountId': 'string',
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'DatasetName': 'string',
'EncryptionKeyArn': 'string',
'EncryptionMode': 'SSE-KMS'|'SSE-S3',
'Name': 'string',
'Type': 'PROFILE'|'RECIPE',
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'LogSubscription': 'ENABLE'|'DISABLE',
'MaxCapacity': 123,
'MaxRetries': 123,
'Outputs': [
{
'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
'PartitionColumns': [
'string',
],
'Location': {
'Bucket': 'string',
'Key': 'string'
},
'Overwrite': True|False
},
],
'ProjectName': 'string',
'RecipeReference': {
'Name': 'string',
'RecipeVersion': 'string'
},
'ResourceArn': 'string',
'RoleArn': 'string',
'Timeout': 123,
'Tags': {
'string': 'string'
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Jobs (list) --
A list of jobs that are defined in the current AWS account.
(dict) --
Represents all of the attributes of an AWS Glue DataBrew job.
AccountId (string) --
The ID of the AWS account that owns the job.
CreatedBy (string) --
The identifier (the user name) of the user who created the job.
CreateDate (datetime) --
The date and time that the job was created.
DatasetName (string) --
A dataset that the job is to process.
EncryptionKeyArn (string) --
The Amazon Resource Name (ARN) of an encryption key that is used to protect a job.
EncryptionMode (string) --
The encryption mode for the job, which can be one of the following:
Name (string) --
The unique name of the job.
Type (string) --
The job type of the job, which must be one of the following:
LastModifiedBy (string) --
The identifier (the user name) of the user who last modified the job.
LastModifiedDate (datetime) --
The modification date and time of the job.
LogSubscription (string) --
The current status of Amazon CloudWatch logging for the job.
MaxCapacity (integer) --
The maximum number of nodes that can be consumed when the job processes data.
MaxRetries (integer) --
The maximum number of times to retry the job after a job run fails.
Outputs (list) --
One or more artifacts that represent output from running the job.
(dict) --
Represents individual output from a particular job run.
CompressionFormat (string) --
The compression algorithm used to compress the output text of the job.
Format (string) --
The data format of the output of the job.
PartitionColumns (list) --
The names of one or more partition columns for the output of the job.
Location (dict) --
The location in Amazon S3 where the job writes its output.
Bucket (string) --
The S3 bucket name.
Key (string) --
The unique name of the object in the bucket.
Overwrite (boolean) --
A value that, if true, means that any data in the location specified for output is overwritten with new output.
ProjectName (string) --
The name of the project that the job is associated with.
RecipeReference (dict) --
A set of steps that the job runs.
Name (string) --
The name of the recipe.
RecipeVersion (string) --
The identifier for the version for the recipe.
ResourceArn (string) --
The unique Amazon Resource Name (ARN) for the job.
RoleArn (string) --
The Amazon Resource Name (ARN) of the role that will be assumed for this job.
Timeout (integer) --
The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .
Tags (dict) --
Metadata tags that have been applied to the job.
NextToken (string) --
A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To obtain the next set of pages, pass in the NextToken from the response object of the previous page call.
Exceptions
Lists all of the DataBrew projects in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.list_projects(
NextToken='string',
MaxResults=123
)
dict
Response Syntax
{
'Projects': [
{
'AccountId': 'string',
'CreateDate': datetime(2015, 1, 1),
'CreatedBy': 'string',
'DatasetName': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'Name': 'string',
'RecipeName': 'string',
'ResourceArn': 'string',
'Sample': {
'Size': 123,
'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
},
'Tags': {
'string': 'string'
},
'RoleArn': 'string',
'OpenedBy': 'string',
'OpenDate': datetime(2015, 1, 1)
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Projects (list) --
A list of projects that are defined in the current AWS account.
(dict) --
Represents all of the attributes of an AWS Glue DataBrew project.
AccountId (string) --
The ID of the AWS account that owns the project.
CreateDate (datetime) --
The date and time that the project was created.
CreatedBy (string) --
The identifier (the user name) of the user who crated the project.
DatasetName (string) --
The dataset that the project is to act upon.
LastModifiedDate (datetime) --
The last modification date and time for the project.
LastModifiedBy (string) --
The identifier (user name) of the user who last modified the project.
Name (string) --
The unique name of a project.
RecipeName (string) --
The name of a recipe that will be developed during a project session.
ResourceArn (string) --
The Amazon Resource Name (ARN) for the project.
Sample (dict) --
The sample size and sampling type to apply to the data. If this parameter isn't specified, then the sample will consiste of the first 500 rows from the dataset.
Size (integer) --
The number of rows in the sample.
Type (string) --
The way in which DataBrew obtains rows from a dataset.
Tags (dict) --
Metadata tags that have been applied to the project.
RoleArn (string) --
The Amazon Resource Name (ARN) of the role that will be assumed for this project.
OpenedBy (string) --
The identifier (the user name) of the user that opened the project for use.
OpenDate (datetime) --
The date and time when the project was opened.
NextToken (string) --
A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.
Exceptions
Lists all of the versions of a particular AWS Glue DataBrew recipe in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.list_recipe_versions(
MaxResults=123,
NextToken='string',
Name='string'
)
[REQUIRED]
The name of the recipe for which to return version information.
dict
Response Syntax
{
'NextToken': 'string',
'Recipes': [
{
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'ProjectName': 'string',
'PublishedBy': 'string',
'PublishedDate': datetime(2015, 1, 1),
'Description': 'string',
'Name': 'string',
'ResourceArn': 'string',
'Steps': [
{
'Action': {
'Operation': 'string',
'Parameters': {
'string': 'string'
}
},
'ConditionExpressions': [
{
'Condition': 'string',
'Value': 'string',
'TargetColumn': 'string'
},
]
},
],
'Tags': {
'string': 'string'
},
'RecipeVersion': 'string'
},
]
}
Response Structure
(dict) --
NextToken (string) --
A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.
Recipes (list) --
A list of versions for the specified recipe.
(dict) --
Represents one or more actions to be performed on an AWS Glue DataBrew dataset.
CreatedBy (string) --
The identifier (the user name) of the user who created the recipe.
CreateDate (datetime) --
The date and time that the recipe was created.
LastModifiedBy (string) --
The identifier (user name) of the user who last modified the recipe.
LastModifiedDate (datetime) --
The last modification date and time of the recipe.
ProjectName (string) --
The name of the project that the recipe is associated with.
PublishedBy (string) --
The identifier (the user name) of the user who published the recipe.
PublishedDate (datetime) --
The date and time when the recipe was published.
Description (string) --
The description of the recipe.
Name (string) --
The unique name for the recipe.
ResourceArn (string) --
The Amazon Resource Name (ARN) for the recipe.
Steps (list) --
A list of steps that are defined by the recipe.
(dict) --
Represents a single step to be performed in an AWS Glue DataBrew recipe.
Action (dict) --
The particular action to be performed in the recipe step.
Operation (string) --
The name of a valid DataBrew transformation to be performed on the data.
Parameters (dict) --
Contextual parameters for the transformation.
ConditionExpressions (list) --
One or more conditions that must be met, in order for the recipe step to succeed.
Note
All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.
(dict) --
Represents an individual condition that evaluates to true or false.
Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.
If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.
Condition (string) --
A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .
Value (string) --
A value that the condition must evaluate to for the condition to succeed.
TargetColumn (string) --
A column to apply this condition to, within an AWS Glue DataBrew dataset.
Tags (dict) --
Metadata tags that have been applied to the recipe.
RecipeVersion (string) --
The identifier for the version for the recipe.
Exceptions
Lists all of the AWS Glue DataBrew recipes in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.list_recipes(
MaxResults=123,
NextToken='string',
RecipeVersion='string'
)
dict
Response Syntax
{
'Recipes': [
{
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'ProjectName': 'string',
'PublishedBy': 'string',
'PublishedDate': datetime(2015, 1, 1),
'Description': 'string',
'Name': 'string',
'ResourceArn': 'string',
'Steps': [
{
'Action': {
'Operation': 'string',
'Parameters': {
'string': 'string'
}
},
'ConditionExpressions': [
{
'Condition': 'string',
'Value': 'string',
'TargetColumn': 'string'
},
]
},
],
'Tags': {
'string': 'string'
},
'RecipeVersion': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Recipes (list) --
A list of recipes that are defined in the current AWS account.
(dict) --
Represents one or more actions to be performed on an AWS Glue DataBrew dataset.
CreatedBy (string) --
The identifier (the user name) of the user who created the recipe.
CreateDate (datetime) --
The date and time that the recipe was created.
LastModifiedBy (string) --
The identifier (user name) of the user who last modified the recipe.
LastModifiedDate (datetime) --
The last modification date and time of the recipe.
ProjectName (string) --
The name of the project that the recipe is associated with.
PublishedBy (string) --
The identifier (the user name) of the user who published the recipe.
PublishedDate (datetime) --
The date and time when the recipe was published.
Description (string) --
The description of the recipe.
Name (string) --
The unique name for the recipe.
ResourceArn (string) --
The Amazon Resource Name (ARN) for the recipe.
Steps (list) --
A list of steps that are defined by the recipe.
(dict) --
Represents a single step to be performed in an AWS Glue DataBrew recipe.
Action (dict) --
The particular action to be performed in the recipe step.
Operation (string) --
The name of a valid DataBrew transformation to be performed on the data.
Parameters (dict) --
Contextual parameters for the transformation.
ConditionExpressions (list) --
One or more conditions that must be met, in order for the recipe step to succeed.
Note
All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.
(dict) --
Represents an individual condition that evaluates to true or false.
Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.
If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.
Condition (string) --
A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .
Value (string) --
A value that the condition must evaluate to for the condition to succeed.
TargetColumn (string) --
A column to apply this condition to, within an AWS Glue DataBrew dataset.
Tags (dict) --
Metadata tags that have been applied to the recipe.
RecipeVersion (string) --
The identifier for the version for the recipe.
NextToken (string) --
A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.
Exceptions
Lists the AWS Glue DataBrew schedules in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.list_schedules(
JobName='string',
MaxResults=123,
NextToken='string'
)
dict
Response Syntax
{
'Schedules': [
{
'AccountId': 'string',
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'JobNames': [
'string',
],
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'ResourceArn': 'string',
'CronExpression': 'string',
'Tags': {
'string': 'string'
},
'Name': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Schedules (list) --
A list of schedules in the current AWS account.
(dict) --
Represents one or more dates and times when a job is to run.
AccountId (string) --
The ID of the AWS account that owns the schedule.
CreatedBy (string) --
The identifier (the user name) of the user who created the schedule.
CreateDate (datetime) --
The date and time that the schedule was created.
JobNames (list) --
A list of jobs to be run, according to the schedule.
LastModifiedBy (string) --
The identifier (the user name) of the user who last modified the schedule.
LastModifiedDate (datetime) --
The date and time when the schedule was last modified.
ResourceArn (string) --
The Amazon Resource Name (ARN) of the schedule.
CronExpression (string) --
The date(s) and time(s), in cron format, when the job will run.
Tags (dict) --
Metadata tags that have been applied to the schedule.
Name (string) --
The name of the schedule.
NextToken (string) --
A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.
Exceptions
Lists all the tags for an AWS Glue DataBrew resource.
See also: AWS API Documentation
Request Syntax
response = client.list_tags_for_resource(
ResourceArn='string'
)
[REQUIRED]
The Amazon Resource Name (ARN) string that uniquely identifies the DataBrew resource.
{
'Tags': {
'string': 'string'
}
}
Response Structure
A list of tags associated with the DataBrew resource.
Exceptions
Publishes a new major version of an AWS Glue DataBrew recipe that exists in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.publish_recipe(
Description='string',
Name='string'
)
[REQUIRED]
The name of the recipe to be published.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the recipe that you published.
Exceptions
Performs a recipe step within an interactive AWS Glue DataBrew session that's currently open.
See also: AWS API Documentation
Request Syntax
response = client.send_project_session_action(
Preview=True|False,
Name='string',
RecipeStep={
'Action': {
'Operation': 'string',
'Parameters': {
'string': 'string'
}
},
'ConditionExpressions': [
{
'Condition': 'string',
'Value': 'string',
'TargetColumn': 'string'
},
]
},
StepIndex=123,
ClientSessionId='string',
ViewFrame={
'StartColumnIndex': 123,
'ColumnRange': 123,
'HiddenColumns': [
'string',
]
}
)
[REQUIRED]
The name of the project to apply the action to.
Represents a single step to be performed in an AWS Glue DataBrew recipe.
The particular action to be performed in the recipe step.
The name of a valid DataBrew transformation to be performed on the data.
Contextual parameters for the transformation.
One or more conditions that must be met, in order for the recipe step to succeed.
Note
All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.
Represents an individual condition that evaluates to true or false.
Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.
If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.
A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .
A value that the condition must evaluate to for the condition to succeed.
A column to apply this condition to, within an AWS Glue DataBrew dataset.
Represents the data being being transformed during an AWS Glue DataBrew project session.
The starting index for the range of columns to return in the view frame.
The number of columns to include in the view frame, beginning with the StartColumnIndex value and ignoring any columns in the HiddenColumns list.
A list of columns to hide in the view frame.
dict
Response Syntax
{
'Result': 'string',
'Name': 'string',
'ActionId': 123
}
Response Structure
(dict) --
Result (string) --
A message indicating the result of performing the action.
Name (string) --
The name of the project that was affected by the action.
ActionId (integer) --
A unique identifier for the action that was performed.
Exceptions
Runs an AWS Glue DataBrew job that exists in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.start_job_run(
Name='string'
)
[REQUIRED]
The name of the job to be run.
{
'RunId': 'string'
}
Response Structure
A system-generated identifier for this particular job run.
Exceptions
Creates an interactive session, enabling you to manipulate an AWS Glue DataBrew project.
See also: AWS API Documentation
Request Syntax
response = client.start_project_session(
Name='string',
AssumeControl=True|False
)
[REQUIRED]
The name of the project to act upon.
dict
Response Syntax
{
'Name': 'string',
'ClientSessionId': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the project to be acted upon.
ClientSessionId (string) --
A system-generated identifier for the session.
Exceptions
Stops the specified job from running in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.stop_job_run(
Name='string',
RunId='string'
)
[REQUIRED]
The name of the job to be stopped.
[REQUIRED]
The ID of the job run to be stopped.
dict
Response Syntax
{
'RunId': 'string'
}
Response Structure
(dict) --
RunId (string) --
The ID of the job run that you stopped.
Exceptions
Adds metadata tags to an AWS Glue DataBrew resource, such as a dataset, job, project, or recipe.
See also: AWS API Documentation
Request Syntax
response = client.tag_resource(
ResourceArn='string',
Tags={
'string': 'string'
}
)
[REQUIRED]
The DataBrew resource to which tags should be added. The value for this parameter is an Amazon Resource Name (ARN). For DataBrew, you can tag a dataset, a job, a project, or a recipe.
[REQUIRED]
One or more tags to be assigned to the resource.
dict
Response Syntax
{}
Response Structure
Exceptions
Removes metadata tags from an AWS Glue DataBrew resource.
See also: AWS API Documentation
Request Syntax
response = client.untag_resource(
ResourceArn='string',
TagKeys=[
'string',
]
)
[REQUIRED]
An DataBrew resource from which you want to remove a tag or tags. The value for this parameter is an Amazon Resource Name (ARN).
[REQUIRED]
The tag keys (names) of one or more tags to be removed.
dict
Response Syntax
{}
Response Structure
Exceptions
Modifies the definition of an existing AWS Glue DataBrew dataset in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.update_dataset(
Name='string',
FormatOptions={
'Json': {
'MultiLine': True|False
},
'Excel': {
'SheetNames': [
'string',
],
'SheetIndexes': [
123,
]
}
},
Input={
'S3InputDefinition': {
'Bucket': 'string',
'Key': 'string'
},
'DataCatalogInputDefinition': {
'CatalogId': 'string',
'DatabaseName': 'string',
'TableName': 'string',
'TempDirectory': {
'Bucket': 'string',
'Key': 'string'
}
}
}
)
[REQUIRED]
The name of the dataset to be updated.
Options that define how Microsoft Excel input is to be interpreted by DataBrew.
Options that define how JSON input is to be interpreted by DataBrew.
A value that specifies whether JSON input contains embedded new line characters.
Options that define how Excel input is to be interpreted by DataBrew.
Specifies one or more named sheets in the Excel file, which will be included in the dataset.
Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.
[REQUIRED]
Information on how AWS Glue DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.
The Amazon S3 location where the data is stored.
The S3 bucket name.
The unique name of the object in the bucket.
The AWS Glue Data Catalog parameters for the data.
The unique identifier of the AWS account that holds the Data Catalog that stores the data.
The name of a database in the Data Catalog.
The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.
An Amazon location that AWS Glue Data Catalog can use as a temporary directory.
The S3 bucket name.
The unique name of the object in the bucket.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the dataset that you updated.
Exceptions
Modifies the definition of an existing AWS Glue DataBrew job in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.update_profile_job(
EncryptionKeyArn='string',
EncryptionMode='SSE-KMS'|'SSE-S3',
Name='string',
LogSubscription='ENABLE'|'DISABLE',
MaxCapacity=123,
MaxRetries=123,
OutputLocation={
'Bucket': 'string',
'Key': 'string'
},
RoleArn='string',
Timeout=123
)
The encryption mode for the job, which can be one of the following:
[REQUIRED]
The name of the job to be updated.
[REQUIRED]
An Amazon S3 location (bucket name an object key) where DataBrew can read input data, or write output from a job.
The S3 bucket name.
The unique name of the object in the bucket.
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the job that was updated.
Exceptions
Modifies the definition of an existing AWS Glue DataBrew project in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.update_project(
Sample={
'Size': 123,
'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
},
RoleArn='string',
Name='string'
)
Represents the sample size and sampling type for AWS Glue DataBrew to use for interactive data analysis.
The number of rows in the sample.
The way in which DataBrew obtains rows from a dataset.
[REQUIRED]
The Amazon Resource Name (ARN) of the IAM role to be assumed for this request.
[REQUIRED]
The name of the project to be updated.
dict
Response Syntax
{
'LastModifiedDate': datetime(2015, 1, 1),
'Name': 'string'
}
Response Structure
(dict) --
LastModifiedDate (datetime) --
The date and time that the project was last modified.
Name (string) --
The name of the project that you updated.
Exceptions
Modifies the definition of the latest working version of an AWS Glue DataBrew recipe in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.update_recipe(
Description='string',
Name='string',
Steps=[
{
'Action': {
'Operation': 'string',
'Parameters': {
'string': 'string'
}
},
'ConditionExpressions': [
{
'Condition': 'string',
'Value': 'string',
'TargetColumn': 'string'
},
]
},
]
)
[REQUIRED]
The name of the recipe to be updated.
One or more steps to be performed by the recipe. Each step consists of an action, and the conditions under which the action should succeed.
Represents a single step to be performed in an AWS Glue DataBrew recipe.
The particular action to be performed in the recipe step.
The name of a valid DataBrew transformation to be performed on the data.
Contextual parameters for the transformation.
One or more conditions that must be met, in order for the recipe step to succeed.
Note
All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.
Represents an individual condition that evaluates to true or false.
Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.
If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.
A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .
A value that the condition must evaluate to for the condition to succeed.
A column to apply this condition to, within an AWS Glue DataBrew dataset.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the recipe that was updated.
Exceptions
Modifies the definition of an existing AWS Glue DataBrew recipe job in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.update_recipe_job(
EncryptionKeyArn='string',
EncryptionMode='SSE-KMS'|'SSE-S3',
Name='string',
LogSubscription='ENABLE'|'DISABLE',
MaxCapacity=123,
MaxRetries=123,
Outputs=[
{
'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
'PartitionColumns': [
'string',
],
'Location': {
'Bucket': 'string',
'Key': 'string'
},
'Overwrite': True|False
},
],
RoleArn='string',
Timeout=123
)
The encryption mode for the job, which can be one of the following:
[REQUIRED]
The name of the job to update.
[REQUIRED]
One or more artifacts that represent the output from running the job.
Represents individual output from a particular job run.
The compression algorithm used to compress the output text of the job.
The data format of the output of the job.
The names of one or more partition columns for the output of the job.
The location in Amazon S3 where the job writes its output.
The S3 bucket name.
The unique name of the object in the bucket.
A value that, if true, means that any data in the location specified for output is overwritten with new output.
[REQUIRED]
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the job that you updated.
Exceptions
Modifies the definition of an existing AWS Glue DataBrew schedule in the current AWS account.
See also: AWS API Documentation
Request Syntax
response = client.update_schedule(
JobNames=[
'string',
],
CronExpression='string',
Name='string'
)
The name or names of one or more jobs to be run for this schedule.
[REQUIRED]
The date or dates and time or times, in cron format, when the jobs are to be run.
[REQUIRED]
The name of the schedule to update.
dict
Response Syntax
{
'Name': 'string'
}
Response Structure
(dict) --
Name (string) --
The name of the schedule that was updated.
Exceptions
The available paginators are:
paginator = client.get_paginator('list_datasets')
Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_datasets().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
{
'Datasets': [
{
'AccountId': 'string',
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'Name': 'string',
'FormatOptions': {
'Json': {
'MultiLine': True|False
},
'Excel': {
'SheetNames': [
'string',
],
'SheetIndexes': [
123,
]
}
},
'Input': {
'S3InputDefinition': {
'Bucket': 'string',
'Key': 'string'
},
'DataCatalogInputDefinition': {
'CatalogId': 'string',
'DatabaseName': 'string',
'TableName': 'string',
'TempDirectory': {
'Bucket': 'string',
'Key': 'string'
}
}
},
'LastModifiedDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'Source': 'S3'|'DATA-CATALOG',
'Tags': {
'string': 'string'
},
'ResourceArn': 'string'
},
],
}
Response Structure
A list of datasets that are defined in the current AWS account.
Represents a dataset that can be processed by AWS Glue DataBrew.
The ID of the AWS account that owns the dataset.
The identifier (the user name) of the user who created the dataset.
The date and time that the dataset was created.
The unique name of the dataset.
Options that define how DataBrew interprets the data in the dataset.
Options that define how JSON input is to be interpreted by DataBrew.
A value that specifies whether JSON input contains embedded new line characters.
Options that define how Excel input is to be interpreted by DataBrew.
Specifies one or more named sheets in the Excel file, which will be included in the dataset.
Specifies one or more sheet numbers in the Excel file, which will be included in the dataset.
Information on how DataBrew can find the dataset, in either the AWS Glue Data Catalog or Amazon S3.
The Amazon S3 location where the data is stored.
The S3 bucket name.
The unique name of the object in the bucket.
The AWS Glue Data Catalog parameters for the data.
The unique identifier of the AWS account that holds the Data Catalog that stores the data.
The name of a database in the Data Catalog.
The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.
An Amazon location that AWS Glue Data Catalog can use as a temporary directory.
The S3 bucket name.
The unique name of the object in the bucket.
The last modification date and time of the dataset.
The identifier (the user name) of the user who last modified the dataset.
The location of the data for the dataset, either Amazon S3 or the AWS Glue Data Catalog.
Metadata tags that have been applied to the dataset.
The unique Amazon Resource Name (ARN) for the dataset.
paginator = client.get_paginator('list_job_runs')
Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_job_runs().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
Name='string',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
[REQUIRED]
The name of the job.
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'JobRuns': [
{
'Attempt': 123,
'CompletedOn': datetime(2015, 1, 1),
'DatasetName': 'string',
'ErrorMessage': 'string',
'ExecutionTime': 123,
'JobName': 'string',
'RunId': 'string',
'State': 'STARTING'|'RUNNING'|'STOPPING'|'STOPPED'|'SUCCEEDED'|'FAILED'|'TIMEOUT',
'LogSubscription': 'ENABLE'|'DISABLE',
'LogGroupName': 'string',
'Outputs': [
{
'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
'PartitionColumns': [
'string',
],
'Location': {
'Bucket': 'string',
'Key': 'string'
},
'Overwrite': True|False
},
],
'RecipeReference': {
'Name': 'string',
'RecipeVersion': 'string'
},
'StartedBy': 'string',
'StartedOn': datetime(2015, 1, 1)
},
],
}
Response Structure
(dict) --
JobRuns (list) --
A list of job runs that have occurred for the specified job.
(dict) --
Represents one run of an AWS Glue DataBrew job.
Attempt (integer) --
The number of times that DataBrew has attempted to run the job.
CompletedOn (datetime) --
The date and time when the job completed processing.
DatasetName (string) --
The name of the dataset for the job to process.
ErrorMessage (string) --
A message indicating an error (if any) that was encountered when the job ran.
ExecutionTime (integer) --
The amount of time, in seconds, during which a job run consumed resources.
JobName (string) --
The name of the job being processed during this run.
RunId (string) --
The unique identifier of the job run.
State (string) --
The current state of the job run entity itself.
LogSubscription (string) --
The current status of Amazon CloudWatch logging for the job run.
LogGroupName (string) --
The name of an Amazon CloudWatch log group, where the job writes diagnostic messages when it runs.
Outputs (list) --
One or more output artifacts from a job run.
(dict) --
Represents individual output from a particular job run.
CompressionFormat (string) --
The compression algorithm used to compress the output text of the job.
Format (string) --
The data format of the output of the job.
PartitionColumns (list) --
The names of one or more partition columns for the output of the job.
Location (dict) --
The location in Amazon S3 where the job writes its output.
Bucket (string) --
The S3 bucket name.
Key (string) --
The unique name of the object in the bucket.
Overwrite (boolean) --
A value that, if true, means that any data in the location specified for output is overwritten with new output.
RecipeReference (dict) --
The set of steps processed by the job.
Name (string) --
The name of the recipe.
RecipeVersion (string) --
The identifier for the version for the recipe.
StartedBy (string) --
The identifier (the user name) of the user who initiated the job run.
StartedOn (datetime) --
The date and time when the job run began.
paginator = client.get_paginator('list_jobs')
Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_jobs().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
DatasetName='string',
ProjectName='string',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'Jobs': [
{
'AccountId': 'string',
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'DatasetName': 'string',
'EncryptionKeyArn': 'string',
'EncryptionMode': 'SSE-KMS'|'SSE-S3',
'Name': 'string',
'Type': 'PROFILE'|'RECIPE',
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'LogSubscription': 'ENABLE'|'DISABLE',
'MaxCapacity': 123,
'MaxRetries': 123,
'Outputs': [
{
'CompressionFormat': 'GZIP'|'LZ4'|'SNAPPY'|'BZIP2'|'DEFLATE'|'LZO'|'BROTLI'|'ZSTD'|'ZLIB',
'Format': 'CSV'|'JSON'|'PARQUET'|'GLUEPARQUET'|'AVRO'|'ORC'|'XML',
'PartitionColumns': [
'string',
],
'Location': {
'Bucket': 'string',
'Key': 'string'
},
'Overwrite': True|False
},
],
'ProjectName': 'string',
'RecipeReference': {
'Name': 'string',
'RecipeVersion': 'string'
},
'ResourceArn': 'string',
'RoleArn': 'string',
'Timeout': 123,
'Tags': {
'string': 'string'
}
},
],
}
Response Structure
(dict) --
Jobs (list) --
A list of jobs that are defined in the current AWS account.
(dict) --
Represents all of the attributes of an AWS Glue DataBrew job.
AccountId (string) --
The ID of the AWS account that owns the job.
CreatedBy (string) --
The identifier (the user name) of the user who created the job.
CreateDate (datetime) --
The date and time that the job was created.
DatasetName (string) --
A dataset that the job is to process.
EncryptionKeyArn (string) --
The Amazon Resource Name (ARN) of an encryption key that is used to protect a job.
EncryptionMode (string) --
The encryption mode for the job, which can be one of the following:
Name (string) --
The unique name of the job.
Type (string) --
The job type of the job, which must be one of the following:
LastModifiedBy (string) --
The identifier (the user name) of the user who last modified the job.
LastModifiedDate (datetime) --
The modification date and time of the job.
LogSubscription (string) --
The current status of Amazon CloudWatch logging for the job.
MaxCapacity (integer) --
The maximum number of nodes that can be consumed when the job processes data.
MaxRetries (integer) --
The maximum number of times to retry the job after a job run fails.
Outputs (list) --
One or more artifacts that represent output from running the job.
(dict) --
Represents individual output from a particular job run.
CompressionFormat (string) --
The compression algorithm used to compress the output text of the job.
Format (string) --
The data format of the output of the job.
PartitionColumns (list) --
The names of one or more partition columns for the output of the job.
Location (dict) --
The location in Amazon S3 where the job writes its output.
Bucket (string) --
The S3 bucket name.
Key (string) --
The unique name of the object in the bucket.
Overwrite (boolean) --
A value that, if true, means that any data in the location specified for output is overwritten with new output.
ProjectName (string) --
The name of the project that the job is associated with.
RecipeReference (dict) --
A set of steps that the job runs.
Name (string) --
The name of the recipe.
RecipeVersion (string) --
The identifier for the version for the recipe.
ResourceArn (string) --
The unique Amazon Resource Name (ARN) for the job.
RoleArn (string) --
The Amazon Resource Name (ARN) of the role that will be assumed for this job.
Timeout (integer) --
The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT .
Tags (dict) --
Metadata tags that have been applied to the job.
paginator = client.get_paginator('list_projects')
Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_projects().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
{
'Projects': [
{
'AccountId': 'string',
'CreateDate': datetime(2015, 1, 1),
'CreatedBy': 'string',
'DatasetName': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'Name': 'string',
'RecipeName': 'string',
'ResourceArn': 'string',
'Sample': {
'Size': 123,
'Type': 'FIRST_N'|'LAST_N'|'RANDOM'
},
'Tags': {
'string': 'string'
},
'RoleArn': 'string',
'OpenedBy': 'string',
'OpenDate': datetime(2015, 1, 1)
},
],
}
Response Structure
A list of projects that are defined in the current AWS account.
Represents all of the attributes of an AWS Glue DataBrew project.
The ID of the AWS account that owns the project.
The date and time that the project was created.
The identifier (the user name) of the user who crated the project.
The dataset that the project is to act upon.
The last modification date and time for the project.
The identifier (user name) of the user who last modified the project.
The unique name of a project.
The name of a recipe that will be developed during a project session.
The Amazon Resource Name (ARN) for the project.
The sample size and sampling type to apply to the data. If this parameter isn't specified, then the sample will consiste of the first 500 rows from the dataset.
The number of rows in the sample.
The way in which DataBrew obtains rows from a dataset.
Metadata tags that have been applied to the project.
The Amazon Resource Name (ARN) of the role that will be assumed for this project.
The identifier (the user name) of the user that opened the project for use.
The date and time when the project was opened.
paginator = client.get_paginator('list_recipe_versions')
Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_recipe_versions().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
Name='string',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
[REQUIRED]
The name of the recipe for which to return version information.
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'Recipes': [
{
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'ProjectName': 'string',
'PublishedBy': 'string',
'PublishedDate': datetime(2015, 1, 1),
'Description': 'string',
'Name': 'string',
'ResourceArn': 'string',
'Steps': [
{
'Action': {
'Operation': 'string',
'Parameters': {
'string': 'string'
}
},
'ConditionExpressions': [
{
'Condition': 'string',
'Value': 'string',
'TargetColumn': 'string'
},
]
},
],
'Tags': {
'string': 'string'
},
'RecipeVersion': 'string'
},
]
}
Response Structure
(dict) --
Recipes (list) --
A list of versions for the specified recipe.
(dict) --
Represents one or more actions to be performed on an AWS Glue DataBrew dataset.
CreatedBy (string) --
The identifier (the user name) of the user who created the recipe.
CreateDate (datetime) --
The date and time that the recipe was created.
LastModifiedBy (string) --
The identifier (user name) of the user who last modified the recipe.
LastModifiedDate (datetime) --
The last modification date and time of the recipe.
ProjectName (string) --
The name of the project that the recipe is associated with.
PublishedBy (string) --
The identifier (the user name) of the user who published the recipe.
PublishedDate (datetime) --
The date and time when the recipe was published.
Description (string) --
The description of the recipe.
Name (string) --
The unique name for the recipe.
ResourceArn (string) --
The Amazon Resource Name (ARN) for the recipe.
Steps (list) --
A list of steps that are defined by the recipe.
(dict) --
Represents a single step to be performed in an AWS Glue DataBrew recipe.
Action (dict) --
The particular action to be performed in the recipe step.
Operation (string) --
The name of a valid DataBrew transformation to be performed on the data.
Parameters (dict) --
Contextual parameters for the transformation.
ConditionExpressions (list) --
One or more conditions that must be met, in order for the recipe step to succeed.
Note
All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.
(dict) --
Represents an individual condition that evaluates to true or false.
Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.
If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.
Condition (string) --
A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .
Value (string) --
A value that the condition must evaluate to for the condition to succeed.
TargetColumn (string) --
A column to apply this condition to, within an AWS Glue DataBrew dataset.
Tags (dict) --
Metadata tags that have been applied to the recipe.
RecipeVersion (string) --
The identifier for the version for the recipe.
paginator = client.get_paginator('list_recipes')
Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_recipes().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
RecipeVersion='string',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'Recipes': [
{
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'ProjectName': 'string',
'PublishedBy': 'string',
'PublishedDate': datetime(2015, 1, 1),
'Description': 'string',
'Name': 'string',
'ResourceArn': 'string',
'Steps': [
{
'Action': {
'Operation': 'string',
'Parameters': {
'string': 'string'
}
},
'ConditionExpressions': [
{
'Condition': 'string',
'Value': 'string',
'TargetColumn': 'string'
},
]
},
],
'Tags': {
'string': 'string'
},
'RecipeVersion': 'string'
},
],
}
Response Structure
(dict) --
Recipes (list) --
A list of recipes that are defined in the current AWS account.
(dict) --
Represents one or more actions to be performed on an AWS Glue DataBrew dataset.
CreatedBy (string) --
The identifier (the user name) of the user who created the recipe.
CreateDate (datetime) --
The date and time that the recipe was created.
LastModifiedBy (string) --
The identifier (user name) of the user who last modified the recipe.
LastModifiedDate (datetime) --
The last modification date and time of the recipe.
ProjectName (string) --
The name of the project that the recipe is associated with.
PublishedBy (string) --
The identifier (the user name) of the user who published the recipe.
PublishedDate (datetime) --
The date and time when the recipe was published.
Description (string) --
The description of the recipe.
Name (string) --
The unique name for the recipe.
ResourceArn (string) --
The Amazon Resource Name (ARN) for the recipe.
Steps (list) --
A list of steps that are defined by the recipe.
(dict) --
Represents a single step to be performed in an AWS Glue DataBrew recipe.
Action (dict) --
The particular action to be performed in the recipe step.
Operation (string) --
The name of a valid DataBrew transformation to be performed on the data.
Parameters (dict) --
Contextual parameters for the transformation.
ConditionExpressions (list) --
One or more conditions that must be met, in order for the recipe step to succeed.
Note
All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.
(dict) --
Represents an individual condition that evaluates to true or false.
Conditions are used with recipe actions: The action is only performed for column values where the condition evaluates to true.
If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.
Condition (string) --
A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide .
Value (string) --
A value that the condition must evaluate to for the condition to succeed.
TargetColumn (string) --
A column to apply this condition to, within an AWS Glue DataBrew dataset.
Tags (dict) --
Metadata tags that have been applied to the recipe.
RecipeVersion (string) --
The identifier for the version for the recipe.
paginator = client.get_paginator('list_schedules')
Creates an iterator that will paginate through responses from GlueDataBrew.Client.list_schedules().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
JobName='string',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'Schedules': [
{
'AccountId': 'string',
'CreatedBy': 'string',
'CreateDate': datetime(2015, 1, 1),
'JobNames': [
'string',
],
'LastModifiedBy': 'string',
'LastModifiedDate': datetime(2015, 1, 1),
'ResourceArn': 'string',
'CronExpression': 'string',
'Tags': {
'string': 'string'
},
'Name': 'string'
},
],
}
Response Structure
(dict) --
Schedules (list) --
A list of schedules in the current AWS account.
(dict) --
Represents one or more dates and times when a job is to run.
AccountId (string) --
The ID of the AWS account that owns the schedule.
CreatedBy (string) --
The identifier (the user name) of the user who created the schedule.
CreateDate (datetime) --
The date and time that the schedule was created.
JobNames (list) --
A list of jobs to be run, according to the schedule.
LastModifiedBy (string) --
The identifier (the user name) of the user who last modified the schedule.
LastModifiedDate (datetime) --
The date and time when the schedule was last modified.
ResourceArn (string) --
The Amazon Resource Name (ARN) of the schedule.
CronExpression (string) --
The date(s) and time(s), in cron format, when the job will run.
Tags (dict) --
Metadata tags that have been applied to the schedule.
Name (string) --
The name of the schedule.