BedrockRuntime / Client / invoke_model_with_response_stream

invoke_model_with_response_stream#

BedrockRuntime.Client.invoke_model_with_response_stream(**kwargs)#

Invoke the specified Bedrock model to run inference using the input provided. Return the response in a stream.

For more information, see Run inference in the Bedrock User Guide.

For an example request and response, see Examples (after the Errors section).

Request Syntax

response = client.invoke_model_with_response_stream(
    accept='string',
    body=b'bytes'|file,
    contentType='string',
    modelId='string'
)

Parameters:

accept (string) – The desired MIME type of the inference body in the response. The default value is application/json.
body (bytes or seekable file-like object) –
[REQUIRED]

Inference input in the format specified by the content-type. To see the format and content of this field for different models, refer to Inference parameters.
contentType (string) – The MIME type of the input data in the request. The default value is application/json.
modelId (string) –
[REQUIRED]

Id of the model to invoke using the streaming request.

Return type:

dict

Returns:

The response of this operation contains an EventStream member. When iterated the EventStream will yield events based on the structure below, where only one of the top level keys will be present for any given event.

Response Syntax

{
    'body': EventStream({
        'chunk': {
            'bytes': b'bytes'
        },
        'internalServerException': {
            'message': 'string'
        },
        'modelStreamErrorException': {
            'message': 'string',
            'originalMessage': 'string',
            'originalStatusCode': 123
        },
        'modelTimeoutException': {
            'message': 'string'
        },
        'throttlingException': {
            'message': 'string'
        },
        'validationException': {
            'message': 'string'
        }
    }),
    'contentType': 'string'
}

Response Structure

(dict) –
- body (EventStream) –
  
  Inference response from the model in the format specified by Content-Type. To see the format and content of this field for different models, refer to Inference parameters.
  - chunk (dict) –
    
    Content included in the response.
    - bytes (bytes) –
      
      Base64-encoded bytes of payload data.
  - internalServerException (dict) –
    
    An internal server error occurred. Retry your request.
    - message (string) –
  - modelStreamErrorException (dict) –
    
    An error occurred while streaming the response.
    - message (string) –
    - originalMessage (string) –
      
      The original message.
    - originalStatusCode (integer) –
      
      The original status code.
  - modelTimeoutException (dict) –
    
    The request took too long to process. Processing time exceeded the model timeout length.
    - message (string) –
  - throttlingException (dict) –
    
    The number of requests exceeds the limit. Resubmit your request later.
    - message (string) –
  - validationException (dict) –
    
    Input validation failed. Check your request parameters and retry the request.
    - message (string) –
- contentType (string) –
  
  The MIME type of the inference result.

Exceptions

BedrockRuntime.Client.exceptions.AccessDeniedException
BedrockRuntime.Client.exceptions.ResourceNotFoundException
BedrockRuntime.Client.exceptions.ThrottlingException
BedrockRuntime.Client.exceptions.ModelTimeoutException
BedrockRuntime.Client.exceptions.InternalServerException
BedrockRuntime.Client.exceptions.ModelStreamErrorException
BedrockRuntime.Client.exceptions.ValidationException
BedrockRuntime.Client.exceptions.ModelNotReadyException
BedrockRuntime.Client.exceptions.ServiceQuotaExceededException
BedrockRuntime.Client.exceptions.ModelErrorException