NeptuneData / Client / get_sparql_stream

get_sparql_stream#

NeptuneData.Client.get_sparql_stream(**kwargs)#

Gets a stream for an RDF graph.

With the Neptune Streams feature, you can generate a complete sequence of change-log entries that record every change made to your graph data as it happens. GetSparqlStream lets you collect these change-log entries for an RDF graph.

The Neptune streams feature needs to be enabled on your Neptune DBcluster. To enable streams, set the neptune_streams DB cluster parameter to 1.

See Capturing graph changes in real time using Neptune streams.

When invoking this operation in a Neptune cluster that has IAM authentication enabled, the IAM user or role making the request must have a policy attached that allows the neptune-db:GetStreamRecords IAM action in that cluster.

Note that the neptune-db:QueryLanguage:Sparql IAM condition key can be used in the policy document to restrict the use of SPARQL queries (see Condition keys available in Neptune IAM data-access policy statements).

See also: AWS API Documentation

Request Syntax

response = client.get_sparql_stream(
    limit=123,
    iteratorType='AT_SEQUENCE_NUMBER'|'AFTER_SEQUENCE_NUMBER'|'TRIM_HORIZON'|'LATEST',
    commitNum=123,
    opNum=123,
    encoding='gzip'
)
Parameters:

  • limit (integer) –

    Specifies the maximum number of records to return. There is also a size limit of 10 MB on the response that can’t be modified and that takes precedence over the number of records specified in the limit parameter. The response does include a threshold-breaching record if the 10 MB limit was reached.

    The range for limit is 1 to 100,000, with a default of 10.

  • iteratorType (string) –

    Can be one of:

    • AT_SEQUENCE_NUMBER   – Indicates that reading should start from the event sequence number specified jointly by the commitNum and opNum parameters.

    • AFTER_SEQUENCE_NUMBER   – Indicates that reading should start right after the event sequence number specified jointly by the commitNum and opNum parameters.

    • TRIM_HORIZON   – Indicates that reading should start at the last untrimmed record in the system, which is the oldest unexpired (not yet deleted) record in the change-log stream.

    • LATEST   – Indicates that reading should start at the most recent record in the system, which is the latest unexpired (not yet deleted) record in the change-log stream.

  • commitNum (integer) – The commit number of the starting record to read from the change-log stream. This parameter is required when iteratorType is AT_SEQUENCE_NUMBER or AFTER_SEQUENCE_NUMBER, and ignored when iteratorType is TRIM_HORIZON or LATEST.

  • opNum (integer) – The operation sequence number within the specified commit to start reading from in the change-log stream data. The default is 1.

  • encoding (string) – If set to TRUE, Neptune compresses the response using gzip encoding.

Return type:

dict

Returns:

Response Syntax

{
    'lastEventId': {
        'string': 'string'
    },
    'lastTrxTimestampInMillis': 123,
    'format': 'string',
    'records': [
        {
            'commitTimestampInMillis': 123,
            'eventId': {
                'string': 'string'
            },
            'data': {
                'stmt': 'string'
            },
            'op': 'string',
            'isLastOp': True|False
        },
    ],
    'totalRecords': 123
}

Response Structure

  • (dict) –

    • lastEventId (dict) –

      Sequence identifier of the last change in the stream response.

      An event ID is composed of two fields: a commitNum, which identifies a transaction that changed the graph, and an opNum, which identifies a specific operation within that transaction:

      • (string) –

        • (string) –

    • lastTrxTimestampInMillis (integer) –

      The time at which the commit for the transaction was requested, in milliseconds from the Unix epoch.

    • format (string) –

      Serialization format for the change records being returned. Currently, the only supported value is NQUADS.

    • records (list) –

      An array of serialized change-log stream records included in the response.

      • (dict) –

        A serialized SPARQL stream record capturing a change-log entry for the RDF graph.

        • commitTimestampInMillis (integer) –

          The time at which the commit for the transaction was requested, in milliseconds from the Unix epoch.

        • eventId (dict) –

          The sequence identifier of the stream change record.

          • (string) –

            • (string) –

        • data (dict) –

          The serialized SPARQL change record. The serialization formats of each record are described in more detail in Serialization Formats in Neptune Streams.

          • stmt (string) –

            Holds an N-QUADS statement expressing the changed quad.

        • op (string) –

          The operation that created the change.

        • isLastOp (boolean) –

          Only present if this operation is the last one in its transaction. If present, it is set to true. It is useful for ensuring that an entire transaction is consumed.

    • totalRecords (integer) –

      The total number of records in the response.

Exceptions

  • NeptuneData.Client.exceptions.UnsupportedOperationException

  • NeptuneData.Client.exceptions.ExpiredStreamException

  • NeptuneData.Client.exceptions.InvalidParameterException

  • NeptuneData.Client.exceptions.MemoryLimitExceededException

  • NeptuneData.Client.exceptions.StreamRecordsNotFoundException

  • NeptuneData.Client.exceptions.ClientTimeoutException

  • NeptuneData.Client.exceptions.PreconditionsFailedException

  • NeptuneData.Client.exceptions.ThrottlingException

  • NeptuneData.Client.exceptions.ConstraintViolationException

  • NeptuneData.Client.exceptions.InvalidArgumentException

  • NeptuneData.Client.exceptions.IllegalArgumentException

  • NeptuneData.Client.exceptions.TooManyRequestsException