Skip to content

Stream Infotons

Description

If you want to perform a one-time retrieval of a large amount of CM-Well data, you can do this by using the stream operation. You can define the data you want by its path in CM-Well, and optionally add a query filter and query parameters. The data returned is the list of infotons under the path you specified that match your query filter.

Special Parameters

Parameter Description       Values           Example
parallelism (NOTE: Only for the sstream operation.) The number of threads used for concurrent streaming. Positive integers. Default is 10 parallelism=15

Syntax

URL: <CMWellHost>

REST verb: GET

Mandatory parameters: op=stream


Template:

<cm-well-host>?op=stream&format=<format>&qp=<field conditions>

URL example: N/A

Curl example (REST API):

   `curl -X GET '<cm-well-host>/data.com?op=stream&format=ntriples&qp=type.rdf::http://ont.com/mdaas/Organization'`

Code Example

Call

curl -X GET '<cm-well-host>/data.com?op=stream&format=ntriples&qp=type.rdf::http://ont.com/mdaas/Organization'

Results

/data.com/2-d9c64e914747b50cc0580159cafd672977f87a2d8f7c424cc0177d60699faa01
    /data.com/11-5040863809
    /data.com/2-66080e4c7e1711c91d26b608749e7c25a6add39595692be6d001604e2aac28ff
    /data.com/2-d82480d63501cecdd2b252087043e3a694c6cb1c20d126cbacfdc1aafa9478a0
    /data.com/2-91d3d82402a1ef373c3fa47510ffbf3eddab0cb10e8c14061fc1816e0972d8a8
    /data.com/2-f0a213ab46218e324fe706fecef492c2413b7a660bdfd7ce3c283f098a7c7e08
    /data.com/11-5014941732
    /data.com/2-1c1244bb2da2c93b3713c3268797fb3eda3955b794f742baa6c42c9e74a8749a
    /data.com/2-8d210ecb2c6e6c8407e7ad26df6a663bb75f48f1409e569104b54fac94869adc
    /data.com/1-39535720416
    /data.com/2-5755b0fc4a315fe7a4c161e49694aedf7a1e4bf0114b13399427aa3974ca7958
    /data.com/11-5027817174
    /data.com/2-cc45840b28a43791097e0dc092e3c6cc76c54692e55aa4e04585fc18f685cc01
    /data.com/2-26cfa845e59944ebdee5079e8dd622db85b593e214d17915b7a3431c646f3dc2
    /data.com/2-8550d1186554359939b3da9dd8c9809aba7ee03ff0a50d8b2916ac1370face28
    /data.com/2-5b4999182e5243784fedccfda13acbb31e8ef081206dcd3bc9cc6a84dbdc7aec
    /data.com/11-5015222445
    /data.com/2-2b152d666e8b3ed47a974a3212999b81d0c2ffcd483a3ad459a7efa51ce10fbe
    /data.com/2-672d1ece5bb588880c7aeafc23f05927da23fd759914ed4d6e36726eb2059ebf
    /data.com/2-948cad8a8a3bade9af7576495928a292256f3264641361c70c6fc092fe430289
    ...

Note

  • A common way to handle data streams is to use the Chunked-Response HTTP Encoding protocol. As the size of the response is unknown when streaming starts, chunks of data are sent, each preceded by its size. A chunk of zero length indicates the end of the stream.
  • If for any reason the streaming operation fails, you will have to retrieve all the data from the beginning. An alternative method is the Consumer API, which is slower than streaming, but if interrupted allows you to resume from where you left off.
  • There are several different implementations of the streaming feature, with different operation names (stream, nstream, mstream, qstream, sstream). They have different approaches and may work better for different types of data (e.g. a small collection vs. a larger collection).
  • If the format is supplied in the call, the resulting infotons' data is returned. If you omit the format parameter, only the infotons' URIs are returned.
  • Only the following formats are supported for the stream command:
Format format= Mimetype
Text text text/plain
Tab-Separated Values tsv text/plain
NTriples ntriples              text/plain
NQuads nquads text/turtle

Subscribing to Real-Time Updates