NAV
shell

Getting Started

Quick Start

Below is a quick guide on how you can get started as quickly as possible.

Step 1 - Which Endpoint

Decide which of the following endpoints you’re interested in using. Dextro offers two endpoints to analyze pre-recorded content and one that analyzes live streams:

Video Categorization use this endpoint to have Dextro classify visual concepts
Sight, Sound & Motion use this endpoint to analyze both visual and audio components of the video.
Live Stream use this endpoint to analyze visual concepts (not audio) in real-time

Step 2 - Callback Testing

To be able to consume any of our endpoints you’ll need to set up a callback server. Once that’s in place it’s important to know whether it’s ready to receive callbacks from Dextro.

Callback Tester this endpoint allows you to test whether your callback server is able to receive requests

Step 3 - Making Requests

Now that you've confirmed that your callback server is able to receive requests you're all set. Every API request is assigned a uniquely identifiable request_id. Once a request is made you can check in on it's status by referencing the request_id with a call to our Request Tracker service.

Request Tracker this service allows you to track the status of a request

Troubleshooting

Callback Tester


curl -X POST -F api_key='call_me' -F video_url='http://try.dextro.co/cat'
 -F callback_url='callback_url' https://api.dextro.co/v1/video/categorize_async
  

By specifying call_me as your API key and http://try.dextro.co/cat as your video URL you can call any of our endpoints to test whether you're callback server is receiving requests. Once a call is made a standard endpoint payload is returned.

Required Parameters

Name Description
api_key Specify call_me as your API key
video_url Use http://try.dextro.co/cat as your video URL
callback_url This is an asynchronous endpoint. Results will be posted to this URL immediately.

Request Tracker


GET https://api.dextro.co/v1/track/< request_id >

curl -X GET -F api_key='api_key' https://api.dextro.co/v1/track/< request_id >

  

Sample JSON Response:

{
"status": "A sentence with the status of the request",
"tracked_request_id": 91572352-865b-4a97-c913-0bf7f0cd1aad,
"request_id": 42872312-123b-4a67-c212-0ef7b1cd1cad,
}

Sample Error JSON Response:

{
"status": "Could not track request_id: 982fd7de-1862-45a4-b331-e7c232663642",
"tracked_request_id": 982fd7de-1862-45a4-b331-e7c232663642,
"request_id": 777e2f7a-7801-43e6-8f60-f8b104fae864,
}

Returns the status of an API request.

Required Parameters

Name Description
api_key This is your unique secret API key.

Response

Name Description
status A message with the status of the request.
tracked_request_id The request_id of the request you're interested in tracking down
request_id A unique id associated with the request made to the Request Tracker endpoint

Status Messages

Messages
The request has been received and is currently in queue for processing
The request is being actively worked on
Video analysis is currently in progress
Video analysis is almost complete
Video analysis complete and a successful callback has been made for the request
Callbacks have been attempted for the request
The request failed

Error Messages

Sample JSON Response:

{
"request_id": "91572352-865b-4a97-c913-0bf7f0cd1aad",
"message": "A sentence about sadness.",
"code": 4XX
}

Below are a list of error messages you can expect to receive.

Error Code Message
400 We either didn't recognize a parameter or a required parameter is missing.
401 Invalid API key: Please check your API key
402 Credit Limit: API credit limit reached
415 Sorry, looks like the video video_url you sent us might be corrupt. Please review the video and try resending the request
420 We’re experiencing a high volume of requests, please excuse us for sluggish responses
422 Exceeds File Size Limit - The video video_url exceeds our file size limit of 512MB
422 Exceeds Permitted Video Length - video video_url exceeds the 30min video length limit
500 There was a problem processing the video.

Video

Video Categorization


POST https://api.dextro.co/v1/video/categorize_async

curl -X POST -F api_key='api_key' -F video_url='video_url'
-F callback_url='callback_url' https://api.dextro.co/v1/video/categorize_async

Sample JSON Response:


{
  "request_id": "91572352-865b-4a97-c913-0bf7f0cd1aad"
}

Sample JSON Callback Response:

{
  "request_id": "91572352-865b-4a97-c913-0bf7f0cd1aad",
  "length": 3652.467,
  "thumbnail": "video_level_thumbnail",
  "original_url": "source_url",
  "detections":[
      {
        "id": 2,
        "name": "Bicycles",
        "salience": 0.9,
        "thumbnail": "thumbnail_url",
        "instance_occurrences": [
            [
              9.56,
              10.52
            ],
            [
              11.04,
              11.48
            ],
            ...
            [
              75.04,
              84.52
            ]
          ]
      },
      {
        "id": 3,
        "name": "Bottles",
        "salience": 0.2,
        "thumbnail": "thumbnail_url",
        "instance_occurrences": [
            [
              12.0,
              12.44
            ],
            [
              32.12,
              32.96
            ],
            ...
            [
              112.4,
              115.24
            ]
          ]
      },
        ...
    ]
}

Returns list of detected items/scenes along with a salience metric for each and the time interval of the video they are present in.

Required Parameters

Name Description
api_key This is your unique secret API key.
video_url The URL of the video on which you want to run the detection.
callback_url This is an asynchronous endpoint. Results will be posted to this URL when the detection is complete.

Optional Parameters

Name Description
get_thumbnail Passing this parameter will return a thumbnail URL to a representative image of the detected item/scene. Please note that the URL will expire after a week.

Response

Name Description
request_id A unique id for that references the request that was made.

Callback Response

Name Description
request_id When the detection results are delivered to the callback address, they will be tagged with the same request id
length The duration of the video in seconds
original_url The source url for the video that was sent through the API
thumbnail A thumbnail URL to a representative image of the video. Please note that the URL will expire after a week.

One detections field containing a list of n detections, each with the following fields:

Name Description
id The category id of the model corresponding to the object/scene that was detected.
name The human-readable name of the object category.
salience A value between 0,1 representing the ratio between not only the percentage of time the instance appears in the video but also its prominence when in the field of view.
thumbnail Image URL of detected instance which will expire after a week.
instance_occurrences Contains a list of time intervals the instance is visible in the video. The units are seconds.

Sight Sound & Motion


POST https://api.dextro.co/v1/video/analyze_async

curl -X POST -F api_key='api_key' -F video_url='video_url' -F get_thumbnail='TRUE'
 -F callback_url='callback_url' https://api.dextro.co/v1/video/analyze_async

Sample JSON Response:

{
  "request_id": "91572352-865b-4a97-c913-0bf7f0cd1aad"
}

Sample JSON Callback Response:

{
  "request_id": "91572352-865b-4a97-c913-0bf7f0cd1aad",
  "length": 3652.467,
  "original_url": "source_url",
  "thumbnail": "video_level_thumbnail",
  "transcript_file": "srt_file_url",
  "ugc_flag": true",
  "topics": [...],
  "related_topics": ["Bands","Holiday","Skiing"],
  "highlights":[
                  {
                    "start": 2245.65,
                    "stop": 2257.95,
                    "support": ["audio","visual","motion"],
                    "topics":{
                                "Selfie": {
                                            "prominence": 0.961626,
                                            "start": 2114.3929,
                                            "stop": 2253.88,
                                            "support":["visual"],
                                            "thumbnail": "thumbnail_url"
                                          },
                                "Snow": {
                                            "prominence": 0.424,
                                            "start": 2246.42,
                                            "stop": 2249.29,
                                            "support":["audio"],
                                            "thumbnail": "thumbnail_url",
                                            "after": "castle.",
                                            "before": "Let's build a",
                                            "parents":["Snow","Precipitation","Forms of water"]
                                        }
                              }
                    }
                ],
  "timeline":[
                  {
                  "concept": "Selfie",
                  "prominence": 0.96162631,
                  "start": 0,
                  "stop": 209.2368,
                  "type": "visual"
                  },
                  {
                  "speech": "Oh",
                  "start": 5.77,
                  "stop": 6.31,
                  "type": "audio"
                  },
                  {
                  "concept": "Westin",
                  "parents":["Hotel","Resorts"],
                  "start": 47.84,
                  "stop": 48.59,
                  "type": "audio"
                  }
            ]
}

Returns list of detected items/scenes visually present and heard along with a prominence metric for each and the time interval of the video where they were detected.

Required Parameters

Name Description
api_key This is your unique secret API key.
video_url The URL of the video on which you want to run the detection.
callback_url This is an asynchronous endpoint. Results will be posted to this URL when the detection is complete.

Optional Parameters

Name Description
get_thumbnail Passing this parameter will return a thumbnail URL to a representative image of the detected item/scene. Please note that the URL will expire after a week.

Response

Name Description
request_id A unique id for that references the request that was made.

Callback Response

Name Description
request_id When the detection results are delivered to the callback address, they will be tagged with the same request id
length The duration of the video in seconds
original_url The source url for the video that was sent through the API
thumbnail A thumbnail URL to a representative image of the video. Please note that the URL will expire after a week.
transcript_url The url to the srt file containing the transcript. Please note that the transcript file URL will expire after a week.
related_topics A list of topics related to the concepts detected in the video

One video highlights field containing a list of n highlights each with the following fields:

Name Description
start/stop The time interval of the video highlight.
support A list of the type of detections that influenced the highlight selection ("visual", "audio", "motion").
topics A list of concepts detected within the highlight along with their instance occurence, their type of detection ("visual", "audio") and a representative thumbnail URL. Please note that the thumbnail URL will expire after a week.

One timeline field containing a list of n instance occurrences, each with the following fields:

Name Description
concept The human-readable name of the concept detected whether visually present or said in the video.
start/stop The time interval of when the concept was detected.
prominence A value between 0,1 representing the ratio between not only the percentage of time the instance appears in the video but also its prominence when in the field of view.
type Where the concept was detected whether, it was seen in the video ("visual") or said ("audio").
parents Contains a list of parent categories for the entity that was detected. This field is only returned for certain detections.

Live Stream

Live Stream Categorization


POST https://api.dextro.co/v1/livestream/categorize_async

curl -X POST -F api_key='api_key' -F livestream_url='livestream_url' -F callback_url='callback_url'
https://api.dextro.co/v1/livestream/categorize_async

Sample JSON Response:


{
"request_id": "91572352-865b-4a97-c913-0bf7f0cd1aad"
}

Sample JSON Callback Response for active live stream:

{
  "request_id": "91572352-865b-4a97-c913-0bf7f0cd1aad",
  "status": "live",
  "detections":[
    {
      "id": 726,
      "name": "Newsroom"
    },
    {
      "id": 521,
      "name": "Podium"
    }
  ]
}

Sample JSON Callback Response for terminated live stream:

{
  "request_id": "91572352-865b-4a97-c913-0bf7f0cd1aad",
  "status": "terminated"
}

Returns regular JSON callbacks with a list of applicable categorizations of the live stream.

Required Parameters

Name Description
api_key This is your unique secret API key.
livestream_url The URL of the video live stream on which you want to run detections. Supported streaming protocols include HLS, RTSP, RTMP, MMS among others
callback_url This is an asynchronous endpoint. PUT requests with the latest results will be made to this URL.

Response

Name Description
request_id A unique id for that references the request that was made.

Callback Response

Name Description
request_id When the detection results are delivered to the callback address, they will be tagged with the same request id
status The status will change depending on whether a live stream is still active or not. If the latter, the status will change from live to terminated

One detections field containing a list of n detections, each with the following fields:

Name Description
id The category id of the model corresponding to the object/scene that was detected.
name The human-readable name of the object category.