Microsoft Seeing AI API v1

Published July 9, 2019 · Last revised August 30, 2021

This API was reverse engineered from the Microsoft Seeing AI app available for iOS, version 3.0.0.0 and 4.1.1.0.

Endpoints

https://seeingai.trafficmanager.net
https://seeingai.azurewebsites.net
https://waws-prod-bay-071.vip.azurewebsites.windows.net

Get product name from barcode digits

POST /api/v1/query?intent=Product

Required headers:

signature: gvoKXKGmEO7YhqzQWaXoN/v1ieHQ00mh2vnKKEFpQ0o= (This is produced via Base64(HMAC-SHA256("intent=Product")).)
Content-Type: application/json

Example request:

{ "barcode": "07895502" }

Example response:

{
  "Product":{
    "Content":"Diet Dr Pepper Cherry",
    "Format":"Text",
    "Metadata":{
        "MoreInfo":"Amazingly Smooth Zero calories. Zero calories Amazingly Smooth One 6-pack of 12 fluid ounce cans"
    },
    "Success":true
  }
}

Get text content from document photo

POST /api/v1/query?intent=Document

Required headers:

signature: /y2bpHR6XIbBr7ZKUFjehmY6btoq8x4BjzahWwN26n0= (This is produced via Base64(HMAC-SHA256("intent=Document")).)
Content-Type: image/jpeg

Example request:

[JPEG image bytes]

Example response:

{
  "Document": {
    "Content": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\r\n<html xmlns=\"http://www.w3.org/XML/XMLSchema/v1.1\">\r\n  <head>\r\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />\r\n<style type=\"text/css\">body{font-family: arial; font-size: 12pt}</style></head>\r\n<body>\r\n<p style=\"font-size:54px;margin-top:0px;margin-bottom:2px;line-height:56px;text-indent:2px;text-align:justify\">IMPORTANT INFORMATION: This product subject to terms &amp; conditions (including an arbitration agreement); if you open the package, ...</p>\r\n\r\n</body>\r\n</html>",
    "Format": "HTML",
    "Success": true
  }
}

Get facial attributes from image of face(s)

POST /api/v1/query?intent=Person

Required headers:

signature: b8IJ82rIZW9sw7tILmKyalgbTMfLDSezi3L6T0rlXJ0= (This is produced via Base64(HMAC-SHA256("intent=Person")).)
Content-Type: image/jpeg

Example request:

[JPEG image bytes]

Example response:

{
  "Person": {
    "Content": "34 year old man with Brown hair wearing glasses looking happy.",
    "Format": "Text",
    "Regions": [
      {
        "Height": 335,
        "Properties": {
          "age": "34",
          "emotion": "happy",
          "gender": "male",
          "glasses": "glasses",
          "hairColor": "Brown",
          "label": "34 year old man with Brown hair wearing glasses looking happy."
        },
        "Width": 335,
        "X": 15,
        "Y": 197
      }
    ],
    "Success": true
  }
}

Get text content from handwritten text

POST /api/v1/query?intent=Handwriting

Required headers:

signature: sBmjwvbhtvXsLKyJsIZ/Mrodp9To4rWPQrlRHWWzSog= (This is produced via Base64(HMAC-SHA256("intent=Handwriting")).)
Content-Type: image/jpeg

Example request:

[JPEG image bytes]

Example response:

{
  "Handwriting": {
    "Content": "Elephant\nShoe",
    "Format": "Text",
    "Regions": [
      {
        "Height": 8,
        "Properties": {
          "label": "Elephant"
        },
        "Width": 18,
        "X": 113,
        "Y": 8
      },
      {
        "Height": 9,
        "Properties": {
          "label": "Shoe"
        },
        "Width": 24,
        "X": 237,
        "Y": 12
      },
    ],
    "Success": true
  }
}

Get text description of a scene

POST /api/v1/query?intent=Scene

Required headers:

signature: AfrBqBu/F0Tteqm94TQtQ5XBojxhhzcPh7ln5Orcanc= (This is produced via Base64(HMAC-SHA256("intent=Scene")).)
Content-Type: image/jpeg

Example request:

[JPEG image bytes]

Example response:

{
  "Scene": {
      "Content": "Probably a white square on a black wall",
      "Format": "Text",
      "Success": true
  }
}