[stephenplusplus]

Fixes #1144

The Vision API. Which has one endpoint.

Docs

http://stephenplusplus.github.io/gcloud-node/#/docs/master/vision

To Dos

• Docs
  • Add beta warning
  • JSDocs
  • Write overview
  • Update manifest.json
  • Update readme
• Support IMAGE_PROPERTIES
• Tests
  • System
    • Test single image
    • Test multiple image
    • Test verbose mode
  • Unit
    • vision/index.js
    • index.js

Examples

All Vision methods take a single or multiple images, which may be File objects or file paths. Therefore, these are all acceptable...

var input = './path/to/image.jpg';
var input = gcs.file('image.jpg');
var input = [
  './path/to/image.jpg',
  gcs.file('image.jpg')
];

To detect text from an image...

vision.detectText('./path/to/image.jpg', function(err, text) {
  text = [
    'This was text from the image',
    'This was more text found in the image'
  ]
});

To detect faces in an image... (this one returns a super long object)

vision.detectFaces(gcs.file('image.jpg'), function(err, faces) {
  faces = [
    {
      "angles": {
        "pan": -8.1090336,
        "roll": -5.0002542,
        "tilt": 18.012161
      },
      "bounds": {
        "head": [
          {
            "x": 1
          },
          {
            "x": 295
          },
          {
            "x": 295,
            "y": 301
          },
          {
            "x": 1,
            "y": 301
          }
        ],
        "face": [
          {
            "x": 28,
            "y": 40
          },
          {
            "x": 250,
            "y": 40
          },
          {
            "x": 250,
            "y": 262
          },
          {
            "x": 28,
            "y": 262
          }
        ]
      },
      "features": {
        "confidence": 0.34489909,
        "chin": {
          "center": {
            "x": 143.34183,
            "y": 262.22998,
            "z": -57.388493
          },
          "left": {
            "x": 63.102425,
            "y": 248.99081,
            "z": 44.207638
          },
          "right": {
            "x": 241.72728,
            "y": 225.53488,
            "z": 19.758242
          }
        },
        "ears": {
          "left": {
            "x": 54.872219,
            "y": 207.23712,
            "z": 97.030685
          },
          "right": {
            "x": 252.67567,
            "y": 180.43124,
            "z": 70.15992
          }
        },
        "eyebrows": {
          "left": {
            "left": {
              "x": 58.790176,
              "y": 113.28249,
              "z": 17.89735
            },
            "right": {
              "x": 106.14151,
              "y": 98.593758,
              "z": -13.116687
            },
            "top": {
              "x": 80.248711,
              "y": 94.04303,
              "z": 0.21131183
            }
          },
          "right": {
            "left": {
              "x": 148.61565,
              "y": 92.294594,
              "z": -18.804882
            },
            "right": {
              "x": 204.40808,
              "y": 94.300117,
              "z": -2.0009689
            },
            "top": {
              "x": 174.70135,
              "y": 81.580917,
              "z": -12.702137
            }
          }
        },
        "eyes": {
          "left": {
            "bottom": {
              "x": 84.883934,
              "y": 134.59479,
              "z": -2.8677137
            },
            "center": {
              "x": 83.707092,
              "y": 128.34,
              "z": -0.00013388535
            },
            "left": {
              "x": 72.213913,
              "y": 132.04138,
              "z": 9.6985674
            },
            "pupil": {
              "x": 86.531624,
              "y": 126.49807,
              "z": -2.2496929
            },
            "right": {
              "x": 105.28892,
              "y": 125.57655,
              "z": -2.51554
            },
            "top": {
              "x": 86.706947,
              "y": 119.47144,
              "z": -4.1606765
            }
          },
          "right": {
            "bottom": {
              "x": 179.30353,
              "y": 121.03307,
              "z": -14.843414
            },
            "center": {
              "x": 181.17694,
              "y": 115.16437,
              "z": -12.82961
            },
            "left": {
              "x": 158.2863,
              "y": 118.491,
              "z": -9.723031
            },
            "pupil": {
              "x": 175.99976,
              "y": 114.64407,
              "z": -14.53744
            },
            "right": {
              "x": 194.59413,
              "y": 115.91954,
              "z": -6.952745
            },
            "top": {
              "x": 173.99446,
              "y": 107.94287,
              "z": -16.050705
            }
          }
        },
        "forehead": {
          "x": 126.53813,
          "y": 93.812057,
          "z": -18.863352
        },
        "lips": {
          "bottom": {
            "x": 137.28528,
            "y": 219.23564,
            "z": -56.663128
          },
          "top": {
            "x": 134.74164,
            "y": 192.50438,
            "z": -53.876408
          }
        },
        "mouth": {
          "center": {
            "x": 136.43481,
            "y": 204.37952,
            "z": -51.620205
          },
          "left": {
            "x": 104.53558,
            "y": 214.05037,
            "z": -30.056231
          },
          "right": {
            "x": 173.79134,
            "y": 204.99333,
            "z": -39.725758
          }
        },
        "nose": {
          "bottom": {
            "center": {
              "x": 133.81947,
              "y": 173.16437,
              "z": -48.287724
            },
            "left": {
              "x": 110.98372,
              "y": 173.61331,
              "z": -29.7784
            },
            "right": {
              "x": 161.31354,
              "y": 168.24527,
              "z": -36.1628
            }
          },
          "tip": {
            "x": 128.14919,
            "y": 153.68129,
            "z": -63.198204
          },
          "top": {
            "x": 127.83745,
            "y": 110.17557,
            "z": -22.650913
          }
        }
      },
      "confidence": 0.56748849,
      "blurry": false,
      "dark": false,
      "happy": false,
      "hat": false,
      "mad": false,
      "sad": false,
      "surprised": false
    }
  ]
});

All methods also accept an object to activate "verbose" mode...

// Without verbose mode (default):
vision.detectLabels('./image.jpg', function(err, labels) {
  labels = [
    'classical sculpture',
    'statue',
    'landmark',
    'ancient history',
    'artwork'
  ];
});

// With verbose mode:
vision.detectLabels('./image.jpg', { verbose: true }, function(err, labels) {
  labels = [
    {
      id: '/m/095yjj',
      desc: 'classical sculpture',
      score: 0.98092282
    },
    {
      id: '/m/013_1c',
      desc: 'statue',
      score: 0.906611
    },
    {
      id: '/m/05_5t0l',
      desc: 'landmark',
      score: 0.85323846
    },
    {
      id: '/m/0djmp',
      desc: 'ancient history',
      score: 0.71385759
    },
    {
      id: '/m/0dgsmq8',
      desc: 'artwork',
      score: 0.69560623
    }
  ]
});

[dhermes]

Fancy! 🌈

[stephenplusplus]

@callmehiphop I'm going to work on locking this up with tests. Should I wait a tick for you to check out the API?

@dhermes 🐶❤️

[callmehiphop]

@stephenplusplus sure, I'll take a peak!

[callmehiphop]

I like it, the convenience methods look great! Do we know if there's a common use case where a user might want to leverage more than one feature on a given image (e.g. face + text)? If there is, it might be nice to make the annotate() options a little bit easier to work with? Thoughts?

[stephenplusplus]

Wow, I totally misread the API and thought only a single type of annotation could be requested at a time. I will definitely make it easier.

[stephenplusplus]

Docs are up! http://stephenplusplus.github.io/gcloud-node/#/docs/master/vision

[JustinBeckwith]

Awesome! @bradabrams I think you were using this too, right?

[stephenplusplus]

don't merge lifted!

[chrisweb]

Thank you @stephenplusplus for implementing the vision API in nodejs.

I used the code of your fork (git://github.com/stephenplusplus/gcloud-node.git#spp--vision), Is this the latest version? because I noticed that the face detect function does not return the original response

Looks like the splice line 334 removes the objects from it. To fix this I cloned the original response object in the Vision.prototype.annotate function, like this:

var respClone = extend(true, {}, resp);

callback(null, resp.responses, respClone);

I also wondered why the vision API always told me that I was not happy when analyzing my picture, so I tried to smile even more but this did not help, it always told me happy = false ... finally I noticed that line 1134 the likehood was not transformed into a value before the comparison, I changed it to this:

var value = Vision.likelihood[object[prop]];

[stephenplusplus]

Thank you very much for catching this stuff. I will fix this up so you will be happy in no time!

[chrisweb]

@stephenplusplus works perfectly now, well done and thank you

[stephenplusplus]

@chrisweb great, thanks again for testing this out!

[callmehiphop]

woo 👍