Upload binary files via `multipart/form-data` from REST API Explorer

[christo911]

Description / Steps to reproduce / Feature proposal

Unable to use local swagger-ui to upload files (Button missing).
Loopback requestBody does not accept "format: binary" used by openAPI to determine that the input field is for file upload.

Current Behavior

When using upload controller from example, the swagger-ui upload button does not appear and only shows a text box (no Upload button)
However if following openAPI spec, the button shows, but on submission of the file, errors occur unknown format "binary" is used in schema at path "#"

Expected Behavior

Either code should work or?

See Reporting Issues for more tips on writing good issues

From example - Button not showing

export class UploadController {
    @post("/image", {
        responses: {
            200: {
                content: {
                    "application/json": {
                        schema: {
                            type: "object"
                        },
                    },
                },
                description: "",
            },
        },
    })

    async image(
        @requestBody({
            description: 'multipart/form-data value.',
            required: true,
            content: {
                 "multipart/form-data": {
                     // Skip body parsing
                     "x-parser': 'stream",
                     schema: {
                         type: "object",
                     },
                 },
             },
        })
            request: Request,
        @inject(RestBindings.Http.RESPONSE) response: Response,
    ): Promise<object> {
        //const storage = multer.memoryStorage();
        
        const upload = multer({dest: "/"});

        return new Promise<object>((resolve, reject) => {
            upload.any()(request, response, (err: any) => {
                if (err) {
                    reject(err);
                } else {
                   resolve({
                        files: request.body.file,
                        fields: (request as any).fields,
                    });
                }
            });
        });
    }
}

But change the request body to:

@requestBody({
            description: 'multipart/form-data value.',
            required: true,
            content: {
                'multipart/form-data': {
                    'x-parser': 'stream',
                    schema: {
                        type: 'object',
                        properties: {
                            file: {
                                type: 'string',
                                format: 'binary'
                            }
                        }
                    }
                }
            }

        })

the choose file button appears, but on submission I get the error...
Unhandled error in POST /image: 500 Error: unknown format "binary" is used in schema at path "#/properties/file"

A bit lost here... thanks!

[christo911]

found out that OpenAPI 3.0 requestBody spec needed for recognising as upload requires properties, with format: 'binary'.
However, this is validated by AJV, which does not recognise format: binary as a known format.
This can be manually overridden in AJV via options, in the request-body.validator.ts file, by setting -

options = { unknownFormats: ["binary"]}
const ajv = new AJV(Object.assign({}, {
        allErrors: true,
    }, options));

After adding this, swagger-ui works as expected, with upload button and file uploaded no prob.

But it seems , there is no way to set the AJV options.

Only he following formats are supported for string validation with AJV "format" keyword:

date: full-date according to RFC3339.
time: time with optional time-zone.
date-time: date-time from the same source (time-zone is mandatory). date, time and date-time validate ranges in full mode and only regexp in fast mode (see options).
uri: full URI.
uri-reference: URI reference, including full and relative URIs.
uri-template: URI template according to RFC6570
url (deprecated): URL record.
email: email address.
hostname: host name according to RFC1034.
ipv4: IP address v4.
ipv6: IP address v6.
regex: tests whether a string is a valid regular expression by passing it to RegExp constructor.
uuid: Universally Unique IDentifier according to RFC4122.
json-pointer: JSON-pointer according to RFC6901.
relative-json-pointer: relative JSON-pointer according to this draft.

Possible to add way to enter the Current OpenApi Data Types to default options?

type	format	Comments
integer	int32	signed 32 bits
integer	int64	signed 64 bits (a.k.a long)
number	float
number	double
string
string	byte	base64 encoded characters
string	binary	any sequence of octets
boolean
string	date	As defined by full-date - RFC3339
string	date-time	As defined by date-time - RFC3339
string	password	A hint to UIs to obscure input.

[bajtos]

Thank you @stoffason for reporting the issue and investigating further information. We don't have first-class support for file uploads yet, this feature is tracked by #1873.

Let's keep this issue open as a place where to investigate and discuss different options for enabling REST API Explorer to upload files to LB4 applications.

The request-body schema you posted in your last comments seems reasonable to me.

@requestBody({
  description: 'multipart/form-data value.',
  required: true,
  content: {
    'multipart/form-data': {
      'x-parser': 'stream',
      schema: {
        type: 'object',
        properties: {
          file: {
              type: 'string',
              format: 'binary'
          }
        }
      }
    }
  }
})

found out that OpenAPI 3.0 requestBody spec needed for recognising as upload requires properties, with format: 'binary'.
However, this is validated by AJV, which does not recognise format: binary as a known format.

This looks like a bug in our validation layer. IMO, we should skip validation of request bodies that are parsed as a stream. It's not possible to validate value inside a stream that we cannot read because it's passed directly to the controller method.

I think we should to the same for raw parser that reads the request body into a Buffer.

Would you like to contribute such change(s) yourself? You can find the code parsing request bodies in packages/rest/src/body-parsers/body-parser.ts. See https://loopback.io/doc/en/contrib/code-contrib-lb4.html and https://loopback.io/doc/en/lb4/submitting_a_pr.html to get started.

[bajtos]

As a temporary workaround, you can leverage the recently-landed #2992 to configure AJV with unknownFormats: ["binary"]. See Customize parsing options in our docs.

[raymondfeng]

The following works for me:

this.bind(RestBindings.REQUEST_BODY_PARSER_OPTIONS).to({
      validation: {unknownFormats: ['binary']},
    });

[raymondfeng]

I'm blocked by swagger-api/swagger-ui#4600

[raymondfeng]

See a working example at #4801