Implement pipelines for FIM

We can call directly the endpoint in our gcloud instance from vscode with the following config

"tabAutocompleteModel": {
    "title": "Stacklok Hosted - Non Instruct",
    "provider": "openai",
    "model": "Qwen/Qwen2.5-Coder-14B",
    "apiKey": "$(token)",
    "apiBase": "https://inference.codegate.ai/v1"
  },
  "tabAutocompleteOptions": {
    "debounceDelay": 500
  }

Make the necessary code changes so that the request first passes through CodeGate and then goes to gcloud

Add a FIM pipeline to providers

The description of the task can be generalized to be able to call any model with FIM. The current logic of CodeGate should be already prepared to handle such requests. There are 2 changes that need to be made:

Message normalizer. We now expect all messages to have the key messages. However, there are incoming messages with key prompt. We should standardize the messages before calling our pipeline. Jakub is working on this.
Implement a pipeline for FIM messages independent to chat completion. Potentially in the future the pipelines can be merged. To keep things separated for the moment we should have 2 independent pipelines.

Add a FIM pipeline to providers

Closed via #102

Navigation Menu

Repository navigation

Implement pipelines for FIM #87

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Implement pipelines for FIM #87

Description

Activity

aponcedeleonch commented on Nov 27, 2024

aponcedeleonch commented on Nov 28, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions