We introduce DeepSeek-R1, which incorporates cold-start data before RL. DeepSeek-R1 achieves performance comparable to OpenAI-o1 across math, code, and reasoning tasks. 

DeepSeek-R1-Turbo

DeepSeek-R1

QwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini.

QwQ-32B

Gemma 3 introduces multimodality, supporting vision-language input and text outputs. It handles context windows up to 128k tokens, understands over 140 languages, and offers improved math, reasoning, and chat capabilities, including structured outputs and function calling. Gemma 3 27B is Google's latest open source model, successor to Gemma 2

gemma-3-27b-it

Phi-4-multimodal-instruct is a lightweight open multimodal foundation model that leverages the language, vision, and speech research and datasets used for Phi-3.5 and 4.0 models. The model processes text, image, and audio inputs, generating text outputs, and comes with 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning, direct preference optimization and RLHF (Reinforcement Learning from Human Feedback) to support precise instruction adherence and safety measures. The languages that each modal supports are the following: - Text: Arabic, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Hebrew, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, Turkish, Ukrainian - Vision: English - Audio: English, Chinese, German, French, Italian, Japanese, Spanish, Portuguese

Phi-4-multimodal-instruct

DeepSeek-R1-Distill-Llama-70B is a highly efficient language model that leverages knowledge distillation to achieve state-of-the-art performance. This model distills the reasoning patterns of larger models into a smaller, more agile architecture, resulting in exceptional results on benchmarks like AIME 2024, MATH-500, and LiveCodeBench. With 70 billion parameters, DeepSeek-R1-Distill-Llama-70B offers a unique balance of accuracy and efficiency, making it an ideal choice for a wide range of natural language processing tasks. 

DeepSeek-R1-Distill-Llama-70B

DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. To achieve efficient inference and cost-effective training, DeepSeek-V3 adopts Multi-head Latent Attention (MLA) and DeepSeekMoE architectures, which were thoroughly validated in DeepSeek-V2. 

DeepSeek-V3

Llama 3.3-70B Turbo is a highly optimized version of the Llama 3.3-70B model, utilizing FP8 quantization to deliver significantly faster inference speeds with a minor trade-off in accuracy. The model is designed to be helpful, safe, and flexible, with a focus on responsible deployment and mitigating potential risks such as bias, toxicity, and misinformation. It achieves state-of-the-art performance on various benchmarks, including conversational tasks, language translation, and text generation.

Llama-3.3-70B-Instruct-Turbo

Llama 3.3-70B is a multilingual LLM trained on a massive dataset of 15 trillion tokens, fine-tuned for instruction-following and conversational dialogue. The model is designed to be helpful, safe, and flexible, with a focus on responsible deployment and mitigating potential risks such as bias, toxicity, and misinformation. It achieves state-of-the-art performance on various benchmarks, including conversational tasks, language translation, and text generation.

Llama-3.3-70B-Instruct

Mistral Small 3 is a 24B-parameter language model optimized for low-latency performance across common AI tasks. Released under the Apache 2.0 license, it features both pre-trained and instruction-tuned versions designed for efficient local deployment.  The model achieves 81% accuracy on the MMLU benchmark and performs competitively with larger models like Llama 3.3 70B and Qwen 32B, while operating at three times the speed on equivalent hardware.

Mistral-Small-24B-Instruct-2501

DeepSeek R1 Distill Qwen 32B is a distilled large language model based on Qwen 2.5 32B, using outputs from DeepSeek R1. It outperforms OpenAI's o1-mini across various benchmarks, achieving new state-of-the-art results for dense models.  Other benchmark results include:  AIME 2024: 72.6 | MATH-500: 94.3 | CodeForces Rating: 1691.

DeepSeek-R1-Distill-Qwen-32B

Phi-4 is a model built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach was to ensure that small capable models were trained with data focused on high quality and advanced reasoning.

phi-4

Meta developed and released the Meta Llama 3.1 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8B, 70B and 405B sizes

Meta-Llama-3.1-70B-Instruct

Meta-Llama-3.1-8B-Instruct

Meta-Llama-3.1-405B-Instruct

Meta-Llama-3.1-8B-Instruct-Turbo

Meta-Llama-3.1-70B-Instruct-Turbo

Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). It has significant improvements in code generation, code reasoning and code fixing. A more comprehensive foundation for real-world applications such as Code Agents. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.

Qwen2.5-Coder-32B-Instruct

Llama-3.1-Nemotron-70B-Instruct is a large language model customized by NVIDIA to improve the helpfulness of LLM generated responses to user queries. This model reaches Arena Hard of 85.0, AlpacaEval 2 LC of 57.6 and GPT-4-Turbo MT-Bench of 8.98, which are known to be predictive of LMSys Chatbot Arena Elo.  As of 16th Oct 2024, this model is #1 on all three automatic alignment benchmarks (verified tab for AlpacaEval 2 LC), edging out strong frontier models such as GPT-4o and Claude 3.5 Sonnet.

Llama-3.1-Nemotron-70B-Instruct

Qwen2.5 is a model pretrained on a large-scale dataset of up to 18 trillion tokens, offering significant improvements in knowledge, coding, mathematics, and instruction following compared to its predecessor Qwen2. The model also features enhanced capabilities in generating long texts, understanding structured data, and generating structured outputs, while supporting multilingual capabilities for over 29 languages.

Qwen2.5-72B-Instruct

The Llama 90B Vision model is a top-tier, 90-billion-parameter multimodal model designed for the most challenging visual reasoning and language tasks. It offers unparalleled accuracy in image captioning, visual question answering, and advanced image-text comprehension. Pre-trained on vast multimodal datasets and fine-tuned with human feedback, the Llama 90B Vision is engineered to handle the most demanding image-based AI tasks.  This model is perfect for industries requiring cutting-edge multimodal AI capabilities, particularly those dealing with complex, real-time visual and textual analysis.

Llama-3.2-90B-Vision-Instruct

Llama 3.2 11B Vision is a multimodal model with 11 billion parameters, designed to handle tasks combining visual and textual data. It excels in tasks such as image captioning and visual question answering, bridging the gap between language generation and visual reasoning. Pre-trained on a massive dataset of image-text pairs, it performs well in complex, high-accuracy image analysis.  Its ability to integrate visual understanding with language processing makes it an ideal solution for industries requiring comprehensive visual-linguistic AI applications, such as content creation, AI-driven customer service, and research.

Llama-3.2-11B-Vision-Instruct

At 8 billion parameters, with superior quality and prompt adherence, this base model is the most powerful in the Stable Diffusion family. This model is ideal for professional use cases at 1 megapixel resolution

sd3.5

Black Forest Labs' latest state-of-the art proprietary model sporting top of the line prompt following, visual quality, details and output diversity.

FLUX-1.1-pro

FLUX.1 [schnell] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions. This model offers cutting-edge output quality and competitive prompt following, matching the performance of closed source alternatives. Trained using latent adversarial diffusion distillation, FLUX.1 [schnell] can generate high-quality images in only 1 to 4 steps. 

FLUX-1-schnell

FLUX.1-dev is a state-of-the-art 12 billion parameter rectified flow transformer developed by Black Forest Labs. This model excels in text-to-image generation, providing highly accurate and detailed outputs. It is particularly well-regarded for its ability to follow complex prompts and generate anatomically accurate images, especially with challenging details like hands and faces.

FLUX-1-dev

Black Forest Labs' first flagship model based on Flux latent rectified flow transformers

FLUX-pro

  At 2.5 billion parameters, with improved MMDiT-X architecture and training methods, this model is designed to run “out of the box” on consumer hardware, striking a balance between quality and ease of customization. It is capable of generating images ranging between 0.25 and 2 megapixel resolution. 

sd3.5-medium

Kokoro is a frontier TTS model for its size of 82 million parameters (text in/audio out).  On 25 Dec 2024, Kokoro v0.19 weights were permissively released in full fp32 precision under an Apache 2.0 license. As of 2 Jan 2025, 10 unique Voicepacks have been released, and a .onnx version of v0.19 is available.

Kokoro-82M

Whisper is a state-of-the-art model for automatic speech recognition (ASR) and speech translation, proposed in the paper "Robust Speech Recognition via Large-Scale Weak Supervision" by Alec Radford  et al. from OpenAI. Trained on >5M hours of labeled data, Whisper demonstrates a strong ability to generalise to many datasets and domains in a zero-shot setting. Whisper large-v3-turbo is a finetuned version of a pruned Whisper large-v3. In other words, it's the exact same model, except that the number of decoding layers have reduced from 32 to 4. As a result, the model is way faster, at the expense of a minor quality degradation.

whisper-large-v3-turbo

Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multi-task model that can perform multilingual speech recognition as well as speech translation and language identification.

whisper-large-v3

WizardLM-2 8x22B is Microsoft AI's most advanced Wizard model. It demonstrates highly competitive performance compared to those leading proprietary models.

WizardLM-2-8x22B

BGE-M3 is a multilingual text embedding model developed by BAAI, distinguished by its Multi-Linguality (supporting 100+ languages), Multi-Functionality (unified dense, multi-vector, and sparse retrieval), and Multi-Granularity (handling inputs from short queries to 8192-token documents). It achieves state-of-the-art retrieval performance across diverse benchmarks while maintaining a single model for multiple retrieval modes.

DeepInfra supports the OpenAI embeddings API.
The following creates an embedding vector representing the input text

```bash
curl "https://api.deepinfra.com/v1/openai/embeddings" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DEEPINFRA_TOKEN" \
  -d '{
    "input": "The food was delicious and the waiter...",
    "model": "BAAI/bge-m3-multi",
    "encoding_format": "float"
  }'
```

which will return something similar to

```json
{
  "object":"list",
  "data":[
    {
      "object": "embedding",
      "index":0,
      "embedding":[
        -0.010480394586920738,
        -0.0026091758627444506
        ...
        0.031979579478502274,
        0.02021978422999382
      ]
    }
  ],
  "model": "BAAI/bge-m3-multi",
  "usage": {
    "prompt_tokens":12,
    "total_tokens":12
  }
}
```


You can use the official OpenAI python client to create embedding vectors

```python
from openai import OpenAI

# Create an OpenAI client with your deepinfra token and endpoint
openai = OpenAI(
    api_key="$DEEPINFRA_TOKEN",
    base_url="https://api.deepinfra.com/v1/openai",
)

input = "The food was delicious and the waiter...", # or an array ["hello", "world"]

embeddings = openai.embeddings.create(
  model="BAAI/bge-m3-multi",
  input=input,
  encoding_format="float"
)

if isinstance(input, str):
    print(embeddings.data[0].embedding)
else:
    for i in range(len(input)):
        print(embeddings.data[i].embedding)

print(embeddings.usage.prompt_tokens)

# [0.0241, 0.5, 1.0, ..., 0.3123, 0.743564]
# 42
```


You can use the official OpenAI Node.js client to create embedding vectors

```javascript
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: 'https://api.deepinfra.com/v1/openai',
  apiKey: "$DEEPINFRA_TOKEN",
});

const input = "The quick brown fox jumped over the lazy dog" // or an array ["hello", "world"]

async function main() {
  const embedding = await openai.embeddings.create({
    model: "BAAI/bge-m3-multi",
    input: input,
    encoding_format: "float",
  });

  // check if input is a string or array
  if (typeof input === "string") {
    console.log(embedding.data[0].embedding);
  } else {
    console.log(embedding.data.map((data) => data.embedding));
  }

  console.log(embedding.usage.prompt_tokens);
}

main();

// [0.0241, 0.5, 1.0, ..., 0.3123, 0.743564]
// 42
```


You can use cURL or any other http client to run inferences:

```bash
curl -X POST \
    -d '{"inputs": ["I like chocolate"]}'  \
    -H "Authorization: bearer $DEEPINFRA_TOKEN"  \
    -H 'Content-Type: application/json'  \
    'https://api.deepinfra.com/v1/inference/BAAI/bge-m3-multi'
```

which will give you back something similar to:

```json
{
  "input_tokens": 42,
  "embeddings": [
    [
      0.0,
      0.5,
      1.0
    ],
    [
      1.0,
      0.5,
      0.0
    ]
  ],
  "sparse": [
    [
      0.0,
      0.0,
      1.0
    ],
    [
      0.0,
      0.6,
      0.0
    ]
  ],
  "colbert": [
    [
      [
        0.5,
        0.1,
        1.0
      ],
      [
        0.3,
        0.6,
        0.5
      ]
    ],
    [
      [
        0.3,
        0.6,
        0.5
      ]
    ]
  ],
  "embedding_jsons": [
    "[0.0, 0.5, 1.0]",
    "[1.0, 0.5, 0.0]"
  ],
  "request_id": null,
  "inference_status": {
    "status": "unknown",
    "runtime_ms": 0,
    "cost": 0.0,
    "tokens_generated": 0,
    "tokens_input": 0
  }
}

```


You can use our command-line tool [deepctl](/docs/advanced/deepctl) to run
inferences:

```bash
deepctl infer \
    -m 'BAAI/bge-m3-multi'  \
    -i 'inputs=["I like chocolate"]'
```

which will give you back something similar to:

```json
{
  "input_tokens": 42,
  "embeddings": [
    [
      0.0,
      0.5,
      1.0
    ],
    [
      1.0,
      0.5,
      0.0
    ]
  ],
  "sparse": [
    [
      0.0,
      0.0,
      1.0
    ],
    [
      0.0,
      0.6,
      0.0
    ]
  ],
  "colbert": [
    [
      [
        0.5,
        0.1,
        1.0
      ],
      [
        0.3,
        0.6,
        0.5
      ]
    ],
    [
      [
        0.3,
        0.6,
        0.5
      ]
    ]
  ],
  "embedding_jsons": [
    "[0.0, 0.5, 1.0]",
    "[1.0, 0.5, 0.0]"
  ],
  "request_id": null,
  "inference_status": {
    "status": "unknown",
    "runtime_ms": 0,
    "cost": 0.0,
    "tokens_generated": 0,
    "tokens_input": 0
  }
}

```


We recommend using our NodeJS client https://github.com/deepinfra/deepinfra-node.

You can install it with

```bash
npm install deepinfra
```

and then

```javascript
import { Embeddings } from "deepinfra";

const DEEPINFRA_API_KEY = "$DEEPINFRA_TOKEN";
const MODEL = "BAAI/bge-m3-multi";

const main = async () => {
  const client = new Embeddings(MODEL, DEEPINFRA_API_KEY);
  const body = {
    inputs: [
      "What is the capital of France?",
      "What is the capital of Germany?",
      "What is the capital of Italy?",
    ],
  };
  const output = await client.generate(body);
  console.log(output.embeddings[0]);
};

main();
```



### 1. Only Dense Embeddings
This request returns only dense embeddings (`dense: true`, others `false`).

```bash
curl https://api.deepinfra.com/v1/inference/BAAI/bge-m3-multi \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DEEPINFRA_TOKEN" \
  -d '{
    "inputs": "I like chocolate",
    "dense": true,
    "sparse": false,
    "colbert": false
  }'
```

#### Response Schema
```json
{
  "request_id": "RJ9X08LaqG3Q4tl5fp2eM2cp",
  "inference_status": {
    "status": "succeeded",
    "runtime_ms": 34,
    "cost": 0.0,
    "tokens_generated": null,
    "tokens_input": null
  },
  "input_tokens": 13,
  "embeddings": [
    [0.0088958740234375, 0.01234, ...]
  ],
  "embedding_jsons": [
    "[0.0088958740234375, 0.01234, ...]",
  ],
  "sparse": null,
  "colbert": null
}
```

---

### 2. Only Sparse Embeddings
This request returns sparse embeddings (`dense: false`, `sparse: true`, `colbert: false`).
Corpus list can be found [here](https://gist.github.com/ovuruska/d3dd8e8508410725f3cbf8a721a5cb1d).


```bash
curl https://api.deepinfra.com/v1/inference/BAAI/bge-m3-multi \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DEEPINFRA_TOKEN" \
  -d '{
    "inputs": ["I like chocolate", "Another sequence"],
    "dense": false,
    "sparse": true,
    "colbert": false
  }'
```






#### Response Schema
```json
{
  "request_id": "RJ9X08LaqG3Q4tl5fp2eM2cp",
  "inference_status": {
    "status": "succeeded",
    "runtime_ms": 34,
    "cost": 0.0,
    "tokens_generated": null,
    "tokens_input": null
  },
  "input_tokens": 13,
  "sparse": [
    [0.0, 0.0, 1.0, ...],
    [0.0, 0.6, 0.0, ...]
  ],
  "embedding_jsons": null,
  "colbert": null
}
```

---

### 3. Dense, Sparse, and ColBERT Embeddings
This request returns all embedding types (`dense: true`, `sparse: true`, `colbert: true`).

```bash
curl https://api.deepinfra.com/v1/inference/BAAI/bge-m3-multi \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DEEPINFRA_TOKEN" \
  -d '{
    "inputs": "I like chocolate",
    "dense": true,
    "sparse": true,
    "colbert": true
  }'
```

#### Response Schema
```json
{
  "request_id": "RJ9X08LaqG3Q4tl5fp2eM2cp",
  "inference_status": {
    "status": "succeeded",
    "runtime_ms": 34,
    "cost": 0.0,
    "tokens_generated": null,
    "tokens_input": null
  },
  "input_tokens": 13,
  "embeddings": [
    [0.0088958740234375, 0.01234, ...]
  ],
  "embeddings_jsons": [
    "[0.0088958740234375, 0.01234, ...]"
  ],
  "sparse": [
    [0.0, 0.0, 1.0, ...]
  ],
  "colbert": [
    [
      [0.5, 0.1, 1.0],
      [0.3, 0.6, 0.5],
      ...
    ]
  ]
}
```



inputs

Dense is a set of low-dimensional vectors where every token in the input is represented by a fully populated embedding derived from a neural model.

dense

Sparse is a collection of high-dimensional vectors where each word in the input is assigned a lexical weight, with most values being zero.

sparse

Colbert is a system of contextualized vectors where every token in the input is represented by its own BERT-derived embedding.

colbert

whether to normalize the computed embeddings

normalize

The webhook to call when inference is done, by default you will get the output in the response of your inference request