Documentation

ByteCompute API

ByteCompute's API is more advanced but gives you access to every model we provide unlike OpenAI which only works with LLMs and embeddings. You can also do Image Generation, Speech Recognition, Object detection, Token classification, Fill mask, Image classification, Zero-shot image classification and Text classification

JavaScript

JavaScript is a first class citizen at ByteCompute. You can install our official client https://github.com/byteCompute/byteCompute-node with

Copy 
npm install ByteCompute

HTTP/Curl

Don't want another dependency?

You prefer Go, C#, Java, PHP, Swift, Ruby, C++ or something exotic?

No problem. You can always use HTTP and have full access to all features by ByteCompute.

Completions/Text Generation

List of text generation models

You should know how to format the input to make completions work. Different models might have a different input format. The example below is for meta-llama/Meta-Llama-3-8B-Instruct

javascript Copy 
import { TextGeneration } from "ByteCompute";

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL_URL = "https://api.byteCompute.com/v1/inference/meta-llama/Meta-Llama-3-8B-Instruct";

async function main() {
  const client = new TextGeneration(MODEL_URL, ByteCompute_API_KEY);
  const res = await client.generate({
    input:
      "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nHello!<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
    stop: ["<|eot_id|>"]
  });

  console.log(res.results[0].generated_text);
  console.log(res.inference_status.tokens_input, res.inference_status.tokens_generated);
}

main();

For every model you can check its input format in its API section.

Embeddings

List of embeddings models

The following creates an embedding vector representing the input text

javascript Copy 
import { Embeddings } from "ByteCompute";

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL = "BAAI/bge-large-en-v1.5";

const main = async () => {
  const client = new Embeddings(MODEL, ByteCompute_API_KEY);
  const body = {
    inputs: ["What is the capital of France?", "What is the capital of Germany?", "What is the capital of Italy?"]
  };
  const output = await client.generate(body);
  console.log(output.embeddings[0]);
};

main();

Image Generation

List of image generation models

javascript Copy 
import { TextToImage } from "ByteCompute";
import { createWriteStream } from "fs";
import { Readable } from "stream";

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL = "stabilityai/stable-diffusion-2-1";

const main = async () => {
  const model = new TextToImage(MODEL, ByteCompute_API_KEY);
  const response = await model.generate({
    prompt: "a burger with a funny hat on the beach"
  });

  const result = await fetch(response.images[0]);

  if (result.ok && result.body) {
    let writer = createWriteStream("image.png");
    Readable.fromWeb(result.body).pipe(writer);
  }
};

main();

Speech Recognition

List of speech recognition models

Text to speed for a locally stored audio.mp3 file

javascript Copy 
import { AutomaticSpeechRecognition } from "ByteCompute";
import path from "path";
import { fileURLToPath } from "url";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL = "openai/whisper-large";

const main = async () => {
  const client = new AutomaticSpeechRecognition(MODEL, ByteCompute_API_KEY);

  const input = {
    audio: path.join(__dirname, "audio.mp3")
  };
  const response = await client.generate(input);
  console.log(response.text);
};

main();

Object Detection

List of object detection models

Send an image for detection

javascript Copy 
import { ObjectDetection } from "ByteCompute";
import path from "path";
import { fileURLToPath } from "url";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL = "hustvl/yolos-small";

const main = async () => {
  const model = new ObjectDetection(MODEL, ByteCompute_API_KEY);

  const input = {
    image: path.join(__dirname, "image.jpg")
  };
  const response = await model.generate(input);

  for (const result of response.results) {
    console.log(result.label, result.score, result.box);
  }
};

main();

Token Classification

List of token classification models

javascript Copy 
import { TokenClassification } from "ByteCompute";

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL = "Davlan/bert-base-multilingual-cased-ner-hrl";

const main = async () => {
  const model = new TokenClassification(MODEL, ByteCompute_API_KEY);

  const input = {
    input: "My name is John Doe and I live in San Francisco."
  };
  const response = await model.generate(input);

  console.log(response.results);
};

main();

Fill Mask

List of fill mask models

javascript Copy 
import { FillMask } from "ByteCompute";

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL = "bert-base-cased";

const main = async () => {
  const model = new FillMask(MODEL, ByteCompute_API_KEY);

  const body = {
    input: "I need my [MASK] right now!"
  };
  const response = await model.generate(body);

  console.log(response.results);
};

main();

Image Classification

List of image classification models

javascript Copy 
import { ImageClassification } from "ByteCompute";
import path from "path";
import { fileURLToPath } from "url";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL = "google/vit-base-patch16-224";

const main = async () => {
  const model = new ImageClassification(MODEL, ByteCompute_API_KEY);

  const input = {
    image: path.join(__dirname, "image.jpg")
  };
  const response = await model.generate(input);

  console.log(response.results);
};

main();

Zero-Shot Image Classification

List of zero-shot image classification models

javascript Copy 
import { ZeroShotImageClassification } from "ByteCompute";
import path from "path";
import { fileURLToPath } from "url";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL = "openai/clip-vit-base-patch32";

const main = async () => {
  const model = new ZeroShotImageClassification(MODEL, ByteCompute_API_KEY);

  const body = {
    image: path.join(__dirname, "image.jpg"),
    candidate_labels: ["dog", "cat", "car", "horse", "person"]
  };

  const response = await model.generate(body);
  console.log(response.results);
};

main();

Text Classification

List of text classification models

javascript Copy 
import { TextClassification } from "ByteCompute";

const ByteCompute_API_KEY = "$ByteCompute_TOKEN";
const MODEL = "ProsusAI/finbert";

const main = async () => {
  const model = new TextClassification(MODEL, ByteCompute_API_KEY);

  const body = {
    input: "Nvidia announces new AI chips months after latest launch as market competition heats up"
  };

  const response = await model.generate(body);
  console.log(response.results);
};

main();
OpenAI-Compatible APIRate Limits