Ektos AI is now in Early Access!Join our Discord 

Available models for Serverless Inference #

Access the most popular models instantly, with no cold starts. Pay only for what you use (by tokens, minutes, steps) ensuring cost efficiency and seamless performance.

Text Models #

NameString in APISizeContext LengthQuantizationLicense
Llama 3.1 70B Instructllama-3.1-70b-instruct70.60B8kbf16Llama 3.1 Community License Agreement
Qwen 2.5 Coder 32B Instructqwen2.5-coder-32b-instruct32.80B8kbf16Apache License 2.0
Pixtral 12b 2409pixtral-12b-240912.40B8kbf16Apache License 2.0
Mistral Nemo Instruct 2407mistral-nemo-instruct-240712.20B8kbf16Apache License 2.0
Llama 3.1 8B Instructllama-3.1-8b-instruct8.03B8kbf16Llama 3.1 Community License Agreement

Important: Context Length is intentionally set low during the Early Access phase and will be updated once General Availability is released

Ektos AI offers the most popular and trending open source models. We add new models on our platform immediately after they are released.

If you would like to use a model that is not currently supported, please let us know on Discord!

Next steps #