freshcrate — #llm-inference

Home > #llm-inference

Tag: #llm-inference

7 packages • ⭐ 12,106 total stars

plano0.4.20🌿 Growing⭐6,241

Plano is an AI-native proxy and data plane for agentic apps — with built-in orchestration, safety, observability, and smart LLM routing so you stay focused on your agents core logic.

ai-gateway ai-gateway-support envoy envoyproxy gateway generative-ai llm-gateway llm-inference rustby katanemo

spiceaiv1.11.5🌱 Seedling⭐2,868

A portable accelerated SQL query, search, and LLM-inference engine, written in Rust, for data-grounded AI apps and agents.

artificial-intelligence data data-federation developers full-text-search infrastructure llm-inference machine-learning rustby spiceai

neuron-ai3.3.9🌿 Growing⭐1,834

The PHP Agentic Framework to build production-ready AI driven applications. Connect components (LLMs, vector DBs, memory) to agents that can interact with your data. With its modular architecture it's

agent agentic-ai agentic-framework agents ai llm llm-inference llms phpby neuron-core

MiniSearchmain@2026-04-20🌿 Growing⭐553

Minimalist web-searching platform with an AI assistant that runs directly from your browser. Uses WebLLM, Wllama and SearXNG. Demo: https://felladrin-minisearch.hf.space

ai ai-search-engine artificial-intelligence generative-ai gpu-accelerated information-retrieval llm llm-inference rag typescriptby felladrin

vllm-cliv0.2.5💤 Dormant⭐487

A command-line interface tool for serving LLM using vLLM.

llm llm-inference llm-tools python vllmby Chen-zexi

monoclev0.7.8🌿 Growing⭐72

Monocle is a framework for tracing GenAI app code. This repo contains implementation of Monocle for GenAI apps written in Python.

generative-ai linux-foundation llm-agent llm-inference llms observability opentelemetry oss pythonby monocle2ai

sample-genai-on-eks-starter-kitv1.1.0🌿 Growing⭐51

A comprehensive toolkit for deploying production-ready Generative AI infrastructure on Amazon EKS. Includes pre-configured components for: 🚀 AI Gateway (LiteLLM) 🤖 LLM Serving (vLLM, SGLang, Ollama

agentic-ai ai-engineering ai-platform javascript kubernetes llm-inference llmopsby aws-samples