freshcrate
Home > Frameworks > SimpleLLMFunc

SimpleLLMFunc

A simple and well-tailored LLM application framework that enables you to seamlessly integrate LLM capabilities in the most "Code-Centric" manner. LLM As Function, Prompt As Code. ไธ€ไธช็ฎ€ๅ•็š„ๆฐๅˆฐ

Description

A simple and well-tailored LLM application framework that enables you to seamlessly integrate LLM capabilities in the most "Code-Centric" manner. LLM As Function, Prompt As Code. ไธ€ไธช็ฎ€ๅ•็š„ๆฐๅˆฐๅฅฝๅค„LLMๅบ”็”จๆก†ๆžถ๏ผŒ่ƒฝๅคŸ่ฎฉไฝ ไปฅๆœ€โ€œCode Centerโ€œ็š„ๆ–นๅผๆ— ็ผ้›†ๆˆLLM่ƒฝๅŠ›ใ€‚LLM As Function, Prompt As Code

README

SimpleLLMFunc

LLM as Function, Prompt as Code


Github StarsGithub ForksLicense: MIT Python Version PyPI Version Maintenance PRs Welcome

Update Notes (0.7.7)

๐Ÿ“š Mintlify Documentation Migration: the docs site is now fully migrated to Mintlify with Chinese as the default language and English pages under /en.

๐Ÿงฐ Skills + Quickstart Workflow: quickstart now surfaces packaged Agent Skills immediately after installation, and the bundled skills now include stronger guidance on provider organization, strong typing, Pydantic, and Harness Engineering.

๐Ÿงน Legacy Docs Cleanup: the old Read the Docs / Sphinx documentation tree, translation scripts, and related release baggage have been removed.

๐Ÿ“˜ Release Refresh: README, Mintlify docs, skills, specs, and release metadata have been aligned around the Mintlify workflow. See CHANGELOG for details.

๐Ÿ“š Complete Documentation

Read detailed documentation: Chinese Docs | English Docs


๐Ÿ’ก Project Introduction

SimpleLLMFunc is a lightweight yet comprehensive LLM/Agent application development framework. Its core philosophy is:

๐ŸŽฏ Core Design Philosophy

  • "LLM as Function" - Treat LLM calls as ordinary Python function calls
  • "Prompt as Code" - Prompts are directly written in function DocStrings, clear at a glance
  • "Code as Doc" - Function definitions serve as complete documentation

Through simple decorators, you can integrate LLM capabilities into Python applications with minimal code and the most intuitive approach.

๐Ÿค” Problems Solved

If you've encountered these dilemmas in LLM development:

  1. Over-abstraction - Low-code frameworks introduce too much abstraction for custom functionality, making code difficult to understand and maintain
  2. Lack of type safety - Workflow frameworks lack type hints, leading to errors in complex flows and uncertainty about return formats
  3. Steep learning curve - Frameworks like LangChain have cumbersome documentation, requiring extensive reading just to implement simple requirements
  4. Flow limitations - Many frameworks only support DAG (Directed Acyclic Graph), unable to build complex logic with loops or branches
  5. Code duplication - Without frameworks, you have to manually write API call code, repeating the same work every time, with prompts scattered throughout the code
  6. Insufficient observability - Lack of complete log tracking and performance monitoring capabilities

SimpleLLMFunc is designed specifically to solve these pain points.

โœจ Core Advantages

  • โœ… Code as Documentation - Prompts in function DocStrings, clear at a glance
  • โœ… Type Safety - Python type annotations + Pydantic models, enjoy IDE code completion and type checking
  • โœ… Extremely Simple - Only one decorator needed, automatically handles API calls, message building, response parsing
  • โœ… Complete Freedom - Function-based design, supports arbitrary flow control logic (loops, branches, recursion, etc.)
  • โœ… Async Native - Full async support, naturally adapts to high-concurrency scenarios, no additional configuration needed
  • โœ… Complete Features - Built-in tool system, multimodal support, API key management, traffic control, structured logging, observability integration
  • โœ… Provider Agnostic - OpenAI-compatible adaptation, easily switch between multiple model vendors
  • โœ… Easy to Extend - Modular design, supports custom LLM interfaces and tools

โš ๏ธ Important - All LLM interaction decorators (@llm_function, @llm_chat, @tool, etc.) support decorating both sync and async functions, but all returned results are async functions. Please call them using await or asyncio.run().


๐Ÿš€ Quick Start

Installation

Method 1: PyPI (Recommended)

pip install SimpleLLMFunc

Method 2: Source Installation

git clone https://github.com/NiJingzhe/SimpleLLMFunc.git
cd SimpleLLMFunc
poetry install

Initial Configuration

  1. Copy configuration template:
cp env_template .env
  1. Configure API keys and other parameters in .env. It's recommended to configure LOG_DIR and LANGFUSE_BASE_URL, LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY for logging and Langfuse tracking.

  2. Check examples/provider_template.json to understand how to configure multiple LLM providers

Export Packaged Agent Skills

If you want to install the bundled Agent Skills into tools such as OpenCode, export them with:

simplellmfunc-skill usage ~/.config/opencode/skills
simplellmfunc-skill developer ~/.config/opencode/skills

This creates:

  • ~/.config/opencode/skills/simplellmfunc
  • ~/.config/opencode/skills/simplellmfunc-developer

Use --force if you want to overwrite an existing exported skill folder.

A Simple Example

import asyncio
from SimpleLLMFunc import llm_function, OpenAICompatible

# Load LLM interface from configuration file
llm = OpenAICompatible.load_from_json_file("provider.json")["your_provider"]["model"]

@llm_function(llm_interface=llm)
async def classify_sentiment(text: str) -> str:
    """
    Analyze the sentiment tendency of text.

    Args:
        text: Text to analyze

    Returns:
        Sentiment classification, can be 'positive', 'negative', or 'neutral'
    """
    pass  # Prompt as Code!

async def main():
    result = await classify_sentiment("This product is amazing!")
    print(f"Sentiment classification: {result}")

asyncio.run(main())

โœจ Core Features

Feature Description
@llm_function decorator Transform any async function into an LLM-driven function, automatically handles Prompt building, API calls, and response parsing
@llm_chat decorator Build conversational Agents, supports streaming responses and tool calls
@tool decorator Register async functions as LLM-available tools, supports multimodal returns (images, text, etc.)
Type Safety Python type annotations + Pydantic models ensure type correctness, enjoy IDE code completion
Async Native Fully async design, native asyncio support, naturally adapts to high-concurrency scenarios
Multimodal Support Supports Text, ImgUrl, ImgPath multimodal input/output
OpenAI Compatible Supports any OpenAI API-compatible model service (OpenAI, Deepseek, Claude, LocalLLM, etc.)
API Key Management Automatic load balancing of multiple API keys, optimize resource utilization
Traffic Control Token bucket algorithm implements intelligent traffic smoothing, prevents rate limiting
Structured Logging Complete trace_id tracking, automatically records requests/responses/tool calls
Observability Integration Integrated Langfuse, complete LLM observability support
Flexible Configuration JSON format provider configuration, easily manage multiple models and vendors

๐Ÿ“– Detailed Guide

1. LLM Function Decorator - "Prompt As Code"

The core philosophy of SimpleLLMFunc is "Prompt as Code, Code as Doc". By writing Prompts directly in function DocStrings, it achieves:

Advantage Description
Code Readability Prompts are tightly integrated with functions, no need to search for Prompt variables everywhere
Type Safety Type annotations + Pydantic models ensure input/output correctness
IDE Support Complete code completion and type checking
Self-documenting DocString serves as both function documentation and LLM Prompt

@llm_function - Stateless Functions

"""
Example using LLM function decorator
"""
import asyncio
from typing import List
from pydantic import BaseModel, Field
from SimpleLLMFunc import llm_function, OpenAICompatible, app_log

# Define a Pydantic model as return type
class ProductReview(BaseModel):
    rating: int = Field(..., description="Product rating, 1-5 points")
    pros: List[str] = Field(..., description="List of product advantages")
    cons: List[str] = Field(..., description="List of product disadvantages")
    summary: str = Field(..., description="Review summary")

# Use decorator to create an LLM function
@llm_function(
    llm_interface=OpenAICompatible.load_from_json_file("provider.json")["volc_engine"]["deepseek-v3-250324"]
)
async def analyze_product_review(product_name: str, review_text: str) -> ProductReview:
    """You are a professional product review expert who needs to objectively analyze the following product review and generate a structured review report.
    
    The report should include:
    1. Overall product rating (1-5 points)
    2. List of main product advantages
    3. List of main product disadvantages
    4. Summary evaluation
    
    Rating rules:
    - 5 points: Perfect, almost no disadvantages
    - 4 points: Excellent, advantages clearly outweigh disadvantages
    - 3 points: Average, advantages and disadvantages are basically equal
    - 2 points: Poor, disadvantages clearly outweigh advantages
    - 1 point: Very poor, almost no advantages
    
    Args:
        product_name: Name of the product to review
        review_text: User's review content of the product
        
    Returns:
        A structured ProductReview object containing rating, advantages list, disadvantages list, and summary
    """
    pass  # Prompt as Code, Code as Doc

async def main():
    
    app_log("Starting example code")
    # Test product review analysis
    product_name = "XYZ Wireless Headphones"
    review_text = """
    I've been using these XYZ wireless headphones for a month. The sound quality is very good, especially the bass performance is excellent,
    and they're comfortable to wear, can be used for long periods without fatigue. The battery life is also strong, can last about 8 hours after full charge.
    However, the connection is occasionally unstable, sometimes suddenly disconnects. Also, the touch controls are not sensitive enough, often need to click multiple times to respond.
    Overall, these headphones have great value for money, suitable for daily use, but if you need them for professional audio work, they might not be enough.
    """
    
    try:
        print("\n===== Product Review Analysis =====")
        result = await analyze_product_review(product_name, review_text)
        # result is directly a Pydantic model instance
        # no need to deserialize
        print(f"Rating: {result.rating}/5")
        print("Advantages:")
        for pro in result.pros:
            print(f"- {pro}")
        print("Disadvantages:")
        for con in result.cons:
            print(f"- {con}")
        print(f"Summary: {result.summary}")
    except Exception as e:
        print(f"Product review analysis failed: {e}")

if __name__ == "__main__":
    asyncio.run(main())

Output:

===== Product Review Analysis =====
Rating: 4/5
Advantages:
- Very good sound quality, especially excellent bass performance
- Comfortable to wear, can be used for long periods without fatigue
- Strong battery life, can last about 8 hours after full charge
- Great value for money, suitable for daily use
Disadvantages:
- Connection occasionally unstable, sometimes suddenly disconnects
- Touch controls not sensitive enough, often need to click multiple times to respond
- Might not be enough for professional audio work
Summary: Excellent sound quality and battery life, comfortable to wear, but insufficient connection stability and touch control sensitivity, suitable for daily use but not for professional audio work.

Key Points:

  • โœ… Only need to declare function, types, and DocString, decorator handles the rest automatically
  • โœ… Directly returns Pydantic object, no manual deserialization needed
  • โœ… Supports complex nested Pydantic models
  • โœ… Small models may not output correct JSON, framework will automatically retry

@llm_chat - Conversations and Agents

Also supports creating conversational functions and Agent systems. llm_chat supports:

  • Multi-turn conversation history management
  • Real-time streaming responses
  • LLM tool calls and automatic execution
  • Flexible return modes (text or raw response)

If you want to build a complete Agent framework, you can refer to our sister project SimpleManus.

@tui - Out-of-the-box terminal UI for llm_chat

SimpleLLMFunc provides a ready-to-use Textual TUI powered by event streaming:

  • Alternating user/assistant chat timeline
  • Streaming markdown rendering
  • Tool call argument/result panels
  • Model and tool stats (latency, token usage)
  • Custom tool-event hooks for live tool output updates
  • Origin-aware event routing for parent and forked agent calls
  • Built-in selfref fork lifecycle and stream visualization
  • Built-in quit controls: /exit /quit /q, Ctrl+Q, Ctrl+C
from SimpleLLMFunc import llm_chat, tui


@tui(custom_event_hook=[...])
@llm_chat(llm_interface=my_llm_interface, stream=True, enable_event=True)
async def agent(message: str, history=None):
    """Your agent prompt"""


if __name__ == "__main__":
    agent()

See examples/tui_chat_example.py for a full example.

When enable_event=True, each EventYield includes origin metadata. This is especially useful for forked agent trees:

from SimpleLLMFunc.hooks import is_event_yield

async for output in agent("split this into parallel subtasks"):
    if not is_event_yield(output):
        continue

    if output.origin.fork_id:
        print(
            f"[fork:{output.origin.fork_id} depth={output.origin.fork_depth}] "
            f"{output.event.event_type}"
        )
    else:
        print(f"[main] {output.event.event_type}")

Async Native Design

Both llm_function and llm_chat are natively async designed, no additional configuration needed:

from SimpleLLMFunc import llm_function, llm_chat


@llm_function(llm_interface=my_llm_interface)
async def async_analyze_text(text: str) -> str:
    """Async text content analysis"""
    pass


@llm_chat(llm_interface=my_llm_interface, stream=True)
async def async_chat(message: str, history: List[Dict[str, str]]):
    """Async chat functionality, supports streaming responses"""
    pass


async def main():
    result = await async_analyze_text("Text to analyze")

    async for response, updated_history in async_chat("Hello", []):
        print(response)

Tool-Call Limit Default

llm_function and llm_chat now default to max_tool_calls=None.

  • None means SimpleLLMFunc does not impose a framework-level tool-call iteration cap by default
  • This is better for long-horizon agents and deep tool-using workflows
  • If you want stricter protection against looping or runaway tool plans, pass an explicit integer such as max_tool_calls=8
@llm_function(llm_interface=my_llm_interface, max_tool_calls=None)
async def analyze(text: str) -> str:
    """Analyze the text."""
    pass


@llm_chat(llm_interface=my_llm_interface, stream=True, max_tool_calls=12)
async def cautious_agent(message: str, history=None):
    """Chat agent with an explicit safety cap."""
    pass

Multimodal Support

SimpleLLMFunc supports multiple modalities of input and output, allowing LLMs to process text, images, and other content:

from SimpleLLMFunc import llm_function
from SimpleLLMFunc.type import ImgPath, ImgUrl, Text

@llm_function(llm_interface=my_llm_interface)
async def analyze_image(
    description: Text,           # Text description
    web_image: ImgUrl,          # Web image URL
    local_image: ImgPath        # Local image path
) -> str:
    """Analyze images and provide detailed explanations based on descriptions
    
    Args:
        description: Specific requirements for image analysis
        web_image: Web image URL to analyze
        local_image: Local reference image path for comparison
        
    Returns:
        Detailed image analysis results
    """
    pass

import asyncio


async def main():
    result = await analyze_image(
        description=Text("Please describe the differences between these two images in detail"),
        web_image=ImgUrl("https://example.com/image.jpg"),
        local_image=ImgPath("./reference.jpg")
    )
    print(result)


asyncio.run(main())

Decorator Parameters and Advanced Features

@llm_function and @llm_chat support rich configuration parameters:

@llm_function(
    llm_interface=llm_interface,          # LLM interface instance
    toolkit=[tool1, tool2],                # Tool list
    retry_on_exception=True,               # Auto retry on exception
    timeout=60                              # Timeout setting
)
async def my_function(param: str) -> str:
    """Supports {language} {style} analysis"""
    pass

result = await my_function(
    "some input",
    _template_params={
        "language": "English",
        "style": "Professional",
    },
)

_template_params is passed at call time and only used to format the function DocString via str.format. It is removed before signature binding and is not part of the LLM input. If a placeholder is missing, the original DocString is used (with a warning).

2. LLM Provider Interface

SimpleLLMFunc provides flexible LLM interface support:

Supported Providers (via OpenAI Compatible adaptation):

  • โœ… OpenAI (GPT-4, GPT-3.5, etc.)
  • โœ… Deepseek
  • โœ… Anthropic Claude
  • โœ… Volc Engine Ark
  • โœ… Baidu Qianfan
  • โœ… Local LLM (Ollama, vLLM, etc.)
  • โœ… Any OpenAI API-compatible service

Quick Integration Example

from SimpleLLMFunc import OpenAICompatible

# Method 1: Load from JSON configuration file
provider_config = OpenAICompatible.load_from_json_file("provider.json")
llm = provider_config["deepseek"]["v3-turbo"]

# Method 2: Direct creation
llm = OpenAICompatible(
    api_key="sk-xxx",
    base_url="https://api.deepseek.com/v1",
    model="deepseek-chat"
)

@llm_function(llm_interface=llm)
async def my_function(text: str) -> str:
    """Process text"""
    pass

provider.json Configuration File

{
    "deepseek": [
        {
            "model_name": "deepseek-v3.2",
            "api_keys": ["sk-your-api-key-1", "sk-your-api-key-2"],
            "base_url": "https://api.deepseek.com/v1",
            "max_retries": 5,
            "retry_delay": 1.0,
            "rate_limit_capacity": 10,
            "rate_limit_refill_rate": 1.0
        }
    ],
    "openai": [
        {
            "model_name": "gpt-4",
            "api_keys": ["sk-your-api-key"],
            "base_url": "https://api.openai.com/v1",
            "max_retries": 5,
            "retry_delay": 1.0,
            "rate_limit_capacity": 10,
            "rate_limit_refill_rate": 1.0
        }
    ]
}

Custom LLM Interface

You can implement completely custom LLM interfaces by inheriting from the LLM_Interface base class:

from SimpleLLMFunc.interface import LLM_Interface

class CustomLLMInterface(LLM_Interface):
    async def call_llm(self, messages, **kwargs):
        # Implement your own LLM calling logic
        pass

3. Logging and Observability System

SimpleLLMFunc includes complete log tracking and observability capabilities to help you gain deep insights into LLM application performance.

Core Features

Feature Description
Trace ID Auto Tracking Each call automatically generates a unique trace_id, associating all related logs
Structured Logging Supports multiple log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)
Context Propagation Automatically preserves context in async environments, trace_id automatically associated
Colored Output Beautified console output, improves readability
File Persistence Automatically writes to local log files, supports rotation and archiving
Langfuse Integration Out-of-the-box observability integration, visualizes LLM call chains

Trace Example

GLaDos_c790a5cc-e629-4cbd-b454-ab102c42d125  <- Auto-generated trace_id
โ”œโ”€โ”€ Function call input parameters
โ”œโ”€โ”€ LLM request content
โ”œโ”€โ”€ Token usage statistics
โ”œโ”€โ”€ Tool calls (if any)
โ”œโ”€โ”€ LLM response content
โ””โ”€โ”€ Execution time and performance metrics

Logging Usage Example

from SimpleLLMFunc.logger import app_log, push_error, log_context

# 1. Basic logging
app_log("Starting request processing", trace_id="request_123")
push_error("Error occurred", trace_id="request_123", exc_info=True)

# 2. Use context manager to automatically associate logs
with log_context(trace_id="task_456", function_name="analyze_text"):
    app_log("Starting text analysis")  # Automatically inherits context trace_id
    try:
        # Execute operations...
        app_log("Analysis completed")
    except Exception:
        push_error("Analysis failed", exc_info=True)  # Also automatically inherits trace_id

4. Tool System - Let LLMs Interact with Environment

SimpleLLMFunc implements a complete tool system, allowing LLMs to call external functions and APIs. Tools support two definition methods.

@tool Decorator Method (Recommended)

The most concise way: use the @tool decorator to register async functions as LLM-available tools.

โš ๏ธ The @tool decorator only supports decorating functions defined with async def

from pydantic import BaseModel, Field
from SimpleLLMFunc.tool import tool

# Define Pydantic model for complex parameters
class Location(BaseModel):
    latitude: float = Field(..., description="Latitude")
    longitude: float = Field(..., description="Longitude")

# Use decorator to create tool
@tool(name="get_weather", description="Get weather information for specified location")
async def get_weather(location: Location, days: int = 1) -> dict:
    """
    Get weather forecast for specified location
    
    Args:
        location: Location information, including latitude and longitude
        days: Forecast days, default is 1 day
        
    Returns:
        Weather forecast information
    """
    # Actual implementation would call weather API
    return {
        "location": f"{location.latitude},{location.longitude}",
        "forecast": [{"day": i, "temp": 25, "condition": "Sunny"} for i in range(days)]
    }

Advantages:

  • โœ… Concise and intuitive, automatically extracts parameter information from function signature
  • โœ… Supports Python native types and Pydantic models
  • โœ… Can still be called directly after decoration, convenient for unit testing
  • โœ… Supports multimodal returns (text, images, etc.)
  • โœ… Can be stacked: one function can be decorated with both @llm_function and @tool

Multimodal Tool Example

from SimpleLLMFunc.tool import tool
from SimpleLLMFunc.type import ImgPath, ImgUrl

@tool(name="generate_chart", description="Generate charts based on data")
async def generate_chart(data: str, chart_type: str = "bar") -> ImgPath:
    """
    Generate charts based on provided data
    
    Args:
        data: CSV format data
        chart_type: Chart type, default is bar chart
        
    Returns:
        Generated chart file path
    """
    # Actual implementation would generate chart and save locally
    chart_path = "./generated_chart.png"
    # ... Chart generation logic
    return ImgPath(chart_path)

@tool(name="search_web_image", description="Search web images")
async def search_web_image(query: str) -> ImgUrl:
    """
    Search web images
    
    Args:
        query: Search keywords
        
    Returns:
        Found image URL
    """
    # Actual implementation would call image search API
    image_url = "https://example.com/search_result.jpg"
    return ImgUrl(image_url)

Class Inheritance Method (Compatible)

Release History

VersionChangesUrgencyDate
v0.7.8# Change log for SimpleLLMFunc ## 0.7.8 (2026-04-16) - Responses Adapter and Selfref Fork Context Refinement ### โœจ New Features 1. **OpenAI Responses API adapter**: - Added `OpenAIResponsesCompatible` as a first-class interface adapter. - Supports `provider.json` loading, direct construction with `APIKeyPool`, Responses tool schema translation, and reasoning passthrough. - Keeps decorator-facing authoring unchanged while mapping system prompts to Responses `instructions`. 2. **RespoHigh4/16/2026
v0.7.7## Highlights - fully migrate documentation to Mintlify and remove the legacy Read the Docs / Sphinx pipeline - add bilingual Mintlify docs with Chinese as default and English under `/en` - improve packaged skills and quickstart guidance around skill export, provider organization, strong typing, Pydantic, and Harness Engineering ## Details - update README and README_ZH release notes for 0.7.7 - align skills, specs, and contributor docs with the Mintlify workflow - clean up obsolete docs trees, High4/3/2026

Similar Packages

agent-archNo descriptionmain@2026-04-21
agent-frameworkA framework for building, orchestrating and deploying AI agents and multi-agent workflows with support for Python and .NET.python-1.1.0
adk-pythonAn open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.v1.31.1
adk-javaAn open-source, code-first Java toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.v1.1.0
autogenA programming framework for agentic AIpython-v0.7.5