DiffuGen - Advanced Local Image Generator with MCP Integration

Your AI art studio embedded directly in code. Generate, iterate, and perfect visual concepts through this powerful MCP server for Cursor, Windsurf, and other compatible IDEs, utilizing cutting-edge Flux and Stable Diffusion models without disrupting your development process.

⭐ New: Now includes OpenAPI server support and OpenWebUI OpenAPI Tools (OWUI Version 0.60.0 Required) integration for seamless image generation and display in chat interfaces! The OpenAPI is seperate from the MCP server and allowss for initigrations into your own projects!

📃 Table of Contents

• Introduction
• Understanding MCP and DiffuGen
• Features
• System Requirements
• Installation
• IDE Setup Instructions
• Usage
  • OpenAPI Server Usage
  • Default Parameters by Model
  • Asking a LLM to Generate Images
  • Parameter Reference
  • Model-Specific Parameter Recommendations
  • Default Parameter Changes
  • Command Line Usage Notes
• Configuration
  • Configuration Approach
  • Environment Variable Overrides
  • Setting IDE-Specific Configurations
  • Key Configuration Elements
  • IDE-Specific Options
  • Customizing Default Parameters
  • Updating Configuration Files
• Advanced Usage
  • Using the OpenAPI Server
• License
• Acknowledgments
• Contact

🚀 Introduction

DiffuGen is a powerful MCP-based image generation system that brings cutting-edge AI models directly into your development workflow. It seamlessly integrates both Flux models (Flux Schnell, Flux Dev) and Stable Diffusion variants (SDXL, SD3, SD1.5) into a unified interface, allowing you to leverage the unique strengths of each model family without switching tools. With comprehensive parameter control and multi-GPU support, DiffuGen scales from rapid concept sketches on modest hardware to production-quality visuals on high-performance systems.

Built on top of the highly optimized stable-diffusion.cpp implementation, DiffuGen offers exceptional performance even on modest hardware while maintaining high-quality output.

🧠 Understanding MCP and DiffuGen

What is MCP?

MCP (Model Context Protocol) is a protocol that enables LLMs (Large Language Models) to access custom tools and services. In simple terms, an MCP client (like Cursor, Windsurf, Roo Code, or Cline) can make requests to MCP servers to access tools that they provide.

DiffuGen as an MCP Server

DiffuGen functions as an MCP server that provides text-to-image generation capabilities. It implements the MCP protocol to allow compatible IDEs to send generation requests and receive generated images.

The server exposes two main tools:

1. generate_stable_diffusion_image: Generate with Stable Diffusion models
2. generate_flux_image: Generate with Flux models

Technical Architecture

DiffuGen consists of several key components:

• setup-diffugen.sh: The complete install utility and model downloader and manager
• diffugen.py: The core Python script that implements the MCP server functionality and defines the generation tools
• diffugen.sh: A shell script launcher that sets up the environment and launches the Python server
• diffugen.json: Template configuration file for MCP integration with various IDEs (to be copied into IDE's MCP configuration)
• stable-diffusion.cpp: The optimized C++ implementation of Stable Diffusion used for actual image generation

The system works by:

1. Receiving prompt and parameter data from an MCP client
2. Processing the request through the Python server
3. Calling the stable-diffusion.cpp binary with appropriate parameters
4. Saving the generated image to a configured output directory
5. Returning the path and metadata of the generated image to the client

About stable-diffusion.cpp

stable-diffusion.cpp is a highly optimized C++ implementation of the Stable Diffusion algorithm. Compared to the Python reference implementation, it offers:

• Significantly faster inference speed (up to 3-4x faster)
• Lower memory usage (works on GPUs with as little as 4GB VRAM)
• Optimized CUDA kernels for NVIDIA GPUs
• Support for various sampling methods and model formats
• Support for model quantization for better performance
• No Python dependencies for the core generation process

This allows DiffuGen to provide high-quality image generation with exceptional performance, even on modest hardware setups.

✨ Features

• Multiple Model Support: Generate images using various models including Flux Schnell, Flux Dev, SDXL, SD3, and SD1.5
• MCP Integration: Seamlessly integrates with IDEs that support MCP (Cursor, Windsurf, Roo Code, Cline, etc.)
• OpenAPI Server: Additional REST API interface for direct HTTP access to image generation capabilities
• Cross-Platform: Works on Linux, macOS, and Windows (via native or WSL)
• Parameter Control: Fine-tune your generations with controls for:
  • Image dimensions (width/height)
  • Sampling steps
  • CFG scale
  • Seed values
  • Negative prompts (for SD models only, Flux does not support negative prompts.)
  • Sampling methods
• CUDA Acceleration: Utilizes GPU acceleration for faster image generation
• Natural Language Interface: Generate images using simple natural language commands
• Smart Error Recovery: Robust error handling with operation-aware recovery procedures
• User-Friendly Setup: Interactive setup script with improved interrupt handling
• Resource Tracking: Session-aware resource management for efficient cleanup
• Customizable Interface: Support for custom ANSI art logos and visual enhancements

💻 System Requirements

Minimum Requirements:

• CPU: 4-core processor (Intel i5/AMD Ryzen 5 or equivalent)
• RAM: 8GB system memory
• Storage: 5GB free disk space (SSD preferred for faster model loading)
• Python: 3.8 or newer
• GPU: Integrated graphics or entry-level dedicated GPU (optional)
• Network: Broadband connection for model downloads (5+ Mbps)

Recommended Requirements:

• CPU: 8+ core processor (Intel i7/i9 or AMD Ryzen 7/9)
• RAM: 16GB+ system memory
• GPU: NVIDIA GPU with 6GB+ VRAM (RTX 2060 or better for optimal performance)
• Storage: 20GB+ free SSD space
• Python: 3.10 or newer (3.11 offers best performance)
• Network: High-speed connection (20+ Mbps) for efficient model downloads

📥 Installation

Automatic Installation (Recommended)

The easiest way to install DiffuGen is using the provided setup script:

git clone https://github.com/CLOUDWERX-DEV/diffugen.git
cd DiffuGen
chmod +x diffugen.sh
chmod +x setup_diffugen.sh
./setup_diffugen.sh

Follow the interactive prompts to complete the installation.

The setup script will:

• Install necessary dependencies
• Clone and build stable-diffusion.cpp
• Set up a Python virtual environment
• Download selected models (Note: Some models require Clip\VAE Models as well)
• Configure file paths for your system

Manual Installation

If you prefer to install manually, follow these steps:

1. Clone the repositories:

git clone https://github.com/CLOUDWERX-DEV/diffugen.git
cd DiffuGen
git clone --recursive https://github.com/leejet/stable-diffusion.cpp

2. Build stable-diffusion.cpp:

cd stable-diffusion.cpp
mkdir -p build && cd build

With CUDA:

cmake .. -DCMAKE_BUILD_TYPE=Release -DSD_CUDA=ON
make -j$(nproc)
cd ../..

Without CUDA:

cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
cd ../..

3. Create and activate a Python virtual environment:

python3 -m venv diffugen_env
source diffugen_env/bin/activate  # On Windows: diffugen_env\Scripts\activate
pip install -r requirements.txt

4. Download required models (structure shown below):

stable-diffusion.cpp/models/
├── ae.sft                           # VAE model
├── clip_l.safetensors               # CLIP model
├── flux/
│   ├── flux1-schnell-q8_0.gguf     # Flux Schnell model (default)
│   └── flux1-dev-q8_0.gguf          # Flux Dev model
├── sd3-medium.safetensors           # SD3 model
├── sdxl-1.0-base.safetensors        # SDXL model
├── sdxl_vae-fp16-fix.safetensors    # SDXL VAE
├── t5xxl_fp16.safetensors           # T5 model
└── v1-5-pruned-emaonly.safetensors  # SD1.5 model

You can download the models from the following sources:

# Create model directories
mkdir -p stable-diffusion.cpp/models/flux

# Flux models
# Flux Schnell - Fast generation model (Q8 Quantized,requires t5xxl, clip-l, vae)
curl -L https://huggingface.co/leejet/FLUX.1-schnell-gguf/resolve/main/flux1-schnell-q8_0.gguf -o stable-diffusion.cpp/models/flux/flux1-schnell-q8_0.gguf

# Flux Dev - Development model with better quality (Q8 QUantized, requires t5xxl, clip-l, vae)
curl -L https://huggingface.co/leejet/FLUX.1-dev-gguf/resolve/main/flux1-dev-q8_0.gguf -o stable-diffusion.cpp/models/flux/flux1-dev-q8_0.gguf

# Required models for Flux
# T5XXL Text Encoder
curl -L https://huggingface.co/Sanami/flux1-dev-gguf/resolve/main/t5xxl_fp16.safetensors -o stable-diffusion.cpp/models/t5xxl_fp16.safetensors

# CLIP-L Text Encoder
curl -L https://huggingface.co/Sanami/flux1-dev-gguf/resolve/main/clip_l.safetensors -o stable-diffusion.cpp/models/clip_l.safetensors

# VAE for image decoding
curl -L https://huggingface.co/pretentioushorsefly/flux-models/resolve/main/models/vae/ae.safetensors -o stable-diffusion.cpp/models/ae.sft

# Stable Diffusion models
# SDXL 1.0 Base Model (requires sdxl-vae)
curl -L https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors -o stable-diffusion.cpp/models/sd_xl_base_1.0.safetensors

# SDXL VAE (required for SDXL)
curl -L https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae-fp16-fix.safetensors -o stable-diffusion.cpp/models/sdxl_vae-fp16-fix.safetensors

# Stable Diffusion 1.5 (standalone)
curl -L https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors -o stable-diffusion.cpp/models/v1-5-pruned-emaonly.safetensors

# Stable Diffusion 3 Medium (standalone)
curl -L https://huggingface.co/leo009/stable-diffusion-3-medium/resolve/main/sd3_medium_incl_clips_t5xxlfp16.safetensors -o stable-diffusion.cpp/models/sd3_medium_incl_clips_t5xxlfp16.safetensors

Note: Model download may take a long time depending on your internet connection. The SDXL model is approximately 6GB, SD3 is about 13GB, SD1.5 is around 4GB, and Flux models are 8-13GB each.

5. Update file paths in configuration:

Set shell script as Executable

chmod +x diffugen.sh

Configuration Approach: DiffuGen uses a single configuration file (diffugen.json) as the source of truth for all settings. The workflow is:

1. Edit diffugen.json in the DiffuGen root directory with your desired settings
2. Run option 5 in setup_diffugen.sh to automatically update paths in this file
3. Copy the content of diffugen.json to your IDE's MCP configuration file

The file contains all necessary settings:

• File paths (command, SD_CPP_PATH, models_dir, output_dir)
• Default model parameters (steps, cfg_scale, sampling_method)
• VRAM usage settings
• Metadata for IDE integration

{
  "mcpServers": {
    "diffugen": {
      "command": "/home/cloudwerxlab/Desktop/Servers/MCP/Tools/DiffuGen/diffugen.sh",
      "args": [],
      "env": {
        "CUDA_VISIBLE_DEVICES": "0",
        "SD_CPP_PATH": "path/to/stable-diffusion.cpp",
        "default_model": "flux-schnell"
      },
      "resources": {
        "models_dir": "path/to/stable-diffusion.cpp/models",
        "output_dir": "path/to/outputs",
        "vram_usage": "adaptive"
      },
      "metadata": {
        "name": "DiffuGen",
        "version": "1.0",
        "description": "Your AI art studio embedded directly in code. Generate, iterate, and perfect visual concepts through this powerful MCP server for Cursor, Windsurf, and other compatible IDEs, utilizing cutting-edge Flux and Stable Diffusion models without disrupting your development process.",
        "author": "CLOUDWERX LAB",
        "homepage": "https://github.com/CLOUDWERX-DEV/diffugen",
        "usage": "Generate images using two primary methods:\n1. Standard generation: 'generate an image of [description]' with optional parameters:\n   - model: Choose from flux-schnell (default), flux-dev, sdxl, sd3, sd15\n   - dimensions: width and height (default: 512x512)\n   - steps: Number of diffusion steps (default: 20, lower for faster generation)\n   - cfg_scale: Guidance scale (default: 7.0, lower for more creative freedom)\n   - seed: For reproducible results (-1 for random)\n   - sampling_method: euler, euler_a (default), heun, dpm2, dpm++2s_a, dpm++2m, dpm++2mv2, lcm\n   - negative_prompt: Specify elements to avoid in the image\n2. Quick Flux generation: 'generate a flux image of [description]' for faster results with fewer steps (default: 4)"
      },
      "cursorOptions": {
        "autoApprove": true,
        "category": "Image Generation",
        "icon": "🖼️",
        "displayName": "DiffuGen"
      },
      "windsurfOptions": {
        "displayName": "DiffuGen",
        "icon": "🖼️",
        "category": "Creative Tools"
      },
      "default_params": {
        "steps": {
          "flux-schnell": 8,
          "flux-dev": 20,
          "sdxl": 20,
          "sd3": 20,
          "sd15": 20
        },
        "cfg_scale": {
          "flux-schnell": 1.0,
          "flux-dev": 1.0,
          "sdxl": 7.0,
          "sd3": 7.0, 
          "sd15": 7.0
        },
        "sampling_method": {
          "flux-schnell": "euler",
          "flux-dev": "euler",
          "sdxl": "euler",
          "sd3": "euler",
          "sd15": "euler"
        }
      }
    }
  }
}

🔧 IDE Setup Instructions

Setting up with Cursor

1. Download and install Cursor
2. Go to Cursor Settings > MCP and click "Add new global MCP server"
3. Copy the contents of your DiffuGen's diffugen.json file and paste it into ~/.cursor/mcp.json
4. Refresh MCP Servers in Settings > MCP
5. Use DiffuGen by opening the AI chat panel (Ctrl+K or Cmd+K) and requesting image generation

Setting up with Windsurf

1. Download and install Windsurf
2. Navigate to Windsurf > Settings > Advanced Settings or Command Palette > Open Windsurf Settings Page
3. Scroll down to the Cascade section and click "Add Server" > "Add custom server +"
4. Copy the contents of your DiffuGen's diffugen.json file and paste into ~/.codeium/windsurf/mcp_config.json
5. Use DiffuGen through the Cascade chat interface

Setting up with Roo Code

1. Download and install Roo Code
2. Locate the MCP configuration file for Roo Code
3. Copy the contents of your DiffuGen's diffugen.json file into Roo Code's MCP configuration
4. Use DiffuGen through the AI assistant feature

Setting up with Cline

1. Download and install Cline
2. Copy the contents of your DiffuGen's diffugen.json file into Cline's MCP settings
3. Use DiffuGen through the AI chat or command interface

Setting up with Claude in Anthropic Console

Claude can use DiffuGen if you've set it up as an MCP server on your system. When asking Claude to generate images, be specific about using DiffuGen and provide the parameters you want to use.

🎮 Usage

To start the DiffuGen server manually:

cd /path/to/diffugen
./diffugen.sh

Or using Python directly:

cd /path/to/diffugen
python -m diffugen

You should see: DiffuGen ready when the server is successfully started.

OpenAPI Server Usage

The OpenAPI server provides a REST API interface for direct HTTP access to DiffuGen's image generation capabilities. This is in addition to the MCP integration and can be useful for:

• Direct HTTP API access
• Integration with other tools that don't support MCP
• Custom applications that need programmatic access

For detailed setup instructions and advanced configuration options, see the OpenAPI Integration Guide.

To start the OpenAPI server:

python diffugen_openapi.py

The server can be configured to use a different host or port if needed. By default, it runs on:

• Host: 0.0.0.0
• Port: 8080

The server will be available at http://0.0.0.0:8080 with interactive documentation at http://0.0.0.0:8080/docs.

Generated images are saved to the /output directory by default. If this directory is not accessible, the server will automatically create an output directory in the current working directory. Images are served through the /images endpoint.

OpenWebUI Integration

1. Open OpenWebUI Settings (gear icon)
2. Navigate to the "Tools" section
3. Click the "+" button to add a new tool server
4. Enter the following details:
   • URL: http://0.0.0.0:5199
   • API Key: (leave empty)
5. Click "Save"

Once added, DiffuGen will appear in the available tools list when clicking the tools icon in the chat interface. The following endpoints will be available:

• generate_stable_image_generate_stable_post: Generate with Stable Diffusion
• generate_flux_image_endpoint_generate_flux_post: Generate with Flux Models
• list_models_models_get: List Available Models

Example using curl:

curl -X POST "http://0.0.0.0:5199/generate/flux" \
     -H "Content-Type: application/json" \
     -d '{"prompt": "A beautiful sunset", "model": "flux-schnell"}'

Example using Python requests:

import requests

response = requests.post(
    "http://0.0.0.0:5199/generate/flux",
    json={
        "prompt": "A beautiful sunset",
        "model": "flux-schnell"
    }
)
result = response.json()

Default Parameters by Model

Each model has specific default parameters optimized for best results:

Model	Default Steps	Default CFG Scale	Best For
flux-schnell	8	1.0	Fast drafts, conceptual images
flux-dev	20	1.0	Better quality flux generations
sdxl	20	7.0	High-quality detailed images
sd3	20	7.0	Latest generation with good quality
sd15	20	7.0	Classic baseline model

These default parameters can be customized by adding a default_params section to your IDE's MCP configuration file:

"default_params": {
  "steps": {
    "flux-schnell": 12,  // Customize steps for better quality
    "sdxl": 30           // Increase steps for more detailed SDXL images
  },
  "cfg_scale": {
    "sd15": 9.0          // Higher cfg_scale for stronger prompt adherence
  }
}

You only need to specify the parameters you want to override - any unspecified values will use the built-in defaults.

Note: For model-specific command line examples and recommendations, see Model-Specific Parameter Recommendations section.

Asking a LLM to Generate Images

Here are examples of how to ask an AI assistant to generate images with DiffuGen:

Basic Requests:

Generate an image of a cat playing with yarn

Create a picture of a futuristic cityscape with flying cars

With Model Specification:

Generate an image of a medieval castle using the sdxl model

Create a flux image of a sunset over mountains

With Advanced Parameters:

Generate an image of a cyberpunk street scene, model=flux-dev, width=768, height=512, steps=25, cfg_scale=1.0, seed=42

Create an illustration of a fantasy character with model=sd15, width=512, height=768, steps=30, cfg_scale=7.5, sampling_method=dpm++2m, negative_prompt=blurry, low quality, distorted

Parameter Reference

DiffuGen can be used from the command line with the following basic syntax:

./diffugen.sh "Your prompt here" [options]

Example:

./diffugen.sh "A futuristic cityscape with flying cars"

This command generates an image using default parameters (flux-schnell model, 512x512 resolution, etc.) and saves it to the configured output directory.

Below are the parameters that can be used with DiffuGen (applicable to both MCP interface and command line):

Parameter	Description	Default	Valid Values	Command Line Flag
model	The model to use for generation	flux-schnell/sd15	flux-schnell, flux-dev, sdxl, sd3, sd15	--model
width	Image width in pixels	512	256-2048	--width
height	Image height in pixels	512	256-2048	--height
steps	Number of diffusion steps	model-specific	1-100	--steps
cfg_scale	Classifier-free guidance scale	model-specific	0.1-30.0	--cfg-scale
seed	Random seed for reproducibility	-1 (random)	-1 or any integer	--seed
sampling_method	Diffusion sampling method	euler	euler, euler_a, heun, dpm2, dpm++2s_a, dpm++2m, dpm++2mv2, lcm	--sampling-method
negative_prompt	Elements to avoid in the image	"" (empty)	Any text string	--negative-prompt
output_dir	Directory to save images	Config-defined	Valid path	--output-dir

These parameters can be specified when asking an AI assistant to generate images or when using the command line interface. Parameters are passed in different formats depending on the interface:

• In MCP/AI Assistant: parameter=value (e.g., model=sdxl, width=768, height=512)
• In Command Line: --parameter value (e.g., --model sdxl --width 768 --height 512)

The default values are chosen to provide good results out-of-the-box with minimal waiting time. For higher quality images, consider increasing steps or switching to models like sdxl.

Model-Specific Parameter Recommendations

Note: These recommendations build on the Default Parameters by Model section and provide practical examples.

For best results when using specific models via command line:

Flux Models (flux-schnell, flux-dev)

# Flux-Schnell (fastest)
./diffugen.sh "Vibrant colorful abstract painting" \
  --model flux-schnell \
  --cfg-scale 1.0 \
  --sampling-method euler \
  --steps 8

# Flux-Dev (better quality)
./diffugen.sh "Detailed fantasy landscape with mountains and castles" \
  --model flux-dev \
  --cfg-scale 1.0 \
  --sampling-method euler \
  --steps 20

Standard SD Models (sdxl, sd3, sd15)

# SDXL (highest quality)
./diffugen.sh "Hyperrealistic portrait of a Celtic warrior" \
  --model sdxl \
  --cfg-scale 7.0 \
  --sampling-method dpm++2m \
  --steps 30

# SD15 (classic model)
./diffugen.sh "Photorealistic landscape at sunset" \
  --model sd15 \
  --cfg-scale 7.0 \
  --sampling-method euler_a \
  --steps 20

Default Parameter Changes

The command-line interface of DiffuGen uses the following defaults if not otherwise specified in configuration:

• Default Model: If not specified, function-appropriate models are used (flux-schnell for Flux functions, sd15 for SD functions)
• Default Sampling Method: euler (best for Flux models)
• Default CFG Scale: 1.0 for Flux models, 7.0 for standard SD models
• Default Steps: 8 for flux-schnell, 20 for other models
• Default Dimensions: 512x512 pixels

When using the command line, you don't need to specify these parameters unless you want to override the defaults. If you frequently use specific parameters, consider adding them to your configuration file rather than specifying them on each command line.

Command Line Usage Notes

• Generated images are saved to the configured output directory with filenames based on timestamp and parameters
• You can generate multiple images in sequence by running the command multiple times
• For batch processing, consider creating a shell script that calls DiffuGen with different parameters
• To see all available command-line options, run ./diffugen.sh --help
• The same engine powers both the MCP interface and command-line tool, so quality and capabilities are identical

⚙️ Configuration

Configuration Approach

DiffuGen uses a single configuration approach centered around the diffugen.json file:

1. Primary Configuration File: diffugen.json in the DiffuGen root directory is the single source of truth for all settings
2. IDE Integration: Copy the contents of diffugen.json to your IDE's MCP configuration file
3. Environment Variables: For advanced usage, you can override settings with environment variables

Environment Variable Overrides

For advanced usage, you can override settings using environment variables:

• SD_CPP_PATH: Override the path to stable-diffusion.cpp
• DIFFUGEN_OUTPUT_DIR: Override the output directory
• DIFFUGEN_DEFAULT_MODEL: Override the default model
• DIFFUGEN_VRAM_USAGE: Override VRAM usage settings
• CUDA_VISIBLE_DEVICES: Control which GPUs are used for generation

Setting IDE-Specific Configurations

DiffuGen allows you to have different configurations for different IDEs by using environment variables in each IDE's MCP configuration. This lets you maintain a single base diffugen.json while customizing parameters per IDE.

The configuration priority works as follows:

1. Environment variables (highest priority)
2. Settings from local diffugen.json file (base configuration)

Example: Different Output Directories for Different IDEs

For Cursor (in ~/.cursor/mcp.json):

"env": {
  "CUDA_VISIBLE_DEVICES": "0",
  "SD_CPP_PATH": "/path/to/stable-diffusion.cpp",
  "DIFFUGEN_OUTPUT_DIR": "/cursor/specific/output/directory",
  "default_model": "flux-schnell"
}

For Windsurf (in ~/.codeium/windsurf/mcp_config.json):

"env": {
  "CUDA_VISIBLE_DEVICES": "0",
  "SD_CPP_PATH": "/path/to/stable-diffusion.cpp",
  "DIFFUGEN_OUTPUT_DIR": "/windsurf/specific/output/directory",
  "default_model": "sdxl"
}

Example: Different Default Models and VRAM Settings

"env": {
  "CUDA_VISIBLE_DEVICES": "0",
  "SD_CPP_PATH": "/path/to/stable-diffusion.cpp",
  "default_model": "flux-dev", 
  "DIFFUGEN_VRAM_USAGE": "maximum"
}

This approach lets you customize DiffuGen's behavior per IDE while still using the same underlying installation.

Key Configuration Elements

Command and Arguments

• command: Full path to the diffugen.sh script (must be absolute path)
• args: Additional command-line arguments to pass to the script (usually left empty)

Environment Variables

• CUDA_VISIBLE_DEVICES: Controls which GPUs are used for generation

  • "0": Use only the first GPU
  • "1": Use only the second GPU
  • "0,1": Use both first and second GPUs
  • "-1": Disable CUDA and use CPU only

• SD_CPP_PATH: Path to the stable-diffusion.cpp installation directory

  • This is used to locate the stable-diffusion.cpp binary and models

• default_model: The default model to use when none is specified

Resource Configuration

• models_dir: Directory containing the model files

  • Should point to the models directory inside your stable-diffusion.cpp installation

• output_dir: Directory where generated images will be saved

  • Must be writable by the user running DiffuGen

• vram_usage: Controls VRAM usage strategy

  • "adaptive": Automatically adjust memory usage based on available VRAM
  • "minimal": Use minimal VRAM at the cost of speed
  • "balanced": Balance memory usage and speed (default)
  • "maximum": Use maximum available VRAM for best performance

IDE-Specific Options

Each IDE has specific options you can customize in the diffugen.json file:

Cursor Options

"cursorOptions": {
  "autoApprove": true,
  "category": "Image Generation",
  "icon": "🖼️",
  "displayName": "DiffuGen"
}

Windsurf Options

"windsurfOptions": {
  "displayName": "DiffuGen",
  "icon": "🖼️",
  "category": "Creative Tools"
}

Customizing Default Parameters

You can customize default parameters for each model in the default_params section:

"default_params": {
  "steps": {
    "flux-schnell": 12,
    "sdxl": 30
  },
  "cfg_scale": {
    "sd15": 9.0
  },
  "sampling_method": {
    "flux-schnell": "euler",
    "sdxl": "dpm++2m"
  }
}

Updating Configuration Files

When using the automatic setup script, a properly configured diffugen.json file is created with the correct paths for your system when you run option 5. To integrate DiffuGen with your IDE:

1. Run option 5 in setup_diffugen.sh to update paths in diffugen.json
2. Copy the entire contents of the generated diffugen.json file
3. Paste it into your IDE's MCP configuration file (e.g., ~/.cursor/mcp.json)
4. Restart your IDE to apply changes

The key advantage of this approach is a single source of truth for configuration, making it easier to maintain and update your DiffuGen setup.

📃 Advanced Usage

The DiffuGen Python module can be imported and used programmatically in your own Python scripts:

from diffugen import generate_image

# Generate an image programmatically
result = generate_image(
    prompt="A starry night over a quiet village",
    model="sdxl",
    width=1024,
    height=768,
    steps=30,
    cfg_scale=7.0,
    seed=42,
    sampling_method="dpm++2m",
    negative_prompt="blurry, low quality"
)

print(f"Image saved to: {result['file_path']}")

Using the OpenAPI Server

You can also use the OpenAPI server programmatically in your applications:

import requests

def generate_image_via_api(prompt, model="flux-schnell", width=512, height=512):
    response = requests.post(
        "http://0.0.0.0:5199/generate/flux",
        json={
            "prompt": prompt,
            "model": model,
            "width": width,
            "height": height
        }
    )
    return response.json()

# Example usage
result = generate_image_via_api(
    prompt="A magical forest at night",
    model="flux-schnell",
    width=768,
    height=512
)
print(f"Generated image: {result['file_path']}")

🔍 Troubleshooting

Common Issues and Solutions

1. Missing models or incorrect paths

   • Ensure all model files are downloaded and placed in the correct directories
   • Check that paths in the configuration file are correctly set
   • Verify file permissions allow read access to model files

2. CUDA/GPU issues

   • Make sure your NVIDIA drivers are up-to-date
   • Set CUDA_VISIBLE_DEVICES to target a specific GPU
   • If running out of VRAM, try using a smaller model or reducing dimensions

3. Image quality issues

   • Increase steps for better quality (at the cost of generation time)
   • Adjust CFG scale: higher for more prompt adherence, lower for creativity
   • Try different sampling methods (dpm++2m often provides good results)
   • Use more detailed prompts with specific style descriptions

4. File permission errors

   • Ensure the output directory is writable by the user running DiffuGen
   • Check that all scripts have execution permissions (chmod +x diffugen.sh)

Getting Help

If you encounter issues not covered here, you can:

• Check the GitHub repository for issues and solutions
• Run with debug logging enabled: DEBUG=1 ./diffugen.sh "your prompt"
• Contact the developers via GitHub issues

🌟 Contributing

Contributions to DiffuGen are welcome! To contribute:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request

Please ensure your code follows the project's coding standards and includes appropriate tests.

📄 License

This project is licensed under the Apache License - see the LICENSE file for details.

• All models are licensed under their respective distribution and are not in any way licensed or provided by CLOUDWERX.DEV
• HuggingFace.co is used to download models and is not affiliated in any way with CLOUDWERX.DEV

🙏 Acknowledgments

• stable-diffusion.cpp for the optimized C++ implementation
• Stability AI for Stable Diffusion models
• Black Forest Labs for their Flux Models
• Hugging Face for the download links
• All contributors to the MCP protocol

📬 Contact

• GitHub: CLOUDWERX-DEV
• Website: cloudwerx.dev
• Mail: [email protected]
• Discord: Join our server

                   ______   __   ___   ___         _______              
                  |   _  \ |__|.'  _|.'  _|.--.--.|   _   |.-----.-----.
                  |.  |   \|  ||   _||   _||  |  ||.  |___||  -__|     |
                  |.  |    \__||__|  |__|  |_____||.  |   ||_____|__|__|
                  |:  1    /                      |:  1   |             
                  |::.. . /                       |::.. . |             
                  `------'                        `-------'             

Made with ❤️ by CLOUDWERX LAB