Skip to main content

Deploying AI on NuNet

This tutorial shows you how to deploy AI models and workloads on the NuNet decentralized compute platform.

What You'll Learn​

  • Preparing AI models for deployment
  • Creating AI job specifications
  • Managing GPU resources
  • Monitoring AI workloads
  • Optimizing performance

Prerequisites​

  • Completed Introduction to NuNet
  • Basic understanding of AI/ML concepts
  • Familiarity with Docker containers
  • GPU-enabled device (optional but recommended)

AI Workload Types​

NuNet supports various AI workloads:

Machine Learning Training​

  • Model training jobs
  • Hyperparameter optimization
  • Distributed training

Model Inference​

  • Real-time inference
  • Batch processing
  • Model serving

Data Processing​

  • Data preprocessing
  • Feature engineering
  • Model evaluation

Preparing Your AI Model​

1. Containerize Your Model​

Create a Dockerfile for your AI application:

FROM python:3.9-slim

# Install dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy model and code
COPY model/ /app/model/
COPY src/ /app/src/

# Set working directory
WORKDIR /app

# Define entry point
CMD ["python", "src/inference.py"]

2. Create Model Specification​

Define your model requirements in a YAML file:

name: "my-ai-model"
version: "1.0.0"
description: "Custom AI model for image classification"

resources:
  cpu:
    cores: 4
    threads: 8
  memory:
    gb: 16
  gpu:
    count: 1
    memory: 8
  storage:
    gb: 50

runtime:
  image: "my-ai-model:latest"
  command: ["python", "src/inference.py"]
  environment:
    MODEL_PATH: "/app/model"
    BATCH_SIZE: "32"

Deploying AI Jobs​

1. Submit Your Job​

nunet-dms jobs submit --spec model-spec.yaml

2. Monitor GPU Usage​

nunet-dms resources gpu --monitor

3. Check Job Status​

nunet-dms jobs status <job-id> --detailed

GPU Resource Management​

GPU Configuration​

Configure GPU resources for AI workloads:

# Enable GPU support
nunet-dms config set gpu.enabled true

# Set GPU memory limits
nunet-dms config set gpu.memory-limit 8

# Configure GPU scheduling
nunet-dms config set gpu.scheduler "fifo"

Multi-GPU Support​

For distributed AI training:

nunet-dms config set gpu.count 4
nunet-dms config set gpu.topology "nvlink"

AI-Specific Features​

Model Versioning​

Manage different versions of your models:

nunet-dms models list
nunet-dms models deploy --version 1.2.0

A/B Testing​

Test different model versions:

nunet-dms experiments create --model-a v1.0 --model-b v1.1 --traffic-split 50

Auto-scaling​

Configure automatic scaling based on demand:

nunet-dms config set scaling.enabled true
nunet-dms config set scaling.min-instances 1
nunet-dms config set scaling.max-instances 10

Performance Optimization​

Resource Optimization​

Optimize resource usage for AI workloads:

# Enable mixed precision training
nunet-dms config set ai.mixed-precision true

# Configure batch size optimization
nunet-dms config set ai.auto-batch-size true

# Enable memory optimization
nunet-dms config set ai.memory-optimization true

Monitoring and Metrics​

Track AI-specific metrics:

# View model performance metrics
nunet-dms metrics model --job <job-id>

# Monitor GPU utilization
nunet-dms metrics gpu --real-time

# Check inference latency
nunet-dms metrics latency --model <model-id>

Example: Deploying a Computer Vision Model​

1. Prepare the Model​

# model/inference.py
import torch
import torchvision.transforms as transforms
from PIL import Image

class ImageClassifier:
    def __init__(self, model_path):
        self.model = torch.load(model_path)
        self.model.eval()
        self.transform = transforms.Compose([
            transforms.Resize(224),
            transforms.ToTensor(),
            transforms.Normalize(mean=[0.485, 0.456, 0.406],
                               std=[0.229, 0.224, 0.225])
        ])
    
    def predict(self, image_path):
        image = Image.open(image_path)
        input_tensor = self.transform(image).unsqueeze(0)
        
        with torch.no_grad():
            output = self.model(input_tensor)
            prediction = torch.argmax(output, dim=1)
        
        return prediction.item()

2. Create Job Specification​

name: "image-classifier"
version: "1.0.0"
description: "ResNet-based image classification"

resources:
  cpu:
    cores: 2
  memory:
    gb: 8
  gpu:
    count: 1
    memory: 4
  storage:
    gb: 20

runtime:
  image: "image-classifier:latest"
  command: ["python", "model/inference.py"]
  ports:
    - 8080:8080

3. Deploy and Test​

# Deploy the model
nunet-dms jobs submit --spec image-classifier.yaml

# Test with sample image
curl -X POST http://localhost:8080/predict \
  -F "image=@sample.jpg"

Troubleshooting AI Workloads​

Common Issues​

  1. GPU memory errors: Reduce batch size or model size
  2. Slow inference: Check GPU utilization and optimize model
  3. Model loading failures: Verify model format and dependencies

Debugging Commands​

# Check GPU status
nunet-dms gpu status

# View model logs
nunet-dms logs --job <job-id> --component model

# Profile performance
nunet-dms profile --job <job-id> --duration 60

Best Practices​

Model Optimization​

  • Use quantization for smaller models
  • Implement proper caching
  • Optimize input preprocessing

Resource Management​

  • Monitor GPU memory usage
  • Use appropriate batch sizes
  • Implement graceful shutdowns

Security​

  • Validate input data
  • Use secure model serving
  • Implement access controls

Next Steps​

Additional Resources​