Why are thousands of consumer GPUs idle while cloud GPUs remain expensive?

Concept: Distributed Compute & Sandboxing
Status: Design & Working Prototypes

---

The Question

Cloud GPU instances (AWS, RunPod, Lambda) charge a premium for compute resources because they package them with enterprise service level agreements, high-speed fiber loops, and dedicated physical facilities. Meanwhile, consumer gaming PCs and homelabs sit idle for most of the day.

Can we harvest these idle, heterogeneous consumer GPUs into a decentralized compute grid for non-latency-critical workloads?

---

Constraints

Building a compute marketplace on top of home networks introduces two severe constraints:

1. Network Latency & NATs: Home connections reside behind symmetric NATs and dynamic IPs, with high packet latency and frequent disconnects.
2. Double-Sided Trust:
   • Host-to-Client: A provider hosting a GPU has physical access. They can dump GPU VRAM or intercept local execution processes to steal proprietary model weights or inputs.
   • Client-to-Host: User-submitted compute scripts can execute privilege escalations or launch network attacks from the host's residential IP.

---

Tradeoffs: Why Reject Kubernetes?

Standard distributed computing relies on Kubernetes (K8s) or similar container orchestrators. We rejected K8s for CampuGrid based on three architectural mismatches:

• Control Plane Reliability: K8s expects low-latency, static node pools. If a student turns off their gaming rig mid-run, a standard K8s api-server would trigger continuous, expensive reschedule cascades.
• Privileged Worker Daemons: The kubelet agent requires root privileges on the host to manage local process namespaces. Hosting providers cannot run a privileged root daemon controlled by a foreign master plane on their private machines.
• Network Topology: K8s assumes flat network namespaces. Home NAT routing breaks standard K8s container overlays.

The Decoupled Push-Pull Model

Instead of K8s, CampuGrid uses a decentralized push-pull scheduling model. Worker nodes run a lightweight, non-privileged agent that pulls self-contained jobs from a central task queue, checkpoints state locally, and reports heartbeat telemetry via secure gRPC channels.

---

Architecture

Below is the execution flow of a job scheduled on a zero-trust provider node:

┌────────────────────────────────────────────────────────┐
│               CENTRAL COORDINATOR & QUEUE              │
└──────────────────────────┬─────────────────────────────┘
                           │ (Pull Task)
┌──────────────────────────▼─────────────────────────────┐
│  PROVIDER AGENT (Non-Privileged User-space)            │
├────────────────────────────────────────────────────────┤
│  SANDBOX ENVELOPE (gVisor syscall virtualizer)         │
├────────────────────────────────────────────────────────┤
│  ROOTLESS CONTAINER EXECUTION (Podman namespace)       │
└────────────────────────────────────────────────────────┘

1. Kaniko Build: User submits a job with a Dockerfile. The worker agent builds the container image in user-space using Kaniko (which does not require a root-privileged Docker daemon), pushing layers directly to static storage.
2. gVisor Sandboxing: The container runs inside a gVisor runsc sandbox wrapper on the provider's host, intercepting and virtualizing all Linux kernel syscalls in user space.
3. GPU Mapping: The local agent maps the specific /dev/nvidia* CUDA devices into the sandboxed process namespace.

---

Implementation Details

Discovering Hardware via CUDA Runtime

Instead of calling shell commands to parse nvidia-smi logs, the provider agent uses direct dynamic library bindings to query CUDA properties at runtime, checking VRAM boundaries and thermal indexes:

// Core discovery snippet (Rust/Go prototype conceptual pattern)
import { cudaRuntime } from "cuda-bindings";

export function discoverLocalGPU() {
  const deviceCount = cudaRuntime.getDeviceCount();
  const devices = [];
  for (let i = 0; i < deviceCount; i++) {
    const memory = cudaRuntime.getDeviceMemoryInfo(i);
    const props = cudaRuntime.getDeviceProperties(i);
    devices.push({
      id: i,
      name: props.name,
      totalVRAM: memory.total,
      computeCapability: `${props.major}.${props.minor}`
    });
  }
  return devices;
}

---

Lessons Learned & Retrospectives

• Homomorphic Encryption is Unusable: We initially explored cryptographic memory obfuscation to secure client training data from provider host dumps. The performance overhead was $>1000\times$, rendering it mathematically impractical. We pivoted to explicit workload segregation: CampuGrid is designed strictly for open-source datasets and public training runs where data leakage is a known, accepted constraint.
• Fault Tolerance Requires Chunked Checkpointing: P2P nodes disconnect frequently. We had to inject wrapper hooks into the user's execution code that push epoch checkpoints automatically to MinIO. If a worker goes offline, the coordinator reschedules the run from the last checkpoint on a different peer.