πŸŽ‰ 75% of content is free forever β€” Unlock Premium from $10/mo β†’
CW
Search courses…
πŸ’Ό Servicesℹ️ Aboutβœ‰οΈ ContactView Pricing Plansfrom $10

AI Coding Assistant with Sandboxed Execution

AI AgentsCode Generation Agent🟒 Free Lesson

Advertisement

AI Coding Assistant with Sandboxed Execution

Code Agent PipelineUser RequestLLM GeneratorCode SandboxDebuggerTest RunnerOutput ParserIterative Refinement Loop

What is a Code Generation Agent?

Code generation agents go beyond simple code completion. They understand requirements, generate complete implementations, execute code in sandboxes, analyze results, and iteratively debug until the code works correctly.

The key challenge is safe execution. Running arbitrary LLM-generated code on your system is dangerous. Sandboxed execution environments (containers, subprocesses with restrictions, or cloud sandboxes) isolate generated code from the host system.

The most effective code agents follow a generate-execute-debug loop: generate code, run it, check for errors, fix issues, and repeat until working. This mirrors how human developers actually work.

Project Overview

We will build a code agent that:

  • Generates Python code from natural language descriptions
  • Executes code in a sandboxed subprocess with resource limits
  • Captures stdout, stderr, and return codes
  • Parses errors and iteratively fixes them
  • Runs tests to validate correctness
  • Supports multiple execution environments (local, Docker)

Expected outcome: A safe, iterative code generation agent that produces working code.

Difficulty: Advanced (requires understanding of subprocess management, Docker, and code safety)

Architecture

Code Agent ArchitectureCode GeneratorLLM + contextSandbox ExecutorIsolated executionError AnalyzerParse & fix errorsTest ValidatorOutput FormatterRefinement Loop Controller

Tools & Setup

ToolVersionPurpose
Python3.11+Core language
OpenAI1.0+LLM backbone
docker6.0+Sandboxed execution
subprocessstdlibLocal sandbox execution
aststdlibCode parsing

Step 1: Environment Setup

python -m venv venv
source venv/bin/activate
pip install openai docker
export OPENAI_API_KEY="sk-your-key"

Step 2: Project Structure

Architecture Diagram
code-agent/
β”œβ”€β”€ sandbox/
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ local_sandbox.py
β”‚   └── docker_sandbox.py
β”œβ”€β”€ generator.py
β”œβ”€β”€ debugger.py
β”œβ”€β”€ code_agent.py
β”œβ”€β”€ safety.py
└── main.py

Step 3: Code Safety Checker

# safety.py
import ast
from typing import List

FORBIDDEN_MODULES = {
    "os", "sys", "subprocess", "shutil", "pathlib",
    "socket", "http", "urllib", "requests",
    "ctypes", "multiprocessing", "threading",
}

FORBIDDEN_FUNCTIONS = {
    "exec", "eval", "compile", "__import__",
    "open", "input", "print",  # controlled separately
}

class CodeSafetyChecker:
    def __init__(self, allowed_modules: set = None):
        self.allowed_modules = allowed_modules or {"math", "json", "re", "datetime", "collections"}

    def check(self, code: str) -> tuple[bool, List[str]]:
        issues = []
        try:
            tree = ast.parse(code)
        except SyntaxError as e:
            return False, [f"Syntax error: {str(e)}"]

        for node in ast.walk(tree):
            if isinstance(node, ast.Import):
                for alias in node.names:
                    if alias.name.split(".")[0] in FORBIDDEN_MODULES:
                        issues.append(f"Forbidden import: {alias.name}")

            elif isinstance(node, ast.ImportFrom):
                if node.module and node.module.split(".")[0] in FORBIDDEN_MODULES:
                    issues.append(f"Forbidden import from: {node.module}")

            elif isinstance(node, ast.Call):
                if isinstance(node.func, ast.Name):
                    if node.func.id in FORBIDDEN_FUNCTIONS:
                        issues.append(f"Forbidden function call: {node.func.id}")

            elif isinstance(node, ast.Attribute):
                if isinstance(node.value, ast.Name):
                    if node.value.id in FORBIDDEN_MODULES:
                        issues.append(f"Forbidden attribute access: {node.value.id}.{node.attr}")

        return len(issues) == 0, issues

    def sanitize_code(self, code: str) -> str:
        tree = ast.parse(code)
        sanitized = []
        for node in ast.iter_child_nodes(tree):
            if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
                sanitized.append(ast.get_source_segment(code, node) or "")
        return "\n\n".join(sanitized)

Step 4: Sandbox Executor

# sandbox/local_sandbox.py
import subprocess
import tempfile
import os
from typing import Dict

class LocalSandbox:
    def __init__(
        self,
        timeout: int = 30,
        memory_limit_mb: int = 256,
        max_output_bytes: int = 1024 * 1024,
    ):
        self.timeout = timeout
        self.memory_limit_mb = memory_limit_mb
        self.max_output_bytes = max_output_bytes

    def execute(self, code: str) -> Dict:
        with tempfile.NamedTemporaryFile(
            mode="w", suffix=".py", delete=False, dir="/tmp"
        ) as f:
            f.write(code)
            temp_path = f.name

        try:
            result = subprocess.run(
                ["python", temp_path],
                capture_output=True,
                text=True,
                timeout=self.timeout,
                env={
                    **os.environ,
                    "PYTHONDONTWRITEBYTECODE": "1",
                },
                cwd=tempfile.gettempdir(),
            )
            return {
                "success": result.returncode == 0,
                "stdout": result.stdout[:self.max_output_bytes],
                "stderr": result.stderr[:self.max_output_bytes],
                "return_code": result.returncode,
                "execution_time_ms": 0,
            }
        except subprocess.TimeoutExpired:
            return {
                "success": False,
                "stdout": "",
                "stderr": f"Execution timed out after {self.timeout}s",
                "return_code": -1,
                "execution_time_ms": self.timeout * 1000,
            }
        finally:
            os.unlink(temp_path)

# sandbox/docker_sandbox.py
import docker
import tempfile
import os
from typing import Dict

class DockerSandbox:
    def __init__(
        self,
        image: str = "python:3.11-slim",
        timeout: int = 30,
        memory_limit: str = "256m",
    ):
        self.client = docker.from_env()
        self.image = image
        self.timeout = timeout
        self.memory_limit = memory_limit

    def execute(self, code: str) -> Dict:
        with tempfile.NamedTemporaryFile(
            mode="w", suffix=".py", delete=False
        ) as f:
            f.write(code)
            temp_path = f.name

        try:
            container = self.client.containers.run(
                self.image,
                command=f"python /code/script.py",
                volumes={
                    os.path.dirname(temp_path): {
                        "bind": "/code",
                        "mode": "ro",
                    }
                },
                mem_limit=self.memory_limit,
                network_disabled=True,
                remove=True,
                detach=True,
            )
            result = container.wait(timeout=self.timeout)
            logs = container.logs().decode("utf-8", errors="ignore")
            return {
                "success": result.get("StatusCode", 1) == 0,
                "stdout": logs,
                "stderr": "" if result.get("StatusCode", 1) == 0 else logs,
                "return_code": result.get("StatusCode", 1),
                "execution_time_ms": 0,
            }
        except Exception as e:
            return {
                "success": False,
                "stdout": "",
                "stderr": f"Docker error: {str(e)}",
                "return_code": -1,
                "execution_time_ms": 0,
            }
        finally:
            os.unlink(temp_path)

Step 5: Code Generator and Debugger

# generator.py
from openai import OpenAI

CODE_SYSTEM = """You are an expert Python programmer. Generate clean, 
working Python code based on user requirements.

Rules:
1. Write complete, runnable code
2. Include necessary imports
3. Add a main() function or script entry point
4. Handle errors gracefully
5. Use type hints where helpful
6. Print results to stdout for verification"""

class CodeGenerator:
    def __init__(self, model: str = "gpt-4-turbo-preview"):
        self.client = OpenAI()
        self.model = model

    def generate(
        self,
        requirement: str,
        context: str = "",
        previous_code: str = "",
        error: str = "",
    ) -> str:
        prompt = f"Requirement: {requirement}"
        if context:
            prompt += f"\n\nContext: {context}"
        if previous_code:
            prompt += f"\n\nPrevious code that failed:\n```python\n{previous_code}\n```\n\nError:\n{error}\n\nFix the code:"
        else:
            prompt += "\n\nGenerate the code:"

        response = self.client.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": CODE_SYSTEM},
                {"role": "user", "content": prompt},
            ],
            temperature=0.0,
        )
        return self._extract_code(response.choices[0].message.content)

    def _extract_code(self, content: str) -> str:
        if "```python" in content:
            parts = content.split("```python")
            if len(parts) > 1:
                code = parts[1].split("```")[0]
                return code.strip()
        return content.strip()

# debugger.py
from openai import OpenAI
from typing import Tuple

DEBUG_SYSTEM = """You are a debugging expert. Analyze the error and provide 
a fix. Return ONLY the corrected Python code, no explanation."""

class CodeDebugger:
    def __init__(self, model: str = "gpt-4-turbo-preview"):
        self.client = OpenAI()
        self.model = model

    def debug(self, code: str, error: str, stdout: str = "") -> str:
        prompt = f"""Code:
```python
{code}

Error: {error}

Stdout (if any): {stdout[:500]}

Fix the code and return ONLY the corrected Python code:"""

response = self.client.chat.completions.create( model=self.model, messages=[ {"role": "system", "content": DEBUG_SYSTEM}, {"role": "user", "content": prompt}, ], temperature=0.0, ) return self._extract_code(response.choices[0].message.content)

def _extract_code(self, content: str) -> str: if "python" in content: parts = content.split("python") if len(parts) > 1: return parts[1].split("```")[0].strip() return content.strip()

Architecture Diagram

### Step 6: Complete Code Agent

```python
# code_agent.py
from generator import CodeGenerator
from debugger import CodeDebugger
from sandbox.local_sandbox import LocalSandbox
from safety import CodeSafetyChecker
from typing import Dict

class CodeAgent:
    def __init__(
        self,
        model: str = "gpt-4-turbo-preview",
        max_retries: int = 3,
        use_docker: bool = False,
    ):
        self.generator = CodeGenerator(model)
        self.debugger = CodeDebugger(model)
        self.sandbox = DockerSandbox() if use_docker else LocalSandbox()
        self.safety_checker = CodeSafetyChecker()
        self.max_retries = max_retries

    def run(self, requirement: str) -> Dict:
        code = self.generator.generate(requirement)
        history = []

        for attempt in range(self.max_retries):
            is_safe, issues = self.safety_checker.check(code)
            if not is_safe:
                return {
                    "success": False,
                    "code": code,
                    "error": f"Safety issues: {', '.join(issues)}",
                    "attempts": attempt + 1,
                    "history": history,
                }

            result = self.sandbox.execute(code)
            history.append({
                "attempt": attempt + 1,
                "code": code,
                "result": result,
            })

            if result["success"]:
                return {
                    "success": True,
                    "code": code,
                    "output": result["stdout"],
                    "attempts": attempt + 1,
                    "history": history,
                }

            code = self.debugger.debug(
                code, result["stderr"], result["stdout"]
            )

        return {
            "success": False,
            "code": code,
            "error": "Max retries exceeded",
            "attempts": self.max_retries,
            "history": history,
        }

Mathematical Foundation

Code Generation Confidence:

Where each parameter means:

  • β€” probability the generated code works correctly
  • β€” natural language description
  • β€” existing code, tests, documentation

Intuition: The model's confidence increases with clearer requirements and more context.

Retry Success Probability:

Where is per-attempt success probability and is max retries.

Intuition: With p=0.7 and k=3 retries, success probability is 97.3%.

Advanced Features

# Test generation
class TestGenerator:
    def __init__(self, model: str = "gpt-4-turbo-preview"):
        from openai import OpenAI
        self.client = OpenAI()
        self.model = model

    def generate_tests(self, code: str, requirement: str) -> str:
        response = self.client.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": "Generate pytest tests for the given code."},
                {"role": "user", "content": f"Code:\n{code}\n\nRequirement: {requirement}\n\nGenerate tests:"},
            ],
            temperature=0.0,
        )
        return response.choices[0].message.content

Testing & Evaluation

import pytest
from code_agent import CodeAgent

@pytest.fixture
def agent():
    return CodeAgent(max_retries=3)

def test_simple_code(agent):
    result = agent.run("Write a function that calculates fibonacci numbers")
    assert result["success"]
    assert "fibonacci" in result["code"].lower() or "def " in result["code"]

def test_safety_check():
    from safety import CodeSafetyChecker
    checker = CodeSafetyChecker()
    safe, issues = checker.check("import math\nprint(math.sqrt(4))")
    assert safe
    safe, issues = checker.check("import os\nos.system('rm -rf /')")
    assert not safe

def test_debugging():
    from debugger import CodeDebugger
    debugger = CodeDebugger()
    fixed = debugger.debug(
        "def add(a, b):\n    return a + b",
        "TypeError: unsupported operand",
    )
    assert "def " in fixed

Performance Metrics

MetricValueNotes
First-try Success65%+With GPT-4
Success within 3 tries95%+With debugging loop
Avg Generation Time3-8sDepends on complexity
Sandbox Execution Time1-30sResource dependent
Safety Check Time<10msAST parsing

Deployment

# main.py
from code_agent import CodeAgent

def main():
    agent = CodeAgent(model="gpt-4-turbo-preview", use_docker=True)
    print("Code Agent Ready. Type 'quit' to exit.\n")

    while True:
        req = input("Describe what you want to build: ").strip()
        if req.lower() == "quit":
            break
        result = agent.run(req)
        print(f"\n{'SUCCESS' if result['success'] else 'FAILED'}")
        print(f"Attempts: {result['attempts']}")
        print(f"\nCode:\n{result['code']}")
        if result["success"]:
            print(f"\nOutput:\n{result['output']}")
        else:
            print(f"\nError: {result['error']}")

if __name__ == "__main__":
    main()

Real-World Use Cases

  • Rapid Prototyping: Generate working code from specifications
  • Data Analysis: Create scripts for data processing tasks
  • Automation: Build scripts for repetitive tasks
  • Education: Learn programming through guided code generation
  • Testing: Generate test cases for existing code

Common Pitfalls & Solutions

PitfallSolution
Infinite loops in generated codeSet execution timeouts
Resource exhaustionLimit memory and CPU in sandbox
Security risksUse Docker containers, disable network
Code quality variesInclude linting in validation
Context lossMaintain conversation history across retries

Summary with Key Takeaways

  • Sandboxed execution is essential for running LLM-generated code safely
  • The generate-execute-debug loop achieves 95%+ success rates
  • Safety checking should happen before execution, not after
  • Docker provides stronger isolation than local subprocess execution
  • Test generation validates that code meets requirements, not just runs

Need Expert AI Agents Help?

Get personalized tutoring, project support, or professional consulting.

Advertisement