feat: enhance local development environment with DNS and improved CLI

2025-08-16 18:42:40 +02:00
parent 296d2ec047
commit 8df1335adc
3 changed files with 346 additions and 100 deletions
--- a/6
+++ b/6
@@ -115,9 +115,9 @@ opencode:

 lint:
 	@echo "🔍 Linting project files..."
-	@markdownlint-cli . || true
-	@nixpkgs-fmt --check . || true
+	@nix develop -c markdownlint-cli . || true
+	@nix develop -c nixpkgs-fmt --check . || true

 lint-fix:
 	@echo "🎨 Formatting Nix files..."
-	@nixpkgs-fmt .
+	@nix develop -c nixpkgs-fmt .
--- a/README.md
+++ b/README.md
@@ -1,4 +1,4 @@
-# 🪐 CODE CRISPIES Workshop Infrastructure
+# 🚀 CODE CRISPIES Workshop Infrastructure

 Single-participant learning environments with local practice and cloud deployment capabilities.

@@ -6,7 +6,7 @@ Single-participant learning environments with local practice and cloud deploymen

 ```bash
 # 1. Start local VM for development/testing
-make local-vm
+make vm-run

 # 2. Build USB drives for participants  
 make build-usb
@@ -21,9 +21,10 @@ make deploy-cloud

 ### Local Practice (USB/VM)
 ```bash
+setup-traefik              # REQUIRED: Setup local proxy first!
 recipes                    # Show available apps
 deploy wordpress           # Deploy locally  
-browser                    # View at wordpress.workshop.local
+browser wordpress          # Open directly in Firefox
 ```

 ### Cloud Deployment
@@ -34,21 +35,24 @@ abra app new wordpress -S --domain=blog.hopper.codecrispi.es
 abra app deploy blog.hopper.codecrispi.es
 ```

-## 🏗️ Architecture
+## 🗃️ Architecture

 **Single Participant Model**: Each environment (USB/VM) is complete and self-contained.

 - **USB Boot**: Bootable NixOS with Docker + abra for hands-on learning
 - **Local VM**: Identical environment for development/testing  
 - **Cloud Servers**: 15 production servers (hopper, curie, lovelace, etc.)
+- **Wildcard DNS**: `*.workshop.local` resolves to `127.0.0.1` via dnsmasq

 ## 💾 USB Environment

 Pre-configured with:
 - Docker Swarm + abra installation
 - SSH client for cloud access
+- Wildcard DNS resolution (dnsmasq)
 - Terminal-first interface (`desktop` command for GUI)
- Helper commands: `recipes`, `deploy`, `connect`, `help`
+- Helper commands: `recipes`, `deploy`, `connect`, `browser`, `help`
+- Tab completion for all commands

 Build and flash:
 ```bash
@@ -69,20 +73,76 @@ make status-cloud  # Check health
 ## 🖥️ Local Development

 ```bash
-make local-vm      # Start VM
-make test-vm       # Verify build
+make vm-run        # Start VM
+make vm-build      # Verify build
 ```

 The VM simulates the USB experience with identical configuration and commands.

-## 📚 Available Commands
+## 📚 Complete Recipe Catalog
+
+Based on Co-op Cloud with quality scoring:
+
+### ⭐ Tier 1 - Production Ready (Score 5)
+- **gitea** - Self-hosted Git service
+- **nextcloud** - Personal cloud storage & collaboration  
+- **mealie** - Recipe manager and meal planner
+
+### 🔧 Tier 2 - Stable (Score 4)
+- **gotosocial** - Lightweight Fediverse server
+- **wordpress** - Website & blog platform
+
+### 🧪 Tier 3 - Community (Score 3)
+- **collabora** - Online office suite
+- **croc** - File transfer tool
+- **custom-php** - Custom PHP applications
+- **dokuwiki** - Simple wiki software
+- **engelsystem** - Event coordination
+- **fab-manager** - FabLab management
+- **ghost** - Professional publishing platform
+- **karrot** - Grassroots initiatives platform
+- **lauti** - Calendar software for events
+- **loomio** - Collaborative decision-making
+- **mattermost** / **mattermost-lts** - Team collaboration
+- **mrbs** - Meeting room booking system
+- **onlyoffice** - Document editing suite
+- **open-inventory** - Inventory management
+- **outline** - Team knowledge base
+- **owncast** - Self-hosted live streaming
+- **rallly** - Group meeting scheduler
+
+### 🌐 Extended Catalog
+- **Content**: hedgedoc, mediawiki, seafile
+- **Communication**: jitsi-meet, matrix-synapse, rocketchat  
+- **Business**: prestashop, invoiceninja, kimai, pretix
+- **Development**: drone, n8n, gitlab, jupyter-lab
+- **Analytics**: plausible, matomo, uptime-kuma, grafana
+- **Media & Social**: peertube, funkwhale, mastodon, pixelfed, jellyfin
+
+## 📚 Enhanced Commands

 **In USB/VM environments**:
- `recipes` - Show Co-op Cloud catalog
- `deploy <app>` - Deploy locally (e.g., `deploy wordpress`)
- `connect <server>` - SSH to cloud server
+- `setup-traefik` - **REQUIRED FIRST**: Setup local DNS proxy
+- `recipes` - Show complete Co-op Cloud catalog
+- `deploy <app>` - Deploy locally with tab completion
+- `browser [app]` - Launch Firefox [to specific app]
+- `connect <server>` - SSH to cloud server with tab completion  
 - `desktop` - Start GUI session
- `browser` - Launch Firefox
+- `help` - Show all commands and debug info
+
+**Examples**:
+```bash
+# Deploy and open WordPress
+deploy wordpress
+browser wordpress          # Opens http://wordpress.workshop.local
+
+# Just open browser
+browser                     # Opens blank page
+
+# Use tab completion
+deploy <TAB>               # Shows all available recipes
+connect <TAB>              # Shows all available servers
+```

 ## 🔧 Prerequisites

@@ -97,3 +157,19 @@ The VM simulates the USB experience with identical configuration and commands.
 make clean         # Local artifacts
 make destroy-cloud # Cloud infrastructure
 ```
+
+## 🔍 Troubleshooting
+
+```bash
+# Check DNS resolution
+dig @127.0.0.1 test.workshop.local
+
+# Check running services  
+docker service ls
+
+# Check DNS service
+systemctl status dnsmasq
+
+# Restart if needed
+sudo systemctl restart dnsmasq
+```
--- a/common.nix
+++ b/common.nix
@@ -8,6 +8,27 @@ let
      makeUsbBootable = true;
    };
  };
+  
+  # Complete Co-op Cloud recipe list (based on your ABRA_RECIPES.md and more)
+  allRecipes = [
+    # Tier 1 - Production Ready (Score 5)
+    "gitea" "mealie" "nextcloud"
+    
+    # Tier 2 - Stable (Score 4)
+    "gotosocial" "wordpress"
+    
+    # Tier 3 - Community (Score 3)
+    "collabora" "croc" "custom-php" "dokuwiki" "engelsystem" "fab-manager"
+    "ghost" "karrot" "lauti" "loomio" "mattermost" "mattermost-lts" "mrbs"
+    "onlyoffice" "open-inventory" "outline" "owncast" "rallly"
+    
+    # Additional recipes from Co-op Cloud catalog
+    "hedgedoc" "mediawiki" "seafile" "jitsi-meet" "matrix-synapse" 
+    "rocketchat" "prestashop" "invoiceninja" "kimai" "pretix"
+    "drone" "n8n" "gitlab" "jupyter-lab" "plausible" "matomo"
+    "uptime-kuma" "grafana" "peertube" "funkwhale" "mastodon"
+    "pixelfed" "jellyfin"
+  ];
 in

 isoConfig // {
@@ -19,13 +40,36 @@ isoConfig // {
    hostName = if isLiveIso then "workshop-live" else "workshop-vm";
  };

+  # Enable dnsmasq for wildcard DNS resolution
+  services.dnsmasq = {
+    enable = true;
+    settings = {
+      # Wildcard: *.workshop.local -> 127.0.0.1
+      address = [
+        "/.workshop.local/127.0.0.1"
+      ];
+      # Don't forward queries for .local domains upstream
+      local = [
+        "/workshop.local/"
+      ];
+      # Listen on all interfaces
+      listen-address = "127.0.0.1";
+      # Don't read /etc/hosts (we want full control)
+      no-hosts = true;
+    };
+  };
+
+  # Configure NetworkManager to use our dnsmasq
+  networking.networkmanager.dns = "dnsmasq";
+  networking.nameservers = [ "127.0.0.1" ];
+
  # Enable Docker for local development
  virtualisation.docker.enable = true;

  services.getty.autologinUser = "workshop";
  users.users.workshop = {
    isNormalUser = true;
-    shell = pkgs.bash;  # Simple bash instead of zsh
+    shell = pkgs.bash;
    extraGroups = [ "networkmanager" "wheel" "docker" ];
    password = "";
  };
@@ -46,22 +90,26 @@ isoConfig // {
    jq
    tree
    nano
+    dnsutils
+    dig  # For DNS debugging
  ];

-  # Auto-install abra on boot
+  # Auto-install abra and setup Docker Swarm
  systemd.services.workshop-abra-setup = {
    wantedBy = [ "multi-user.target" ];
-    after = [ "network-online.target" "docker.service" ];
+    after = [ "network-online.target" "docker.service" "dnsmasq.service" ];
    wants = [ "network-online.target" ];
    script = ''
      export HOME=/home/workshop
      
-      # Wait for network
-      for i in {1..10}; do
-        if ${pkgs.curl}/bin/curl -s --max-time 5 google.com >/dev/null 2>&1; then
+      # Wait for network, Docker, and DNS
+      for i in {1..20}; do
+        if ${pkgs.curl}/bin/curl -s --max-time 5 google.com >/dev/null 2>&1 && \
+           ${pkgs.docker}/bin/docker info >/dev/null 2>&1 && \
+           ${pkgs.dnsutils}/bin/dig @127.0.0.1 test.workshop.local +short | grep -q "127.0.0.1"; then
          break
        fi
-        sleep 3
+        sleep 2
      done
      
      # Install abra for workshop user
@@ -71,11 +119,21 @@ isoConfig // {
        sudo -u workshop ${pkgs.curl}/bin/curl -fsSL https://install.abra.coopcloud.tech | sudo -u workshop ${pkgs.bash}/bin/bash
      fi
      
-      # Initialize local Docker Swarm
-      ${pkgs.docker}/bin/docker swarm init --advertise-addr 127.0.0.1 2>/dev/null || true
+      # Initialize Docker Swarm with retry logic
+      for i in {1..5}; do
+        if ${pkgs.docker}/bin/docker swarm init --advertise-addr 127.0.0.1 2>/dev/null; then
+          break
+        elif ${pkgs.docker}/bin/docker info | grep -q "Swarm: active"; then
+          break
+        fi
+        sleep 2
+      done
      
-      # Add workshop user to docker group
+      # Ensure workshop user is in docker group
      usermod -aG docker workshop
+      
+      # Create Docker network for local development
+      ${pkgs.docker}/bin/docker network create --driver bridge workshop-net 2>/dev/null || true
    '';
    serviceConfig = {
      Type = "oneshot";
@@ -84,52 +142,116 @@ isoConfig // {
    };
  };

-  # Simple bash configuration with custom functions
+  # Enhanced bash configuration with complete recipe support
  programs.bash = {
    interactiveShellInit = ''
      # Workshop welcome and command definitions
-      echo "CODE CRISPIES Workshop Environment"
+      echo "🚀 CODE CRISPIES Workshop Environment"
      echo "Mode: Local Development + Cloud Access"
      echo ""
      echo "🏠 Local Development:"
-      echo "  recipes          - Show available app recipes"
-      echo "  deploy <recipe>  - Deploy app locally (e.g., deploy wordpress)"
-      echo "  setup-traefik    - Setup local Traefik (required first!)"
-      echo "  browser          - Launch Firefox"
-      echo "  desktop          - Start GUI session"
+      echo "  setup-traefik       - Setup local Traefik (REQUIRED FIRST!)"
+      echo "  recipes             - Show available app recipes"
+      echo "  deploy <recipe>     - Deploy app locally (e.g., deploy wordpress)"
+      echo "  browser [recipe]    - Launch Firefox [to specific app]"
+      echo "  desktop             - Start GUI session"
      echo ""
      echo "☁️ Cloud Access:"
      echo "  Available servers:"
      ${builtins.concatStringsSep "\n" (map (name: 
        "echo \"    - ${name}.codecrispi.es\""
      ) cloudServerNames)}
-      echo "  connect <name>   - SSH to cloud server"
+      echo "  connect <name>      - SSH to cloud server"
      echo ""
-      echo "📚 Commands: setup-traefik | recipes | deploy | connect | browser | desktop | help"
+      echo "📚 Commands: setup-traefik | recipes | deploy | browser | connect | desktop | help"

      # Ensure abra is in PATH
      export PATH="$HOME/.local/bin:$PATH"

+      # Complete recipe list for bash completion
+      ALL_RECIPES="${builtins.concatStringsSep " " allRecipes}"
+
+      # Enable tab completion for deploy and browser commands
+      _workshop_completion() {
+        local cur prev opts
+        COMPREPLY=()
+        cur="''${COMP_WORDS[COMP_CWORD]}"
+        prev="''${COMP_WORDS[COMP_CWORD-1]}"
+        
+        case "''${prev}" in
+          deploy|browser)
+            opts="$ALL_RECIPES"
+            COMPREPLY=( $(compgen -W "''${opts}" -- ''${cur}) )
+            return 0
+            ;;
+          connect)
+            opts="${builtins.concatStringsSep " " cloudServerNames}"
+            COMPREPLY=( $(compgen -W "''${opts}" -- ''${cur}) )
+            return 0
+            ;;
+        esac
+      }
+      complete -F _workshop_completion deploy browser connect
+
      setup-traefik() {
        echo "🔧 Setting up local Traefik proxy..."
        
        if ! command -v abra &> /dev/null; then
-          echo "❌ Abra not found. Run 'sudo systemctl restart workshop-abra-setup'"
-          return 1
+          echo "❌ Abra not found. Installing..."
+          sudo systemctl restart workshop-abra-setup
+          sleep 5
+          export PATH="$HOME/.local/bin:$PATH"
        fi

-        abra app new traefik -S --domain=traefik.workshop.local
+        # Test DNS resolution
+        if ! dig @127.0.0.1 test.workshop.local +short | grep -q "127.0.0.1"; then
+          echo "⚠️  DNS not ready, restarting dnsmasq..."
+          sudo systemctl restart dnsmasq
+          sleep 2
+        fi
+
+        # Ensure Docker Swarm is ready
+        if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
+          echo "🔄 Initializing Docker Swarm..."
+          docker swarm init --advertise-addr 127.0.0.1 || true
+        fi
+
+        # Create abra context if not exists
+        if ! abra server ls 2>/dev/null | grep -q "workshop-local"; then
+          echo "📝 Creating local abra context..."
+          abra server add workshop-local docker://localhost --local
+        fi
+
+        echo "🚀 Deploying Traefik..."
+        abra app new traefik -S --domain=traefik.workshop.local --server=workshop-local
        abra app deploy traefik.workshop.local
        
-        echo "✅ Traefik deployed! Dashboard: http://traefik.workshop.local"
-        echo "🚀 Now you can deploy apps with 'deploy <recipe>'"
+        # Wait for Traefik to be ready
+        echo "⏳ Waiting for Traefik to start..."
+        for i in {1..30}; do
+          if curl -s http://traefik.workshop.local >/dev/null 2>&1; then
+            break
+          fi
+          sleep 2
+        done
+        
+        if curl -s http://traefik.workshop.local >/dev/null 2>&1; then
+          echo "✅ Traefik deployed! Dashboard: http://traefik.workshop.local"
+          echo "🚀 Now you can deploy apps with 'deploy <recipe>'"
+          echo "🌐 DNS test: $(dig @127.0.0.1 traefik.workshop.local +short)"
+        else
+          echo "⚠️  Traefik deployed but may still be starting..."
+          echo "🔍 Debug: docker service ls | systemctl status dnsmasq"
+        fi
      }

      deploy() {
        if [ -z "$1" ]; then
          echo "Usage: deploy <recipe>"
          echo "Example: deploy wordpress"
-          echo "Run 'recipes' to see available options"
+          echo "Available recipes: $ALL_RECIPES"
+          echo ""
+          echo "🔍 Use tab completion or run 'recipes' for categorized list"
          return 1
        fi
  
@@ -144,94 +266,142 @@ isoConfig // {
          return 1
        fi

-        abra app new "$recipe" -S --domain="$domain"
+        # Check if Traefik is running
+        if ! curl -s http://traefik.workshop.local >/dev/null 2>&1; then
+          echo "⚠️  Traefik not detected. Running setup first..."
+          setup-traefik
+        fi
+  
+        echo "📦 Creating app: $recipe"
+        abra app new "$recipe" -S --domain="$domain" --server=workshop-local
+        
+        echo "🚀 Deploying app: $domain"
        abra app deploy "$domain"
  
-        echo "✅ Deployed! Access at: http://$domain"
-        echo "🌐 Open browser with: browser"
+        echo "⏳ Waiting for deployment..."
+        for i in {1..60}; do
+          if curl -s http://$domain >/dev/null 2>&1; then
+            echo "✅ Deployed! Access at: http://$domain"
+            echo "🌐 Quick launch: browser $recipe"
+            return 0
+          fi
+          sleep 3
+        done
+        
+        echo "⚠️  Deployment completed but app may still be starting..."
+        echo "🔍 Debug: docker service ls | dig @127.0.0.1 $domain +short"
+        echo "🌐 Try: browser $recipe (in a few moments)"
      }

      connect() {
-        [ -z "$1" ] && { echo "Usage: connect <name>"; return 1; }
-        echo "Connecting to $1.codecrispi.es..."
+        [ -z "$1" ] && { echo "Usage: connect <name>"; echo "Available: ${builtins.concatStringsSep " " cloudServerNames}"; return 1; }
+        echo "🔌 Connecting to $1.codecrispi.es..."
        ssh -o StrictHostKeyChecking=no workshop@$1.codecrispi.es
      }

-      recipes() {
-        echo "Available Co-op Cloud Recipes:"
-        echo ""
-        echo "📝 Content Management:"
-        echo "  wordpress ghost hedgedoc dokuwiki mediawiki"
-        echo ""
-        echo "📁 File & Collaboration:" 
-        echo "  nextcloud seafile collabora onlyoffice"
-        echo ""
-        echo "💬 Communication:"
-        echo "  jitsi-meet matrix-synapse rocketchat mattermost"
-        echo ""
-        echo "🛒 E-commerce & Business:"
-        echo "  prestashop invoiceninja kimai pretix"
-        echo ""
-        echo "⚙️  Development & Tools:"
-        echo "  gitea drone n8n gitlab jupyter-lab"
-        echo ""
-        echo "📊 Analytics & Monitoring:"
-        echo "  plausible matomo uptime-kuma grafana"
-        echo ""
-        echo "🎵 Media & Social:"
-        echo "  peertube funkwhale mastodon pixelfed jellyfin"
-        echo ""
-        echo "🚀 Local Deploy: deploy <recipe>"
-        echo "☁️  Cloud Deploy: connect <server> then use abra commands"
-        echo "📖 Browse all: https://recipes.coopcloud.tech"
-      }
-
      browser() {
-        echo "🌐 Starting Firefox..."
+        local target_url="about:blank"
+        
+        if [ -n "$1" ]; then
+          # Specific app requested
+          target_url="http://$1.workshop.local"
+          echo "🌐 Opening $1 at $target_url"
+        else
+          echo "🌐 Opening Firefox browser"
+        fi
+        
        if [ -n "$DISPLAY" ]; then
-          firefox &
+          firefox "$target_url" &
        else
          echo "❌ No GUI session. Run 'desktop' first"
+          echo "🌐 Target was: $target_url"
        fi
      }

-       desktop() {
-         echo "🖥️  Starting GUI session..."
-         if command -v startx &> /dev/null; then
-           if [ -z "$DISPLAY" ]; then
-             startx &
-             export DISPLAY=:0
-             sleep 3
-             echo "✅ GUI started. Check QEMU window or run 'browser'"
-           else
-             echo "ℹ️  GUI already running"
-           fi
-         else
-           echo "💡 GUI available in QEMU window (Alt+Tab to switch)"
-           echo "🖱️  Click on QEMU graphics window to use desktop"
-         fi
-       }
+      recipes() {
+        echo "📚 Complete Co-op Cloud Recipe Catalog:"
+        echo ""
+        echo "⭐ Tier 1 - Production Ready (Score 5):"
+        echo "  gitea mealie nextcloud"
+        echo ""
+        echo "🔧 Tier 2 - Stable (Score 4):" 
+        echo "  gotosocial wordpress"
+        echo ""
+        echo "🧪 Tier 3 - Community (Score 3):"
+        echo "  collabora croc custom-php dokuwiki engelsystem"
+        echo "  fab-manager ghost karrot lauti loomio mattermost"
+        echo "  mattermost-lts mrbs onlyoffice open-inventory outline"
+        echo "  owncast rallly"
+        echo ""
+        echo "🌐 Extended Catalog:"
+        echo "  Content: hedgedoc mediawiki seafile"
+        echo "  Chat: jitsi-meet matrix-synapse rocketchat"
+        echo "  Business: prestashop invoiceninja kimai pretix"
+        echo "  Dev Tools: drone n8n gitlab jupyter-lab"
+        echo "  Analytics: plausible matomo uptime-kuma grafana"
+        echo "  Media: peertube funkwhale mastodon pixelfed jellyfin"
+        echo ""
+        echo "🚀 Usage:"
+        echo "  deploy <recipe>     - Deploy locally"
+        echo "  browser <recipe>    - Open app in browser"
+        echo "  📖 Full catalog: https://recipes.coopcloud.tech"
+        echo ""
+        echo "💡 Use tab completion: type 'deploy <TAB>' or 'browser <TAB>'"
+      }
+
+      desktop() {
+        echo "🖥️ Starting GUI session..."
+        if command -v startx &> /dev/null; then
+          if [ -z "$DISPLAY" ]; then
+            startx &
+            export DISPLAY=:0
+            sleep 3
+            echo "✅ GUI started. Check QEMU window or run 'browser'"
+          else
+            echo "ℹ️  GUI already running"
+          fi
+        else
+          echo "💡 GUI available in QEMU window (Alt+Tab to switch)"
+          echo "🖱️  Click on QEMU graphics window to use desktop"
+        fi
+      }

      help() {
-        echo "CODE CRISPIES Workshop Commands:"
+        echo "🚀 CODE CRISPIES Workshop Commands:"
        echo ""
        echo "🏠 Local Development:"
-        echo "  setup-traefik   - Setup local Traefik proxy (required first!)"
-        echo "  recipes         - Show all available app recipes"
-        echo "  deploy <recipe> - Deploy app locally (e.g., deploy wordpress)"
-        echo "  browser         - Launch Firefox browser"
-        echo "  desktop         - Start GUI desktop session"
+        echo "  setup-traefik      - Setup local Traefik proxy (REQUIRED FIRST!)"
+        echo "  recipes            - Show all available app recipes"
+        echo "  deploy <recipe>    - Deploy app locally (e.g., deploy wordpress)"
+        echo "  browser [recipe]   - Launch Firefox [to specific app]"
+        echo "  desktop            - Start GUI desktop session"
        echo ""
        echo "☁️ Cloud Access:"
-        echo "  connect <name>  - SSH to cloud server (e.g., connect hopper)"
+        echo "  connect <name>     - SSH to cloud server (e.g., connect hopper)"
        echo ""
        echo "Available servers: ${builtins.concatStringsSep " " cloudServerNames}"
        echo ""
        echo "📚 Learning Flow:"
        echo "  1. First time: setup-traefik"
-        echo "  2. Try local: recipes → deploy wordpress → browser"
+        echo "  2. Try local: recipes → deploy wordpress → browser wordpress"
        echo "  3. Try cloud: connect hopper → same abra commands"
+        echo ""
+        echo "🔍 Debug Commands:"
+        echo "  docker service ls                    - Check running services"
+        echo "  dig @127.0.0.1 app.workshop.local   - Test DNS resolution"
+        echo "  systemctl status dnsmasq             - Check DNS service"
+        echo ""
+        echo "💡 Tab completion available for deploy, browser, connect commands"
      }
+
+      # Welcome DNS test
+      if command -v dig &> /dev/null; then
+        if dig @127.0.0.1 test.workshop.local +short 2>/dev/null | grep -q "127.0.0.1"; then
+          echo "✅ DNS wildcard ready: *.workshop.local → 127.0.0.1"
+        else
+          echo "⚠️  DNS not ready yet, services may be starting..."
+        fi
+      fi
    '';
  };