SmartCane/IMPLEMENTATION_GUIDE.md

SmartCane Code-Improvements Merge Guide (Complete)

Target Repo: https://bitbucket.org/sobitnl/smartcane/src/master/
Source Branch: code-improvements (Timon's experimental area)
Status: Ready for merge (all changes, including optional enhancements)

---

📋 SCOPE: CORE + OPTIONAL FEATURES

🔴 CORE (Required for merge)

• Database migration (client_type column)
• File path changes (merged_final_tif → field_tiles_CI)
• Laravel form/job/mailing updates
• Shell script wrappers (5 files)
• Python package files (2 files)

🟡 OPTIONAL (Post-merge enhancements)

• Country-based project organization (MZ/UG/TZ folders)
• Download scheduling (staggered 00:01 per-project)
• Project search feature
• Harvest prediction setup docs

---

🚀 FULL IMPLEMENTATION CHECKLIST

═══════════════════════════════════════════════════════════════
PHASE 1: CORE MERGE (Required)
═══════════════════════════════════════════════════════════════

[1.1] Database Migration (10 min)
  ✓ Create: database/migrations/YYYY_MM_DD_add_client_type_to_projects_table.php
  ✓ Run: php artisan migrate

[1.2] Laravel Model Changes (20 min)
  ✓ Edit: laravel_app/app/Models/Project.php
    - Add 'client_type' to $fillable
    - Update getMergedTiffList() — path change
    - Update startDownload() — path change
    - Update getTifsAsZip() — path change
    - Add getLatestKpiFile() method

[1.3] Laravel Job Changes (15 min)
  ✓ Edit: laravel_app/app/Jobs/ProjectDownloadTiffJob.php
    - Change path in handleForDate()
  ✓ Edit: laravel_app/app/Jobs/ProjectMosiacGeneratorJob.php
    - Replace command array
    - Improve error handling

[1.4] Laravel Forms (15 min)
  ✓ Edit: laravel_app/app/Livewire/Projects/ProjectManager.php
    - Add client_type to form
  ✓ Edit: Project form Blade template
    - Add <select> for client_type
  ✓ Edit: laravel_app/app/Livewire/Forms/MailingForm.php
    - Auto-attach KPI Excel for cane_supply

[1.5] Shell Script Wrappers (15 min)
  ✓ Create: 10_create_per_field_tiffs.sh
  ✓ Create: 21_convert_ci_rds_to_csv.sh
  ✓ Create: 22_harvest_baseline_prediction.sh
  ✓ Create: 23_convert_harvest_format.sh
  ✓ Create: 31_harvest_imminent_weekly.sh

[1.6] Python Package Files (5 min)
  ✓ Create: python_app/requirements_harvest.txt
  ✓ Create: python_app/environment_pytorch.yml

[1.7] Testing Core (20 min)
  ✓ Run migration ✓ Download test ✓ Mosaic test ✓ Mail test

═══════════════════════════════════════════════════════════════
PHASE 2: ENHANCEMENTS 
═══════════════════════════════════════════════════════════════

[2.1] Country Organization (45 min)
  - Add DB migration (country, country_code columns)
  - Update Project.php fillable
  - Add ProjectManager form fields
  - Add Blade country selector + auto-populate
  - Add country filtering to ProjectList
  - Create "Add New Country" feature

[2.2] Download Scheduling (30 min)
  - Option A: Windows Task Scheduler
  - Option B: Linux cron
  - Option C: Laravel Task Scheduler

[2.3] Project Search (30 min)
  - Add search/filter to ProjectList.php
  - Add Blade input fields

[2.4] Harvest Prediction Setup (20 min)
  - Document conda env setup
  - Document script execution

---

🔴 PHASE 1: CORE MERGE

STEP 1: Create & Run Database Migration

File: database/migrations/2024_02_19_000000_add_client_type_to_projects_table.php

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up(): void
    {
        Schema::table('projects', function (Blueprint $table) {
            $table->enum('client_type', ['agronomic_support', 'cane_supply'])
                ->default('agronomic_support')
                ->after('download_path');
        });
    }

    public function down(): void
    {
        Schema::table('projects', function (Blueprint $table) {
            $table->dropColumn('client_type');
        });
    }
};

Run migration:

php artisan migrate

---

STEP 2: Edit laravel_app/app/Models/Project.php

Edit 2.1: Add to $fillable array

Find and add 'client_type' after 'download_path':

protected $fillable = [
    'name',
    'download_path',
    'client_type',  // ADD THIS
    'mail_template',
    'mail_subject',
    'mail_frequency',
    'mail_day',
    'mail_scheduled',
    'pivot_json_path',
    'span_json_path',
    'harvest_json_path',
    'min_harvest_date',
    'borders',
];

Edit 2.2: Update getMergedTiffList() method (around line 232)

public function getMergedTiffList()
{
    return collect(Storage::files($this->download_path.'/field_tiles_CI'))  // CHANGED
        ->filter(fn($file) => Str::endsWith($file, '.tif'))
        ->sortByDesc(function ($file) {
            $parts = explode('_', str_replace('.tif', '', $file));
            $date = $parts[1];
            return $date;
        })
        ->values();
}

Edit 2.3: Update startDownload() method (around line 265)

public function startDownload(Carbon $date)
{
    $downloadRequest = $this->downloads()->updateOrCreate(
        [
            'project_id' => $this->id,
            'name'       => sprintf('%s.tif', $date->format('Y-m-d')),
        ],
        [
            'path' => sprintf('%s/%s/%s.tif', $this->download_path, 'field_tiles_CI', $date->format('Y-m-d')),  // CHANGED
        ]
    );
    ProjectDownloadTiffJob::dispatch($downloadRequest, $date);
}

Edit 2.4: Update getTifsAsZip() method (around line 489)

public function getTifsAsZip()
{
    return collect(Storage::files($this->download_path . '/field_tiles_CI'))  // CHANGED
        ->filter(fn($file) => Str::endsWith($file, '.tif'))
        ->values();
}

Edit 2.5: Add new method getLatestKpiFile()

Add this method at the end of the Project class:

public function getLatestKpiFile(): ?string
{
    $kpiPath = $this->download_path . '/reports/kpis/';
    
    try {
        $files = Storage::files($kpiPath);
    } catch (\Exception $e) {
        return null;
    }
    
    return collect($files)
        ->filter(fn($f) => Str::endsWith($f, '.xlsx'))
        ->sortByDesc(fn($f) => Storage::lastModified($f))
        ->first();
}

---

STEP 3: Edit laravel_app/app/Jobs/ProjectDownloadTiffJob.php

Around line 73 in handleForDate() method:

Change:

$path = $project->download_path . '/merged_final_tif/' . $filename;

To:

$path = $project->download_path . '/field_tiles_CI/' . $filename;

---

STEP 4: Edit laravel_app/app/Jobs/ProjectMosiacGeneratorJob.php

Lines 50-70, replace the $command array initialization:

OLD:

$command = [
    sprintf('%sbuild_mosaic.sh', $projectFolder),
    sprintf('--end_date=%s', $this->mosaic->end_date->format('Y-m-d')),
    sprintf('--offset=%s', $this->mosaic->offset),
    sprintf('--data_dir=%s', $this->mosaic->project->download_path),
    sprintf('--file_name_tif=%s', basename($this->mosaic->path)),
];

NEW:

$command = [
    sprintf('%s40_mosaic_creation_per_field.sh', $projectFolder),
    sprintf('--project=%s', $project->name),
    sprintf('--end_date=%s', $this->mosaic->end_date->format('Y-m-d')),
    sprintf('--offset=%s', $this->mosaic->offset),
];

Also improve exception handling (around line 65-75):

try {
    $process = ProcessNew::timeout(300)
        ->env(['PATH' => $currentPath.':/usr/local/Cellar/pandoc/3.1.8/bin/pandoc'])
        ->start($command, function (string $type, string $output) use ($project) {
            ProjectLogger::log($project, $output);
            $this->throwIfOutputContainsError($output);
        });
    $results = $process->wait();
    if ($results->successful()) {
        $this->mosaic->setStatusSuccess();
    }
} catch (\RuntimeException|ProcessTimedOutException|ProcessFailedException $e) {
    ProjectLogger::log($project, "MOSAIC JOB ERROR: " . $e->getMessage());
    $this->mosaic->setStatusFailed();
    throw $e;
}

---

STEP 5: Edit laravel_app/app/Livewire/Projects/ProjectManager.php

In createProject() method:

public function createProject()
{
    $projectIdentifier = $this->formData['id'] ?? null;
    Validator::make(
        ['name' => $this->formData['name']],
        ['name' => ['required', Rule::unique('projects')->ignore($projectIdentifier), 'string', 'max:255']]
    )->validate();
    
    $project = Project::create([
        'name' => $this->formData['name'],
        'download_path' => $this->makeValidDirectoryName($this->formData['name']),
        'client_type' => $this->formData['client_type'] ?? 'agronomic_support',  // ADD THIS
    ]);
    return redirect()->route('project.show', [$project->name, 'settings']);
}

In resetFormData() method:

private function resetFormData()
{
    $this->formData = [
        'name' => '',
        'client_type' => 'agronomic_support',  // ADD THIS
        'mail_template' => '',
        'mail_subject' => '',
        'mail_frequency' => '',
        'mail_day' => '',
        // ... rest of fields
    ];
}

---

STEP 6: Edit Project Form Blade Template

Find the project create/edit form and add this field:

<div class="form-group mb-3">
    <label for="client_type" class="form-label">Client Type <span class="text-danger">*</span></label>
    <select wire:model="formData.client_type" id="client_type" class="form-control" required>
        <option value="">-- Select Client Type --</option>
        <option value="agronomic_support">Agronomic Support</option>
        <option value="cane_supply">Cane Supply</option>
    </select>
    @error('formData.client_type') 
        <span class="text-danger small">{{ $message }}</span> 
    @enderror
</div>

---

STEP 7: Edit laravel_app/app/Livewire/Forms/MailingForm.php

In saveAndSendMailing() static method:

public static function saveAndSendMailing($report, $subject, $message, $recipients) {
    if ($report->documentExists()) {
        $mailing = $report->project->mailings()->create([
            'subject'   => $subject,
            'message'   => $message,
            'report_id' => $report->id,
        ]);

        // Attach main report
        $mailing->attachments()->create([
            'name' => $report->name,
            'path' => $report->path,
        ]);
        
        // Auto-attach KPI Excel for cane_supply projects
        if ($report->project->client_type === 'cane_supply') {
            $kpiFile = $report->project->getLatestKpiFile();
            if ($kpiFile) {
                $mailing->attachments()->create([
                    'name' => 'KPI Summary',
                    'path' => $kpiFile,
                ]);
            }
        }

        $mailing->recipients()->createMany($recipients);
        Mail::to($mailing->recipients()->pluck('email')->toArray())
            ->send(new \App\Mail\ReportMailer($mailing, $report));
    } else {
        self::sendReportNotFoundNotificationToAdmin($report);
    }
}

---

STEP 8: CREATE 5 Shell Script Wrappers

File: 10_create_per_field_tiffs.sh

#!/bin/bash
# Wrapper for R script 10: Create per-field TIFFs
# Usage: ./10_create_per_field_tiffs.sh --project=angata

set -e

PROJECT=""

while [[ $# -gt 0 ]]; do
    case $1 in
        --project=*) PROJECT="${1#*=}" ;;
        --*) ;; # Ignore other args
    esac
    shift
done

[ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; }

cd "$(dirname "$0")/r_app"
Rscript -e "PROJECT='$PROJECT'; source('parameters_project.R'); source('10_create_per_field_tiffs.R')"

File: 21_convert_ci_rds_to_csv.sh

#!/bin/bash
# Wrapper for R script 21: Convert CI RDS to CSV
# Usage: ./21_convert_ci_rds_to_csv.sh --project=angata

set -e

PROJECT=""

while [[ $# -gt 0 ]]; do
    case $1 in
        --project=*) PROJECT="${1#*=}" ;;
        --*) ;;
    esac
    shift
done

[ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; }

cd "$(dirname "$0")/r_app"
Rscript -e "PROJECT='$PROJECT'; source('parameters_project.R'); source('21_convert_ci_rds_to_csv.R')"

File: 22_harvest_baseline_prediction.sh

#!/bin/bash
# Wrapper for Python script 22: Harvest baseline prediction
# Usage: ./22_harvest_baseline_prediction.sh --project=angata

set -e

PROJECT=""

while [[ $# -gt 0 ]]; do
    case $1 in
        --project=*) PROJECT="${1#*=}" ;;
        --*) ;;
    esac
    shift
done

[ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; }

cd "$(dirname "$0")/python_app"

if command -v conda &> /dev/null; then
    conda run -n pytorch_gpu python 22_harvest_baseline_prediction.py "$PROJECT" 2>&1 || \
    conda run -n pytorch_cpu python 22_harvest_baseline_prediction.py "$PROJECT" 2>&1
else
    python 22_harvest_baseline_prediction.py "$PROJECT"
fi

File: 23_convert_harvest_format.sh

#!/bin/bash
# Wrapper for Python script 23: Convert harvest format
# Usage: ./23_convert_harvest_format.sh --project=angata

set -e

PROJECT=""

while [[ $# -gt 0 ]]; do
    case $1 in
        --project=*) PROJECT="${1#*=}" ;;
        --*) ;;
    esac
    shift
done

[ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; }

cd "$(dirname "$0")/python_app"

if command -v conda &> /dev/null; then
    conda run -n pytorch_gpu python 23_convert_harvest_format.py "$PROJECT" 2>&1 || \
    conda run -n pytorch_cpu python 23_convert_harvest_format.py "$PROJECT" 2>&1
else
    python 23_convert_harvest_format.py "$PROJECT"
fi

File: 31_harvest_imminent_weekly.sh

#!/bin/bash
# Wrapper for Python script 31: Harvest imminent weekly
# Usage: ./31_harvest_imminent_weekly.sh --project=angata

set -e

PROJECT=""

while [[ $# -gt 0 ]]; do
    case $1 in
        --project=*) PROJECT="${1#*=}" ;;
        --*) ;;
    esac
    shift
done

[ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; }

cd "$(dirname "$0")/python_app"

if command -v conda &> /dev/null; then
    conda run -n pytorch_gpu python 31_harvest_imminent_weekly.py "$PROJECT" 2>&1 || \
    conda run -n pytorch_cpu python 31_harvest_imminent_weekly.py "$PROJECT" 2>&1
else
    python 31_harvest_imminent_weekly.py "$PROJECT"
fi

---

STEP 9: CREATE 2 Python Package Files

File: python_app/requirements_harvest.txt

torch>=2.0.0
pandas>=1.5.0
numpy>=1.23.0
scikit-learn>=1.3.0
GDAL>=3.7.0
sentinelhub>=3.9.0
shapely>=2.0.0
pyproj>=3.4.0

File: python_app/environment_pytorch.yml

name: pytorch_gpu
channels:
  - pytorch
  - nvidia
  - conda-forge
dependencies:
  - python=3.10
  - pytorch::pytorch
  - pytorch::torchvision
  - pytorch::torchaudio
  - pytorch::pytorch-cuda=11.8
  - gdal>=3.7.0
  - pip
  - pip:
    - sentinelhub>=3.9.0
    - shapely>=2.0.0
    - pyproj>=3.4.0

---

STEP 10: CORE TESTING CHECKLIST

# 1. Migration
php artisan migrate
# ✅ Expected: No errors, client_type column added

# 2. Download test
# Go to Laravel UI → Create project with client_type=agronomic_support
# → Download Manager → Add image → Download
# Expected: File in laravel_app/storage/app/{project}/field_tiles_CI/

# 3. Mosaic test
# Go to Mosaic Manager → Create mosaic
# Check logs: grep "Unknown option" laravel.log
# Expected: No --data_dir errors, mosaic created

# 4. Mail test
# Create project with client_type=cane_supply
# Generate & send report
# Expected: Email has 2 attachments (report + KPI Excel)

# 5. Shell wrapper test
./10_create_per_field_tiffs.sh --project=angata
# Expected: R script executes without error

✅ CORE MERGE COMPLETE

---

🟡 PHASE 2: ENHANCEMENTS (Post-Merge)

OPTIONAL 1: Country-Based Organization

Why: Organize projects by geographic location (MZ/UG/TZ folders)

Create Migration: database/migrations/YYYY_MM_DD_add_country_to_projects_table.php

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
use Illuminate\Support\Facades\DB;

return new class extends Migration
{
    public function up(): void
    {
        Schema::table('projects', function (Blueprint $table) {
            $table->string('country')->default('Mozambique')->after('name');
            $table->string('country_code', 2)->default('MZ')->after('country');
        });
        
        // Update existing projects
        DB::table('projects')->where('name', 'angata')->update(['country' => 'Mozambique', 'country_code' => 'MZ']);
        DB::table('projects')->where('name', 'aura')->update(['country' => 'Mozambique', 'country_code' => 'MZ']);
        DB::table('projects')->where('name', 'chemba')->update(['country' => 'Mozambique', 'country_code' => 'MZ']);
        DB::table('projects')->where('name', 'xinavane')->update(['country' => 'Tanzania', 'country_code' => 'TZ']);
        DB::table('projects')->where('name', 'esa')->update(['country' => 'Kenya', 'country_code' => 'KQ']);
        DB::table('projects')->where('name', 'simba')->update(['country' => 'Uganda', 'country_code' => 'UG']);
        DB::table('projects')->where('name', 'john')->update(['country' => 'Uganda', 'country_code' => 'UG']);
        DB::table('projects')->where('name', 'huss')->update(['country' => 'Tanzania', 'country_code' => 'TZ']);
    }

    public function down(): void
    {
        Schema::table('projects', function (Blueprint $table) {
            $table->dropColumn(['country', 'country_code']);
        });
    }
};

Update Project.php $fillable:

protected $fillable = [
    'name',
    'country',
    'country_code',
    'download_path',
    'client_type',
    // ... rest
];

Update ProjectManager.php:

public array $countries = [
    'MZ' => 'Mozambique',
    'TZ' => 'Tanzania',
    'UG' => 'Uganda',
    'KQ' => 'Kenya',
    'SA' => 'South Africa',
    'ZW' => 'Zimbabwe',
    'BR' => 'Brazil',
    'MX' => 'Mexico',
    'IN' => 'India',
];

public function createProject()
{
    $projectIdentifier = $this->formData['id'] ?? null;
    Validator::make(
        ['name' => $this->formData['name']],
        ['name' => ['required', Rule::unique('projects')->ignore($projectIdentifier)]]
    )->validate();
    
    $projectPath = $this->formData['country_code'] . '/' . $this->formData['name'];
    Storage::makeDirectory($projectPath, recursive: true);
    
    $project = Project::create([
        'name' => $this->formData['name'],
        'country' => $this->formData['country'],
        'country_code' => $this->formData['country_code'],
        'download_path' => $projectPath,
        'client_type' => $this->formData['client_type'] ?? 'agronomic_support',
    ]);
    return redirect()->route('project.show', [$project->name, 'settings']);
}

Add to Blade template:

<div class="form-group mb-3">
    <label for="country" class="form-label">Country</label>
    <select wire:model="formData.country" id="country" class="form-control" required>
        @foreach($countries as $code => $name)
            <option value="{{ $name }}" @selected($formData['country'] === $name)>
                {{ $name }} ({{ $code }})
            </option>
        @endforeach
    </select>
</div>

<div class="form-group mb-3">
    <label for="country_code" class="form-label">Country Code (Auto-populated)</label>
    <input type="text" wire:model="formData.country_code" id="country_code" readonly class="form-control" />
</div>

---

OPTIONAL 2: Download Scheduling

Why: Avoid API rate limits by staggering downloads per project at 00:01

Option A: Linux Cron (If server is Linux)

Add to /etc/cron.d/smartcane_downloads:

# Stagger downloads by 10 minutes per project
1 0 * * * /usr/bin/python /home/user/smartcane/python_app/00_download_8band_pu_optimized.py angata 2>&1 | logger
15 0 * * * /usr/bin/python /home/user/smartcane/python_app/00_download_8band_pu_optimized.py chemba 2>&1 | logger
25 0 * * * /usr/bin/python /home/user/smartcane/python_app/00_download_8band_pu_optimized.py xinavane 2>&1 | logger
35 0 * * * /usr/bin/python /home/user/smartcane/python_app/00_download_8band_pu_optimized.py esa 2>&1 | logger
45 0 * * * /usr/bin/python /home/user/smartcane/python_app/00_download_8band_pu_optimized.py simba 2>&1 | logger
0 1 * * * /usr/bin/python /home/user/smartcane/python_app/00_download_8band_pu_optimized.py aura 2>&1 | logger

Option B: Windows Task Scheduler (If server is Windows)

# Create task for each project
$taskName = "SmartCane-Download-angata"
$action = New-ScheduledTaskAction -Execute "powershell.exe" -Argument "-NoProfile -WindowStyle Hidden -Command python C:\smartcane\python_app\00_download_8band_pu_optimized.py angata"
$trigger = New-ScheduledTaskTrigger -Daily -At 00:01
Register-ScheduledTask -TaskName $taskName -Action $action -Trigger $trigger -RunLevel Highest

Option C: Laravel Task Scheduler

Add to laravel_app/app/Console/Kernel.php:

protected function schedule(Schedule $schedule)
{
    $schedule->exec('python python_app/00_download_8band_pu_optimized.py angata')
        ->dailyAt('00:01');
    $schedule->exec('python python_app/00_download_8band_pu_optimized.py chemba')
        ->dailyAt('00:15');
    $schedule->exec('python python_app/00_download_8band_pu_optimized.py xinavane')
        ->dailyAt('00:25');
    $schedule->exec('python python_app/00_download_8band_pu_optimized.py esa')
        ->dailyAt('00:35');
    $schedule->exec('python python_app/00_download_8band_pu_optimized.py simba')
        ->dailyAt('00:45');
    $schedule->exec('python python_app/00_download_8band_pu_optimized.py aura')
        ->dailyAt('01:00');
}

---

OPTIONAL 3: Project Search Feature

Why: Find projects quickly if there are many

Add to laravel_app/app/Livewire/Projects/ProjectList.php:

public string $searchQuery = '';

public function getProjectsProperty()
{
    $query = Project::query();
    
    if (!empty($this->searchQuery)) {
        $query->where('name', 'like', '%' . $this->searchQuery . '%')
              ->orWhere('download_path', 'like', '%' . $this->searchQuery . '%');
    }
    
    return $query->orderBy('name')->paginate(15);
}

Add to Blade template:

<div class="search-bar mb-4">
    <input type="text" 
           wire:model.live="searchQuery" 
           placeholder="Search projects..." 
           class="form-control" />
</div>

<div class="projects-list">
    @forelse($this->projects as $project)
        <div class="project-card">
            <h3>{{ $project->name }}</h3>
            <p>{{ $project->download_path }}</p>
            <span class="badge badge-info">{{ $project->client_type }}</span>
        </div>
    @empty
        <p>No projects found</p>
    @endforelse
</div>

{{ $this->projects->links() }}

---

OPTIONAL 4: Harvest Date Prediction Setup

Why: Enable harvest Date forecasting for cane_supply projects

Create conda environment:

conda env create -f python_app/environment_pytorch.yml

# Activate
conda activate pytorch_gpu

# Or CPU-only if no GPU
conda create -n pytorch_cpu python=3.10 pytorch::pytorch torchvision torchaudio -c pytorch
conda activate pytorch_cpu

Run baseline prediction (once):

python python_app/22_harvest_baseline_prediction.py angata
python python_app/23_convert_harvest_format.py angata

Schedule weekly prediction:

# Add to cron (Linux)
0 23 * * 0 conda run -n pytorch_gpu python /home/user/smartcane/python_app/31_harvest_imminent_weekly.py angata 2>&1 | logger

# Or Task Scheduler (Windows)
# Similar to download scheduling above, but Sunday 23:00

---

📊 Summary: What Gets Changed

Category	Files Modified	Changes Required
Database	migrations/	1 file: add client_type column
Models	Project.php	5 edits: fillable, 3 methods, 1 new method
Jobs	2 files	ProjectDownloadTiffJob (1 line), ProjectMosiacGeneratorJob (full array)
Forms	3 files	ProjectManager.php, Blade template, MailingForm.php
Scripts	5 files created	Shell wrappers (R/Python)
Python	2 files created	requirements_harvest.txt, environment_pytorch.yml

TOTAL: 6 files created + 6 files modified + 1 template modified = 13 changes

---

✅ FINAL VERIFICATION

After ALL changes (core + optionals), test:

# 1. Migration worked
php artisan migrate

# 2. Download saves to correct path
# → Download image → check laravel_app/storage/app/{project}/field_tiles_CI/

# 3. Mosaic runs without errors
# → Create mosaic → check logs for no --data_dir errors

# 4. Mail has 2 attachments for cane_supply
# → Send report for cane_supply project → verify report + KPI Excel

# 5. Shell wrappers work
./10_create_per_field_tiffs.sh --project=angata
# → Should execute R script successfully

# 6. Search works (if implemented)
# → Search for project by name on Projects page

# 7. Country filter works (if implemented)
# → Filter projects by country code

---

🌍 POST-MERGE: Data Recreation Strategy

After merge is live, existing projects need new directory structure.

Option A: Delete & Redownload (Small projects)

Projects: aura, chemba, xinavane, esa, simba

1. Backup project folder (optional)
2. Delete project from Laravel UI
3. Recreate with new client_type selector
4. Redownload 2-3 years of data (~50-150 GB per project)
5. Run pipeline normally

Option B: Preserve merged_tif (Large projects)

Projects: angata

1. Backup merged_tif/ folder externally
2. Delete all other folders in project
3. Keep only: merged_tif/
4. Run Scripts 10-80 on existing data
   → Regenerates field_tiles_CI/, reports/, etc.
5. No need to redownload

---

🔴 CORE vs OPTIONAL Quick List

MUST DO (for merge):

• ✅ Database migration
• ✅ Project.php edits (4 path changes + 1 new method)
• ✅ Job edits (ProjectDownloadTiffJob, ProjectMosiacGeneratorJob)
• ✅ Form edits (ProjectManager, MailingForm, Blade template)
• ✅ 5 shell wrappers
• ✅ 2 Python files

NICE-TO-HAVE (post-merge):

• 🟡 Country organization (adds ~45 min)
• 🟡 Download scheduling (adds ~30 min)
• 🟡 Project search (adds ~30 min)
• 🟡 Harvest prediction setup (adds ~20 min)

---

Ready to implement everything? All code is copy-paste ready above.