diff --git a/IMPLEMENTATION_GUIDE.md b/CODE_MERGE_GUIDE_LARAVEL.md similarity index 52% rename from IMPLEMENTATION_GUIDE.md rename to CODE_MERGE_GUIDE_LARAVEL.md index f07a9a8..83a440d 100644 --- a/IMPLEMENTATION_GUIDE.md +++ b/CODE_MERGE_GUIDE_LARAVEL.md @@ -1,111 +1,81 @@ -# SmartCane Code-Improvements Merge Guide (Complete) +# SmartCane Code Merge: Implementation Handoff for Developer + +**For**: Martins moltbot + +**What**: Complete implementation guide for merging `code-improvements` branch into `master` **Target Repo**: https://bitbucket.org/sobitnl/smartcane/src/master/ -**Source Branch**: `code-improvements` (Timon's experimental area) -**Status**: Ready for merge (all changes, including optional enhancements) + +**Date**: February 24, 2026 --- -## 📋 SCOPE: CORE + OPTIONAL FEATURES +New folder structure, new scripts (R and Python) so new .sh and cron, new buttons (client type and area unit). Add kpi excel to email (in addition to the word file). New packages to be installed. nice to have - group projects by country. -### 🔴 CORE (Required for merge) -- Database migration (client_type column) -- File path changes (merged_final_tif → field_tiles_CI) -- Laravel form/job/mailing updates -- Shell script wrappers (5 files) -- Python package files (2 files) +## 📋 TASKS (AI generated, ik weet niet of ze ergens op slaan...) -### 🟡 OPTIONAL (Post-merge enhancements) -- Country-based project organization (MZ/UG/TZ folders) -- Download scheduling (staggered 00:01 per-project) -- Project search feature -- Harvest prediction setup docs +This handoff includes everything you need to do before handing over to production: + +1. Database migration +2. Update Project model +3. Update jobs (2 files) +4. Update ProjectManager form +5. Add form fields to Blade template +6. Update MailingForm +7. Create 5 shell script wrappers +8. Setup Linux cron jobs +9. Run verification tests +10. Country-based project organization (MZ/UG/TZ) +11. Download scheduling (auto-stagger per project) +12. Project search feature +13. Harvest prediction setup +14. Area unit selection UI (hectare/acre) --- -## 🚀 FULL IMPLEMENTATION CHECKLIST +## 📋 WHAT EACH STEP MEANS +### Step 1-6: Laravel Changes +- Steps 1-3: Database + models + jobs (backend logic) +- Steps 4-6: Forms + email logic (UI + user-facing) + +### Step 7: Shell Script Wrappers +Create 5 bash shell scripts that wrap R and Python scripts. These let Laravel jobs call R/Python scripts easily. + +**Example**: When Laravel needs to create field TIFFs, it runs: +```bash +./10_create_per_field_tiffs.sh --project=angata --area_unit=hectare ``` -═══════════════════════════════════════════════════════════════ -PHASE 1: CORE MERGE (Required) -═══════════════════════════════════════════════════════════════ +This script then exports the area unit as an environment variable and calls the R script. -[1.1] Database Migration (10 min) - ✓ Create: database/migrations/YYYY_MM_DD_add_client_type_to_projects_table.php - ✓ Run: php artisan migrate +### Step 8: Setup Cron +Add scheduled job to Linux to run `/10_planet_download.sh` every day at staggered times (00:01, 00:11, etc.) for each project. -[1.2] Laravel Model Changes (20 min) - ✓ Edit: laravel_app/app/Models/Project.php - - Add 'client_type' to $fillable - - Update getMergedTiffList() — path change - - Update startDownload() — path change - - Update getTifsAsZip() — path change - - Add getLatestKpiFile() method - -[1.3] Laravel Job Changes (15 min) - ✓ Edit: laravel_app/app/Jobs/ProjectDownloadTiffJob.php - - Change path in handleForDate() - ✓ Edit: laravel_app/app/Jobs/ProjectMosiacGeneratorJob.php - - Replace command array - - Improve error handling - -[1.4] Laravel Forms (15 min) - ✓ Edit: laravel_app/app/Livewire/Projects/ProjectManager.php - - Add client_type to form - ✓ Edit: Project form Blade template - - Add @error('formData.client_type') - {{ $message }} + {{ $message }} + @enderror + + + +
+ +
+ + + + + +
+ @error('formData.preferred_area_unit') + {{ $message }} @enderror
``` --- -### STEP 7: Edit `laravel_app/app/Livewire/Forms/MailingForm.php` +### STEP 6: Update Mailing Form -**In `saveAndSendMailing()` static method:** +**File**: `laravel_app/app/Livewire/Forms/MailingForm.php` + +Find the `saveAndSendMailing()` static method and update it to auto-attach KPI Excel for cane_supply clients: ```php -public static function saveAndSendMailing($report, $subject, $message, $recipients) { +public static function saveAndSendMailing($report, $subject, $message, $recipients) +{ if ($report->documentExists()) { $mailing = $report->project->mailings()->create([ 'subject' => $subject, @@ -383,7 +386,7 @@ public static function saveAndSendMailing($report, $subject, $message, $recipien $kpiFile = $report->project->getLatestKpiFile(); if ($kpiFile) { $mailing->attachments()->create([ - 'name' => 'KPI Summary', + 'name' => 'KPI Summary - ' . date('Y-m-d'), 'path' => $kpiFile, ]); } @@ -400,23 +403,159 @@ public static function saveAndSendMailing($report, $subject, $message, $recipien --- -### STEP 8: CREATE 5 Shell Script Wrappers +### STEP 6.5: Understanding Script-Client Type Mapping -**File: `10_create_per_field_tiffs.sh`** +**Critical for implementation**: Different client types require different scripts. This section clarifies which scripts run for which client type. + +#### Script Dependencies by Client Type + +**All Projects** (Both agronomic_support and cane_supply): +``` +10_create_per_field_tiffs.sh ← Always run +21_convert_ci_rds_to_csv.sh ← Always run +``` + +**cane_supply ONLY** (Projects with harvest prediction): +``` +22_harvest_baseline_prediction.sh ← Cane supply only +23_convert_harvest_format.sh ← Cane supply only +31_harvest_imminent_weekly.sh ← Cane supply only (weekly) +``` + +**agronomic_support ONLY** (Default projects, farm analysis): +``` +[These 5 scripts are NOT called] +``` + +#### Reference Table + +| Script | Purpose | agronomic_support | cane_supply | +|--------|---------|-------------------|------------| +| 10_create_per_field_tiffs.sh | Extract per-field CI from 4-band imagery | ✅ Always | ✅ Always | +| 21_convert_ci_rds_to_csv.sh | Convert CI data to CSV (for reports) | ✅ Always | ✅ Always | +| 22_harvest_baseline_prediction.sh | Baseline harvest date ML model | ❌ Skip | ✅ Once at setup | +| 23_convert_harvest_format.sh | Convert harvest dates to reporting format | ❌ Skip | ✅ Once at setup | +| 31_harvest_imminent_weekly.sh | Weekly harvest imminence detection | ❌ Skip | ✅ Weekly schedule | + +#### How Dispatch Works + +When a Laravel job needs to run scripts, it should check the project's `client_type`: + +```php +// Example: In a ProjectPipelineJob or similar dispatcher + +public function handle() +{ + $project = $this->project; + + // ALWAYS run for both client types + // (Script 10: Create per-field TIFFs) + // (Script 21: Convert CI to CSV) + + // CONDITIONALLY run only for cane_supply + if ($project->client_type === 'cane_supply') { + // Script 22: Harvest baseline + // Script 23: Harvest format conversion + // Script 31: Weekly harvest imminent (scheduled separately) + } +} +``` + +#### Implementation Locations + +**Where to check client_type**: +1. **Job Dispatchers** (if creating automated job queues) + - File: Any job class that orchestrates multiple scripts + - Check: `if ($project->client_type === 'cane_supply')` + +2. **Manual UI** (if running scripts from Laravel UI) + - File: ProjectManager Blade template or controller + - Show: Only harvest buttons for cane_supply projects + +3. **Scheduled Tasks** (for weekly/baseline runs) + - File: `/etc/cron.d/smartcane_downloads` or Laravel Kernel.php + - Condition: Only queue harvest scripts for cane_supply projects + +#### Example: Conditional Job Dispatch + +If you're creating a job to run the full pipeline, use this pattern: + +```php +runScript('10_create_per_field_tiffs.sh'); + $this->runScript('21_convert_ci_rds_to_csv.sh'); + + // Scripts that run ONLY for cane_supply + if ($this->project->client_type === 'cane_supply') { + $this->runScript('22_harvest_baseline_prediction.sh'); // Once at setup + $this->runScript('23_convert_harvest_format.sh'); // Once at setup + // 31_harvest_imminent_weekly.sh runs separately on schedule + } + } + + private function runScript(string $scriptName) + { + $command = "bash {$scriptName} --project={$this->project->name} --area_unit={$this->project->preferred_area_unit}"; + \Illuminate\Support\Facades\Process::run($command); + } +} +``` + +#### Why This Matters + +- **Cost**: Harvest scripts use high-impact ML models. Running them for agronomic_support projects wastes resources. +- **Clarity**: Future developers need to understand the business logic (harvest prediction = cane_supply only). +- **Maintenance**: If new scripts are added (e.g., SAR analysis), this mapping gets updated in one place. + +--- + +### STEP 7: Create Shell Script Wrappers + +**What**: Create 5 bash shell scripts in the project root +**Why**: These allow Laravel jobs to call R and Python scripts with proper environment variables +**Note**: Make each executable with `chmod +x` after creation + +**When to call each script** (see Step 6.5 above): +- Scripts 10, 21: Always (for all projects) +- Scripts 22, 23, 31: Only if `client_type === 'cane_supply'` + +Create these 5 shell scripts in the project root: + +--- + +#### Script 1: `10_create_per_field_tiffs.sh` ```bash #!/bin/bash -# Wrapper for R script 10: Create per-field TIFFs +# Create per-field TIFFs from merged 4-band imagery # Usage: ./10_create_per_field_tiffs.sh --project=angata set -e PROJECT="" +AREA_UNIT="hectare" # Default while [[ $# -gt 0 ]]; do case $1 in --project=*) PROJECT="${1#*=}" ;; - --*) ;; # Ignore other args + --area_unit=*) AREA_UNIT="${1#*=}" ;; + --*) ;; # Ignore other args esac shift done @@ -424,23 +563,28 @@ done [ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; } cd "$(dirname "$0")/r_app" +export AREA_UNIT="$AREA_UNIT" Rscript -e "PROJECT='$PROJECT'; source('parameters_project.R'); source('10_create_per_field_tiffs.R')" ``` -**File: `21_convert_ci_rds_to_csv.sh`** +--- + +#### Script 2: `21_convert_ci_rds_to_csv.sh` ```bash #!/bin/bash -# Wrapper for R script 21: Convert CI RDS to CSV +# Convert CI RDS file to CSV format # Usage: ./21_convert_ci_rds_to_csv.sh --project=angata set -e PROJECT="" +AREA_UNIT="hectare" while [[ $# -gt 0 ]]; do case $1 in --project=*) PROJECT="${1#*=}" ;; + --area_unit=*) AREA_UNIT="${1#*=}" ;; --*) ;; esac shift @@ -449,23 +593,28 @@ done [ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; } cd "$(dirname "$0")/r_app" +export AREA_UNIT="$AREA_UNIT" Rscript -e "PROJECT='$PROJECT'; source('parameters_project.R'); source('21_convert_ci_rds_to_csv.R')" ``` -**File: `22_harvest_baseline_prediction.sh`** +--- + +#### Script 3: `22_harvest_baseline_prediction.sh` ```bash #!/bin/bash -# Wrapper for Python script 22: Harvest baseline prediction +# Baseline harvest date prediction # Usage: ./22_harvest_baseline_prediction.sh --project=angata set -e PROJECT="" +AREA_UNIT="hectare" while [[ $# -gt 0 ]]; do case $1 in --project=*) PROJECT="${1#*=}" ;; + --area_unit=*) AREA_UNIT="${1#*=}" ;; --*) ;; esac shift @@ -474,6 +623,7 @@ done [ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; } cd "$(dirname "$0")/python_app" +export AREA_UNIT="$AREA_UNIT" if command -v conda &> /dev/null; then conda run -n pytorch_gpu python 22_harvest_baseline_prediction.py "$PROJECT" 2>&1 || \ @@ -483,20 +633,24 @@ else fi ``` -**File: `23_convert_harvest_format.sh`** +--- + +#### Script 4: `23_convert_harvest_format.sh` ```bash #!/bin/bash -# Wrapper for Python script 23: Convert harvest format +# Convert harvest date format # Usage: ./23_convert_harvest_format.sh --project=angata set -e PROJECT="" +AREA_UNIT="hectare" while [[ $# -gt 0 ]]; do case $1 in --project=*) PROJECT="${1#*=}" ;; + --area_unit=*) AREA_UNIT="${1#*=}" ;; --*) ;; esac shift @@ -505,6 +659,7 @@ done [ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; } cd "$(dirname "$0")/python_app" +export AREA_UNIT="$AREA_UNIT" if command -v conda &> /dev/null; then conda run -n pytorch_gpu python 23_convert_harvest_format.py "$PROJECT" 2>&1 || \ @@ -514,20 +669,24 @@ else fi ``` -**File: `31_harvest_imminent_weekly.sh`** +--- + +#### Script 5: `31_harvest_imminent_weekly.sh` ```bash #!/bin/bash -# Wrapper for Python script 31: Harvest imminent weekly +# Weekly harvest prediction (imminent detection) # Usage: ./31_harvest_imminent_weekly.sh --project=angata set -e PROJECT="" +AREA_UNIT="hectare" while [[ $# -gt 0 ]]; do case $1 in --project=*) PROJECT="${1#*=}" ;; + --area_unit=*) AREA_UNIT="${1#*=}" ;; --*) ;; esac shift @@ -536,6 +695,7 @@ done [ -z "$PROJECT" ] && { echo "ERROR: --project required"; exit 1; } cd "$(dirname "$0")/python_app" +export AREA_UNIT="$AREA_UNIT" if command -v conda &> /dev/null; then conda run -n pytorch_gpu python 31_harvest_imminent_weekly.py "$PROJECT" 2>&1 || \ @@ -547,79 +707,147 @@ fi --- -### STEP 9: CREATE 2 Python Package Files - -**File: `python_app/requirements_harvest.txt`** - -``` -torch>=2.0.0 -pandas>=1.5.0 -numpy>=1.23.0 -scikit-learn>=1.3.0 -GDAL>=3.7.0 -sentinelhub>=3.9.0 -shapely>=2.0.0 -pyproj>=3.4.0 -``` - -**File: `python_app/environment_pytorch.yml`** - -```yaml -name: pytorch_gpu -channels: - - pytorch - - nvidia - - conda-forge -dependencies: - - python=3.10 - - pytorch::pytorch - - pytorch::torchvision - - pytorch::torchaudio - - pytorch::pytorch-cuda=11.8 - - gdal>=3.7.0 - - pip - - pip: - - sentinelhub>=3.9.0 - - shapely>=2.0.0 - - pyproj>=3.4.0 -``` - ---- - -### STEP 10: CORE TESTING CHECKLIST +### STEP 8: Make Scripts Executable ```bash -# 1. Migration -php artisan migrate -# ✅ Expected: No errors, client_type column added - -# 2. Download test -# Go to Laravel UI → Create project with client_type=agronomic_support -# → Download Manager → Add image → Download -# Expected: File in laravel_app/storage/app/{project}/field_tiles_CI/ - -# 3. Mosaic test -# Go to Mosaic Manager → Create mosaic -# Check logs: grep "Unknown option" laravel.log -# Expected: No --data_dir errors, mosaic created - -# 4. Mail test -# Create project with client_type=cane_supply -# Generate & send report -# Expected: Email has 2 attachments (report + KPI Excel) - -# 5. Shell wrapper test -./10_create_per_field_tiffs.sh --project=angata -# Expected: R script executes without error +chmod +x 10_create_per_field_tiffs.sh +chmod +x 21_convert_ci_rds_to_csv.sh +chmod +x 22_harvest_baseline_prediction.sh +chmod +x 23_convert_harvest_format.sh +chmod +x 31_harvest_imminent_weekly.sh ``` -✅ **CORE MERGE COMPLETE** - --- -## 🟡 PHASE 2: ENHANCEMENTS (Post-Merge) +### STEP 8: Setup Linux Cron Jobs (5 min) -### OPTIONAL 1: Country-Based Organization +**What**: Add scheduled jobs to run satellite downloads daily +**Where**: Add to `/etc/cron.d/smartcane_downloads` +**When**: Runs at 00:01 each day, staggered by 10 minutes per project + +```bash +# SmartCane Satellite Download - Staggered per project +# Runs at 00:01 each day, offset by 10 minutes per project + +1 0 * * * /path/to/smartcane/10_planet_download.sh --project=angata 2>&1 | logger +11 0 * * * /path/to/smartcane/10_planet_download.sh --project=chemba 2>&1 | logger +21 0 * * * /path/to/smartcane/10_planet_download.sh --project=xinavane 2>&1 | logger +31 0 * * * /path/to/smartcane/10_planet_download.sh --project=esa 2>&1 | logger +41 0 * * * /path/to/smartcane/10_planet_download.sh --project=simba 2>&1 | logger +51 0 * * * /path/to/smartcane/10_planet_download.sh --project=aura 2>&1 | logger +``` + +**Note**: Replace `/path/to/smartcane/` with absolute path. Find it with: +```bash +pwd # Run in project root +``` + + + +--- + +## STEP 9: Verification (15 min) + +Run these checks to confirm all changes work: + +### 10.1: Database + +```bash +php artisan migrate +# ✅ Expected: No errors, 2 new columns added +``` + +### 10.2: Create Project via UI + +``` +1. Visit project create form +2. Enter project name (e.g., "test_project") +3. Select Client Type: "Cane Supply" +4. Select Preferred Area Unit: "Acre (ac)" +5. Click Save +``` + +**✅ Expected**: +- Project created with client_type="cane_supply", preferred_area_unit="acre" +- Directories created: Data/, field_tiles/, field_tiles_CI/, reports/kpis/, etc. +- Project appears in list with correct settings + +### 10.3: Test Shell Wrapper + +```bash +./10_create_per_field_tiffs.sh --project=test_project --area_unit=acre +# ✅ Expected: R script runs without error +``` + +### 10.4: Test Area Unit Env Var + +```bash +cd r_app +export AREA_UNIT="acre" +Rscript -e "source('parameters_project.R'); print(AREA_UNIT_PREFERENCE)" +# ✅ Expected: [1] "acre" +``` + +### 10.5: Test Cron (if configured) + +```bash +sudo systemctl restart cron +grep "smartcane" /var/log/syslog +# ✅ Expected: cron job loaded successfully +``` + +--- + +## ✅ VERIFICATION CHECKLIST + +After all changes: + +```bash +# 1. Database migration ran successfully +php artisan migrate + +# 2. New columns exist +php artisan tinker +>>> DB::select("DESCRIBE projects WHERE Field IN ('client_type', 'preferred_area_unit')") +``` + +**Expected output**: +``` +client_type: enum AGRONOMIC_SUPPORT/CANE_SUPPLY +preferred_area_unit: enum HECTARE/ACRE +``` + +```bash +# 3. Create new project via UI +# → Visit project create form +# → Verify "Client Type" dropdown appears +# → Verify "Preferred Area Unit" toggle appears +# → Select values and save + +# 4. Verify directories created +ls -la laravel_app/storage/app/{your_new_project}/ +# → Should have: Data/, field_tiles/, field_tiles_CI/, reports/, etc. + +# 5. Test shell wrapper +./10_create_per_field_tiffs.sh --project={your_project} +# → Should run R script without error + +# 6. Test cron job (if Linux cron set up) +sudo systemctl restart cron # Or relevant service for your distro +``` + +--- + +These must be completed before production deployment: +- Country-based project organization (MZ/UG/TZ folders) +- Download scheduling (staggered by project) +- Project search feature +- Harvest prediction setup +- Area unit selection UI + +--- + +## STEP 10: Country-Based Organization **Why**: Organize projects by geographic location (MZ/UG/TZ folders) @@ -734,7 +962,7 @@ public function createProject() --- -### OPTIONAL 2: Download Scheduling +## STEP 11: Download Scheduling **Why**: Avoid API rate limits by staggering downloads per project at 00:01 @@ -786,7 +1014,7 @@ protected function schedule(Schedule $schedule) --- -### OPTIONAL 3: Project Search Feature +## STEP 12: Project Search Feature **Why**: Find projects quickly if there are many @@ -835,9 +1063,9 @@ public function getProjectsProperty() --- -### OPTIONAL 4: Harvest Date Prediction Setup +## STEP 13: Harvest Date Prediction Setup -**Why**: Enable harvest Date forecasting for cane_supply projects +**Why**: Enable harvest date forecasting for cane_supply projects **Create conda environment:** @@ -871,7 +1099,7 @@ python python_app/23_convert_harvest_format.py angata --- -### OPTIONAL 5: Area Unit Selection UI (Hectares vs Acres) +## STEP 14: Area Unit Selection UI (Hectares vs Acres) **Why**: Allow projects to choose their preferred area unit (hectares or acres) in reports and dashboards @@ -885,7 +1113,7 @@ python python_app/23_convert_harvest_format.py angata - ✅ Area now included in KPI outputs (CSV/RDS/Excel) from script 80 - ✅ Scripts 90/91 read area from KPI files instead of recalculating -**What needs implementation (for your Laravel colleague)**: +**What needs implementation:** **Step 1: Create database migration** @@ -925,7 +1153,7 @@ protected $fillable = [ ]; ``` -**Step 3: Add form UI** (`laravel_app/app/Livewire/Projects/ProjectManager.php` or Blade template) +**Step 3: Add form UI** (Blade template) ```blade
@@ -962,112 +1190,47 @@ $env:AREA_UNIT = $areaUnit # Set environment variable **Testing checklist**: - [ ] Database migration runs successfully -- [ ] Project form shows area unit radio buttons/dropdown +- [ ] Project form shows area unit radio buttons - [ ] Can select and save area unit preference - [ ] Area unit persists in database - [ ] Run script 80 with one project set to "hectare", observe KPI output - [ ] Run script 80 with another project set to "acre", compare outputs -- [ ] Reports (scripts 90/91) display area in user's chosen unit +- [ ] Reports display area in user's chosen unit **Notes**: - Default preference: "hectare" (metric standard) -- Conversion factor used: 0.404686 (1 hectare = 0.404686 acres) +- Conversion factor: 0.404686 (1 hectare = 0.404686 acres) - All area calculations use EPSG:6933 (equal-area projection) for accuracy - Area column in KPI outputs named dynamically: "Area_ha" or "Area_ac" --- -## 📊 Summary: What Gets Changed -| Category | Files Modified | Changes Required | -|----------|---|---| -| Database | migrations/ | 1 file: add client_type column | -| Models | Project.php | 5 edits: fillable, 3 methods, 1 new method | -| Jobs | 2 files | ProjectDownloadTiffJob (1 line), ProjectMosiacGeneratorJob (full array) | -| Forms | 3 files | ProjectManager.php, Blade template, MailingForm.php | -| Scripts | 5 files created | Shell wrappers (R/Python) | -| Python | 2 files created | requirements_harvest.txt, environment_pytorch.yml | +# SECTION C: POST-MERGE DATA MIGRATION STRATEGY -**TOTAL**: 6 files created + 6 files modified + 1 template modified = **13 changes** ---- +## Approach: Per-Project Options -## ✅ FINAL VERIFICATION +### Small Projects +**Projects**: aura, chemba, xinavane, esa, simba, john, huss, tpc, miwani, etc. -After ALL changes (core + optionals), test: +**Option A: Delete & Redownload** ✅ Recommended (Fastest setup) +1. Backup project folder externally (optional) +2. Delete project from Laravel UI (removes all data) +3. Recreate project with new form fields (client_type + area_unit selector) +4. Redownload satellite data for relevant date range +5. Run full R pipeline (Scripts 10-80) to generate KPI reports -```bash -# 1. Migration worked -php artisan migrate - -# 2. Download saves to correct path -# → Download image → check laravel_app/storage/app/{project}/field_tiles_CI/ - -# 3. Mosaic runs without errors -# → Create mosaic → check logs for no --data_dir errors - -# 4. Mail has 2 attachments for cane_supply -# → Send report for cane_supply project → verify report + KPI Excel - -# 5. Shell wrappers work -./10_create_per_field_tiffs.sh --project=angata -# → Should execute R script successfully - -# 6. Search works (if implemented) -# → Search for project by name on Projects page - -# 7. Country filter works (if implemented) -# → Filter projects by country code -``` - ---- - -## 🌍 POST-MERGE: Data Recreation Strategy - -After merge is live, existing projects need new directory structure. - -### Option A: Delete & Redownload (Small projects) -**Projects**: aura, chemba, xinavane, esa, simba - -``` -1. Backup project folder (optional) -2. Delete project from Laravel UI -3. Recreate with new client_type selector -4. Redownload 2-3 years of data (~50-150 GB per project) -5. Run pipeline normally -``` - -### Option B: Preserve merged_tif (Large projects) +### Large Projects **Projects**: angata -``` -1. Backup merged_tif/ folder externally -2. Delete all other folders in project -3. Keep only: merged_tif/ -4. Run Scripts 10-80 on existing data - → Regenerates field_tiles_CI/, reports/, etc. -5. No need to redownload -``` +**Option B: Preserve merged_tif** ✅ Recommended (No redownload) +1. Backup `merged_tif/` folder externally +2. Keep ONLY: `merged_tif/` folder +3. Delete: everything else (field_tiles/, reports/, logs/, etc.) +4. Run Scripts 10-80 from existing merged_tif data + - Regenerates field_tiles_CI/, reports/kpis/, weekly_mosaic/, etc. + - No satellite redownload needed + - KPI reports automatically use project's preferred_area_unit --- - -## 🔴 CORE vs OPTIONAL Quick List - -**MUST DO** (for merge): -- ✅ Database migration -- ✅ Project.php edits (4 path changes + 1 new method) -- ✅ Job edits (ProjectDownloadTiffJob, ProjectMosiacGeneratorJob) -- ✅ Form edits (ProjectManager, MailingForm, Blade template) -- ✅ 5 shell wrappers -- ✅ 2 Python files - -**NICE-TO-HAVE** (post-merge): -- 🟡 Country organization (adds ~45 min) -- 🟡 Download scheduling (adds ~30 min) -- 🟡 Project search (adds ~30 min) -- 🟡 Harvest prediction setup (adds ~20 min) - ---- - -**Ready to implement everything?** All code is copy-paste ready above. - diff --git a/python_app/environment_pytorch.yml b/python_app/environment_pytorch.yml new file mode 100644 index 0000000..08e81c8 --- /dev/null +++ b/python_app/environment_pytorch.yml @@ -0,0 +1,17 @@ +name: pytorch_gpu +channels: + - pytorch + - nvidia + - conda-forge +dependencies: + - python=3.10 + - pytorch::pytorch + - pytorch::torchvision + - pytorch::torchaudio + - pytorch::pytorch-cuda=11.8 + - gdal>=3.7.0 + - pip + - pip: + - sentinelhub>=3.9.0 + - shapely>=2.0.0 + - pyproj>=3.4.0 diff --git a/python_app/requirements_harvest.txt b/python_app/requirements_harvest.txt new file mode 100644 index 0000000..5af1a44 --- /dev/null +++ b/python_app/requirements_harvest.txt @@ -0,0 +1,8 @@ +torch>=2.0.0 +pandas>=1.5.0 +numpy>=1.23.0 +scikit-learn>=1.3.0 +GDAL>=3.7.0 +sentinelhub>=3.9.0 +shapely>=2.0.0 +pyproj>=3.4.0