Timon
d5fd4bb463
Major Changes: - NEW: Scripts 09 & 10 for KPI calculation and enhanced reporting - NEW: Shell script wrappers (01-10) for easier execution - NEW: R packages flextable and officer for enhanced Word reports - NEW: DEPLOYMENT_README.md with complete deployment guide - RENAMED: Numbered R scripts (02, 03, 04) for clarity - REMOVED: Old package management scripts (using renv only) - UPDATED: Workflow now uses scripts 09->10 instead of 05 Files Changed: 90+ files New Packages: flextable, officer New Scripts: 09_run_calculate_kpis.sh, 10_run_kpi_report.sh Documentation: DEPLOYMENT_README.md, EMAIL_TO_ADMIN.txt See DEPLOYMENT_README.md for full deployment instructions.
300 lines
8.5 KiB
Markdown
300 lines
8.5 KiB
Markdown
# SmartCane Deployment Guide
|
|
**Quick Reference for Bitbucket Push & Server Deployment**
|
|
|
|
---
|
|
|
|
## 🎯 TL;DR - WHAT YOU NEED TO KNOW
|
|
|
|
### What's New:
|
|
- ✅ **Scripts 09 & 10** are NEW - they generate reports WITH KPIs (field uniformity, stress detection)
|
|
- ✅ **2 new packages** to install: `flextable` and `officer` (for better tables in Word reports)
|
|
- ✅ **Shell script wrappers** (01-10) make execution easier
|
|
|
|
### Workflow Change:
|
|
```bash
|
|
# OLD (master branch):
|
|
Manual R script execution
|
|
|
|
# NEW (code-improvements branch):
|
|
./01_run_planet_download.sh
|
|
./02_run_ci_extraction.sh
|
|
./03_run_growth_model.sh
|
|
./04_run_mosaic_creation.sh
|
|
# SKIP 05 (old report without KPIs)
|
|
./09_run_calculate_kpis.sh # NEW - calculate KPIs first
|
|
./10_run_kpi_report.sh # NEW - generate report WITH KPIs
|
|
```
|
|
|
|
### For Your Admin:
|
|
1. Install 2 new R packages: `Rscript -e "renv::restore()"`
|
|
2. Run scripts in order: 01→02→03→04→09→10 (skip 05)
|
|
3. Script 10 parameters are configurable (see below)
|
|
|
|
**That's it!** Read below for details if needed.
|
|
|
|
---
|
|
|
|
## 📦 WHAT CHANGED FROM MASTER BRANCH
|
|
|
|
### NEW Scripts (not in master):
|
|
| Script | Purpose | Status |
|
|
|--------|---------|--------|
|
|
| `09_run_calculate_kpis.sh` | Calculate field KPIs | ⭐ Required |
|
|
| `10_run_kpi_report.sh` | Generate reports WITH KPIs | ⭐ Required |
|
|
| `01-05_run_*.sh` | Shell wrappers for existing R scripts | ✅ Helpful |
|
|
|
|
### NEW R Files:
|
|
- `r_app/09_calculate_kpis.R` - KPI calculation logic
|
|
- `r_app/10_CI_report_with_kpis_simple.Rmd` - Enhanced report template
|
|
- `r_app/kpi_utils.R` - KPI utility functions
|
|
|
|
### NEW R Packages (in renv.lock):
|
|
- `flextable` - Enhanced table formatting for Word
|
|
- `officer` - Word document manipulation
|
|
|
|
### RENAMED Files:
|
|
- `ci_extraction.R` → `02_ci_extraction.R`
|
|
- `interpolate_growth_model.R` → `03_interpolate_growth_model.R`
|
|
- `mosaic_creation.R` → `04_mosaic_creation.R`
|
|
|
|
### DELETED Files:
|
|
- Old package management scripts (now using renv only)
|
|
- Duplicate geometry files
|
|
- Laravel build artifacts (will regenerate)
|
|
|
|
**Total:** 90 files changed, +12,309 lines added, -7,132 lines removed
|
|
|
|
---
|
|
|
|
## 💻 LINUX SERVER DEPLOYMENT
|
|
|
|
### Step 1: Install System Dependencies
|
|
```bash
|
|
sudo apt-get update
|
|
sudo apt-get install -y \
|
|
libgdal-dev libgeos-dev libproj-dev libudunits2-dev \
|
|
libcurl4-openssl-dev libssl-dev libxml2-dev \
|
|
libfontconfig1-dev libharfbuzz-dev libfribidi-dev \
|
|
pandoc pandoc-citeproc
|
|
```
|
|
|
|
### Step 2: Clone & Setup
|
|
```bash
|
|
git clone <bitbucket-url> smartcane
|
|
cd smartcane
|
|
chmod +x *.sh
|
|
dos2unix *.sh # Fix Windows line endings
|
|
```
|
|
|
|
### Step 3: Install R Packages
|
|
```bash
|
|
Rscript -e "renv::restore()"
|
|
```
|
|
|
|
### Step 4: Test Workflow
|
|
```bash
|
|
./09_run_calculate_kpis.sh aura
|
|
./10_run_kpi_report.sh --data_dir=aura --filename=test.docx
|
|
ls laravel_app/storage/app/aura/reports/
|
|
```
|
|
|
|
---
|
|
|
|
## ⚙️ SCRIPT 10 PARAMETERS (for Laravel UI)
|
|
|
|
### Configurable Parameters (add to Laravel project settings):
|
|
|
|
| Parameter | Type | Default | Options | Description |
|
|
|-----------|------|---------|---------|-------------|
|
|
| `borders` | Boolean | FALSE | TRUE/FALSE | Show field borders on maps |
|
|
| `ci_plot_type` | String | both | absolute/cumulative/both | Type of CI plots |
|
|
| `colorblind_friendly` | Boolean | TRUE | TRUE/FALSE | Use accessible color palettes |
|
|
| `facet_by_season` | Boolean | FALSE | TRUE/FALSE | Split plots by season |
|
|
| `x_axis_unit` | String | days | days/weeks | X-axis time unit |
|
|
|
|
### Auto-Set Parameters (managed by system):
|
|
|
|
| Parameter | Source | Description |
|
|
|-----------|--------|-------------|
|
|
| `filename` | Auto-generated | Set by system: `{project}_{date}.docx` |
|
|
| `report_date` | Current date | Automatically uses today's date |
|
|
| `mail_day` | Current day | Automatically uses current weekday |
|
|
| `data_dir` | Project name | Set from Laravel project configuration |
|
|
|
|
### Laravel Implementation Notes:
|
|
|
|
1. **Create settings per project** with the 5 configurable parameters above
|
|
2. **Auto-generate filename**: `${project_name}_report_${date}.docx`
|
|
3. **Auto-set dates**: Use current date/day when script runs
|
|
4. **data_dir**: Pull from project's directory name in Laravel
|
|
|
|
**Example usage:**
|
|
```bash
|
|
./10_run_kpi_report.sh \
|
|
--data_dir=aura \
|
|
--report_date=$(date +%Y-%m-%d) \
|
|
--filename="aura_report_$(date +%Y%m%d).docx" \
|
|
--mail_day=$(date +%A) \
|
|
--borders=FALSE \
|
|
--ci_plot_type=both \
|
|
--colorblind_friendly=TRUE \
|
|
--facet_by_season=FALSE \
|
|
--x_axis_unit=days
|
|
```
|
|
|
|
---
|
|
|
|
## 🚨 COMMON DEPLOYMENT ERRORS
|
|
|
|
### Error 1: Package Compilation Fails
|
|
```
|
|
ERROR: configuration failed for package 'sf'
|
|
```
|
|
**Solution:** Install system dependencies (see Step 1 above)
|
|
|
|
### Error 2: Permission Denied
|
|
```
|
|
bash: ./10_run_kpi_report.sh: Permission denied
|
|
```
|
|
**Solution:** `chmod +x *.sh`
|
|
|
|
### Error 3: Line Ending Issues
|
|
```
|
|
/bin/bash^M: bad interpreter
|
|
```
|
|
**Solution:** `dos2unix *.sh` or `sed -i 's/\r$//' *.sh`
|
|
|
|
### Error 4: Pandoc Missing
|
|
```
|
|
Error: pandoc version 1.12.3 or higher is required
|
|
```
|
|
**Solution:** `sudo apt-get install -y pandoc`
|
|
|
|
### Error 5: Font Errors
|
|
```
|
|
Error in gdtools::...: font family not found
|
|
```
|
|
**Solution:** Install font libraries (libfontconfig1-dev, etc. - see Step 1)
|
|
|
|
---
|
|
|
|
## 📊 SCRIPT COMPARISON: Old vs New
|
|
|
|
### Script 05 (OLD - skip this):
|
|
- Basic CI maps ✅
|
|
- CI trend plots ✅
|
|
- Week-over-week change ✅
|
|
- **NO KPI metrics** ❌
|
|
- **NO field uniformity** ❌
|
|
- **NO priority detection** ❌
|
|
|
|
### Scripts 09 + 10 (NEW - use these):
|
|
- Everything from script 05 ✅
|
|
- **KPI metrics** ✅
|
|
- **Field uniformity (CV, Moran's I)** ✅
|
|
- **Priority classification** (urgent/monitor/no stress) ✅
|
|
- **Enhanced tables** (flextable formatting) ✅
|
|
- **Field stress detection** ✅
|
|
|
|
---
|
|
|
|
## ⚠️ WINDOWS → LINUX COMPATIBILITY
|
|
|
|
**Known issues when moving from Windows to Linux:**
|
|
|
|
| Issue | Windows | Linux | Solution |
|
|
|-------|---------|-------|----------|
|
|
| Path separators | `\` | `/` | Scripts use `here::here()` ✅ |
|
|
| Line endings | CRLF | LF | Run `dos2unix *.sh` |
|
|
| Package compilation | Binary | Source | Install system libs first |
|
|
| File permissions | Auto | Manual | Run `chmod +x *.sh` |
|
|
| R path | Fixed path | In PATH | Scripts auto-detect ✅ |
|
|
|
|
---
|
|
|
|
## ✅ DEPLOYMENT CHECKLIST
|
|
|
|
**Before pushing to Bitbucket:**
|
|
- [ ] Verify scripts 09 and 10 work locally
|
|
- [ ] Check renv.lock is committed
|
|
- [ ] Test workflow: 01→02→03→04→09→10
|
|
|
|
**After pulling on Linux server:**
|
|
- [ ] Install system dependencies (GDAL, GEOS, PROJ, Pandoc, fonts)
|
|
- [ ] Clone repository
|
|
- [ ] Fix line endings: `dos2unix *.sh`
|
|
- [ ] Set permissions: `chmod +x *.sh`
|
|
- [ ] Install R packages: `Rscript -e "renv::restore()"`
|
|
- [ ] Test with one project: `./09_run_calculate_kpis.sh aura`
|
|
- [ ] Generate test report: `./10_run_kpi_report.sh --data_dir=aura`
|
|
- [ ] Create Laravel UI for script 10 parameters
|
|
- [ ] Update any automation scripts to use new workflow
|
|
|
|
---
|
|
|
|
## 📂 KEY FILES TO KNOW
|
|
|
|
```
|
|
smartcane/
|
|
├── 01-04_*.sh # Data acquisition (existing workflow)
|
|
├── 05_*.sh # ❌ Old report (skip)
|
|
├── 09_*.sh # ✅ NEW - KPI calculation
|
|
├── 10_*.sh # ✅ NEW - Report with KPIs
|
|
├── renv.lock # Package versions (includes flextable/officer)
|
|
└── r_app/
|
|
├── 09_calculate_kpis.R # NEW
|
|
├── 10_CI_report_with_kpis_simple.Rmd # NEW
|
|
└── kpi_utils.R # NEW
|
|
```
|
|
|
|
---
|
|
|
|
## 🔄 EXAMPLE: Full Weekly Pipeline
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
# Complete weekly workflow for Aura farm
|
|
|
|
PROJECT="aura"
|
|
DATE=$(date +%Y-%m-%d)
|
|
|
|
# Step 1-4: Data acquisition
|
|
./01_run_planet_download.sh --project_dir=$PROJECT
|
|
./02_run_ci_extraction.sh --project_dir=$PROJECT
|
|
./03_run_growth_model.sh --project_dir=$PROJECT
|
|
./04_run_mosaic_creation.sh --data_dir=$PROJECT
|
|
|
|
# Step 5-6: KPI calculation & reporting (NEW)
|
|
./09_run_calculate_kpis.sh $PROJECT
|
|
./10_run_kpi_report.sh \
|
|
--data_dir=$PROJECT \
|
|
--report_date=$DATE \
|
|
--filename="${PROJECT}_${DATE}.docx" \
|
|
--colorblind_friendly=TRUE
|
|
|
|
echo "✅ Pipeline complete! Check output/"
|
|
```
|
|
|
|
---
|
|
|
|
## 📞 TROUBLESHOOTING
|
|
|
|
**If deployment fails:**
|
|
1. Check error against "Common Errors" section above
|
|
2. Verify system dependencies: `dpkg -l | grep libgdal`
|
|
3. Test R packages: `Rscript -e "library(flextable)"`
|
|
4. Check file structure: `ls laravel_app/storage/app/*/`
|
|
5. Review logs: `./10_run_kpi_report.sh 2>&1 | tee debug.log`
|
|
|
|
**Still stuck?** Contact developer with:
|
|
- Full error message
|
|
- Which script failed
|
|
- Output of `sessionInfo()` in R
|
|
- Server OS and R version
|
|
|
|
---
|
|
|
|
**Version:** 1.0
|
|
**Last Updated:** October 14, 2025
|
|
**Branch:** code-improvements (ready for merge to master)
|