- Consolidate 80_kpi_utils.R, 80_weekly_stats_utils.R, 80_report_building_utils.R into three client-aware files:
- 80_utils_common.R (50+ functions): Shared utilities, constants, and helpers
- 80_utils_agronomic_support.R: AURA-specific KPI calculations (6 KPIs)
- 80_utils_cane_supply.R: ANGATA placeholder for future expansion
- Move all internal helpers (calculate_cv, calculate_change_percentages, calculate_spatial_autocorrelation, extract_ci_values, etc.) to common
- Add MORAN_THRESHOLD_* constants to common
- Fix parameters_project.R field boundaries path (removed extra 'Data' directory)
- Update 80_calculate_kpis.R with conditional client-type sourcing logic
- Validate both ANGATA (cane_supply) and AURA (agronomic_support) workflows with comprehensive testing
- All 96+ functions accounted for; ready for production use
- Changed gap calculation from Q25 to 2σ below median method (kpi_utils.R)
- Integrated gap filling into script 80 with tile-based processing
- Added Gap_score column to field analysis output (Excel/CSV/RDS)
- Fixed memory issues by processing 25 tiles individually instead of merging
- Fixed Field_id matching to ensure gap scores populate correctly
Gap scores now calculate for all 1185 fields with range 0-11.25%
Works with tile-based mosaics (Angata 5x5 grid) without memory errors
- Script 80: Add fallback logic to detect and handle single-file mosaics
- Weekly stats utils: Support both tile-based and single-file mosaic detection
- Pipeline runner: Auto-detect mosaic mode (tiled vs single-file)
- Flexible grid size detection for tile-based projects (5x5, 10x10, etc)
Fixes:
- Script 80 now checks weekly_tile_max/{grid_size} first, falls back to weekly_mosaic
- calculate_field_statistics handles both tile patterns and single-file patterns
- run_full_pipeline detects project mode automatically
- All verification checks are now flexible and don't assume fixed paths
Projects like 'aura' (small ROI < 10km) will use single-file approach automatically
Projects like 'angata' (large ROI >= 10km) will use tile-based approach automatically
- Updated `detect_mosaic_mode` function to check for grid-size subdirectories in addition to tile-named files.
- Added comprehensive tests for DOY reset logic in `test_doy_logic.py`.
- Implemented feature extraction tests in `test_feature_extraction.py`.
- Created tests for growing window method in `test_growing_window_only.py`.
- Developed a complete model inference test in `test_model_inference.py`.
- Added a debug script for testing two-step refinement logic in `test_script22_debug.py`.
- Added a progress bar to the tile download process for better user feedback.
- Simplified the tile overlap checking logic in the R script to improve performance and readability.
Major Changes:
- NEW: Scripts 09 & 10 for KPI calculation and enhanced reporting
- NEW: Shell script wrappers (01-10) for easier execution
- NEW: R packages flextable and officer for enhanced Word reports
- NEW: DEPLOYMENT_README.md with complete deployment guide
- RENAMED: Numbered R scripts (02, 03, 04) for clarity
- REMOVED: Old package management scripts (using renv only)
- UPDATED: Workflow now uses scripts 09->10 instead of 05
Files Changed: 90+ files
New Packages: flextable, officer
New Scripts: 09_run_calculate_kpis.sh, 10_run_kpi_report.sh
Documentation: DEPLOYMENT_README.md, EMAIL_TO_ADMIN.txt
See DEPLOYMENT_README.md for full deployment instructions.
- Add renv.lock with exact package versions for reproducibility
- Add package_manager.R for automated package installation/updates
- Add extract_current_versions.R for package discovery
- Configure renv environment for team collaboration
All team members can now run source('r_app/package_manager.R') to get identical package environment
- Updated all CI maps to use tm_scale_continuous() for proper tmap v4 compatibility
- Added fixed color scale limits (1-8 for CI, -3 to +3 for differences) for consistent field comparison
- Fixed YAML header formatting issues in CI_report_dashboard_planet.Rmd
- Positioned RGB map before CI overview map as requested
- Removed all obsolete use_breaks parameter references
- Enhanced error handling and logging throughout the pipeline
- Added new experimental analysis scripts and improvements to mosaic creation
Added explanatory text for all visualizations
Fixed priority map color scheme (red=high priority, green=low priority)
Improved error handling in farm health data calculations
Added fallback mechanisms for missing data
- Refactored ci_plot and cum_ci_plot functions to accept explicit parameters
- Added default values for backward compatibility
- Updated CI_report_dashboard_planet.Rmd to pass explicit parameters
- Added input validation for better error handling
- Added parameter documentation in function headers
- Removed reliance on global variables for better maintainability
- Replace raster package with terra throughout the codebase
- Update map visualizations with better layout and legends
- Add descriptive headers to report sections
- Improve map legend positioning and sizing
- Enhance error handling for missing data
- Remove redundant legends in field-specific visualizations
- Optimize figure dimensions to prevent page overflow
- Expand documentation of CI index and report components
- Update package dependencies in packages.
Timon
DimitraVeropoulou
