Commit graph

54 commits

Author SHA1 Message Date
Timon 442d1ac46c Update date range in batch pipeline and enhance area unit preference handling 2026-02-24 15:12:03 +01:00
Timon e0bfbccf0e Refactor area calculation functions and enhance error handling; add area conversion utility 2026-02-24 13:07:55 +01:00
Timon b487cc983f Refactor translation function and update reports to use new area unit preference
- Renamed translation function from `t` to `tr_key` for clarity and consistency.
- Updated all instances of translation calls in `90_CI_report_with_kpis_agronomic_support.Rmd` and `91_CI_report_with_kpis_cane_supply.Rmd` to use `tr_key`.
- Introduced a new helper function `get_area_unit_label` to manage area unit preferences across the project.
- Modified area calculations in `91_CI_report_with_kpis_cane_supply.Rmd` to utilize area from analysis data instead of recalculating.
- Added area unit preference setting in `parameters_project.R` to allow for flexible reporting in either hectares or acres.
- Updated `MANUAL_PIPELINE_RUNNER.R` to include language parameter for report generation.
- Adjusted translations in the `translations.xlsx` file to reflect changes in the report structure.
2026-02-24 12:16:44 +01:00
DimitraVeropoulou 32c1bc358c
Merge branch 'code-improvements' into review_perField_code 2026-02-23 14:31:47 +01:00
DimitraVeropoulou d4553693e6 Refactor KPI reporting functions; update directory parameters for historical data loading and fix historical data for four weeks trend 2026-02-23 13:38:41 +01:00
Timon aaea7f62c7 Remove unused KPI field analysis directory from project setup function 2026-02-18 15:26:27 +01:00
Timon 55d67ef503 Merge branch 'code-improvements' into review_perField_code 2026-02-18 15:17:11 +01:00
Timon 29a85357a5 Refactor field analysis utilities and reporting; update column names for alerts, enhance directory setup, and improve KPI calculation robustness. 2026-02-18 14:53:44 +01:00
Timon b2d4093601 Refactor KPI reporting and status alert logic; streamline directory structure for KPI reports and enhance field analysis summaries with additional metrics and alerts. 2026-02-18 12:02:25 +01:00
Timon f1821dab59 Enhance project configuration and reporting utilities; update .gitignore for PNG exceptions, add CI change thresholds, and improve historical data handling in KPI calculations. 2026-02-18 10:54:42 +01:00
Timon 1f677f2626 Update terminology from DOY to DAH across multiple scripts and reports for consistency in crop age calculations 2026-02-18 09:36:54 +01:00
DimitraVeropoulou 8e2d796b41 Added aura in client type map 2026-02-16 14:45:00 +01:00
Timon d075302efa feat: Add spectral extraction and analysis scripts for SC-161 and SC-162
- Implemented `run_spectral_extraction.py` to batch extract spectral indices (NDVI, BSI, NDWI) from 4-band TIFF files, saving results in a structured CSV format.
- Created `spectral_features.py` for calculating various spectral indices, including NDVI, BSI, NDWI, CI_green, CI_red, GNDVI, SAVI, and EVI2.
- Added Jupyter notebook `02_season_normalization_analysis.ipynb` for analyzing season-length normalization, comparing ESA and Angata spectral indices, and visualizing growth patterns.
- Included functions for computing season age, plotting trajectories, and performing peak and amplitude analysis.
2026-02-16 10:11:43 +01:00
DimitraVeropoulou 3106871a81 Remove the creation of weekly_tile_max_mosaic folder 2026-02-12 16:55:34 +01:00
DimitraVeropoulou 4968162bfc Add huss in the client type map 2026-02-11 13:31:56 +01:00
Timon 2d6f062c27 changes again 2026-02-10 22:11:58 +01:00
Timon a40b9c1dfe coderabbit fixes 2026-02-10 22:01:02 +01:00
Timon dd83a9e27f some messy updates to script 90, wil l need to do some cleaning tomorrow... 2026-02-10 21:53:49 +01:00
Timon 3d3df151d3 Update .gitignore to include cache files, enhance R Markdown report paths, and improve KPI calculation script for clarity and error handling 2026-02-10 17:50:12 +01:00
Timon 3ee3f9e31c shaving off some more details... 2026-02-09 10:40:01 +01:00
Timon e16677eb78 aura until word creation works. word cration itself needs more work. 2026-02-04 12:24:02 +01:00
Timon 8d560ff155 SC-112 Phase 2 Complete: Restructure Script 80 utilities by client type
- Consolidate 80_kpi_utils.R, 80_weekly_stats_utils.R, 80_report_building_utils.R into three client-aware files:
  - 80_utils_common.R (50+ functions): Shared utilities, constants, and helpers
  - 80_utils_agronomic_support.R: AURA-specific KPI calculations (6 KPIs)
  - 80_utils_cane_supply.R: ANGATA placeholder for future expansion
- Move all internal helpers (calculate_cv, calculate_change_percentages, calculate_spatial_autocorrelation, extract_ci_values, etc.) to common
- Add MORAN_THRESHOLD_* constants to common
- Fix parameters_project.R field boundaries path (removed extra 'Data' directory)
- Update 80_calculate_kpis.R with conditional client-type sourcing logic
- Validate both ANGATA (cane_supply) and AURA (agronomic_support) workflows with comprehensive testing
- All 96+ functions accounted for; ready for production use
2026-02-03 17:21:35 +01:00
Timon 20e3d5b4a3 worked on sc 110 making a 10 utils file and making general log files. 2026-02-03 16:34:49 +01:00
Timon c313f87959 fixed issue 110 moving stuff to 00 common 2026-02-03 16:14:10 +01:00
Timon c4ef10f44f changing paths CS-109 2026-02-03 15:13:21 +01:00
Timon 14300f6832 updating inbetween stuff, not finished yet 2026-02-02 12:07:10 +01:00
Timon 4d6bba828f refactored 10 20 2026-01-29 21:10:24 +01:00
Timon d1f352f21c updated sc-91 2026-01-29 17:26:03 +01:00
Timon 4445f72e6f works for angata and aura 2026-01-29 16:04:04 +01:00
Timon dd915f9b9e adjusting scripts to work for all projects 2026-01-29 09:04:46 +01:00
Timon fabbf3214d Enhance harvest detection logic and testing framework
- Updated `detect_mosaic_mode` function to check for grid-size subdirectories in addition to tile-named files.
- Added comprehensive tests for DOY reset logic in `test_doy_logic.py`.
- Implemented feature extraction tests in `test_feature_extraction.py`.
- Created tests for growing window method in `test_growing_window_only.py`.
- Developed a complete model inference test in `test_model_inference.py`.
- Added a debug script for testing two-step refinement logic in `test_script22_debug.py`.
2026-01-15 14:30:54 +01:00
Timon 458b8247be Cleanup: Fix CI formula, reorganize shell scripts and test files
- Fixed CI calculation: changed from NDVI (NIR-Red)/(NIR+Red) to correct NIR/Green-1 formula in:
  * process_single_tile() function
  * create_ci_band() utility function
  * Updated create_mask_and_crop() documentation

- Renamed numbered shell scripts for clarity (matching R script numbering):
  * 01_run_planet_download -> 10_planet_download.sh
  * 02_run_ci_extraction -> 20_ci_extraction.sh
  * 03_run_growth_model -> 30_growth_model.sh
  * 04_run_mosaic_creation -> 40_mosaic_creation.sh
  * 09_run_calculate_kpis -> 80_calculate_kpis.sh
  * 10_run_kpi_report -> 90_kpi_report.sh

- Archived obsolete shell scripts to old_sh/:
  * build_mosaic.sh, build_report.sh, interpolate_growth_model.sh
  * 05_run_dashboard_report.sh, 06_run_crop_messaging.sh
  * 11_run_yield_prediction.sh/ps1
  * runcane.sh, runpython.sh, smartcane.sh, update_RDS.sh

- Deleted test/debug files and temporary outputs:
  * analyze_*.R, benchmark_gpu_vs_cpu.py, convert_angata_harvest.py
  * debug_mosaic.R, examine_kpi_results.R, generate_sar_report.R
  * inspect_8band_structure.R, inspect_tif_bands.R
  * old_working_utils.R, predict_harvest_operational.R
  * run_kpi_calculation.R, run_report.R, simple_sar_test.R
  * data_validation_tool/, harvest_ci_pattern_analysis.png, kpi_debug.out

- Enhanced harvest prediction: Added threshold tuning (0.40-0.45) and field type handling

- Enhanced mosaic creation: Improved tile detection and routing logic
2026-01-14 16:58:51 +01:00
Timon dc7c346854 updated whole workflow and architercutre md 2026-01-12 09:34:13 +01:00
Timon d22dc2f96e commit all stuff 2026-01-06 14:17:37 +01:00
Timon d5fd4bb463 Add KPI reporting system and deployment documentation
Major Changes:
- NEW: Scripts 09 & 10 for KPI calculation and enhanced reporting
- NEW: Shell script wrappers (01-10) for easier execution
- NEW: R packages flextable and officer for enhanced Word reports
- NEW: DEPLOYMENT_README.md with complete deployment guide
- RENAMED: Numbered R scripts (02, 03, 04) for clarity
- REMOVED: Old package management scripts (using renv only)
- UPDATED: Workflow now uses scripts 09->10 instead of 05

Files Changed: 90+ files
New Packages: flextable, officer
New Scripts: 09_run_calculate_kpis.sh, 10_run_kpi_report.sh
Documentation: DEPLOYMENT_README.md, EMAIL_TO_ADMIN.txt

See DEPLOYMENT_README.md for full deployment instructions.
2025-10-14 11:49:30 +02:00
Martin Folkerts d258a2bf2e cleaned the folders, made some parameters 2025-09-05 15:23:41 +02:00
Timon 07aee7bed1 –[200~Improve CI report visualization and migrate to terra package
- Replace raster package with terra throughout the codebase
- Update map visualizations with better layout and legends
- Add descriptive headers to report sections
- Improve map legend positioning and sizing
- Enhance error handling for missing data
- Remove redundant legends in field-specific visualizations
- Optimize figure dimensions to prevent page overflow
- Expand documentation of CI index and report components
- Update package dependencies in packages.
2025-04-22 20:55:02 +02:00
Martin Folkerts ddb4fdbc02 updated date format for excel 2024-10-28 15:57:23 +01:00
Martin Folkerts f08da04c3b added DownloadCreditsCommand some updates 2024-10-23 09:34:52 +02:00
Martin Folkerts 056b8dd67b okay 2024-09-17 14:56:33 +02:00
Martin Folkerts 4beca30d2e okay 2024-08-30 16:27:32 +02:00
Martin Folkerts 7bf4f0930d added log as project file; 2024-08-30 16:16:26 +02:00
Martin Folkerts 499744db6d moved logging arroudn 2024-08-30 12:09:13 +02:00
Timon 611bc17017 adjusted all functions and cleaned scripts 2024-08-29 16:35:59 +02:00
Martin Folkerts 2598306643 removed chemba as a option from parameters so it is treathed as a regular project 2024-08-28 10:18:53 +02:00
Martin Folkerts 64a576d638 okay 2024-08-20 17:01:58 +02:00
Martin Folkerts 5645ad83e5 changes 2024-07-17 14:53:46 +02:00
Martin Folkerts 0b1826dccb wip 2024-07-15 11:45:21 +02:00
Martin Folkerts d9fafcea43 wip 2024-07-02 16:51:55 +02:00
Martin Folkerts af1135e52a wip 2024-06-25 13:37:45 +02:00