Commit graph

64 commits

Author SHA1 Message Date
Timon ddeab0e3bd moved files, added sys architecture to webapps 2026-02-19 12:42:54 +01:00
Timon a2d9f00ed1 Merge branch 'review_perField_code' into code-improvements 2026-02-18 15:22:37 +01:00
Timon 55d67ef503 Merge branch 'code-improvements' into review_perField_code 2026-02-18 15:17:11 +01:00
Timon 29a85357a5 Refactor field analysis utilities and reporting; update column names for alerts, enhance directory setup, and improve KPI calculation robustness. 2026-02-18 14:53:44 +01:00
Timon 1500bbcb1c Refactor KPI calculations and reporting utilities; normalize field details columns, update area change metrics, and enhance .gitignore for PNG exceptions. 2026-02-18 13:26:32 +01:00
Timon 1f677f2626 Update terminology from DOY to DAH across multiple scripts and reports for consistency in crop age calculations 2026-02-18 09:36:54 +01:00
Timon e4e19df0c7 Update CI report parameters and improve map legend configurations
- Changed report date in CI report for cane supply to "2026-02-04".
- Updated output file naming convention for agronomic support report to reflect new report date.
- Enhanced map creation functions to allow customizable legend positions and improved layout settings.
- Adjusted widths for map arrangements to ensure better visual representation.
- Fixed minor issues in ggplot aesthetics for clearer legend positioning and improved readability.
- Corrected field size unit from hectares to acres in KPI summary generation.
2026-02-17 13:46:43 +01:00
Timon 8bc151ee4c chore: Remove obsolete .gitignore files from python_app and r_app 2026-02-16 10:28:33 +01:00
DimitraVeropoulou a9840171cb Enhance logging functionality and update report generation scripts
- Improved safe_log function to include timestamps and conditional logging
- Added diagnostic messages for field visualization processing
- Updated CI map rendering parameters for consistency
- Refined raster mapping functions in report_utils for clarity
- Added .png files to .gitignore
2026-02-16 10:21:55 +01:00
Timon d075302efa feat: Add spectral extraction and analysis scripts for SC-161 and SC-162
- Implemented `run_spectral_extraction.py` to batch extract spectral indices (NDVI, BSI, NDWI) from 4-band TIFF files, saving results in a structured CSV format.
- Created `spectral_features.py` for calculating various spectral indices, including NDVI, BSI, NDWI, CI_green, CI_red, GNDVI, SAVI, and EVI2.
- Added Jupyter notebook `02_season_normalization_analysis.ipynb` for analyzing season-length normalization, comparing ESA and Angata spectral indices, and visualizing growth patterns.
- Included functions for computing season age, plotting trajectories, and performing peak and amplitude analysis.
2026-02-16 10:11:43 +01:00
Timon 6f54ed263b wip 2026-02-10 19:56:57 +01:00
Timon cfd29fa84d sat downloader only starts tiling at 25+ fields in geojson 2026-02-10 19:51:02 +01:00
Timon 14300f6832 updating inbetween stuff, not finished yet 2026-02-02 12:07:10 +01:00
Timon dd915f9b9e adjusting scripts to work for all projects 2026-01-29 09:04:46 +01:00
Timon 9a55d2fcf8 Refactor full pipeline script to include intelligent checking of existing outputs and dynamic execution of scripts. Added new Python scripts for RGB validation and evaluation template creation. Enhanced error handling and logging throughout the pipeline. 2026-01-27 08:58:06 +01:00
Timon c11b10a73f working on visualisation, not working too well... 2026-01-18 11:25:31 +01:00
Timon 41bbc370f2 harvest pred working better now 2026-01-18 08:22:02 +01:00
Timon fabbf3214d Enhance harvest detection logic and testing framework
- Updated `detect_mosaic_mode` function to check for grid-size subdirectories in addition to tile-named files.
- Added comprehensive tests for DOY reset logic in `test_doy_logic.py`.
- Implemented feature extraction tests in `test_feature_extraction.py`.
- Created tests for growing window method in `test_growing_window_only.py`.
- Developed a complete model inference test in `test_model_inference.py`.
- Added a debug script for testing two-step refinement logic in `test_script22_debug.py`.
2026-01-15 14:30:54 +01:00
Timon d6448b8703 Refactor: Update system architecture documentation to reflect Python integration and add complete pipeline overview 2026-01-15 10:25:38 +01:00
Timon 4c7ca85d29 Add script 23: Convert harvest format from production to standard, remove diagnostic script 2026-01-15 09:28:13 +01:00
Timon 7b347ddba6 Fix: Revert run_phase1_growing_window to original GitHub version (off-by-one fix) and move test scripts to harvest_detection_experiments/tests 2026-01-15 09:08:07 +01:00
Timon 458b8247be Cleanup: Fix CI formula, reorganize shell scripts and test files
- Fixed CI calculation: changed from NDVI (NIR-Red)/(NIR+Red) to correct NIR/Green-1 formula in:
  * process_single_tile() function
  * create_ci_band() utility function
  * Updated create_mask_and_crop() documentation

- Renamed numbered shell scripts for clarity (matching R script numbering):
  * 01_run_planet_download -> 10_planet_download.sh
  * 02_run_ci_extraction -> 20_ci_extraction.sh
  * 03_run_growth_model -> 30_growth_model.sh
  * 04_run_mosaic_creation -> 40_mosaic_creation.sh
  * 09_run_calculate_kpis -> 80_calculate_kpis.sh
  * 10_run_kpi_report -> 90_kpi_report.sh

- Archived obsolete shell scripts to old_sh/:
  * build_mosaic.sh, build_report.sh, interpolate_growth_model.sh
  * 05_run_dashboard_report.sh, 06_run_crop_messaging.sh
  * 11_run_yield_prediction.sh/ps1
  * runcane.sh, runpython.sh, smartcane.sh, update_RDS.sh

- Deleted test/debug files and temporary outputs:
  * analyze_*.R, benchmark_gpu_vs_cpu.py, convert_angata_harvest.py
  * debug_mosaic.R, examine_kpi_results.R, generate_sar_report.R
  * inspect_8band_structure.R, inspect_tif_bands.R
  * old_working_utils.R, predict_harvest_operational.R
  * run_kpi_calculation.R, run_report.R, simple_sar_test.R
  * data_validation_tool/, harvest_ci_pattern_analysis.png, kpi_debug.out

- Enhanced harvest prediction: Added threshold tuning (0.40-0.45) and field type handling

- Enhanced mosaic creation: Improved tile detection and routing logic
2026-01-14 16:58:51 +01:00
Timon 62106eeb3b Update: source() calls to reference renamed util files (20_, 30_, 40_) 2026-01-14 11:24:27 +01:00
Timon ab19c2b99d Rename: python scripts to match pipeline stage numbers 2026-01-14 11:21:05 +01:00
Timon fc7e5f1ee0 Enhance download progress feedback and optimize tile overlap checks
- Added a progress bar to the tile download process for better user feedback.
- Simplified the tile overlap checking logic in the R script to improve performance and readability.
2026-01-13 11:30:38 +01:00
Timon 4f5bb1be9f test 2026-01-12 16:40:51 +01:00
Timon cb63cf00b8 updated code to fit in workflow better 2026-01-12 16:33:23 +01:00
Timon d1303dd218 testing new branch 2026-01-12 14:56:54 +01:00
Timon 182d13d593 wip 2026-01-12 14:55:25 +01:00
Timon dc7c346854 updated whole workflow and architercutre md 2026-01-12 09:34:13 +01:00
Timon d22dc2f96e commit all stuff 2026-01-06 14:17:37 +01:00
Timon d5fd4bb463 Add KPI reporting system and deployment documentation
Major Changes:
- NEW: Scripts 09 & 10 for KPI calculation and enhanced reporting
- NEW: Shell script wrappers (01-10) for easier execution
- NEW: R packages flextable and officer for enhanced Word reports
- NEW: DEPLOYMENT_README.md with complete deployment guide
- RENAMED: Numbered R scripts (02, 03, 04) for clarity
- REMOVED: Old package management scripts (using renv only)
- UPDATED: Workflow now uses scripts 09->10 instead of 05

Files Changed: 90+ files
New Packages: flextable, officer
New Scripts: 09_run_calculate_kpis.sh, 10_run_kpi_report.sh
Documentation: DEPLOYMENT_README.md, EMAIL_TO_ADMIN.txt

See DEPLOYMENT_README.md for full deployment instructions.
2025-10-14 11:49:30 +02:00
Martin Folkerts d258a2bf2e cleaned the folders, made some parameters 2025-09-05 15:23:41 +02:00
Martin Folkerts b169669d64 fix splitting area was set to wide 2025-07-23 08:30:45 +02:00
Timon 81e68096ba changed the evalscript, less strict version 2025-07-21 11:15:09 +02:00
Timon 6efcc8cfec Fix CI report pipeline: update tmap v4 syntax, add continuous color scales, fix formatting
- Updated all CI maps to use tm_scale_continuous() for proper tmap v4 compatibility
- Added fixed color scale limits (1-8 for CI, -3 to +3 for differences) for consistent field comparison
- Fixed YAML header formatting issues in CI_report_dashboard_planet.Rmd
- Positioned RGB map before CI overview map as requested
- Removed all obsolete use_breaks parameter references
- Enhanced error handling and logging throughout the pipeline
- Added new experimental analysis scripts and improvements to mosaic creation
2025-06-19 20:37:20 +02:00
Timon fbc8f5e11b wip 2025-04-22 17:43:11 +02:00
Timon 29e2867b41 adding docx 2025-03-25 10:42:04 +01:00
Martin Folkerts c8ef406545 added Catalog package with config and changed iteration to use available_slots 2024-11-11 10:26:02 +01:00
Martin Folkerts c131b18cec added source from cleanup sentinal Code branch 2024-11-07 11:48:23 +01:00
Timon 119084ec41 only download images on dates with actual data, not empty dates 2024-11-01 17:04:17 +01:00
Martin Folkerts f08da04c3b added DownloadCreditsCommand some updates 2024-10-23 09:34:52 +02:00
Martin Folkerts dce7344947 updated project 2024-08-26 14:01:57 +02:00
Martin Folkerts 64a576d638 okay 2024-08-20 17:01:58 +02:00
Martin Folkerts 1e841a0aa6 wip 2024-06-18 09:17:57 +02:00
Martin Folkerts e2138a378a some updates 2024-05-13 19:35:17 +02:00
Martin Folkerts 5886f59bd8 wip 2024-04-18 13:43:14 +02:00
Timon 238520ff89 parameterised harvestdata etc 2024-03-12 22:18:57 +01:00
Martin Folkerts e996a274db merged testing 2024-03-05 09:40:46 +01:00
Timon 6807b30312 redady for merge 2024-03-05 09:20:34 +01:00