Timon/SmartCane
1
1
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
340 commits 1 branch 0 tags 252 MiB
Commit graph

340 commits

This branch
This branch
All branches
Author SHA1 Message Date
Timon Weitkamp 24401d436a
Merge pull request #4 from TimonWeitkamp/timon/sc-64-apply-the-new-structure-to-the-code 
Timon/sc 64 apply the new structure to the code. mainly new numbering structure, and cleaned some code.
 2026-01-27 09:12:51 +01:00
Timon 9a55d2fcf8 Refactor full pipeline script to include intelligent checking of existing outputs and dynamic execution of scripts. Added new Python scripts for RGB validation and evaluation template creation. Enhanced error handling and logging throughout the pipeline. 2026-01-27 08:58:06 +01:00
Timon 4143bdf4d7 updateing csv file 2026-01-20 12:06:45 +01:00
Timon c11b10a73f working on visualisation, not working too well... 2026-01-18 11:25:31 +01:00
Timon 9f312131d7 integration lstm 31 and 80 complete - angata kpi file now dynamic 2026-01-18 10:14:08 +01:00
Timon 1f5add7485 refactored script 80 2026-01-18 09:27:16 +01:00
Timon 41bbc370f2 harvest pred working better now 2026-01-18 08:22:02 +01:00
Timon 7975f8ad06 8 week data working now 2026-01-18 07:39:30 +01:00
Timon 3e4430b3be added last columns of trends, only harvest imminent and age left to do 2026-01-16 09:10:39 +01:00
Timon 8d84c8cab5 phase 2 cv trend implemented 2026-01-16 08:30:47 +01:00
Timon 6e88acef25 added some more stats and functions calculate and export csv and rds file for future ref 2026-01-16 08:04:45 +01:00
Timon 4e94a9a78b expanding csv table 2026-01-15 15:35:16 +01:00
Timon fabbf3214d Enhance harvest detection logic and testing framework 
- Updated `detect_mosaic_mode` function to check for grid-size subdirectories in addition to tile-named files.
- Added comprehensive tests for DOY reset logic in `test_doy_logic.py`.
- Implemented feature extraction tests in `test_feature_extraction.py`.
- Created tests for growing window method in `test_growing_window_only.py`.
- Developed a complete model inference test in `test_model_inference.py`.
- Added a debug script for testing two-step refinement logic in `test_script22_debug.py`.
 2026-01-15 14:30:54 +01:00
Timon d6448b8703 Refactor: Update system architecture documentation to reflect Python integration and add complete pipeline overview 2026-01-15 10:25:38 +01:00
Timon 4c7ca85d29 Add script 23: Convert harvest format from production to standard, remove diagnostic script 2026-01-15 09:28:13 +01:00
Timon 7b347ddba6 Fix: Revert run_phase1_growing_window to original GitHub version (off-by-one fix) and move test scripts to harvest_detection_experiments/tests 2026-01-15 09:08:07 +01:00
Timon 458b8247be Cleanup: Fix CI formula, reorganize shell scripts and test files 
- Fixed CI calculation: changed from NDVI (NIR-Red)/(NIR+Red) to correct NIR/Green-1 formula in:
  * process_single_tile() function
  * create_ci_band() utility function
  * Updated create_mask_and_crop() documentation

- Renamed numbered shell scripts for clarity (matching R script numbering):
  * 01_run_planet_download -> 10_planet_download.sh
  * 02_run_ci_extraction -> 20_ci_extraction.sh
  * 03_run_growth_model -> 30_growth_model.sh
  * 04_run_mosaic_creation -> 40_mosaic_creation.sh
  * 09_run_calculate_kpis -> 80_calculate_kpis.sh
  * 10_run_kpi_report -> 90_kpi_report.sh

- Archived obsolete shell scripts to old_sh/:
  * build_mosaic.sh, build_report.sh, interpolate_growth_model.sh
  * 05_run_dashboard_report.sh, 06_run_crop_messaging.sh
  * 11_run_yield_prediction.sh/ps1
  * runcane.sh, runpython.sh, smartcane.sh, update_RDS.sh

- Deleted test/debug files and temporary outputs:
  * analyze_*.R, benchmark_gpu_vs_cpu.py, convert_angata_harvest.py
  * debug_mosaic.R, examine_kpi_results.R, generate_sar_report.R
  * inspect_8band_structure.R, inspect_tif_bands.R
  * old_working_utils.R, predict_harvest_operational.R
  * run_kpi_calculation.R, run_report.R, simple_sar_test.R
  * data_validation_tool/, harvest_ci_pattern_analysis.png, kpi_debug.out

- Enhanced harvest prediction: Added threshold tuning (0.40-0.45) and field type handling

- Enhanced mosaic creation: Improved tile detection and routing logic
 2026-01-14 16:58:51 +01:00
Timon d365b5838b Move: archive old KPI scripts (09, 09b, 09c, 20) to old_scripts/ for testing 2026-01-14 11:37:13 +01:00
Timon 04e1641a18 Create 80_calculate_kpis.R: unified KPI calculation (per-field SC-64 + farm-level metrics) 2026-01-14 11:36:06 +01:00
Timon 62106eeb3b Update: source() calls to reference renamed util files (20_, 30_, 40_) 2026-01-14 11:24:27 +01:00
Timon ab19c2b99d Rename: python scripts to match pipeline stage numbers 2026-01-14 11:21:05 +01:00
Timon b7ef583efa Rename: number utils to match their main scripts 2026-01-14 11:16:23 +01:00
Timon 5aa456fbf5 Rename: 01_ to 10_ for stage consistency 2026-01-14 11:12:11 +01:00
Timon e69a7c5465 Rename: align stage numbering with Bitbucket structure 2026-01-14 11:10:28 +01:00
Timon 252b4049b3 Rename: align stage numbering with Bitbucket structure 2026-01-14 11:08:33 +01:00
Timon ef29cb5c2c Move experimental files to r_app/experiments 2026-01-14 11:07:40 +01:00
Timon 6d5509dd1b Remove: cleanup test/draft files not in Bitbucket 2026-01-14 11:05:57 +01:00
Timon dfa0aa900d updated website and small things 2026-01-14 11:02:45 +01:00
Timon fc7e5f1ee0 Enhance download progress feedback and optimize tile overlap checks 
- Added a progress bar to the tile download process for better user feedback.
- Simplified the tile overlap checking logic in the R script to improve performance and readability.
 2026-01-13 11:30:38 +01:00
Timon 03f269302a SC-58: Implement grid filtering and per-date tiling 
- Filter master grid to only overlapping tiles based on GeoJSON field boundaries
- Implement per-date tiling for resumable processing (can stop/restart anytime)
- Add grid-size versioning (subfolder structure: daily_tiles_split/5x5/, etc)
- Skip already-processed dates for idempotent execution
- Upfront extent checking (one-time), per-date processing (resumable)
- Reduce storage by excluding empty tiles outside field boundaries
 2026-01-13 09:35:51 +01:00
Timon 4f5bb1be9f test 2026-01-12 16:40:51 +01:00
Timon cb63cf00b8 updated code to fit in workflow better 2026-01-12 16:33:23 +01:00
Timon d1303dd218 testing new branch 2026-01-12 14:56:54 +01:00
Timon 182d13d593 wip 2026-01-12 14:55:25 +01:00
Timon dc7c346854 updated whole workflow and architercutre md 2026-01-12 09:34:13 +01:00
Timon d22dc2f96e commit all stuff 2026-01-06 14:17:37 +01:00
Timon d5fd4bb463 Add KPI reporting system and deployment documentation 
Major Changes:
- NEW: Scripts 09 & 10 for KPI calculation and enhanced reporting
- NEW: Shell script wrappers (01-10) for easier execution
- NEW: R packages flextable and officer for enhanced Word reports
- NEW: DEPLOYMENT_README.md with complete deployment guide
- RENAMED: Numbered R scripts (02, 03, 04) for clarity
- REMOVED: Old package management scripts (using renv only)
- UPDATED: Workflow now uses scripts 09->10 instead of 05

Files Changed: 90+ files
New Packages: flextable, officer
New Scripts: 09_run_calculate_kpis.sh, 10_run_kpi_report.sh
Documentation: DEPLOYMENT_README.md, EMAIL_TO_ADMIN.txt

See DEPLOYMENT_README.md for full deployment instructions.
 2025-10-14 11:49:30 +02:00
Martin Folkerts 1795e81b4e updated .gitignore 2025-09-05 16:55:40 +02:00
Martin Folkerts 9a59c6adc1 Remove public/build and public/vendor from Git tracking 2025-09-05 16:53:35 +02:00
Martin Folkerts d258a2bf2e cleaned the folders, made some parameters 2025-09-05 15:23:41 +02:00
Martin Folkerts b169669d64 fix splitting area was set to wide 2025-07-23 08:30:45 +02:00
Martin Folkerts d0b90b079a build assets 2025-07-21 17:03:02 +02:00
Martin Folkerts f8a980a283 Merge branch 'code-improvements' 2025-07-21 17:02:06 +02:00
Timon 81e68096ba changed the evalscript, less strict version 2025-07-21 11:15:09 +02:00
Martin Folkerts af3fd3c6ab Merged feature/readonly-account into master 2025-07-02 11:59:55 +00:00
Martin Folkerts e1e359503d added role viewer and command to create user viewer updated application to hide components for viewer 2025-06-30 14:49:21 +02:00
Martin Folkerts d4648bdd54 updated to laravel 12 2025-06-30 11:01:27 +02:00
Martin Folkerts 77d67c36ca Merge branch 'code-improvements' 2025-06-25 11:03:33 +02:00
Timon a6aa002d33 minor visualistion chagnes 2025-06-25 11:01:37 +02:00
Martin Folkerts 0e7f5cf11d Merge branch 'code-improvements' 2025-06-25 10:47:11 +02:00